Awesome
Vulkan glTF Scene Renderer
Pathtracer | Raster |
---|---|
Overview
This application demonstrates a dual-mode renderer for glTF 2.0 scenes, implementing both ray tracing and rasterization pipelines. It showcases the utilization of shared Vulkan resources across rendering modes, including geometry, materials, and textures.
Key Features
- glTF 2.0 (.gltf/.glb) scene loading
- Pathtracing with global illumination
- PBR-based rasterization
- HDR environment mapping and Sun & Sky simulation
- Advanced tone mapping
- Camera control system
- Extensive debug visualization options
Dependencies
- Vulkan SDK (latest version)
- Nvpro-Core framework
Build Instructions
- Clone the repositories
git clone https://github.com/nvpro-samples/nvpro_core.git
git clone https://github.com/nvpro-samples/vk_gltf_renderer.git
- Build the project
cd vk_gltf_renderer
mkdir build
cd build
cmake ..
cmake --build . --config release
- Run the application
.\bin_x64\Release\vk_gltf_renderer.exe
- Install [optional] : if you want to package the application
cmake --install .
Draco Compression
To enable Draco mesh compression, you need to enable the option CMake. In the GUI interface, you will see the option USE_DRACO
. If you are using the command line, you can add -DUSE_DRACO=ON
to the cmake command. This will download the Draco library and it will be included in the project.
glTF Core features
- glTF 2.0 (.gltf/.glb)
- images (HDR, PNG, JPEG, ...)
- buffers (geometry, animation, skinning, ...)
- textures (base color, normal, metallic, roughness, ...)
- materials (PBR, ...)
- animations
- skins
- cameras
- lights
- nodes
- scenes
- samplers
- textures
- extensions
What is currently not supported are animations and skins, multiple textures coordinates, morph targets, color at vertices, and some extensions.
GLTF Extensions
Here are the list of extensions that are supported by this application
- KHR_animation_pointer
- KHR_draco_mesh_compression
- KHR_lights_punctual
- KHR_materials_anisotropy
- KHR_materials_clearcoat
- KHR_materials_dispersion
- KHR_materials_emissive_strength
- KHR_materials_ior
- KHR_materials_iridescence
- KHR_materials_sheen
- KHR_materials_specular
- KHR_materials_transmission
- KHR_materials_unlit
- KHR_materials_variants
- KHR_materials_volume
- KHR_mesh_quantization
- KHR_texture_basisu
- KHR_texture_transform
- KHR_xmp_json_ld
- EXT_mesh_gpu_instancing
- KHR_node_visibility
Pathtracer
Implements a path tracer with global illumination.
The options are:
- Max Depth : number of bounces the path can do
- Max Samples: how many samples per pixel at each frame iteration
- Aperture: depth-of-field
- Debug Method: shows information like base color, metallic, roughness, and some attributes
- Choice between indirect and RTX pipeline.
- Denoiser: A-trous denoiser
Raster
Utilizes shared Vulkan resources with the path tracer, including:
- Scene geometry
- Material data
- Textures
- Shading functions
The options are:
- Show wireframe: display wireframe on top of the geometry
- Super-Sampling: render the image 2x and blit it with linear filter.
- Debug Method: shows information like base color, metallic, roughness, and some attributes
Example with wireframe option turned on
Debug
There is also the ability to debug various out channels, such as:
metallic | roughness | normal | base | emissive |
---|---|---|---|---|
Environment
It is possible to modify the environment, either by choosing an integrated sun and sky, or by lighting the scene using Image Base Lighting. In the latter case, you need an image (.hdr). You can find examples of such images at the following address Poly Haven
Sun & Sky | HDRi |
---|---|
Having HDRi (High Dynamic Range Imaging) to illuminate the scene greatly simplifies complex lighting environments. It also helps to integrate 3D objects into its environment.
This example loads HDR images, then creates an importance sampling acceleration structure used by the ray tracer and stores the PDF in the alpha channel of the RGBA32F image.
For real-time rendering, we use the created acceleration structure and create two cubemaps. One containing the diffuse irradiance and the other, storing the glossy reflection, where the different levels of glossiness are stored in separate mipmap levels.
The HDR can also be blured and rotated to better fit the scene.
An option to replace the background with a solid color is also available.
Tonemapper
We could not get good results without a tone mapper. This is done with a compute shader and different settings can be made.
Multiple tonemapper are supported:
- Filmic
- Uncharted 2
- Clip : Simple Gamma correction (linear to sRGB)
- ACES: Academy Color Encoding System
- AgX
- Khronos PBR : PBR Neutral Specification
Camera
The camera navigation follows the Softimage default behavior. This means, the camera is always looking at a point of interest and orbit around it.
Here are the default navigations:
The camera information can be fine tune by editing its values.
Note: copy will copy in text the camera in the clipboard, and pressing the paste button will parse the clipboard to set the camera.
Ex: {0.47115, 0.32620, 0.52345}, {-0.02504, -0.12452, 0.03690}, {0.00000, 1.00000, 0.00000}
Save and Restore Cameras
It is also possible to save and restore multiple cameras in the second tab. Press the +
button to save a camera, the middle button to delete it. By pressing one of the saved cameras, its position, interests, orientation and FOV will be changed smoothly.
Note: If the glTF scene contains multiple cameras, they will be showing here.
Other modes
Other navigation modes also exist, like fly, where the w
, a
, s
, d
keys also moves the camera.
Schema of the Program
The nvvk::Application is a class that provides a framework for creating Vulkan applications. It encapsulates the Vulkan instance, device, and surface creation, as well as window management and event handling.
When using nvvk::Application
, you can attach nvvkhl::IAppElement
to it and each element will be called for the different state, allowing to customize the behavior of your application. The nvvkhl::IAppElement
class provides default implementations for these functions, so you only need to override the ones you need.
Here is a brief overview of how nvvk::Application
works:
Initialization:
When you create an instance of nvvk::Application
, it sets up the Vulkan instance, device, and surface. It also creates a window and sets up event handling.
Attaching Elements
In main()
we are attaching many elements, like:
ElementCamera
: this allow to control a singleton cameraElementProfiler
: allow to time the execution on the GPUElementBenchmarkParameters
: command line arguments and test purposeElementLogger
: redirect log information in a windowElementNvml
: shows the status of the GPU
But the main one that interest us, and which is the main of this application is GltfRendererElement
. This is the one that will be controlling the scene and rendering.
Main Loop:
The nvvk::Application
class provides a main loop that continuously processes events and updates the application state. Inside the main loop, it calls the following functions:
-
onAttach():<br> This function is called whenever the element is attached to the application. In
GltfRendererElement
, we are creating the resource needed internally. -
onDetach(): <br> This function is called when the user tries to close the window. You can override this function to handle window close events.
-
onRender(VkCommandBuffer): <br> This function is called to render the frame using the current command buffer of the frame. You can override this function to perform rendering operations using Vulkan. In
GltfRendererElement
this is where the active renderer is called. -
onResize(): <br> This function is called when the
viewport
is resized. You can override this function to handle window resize events. InGltfRendererElement
the G-Buffer will be re-created -
onUIRender(): <br> This function is called to allow the
IAppElement
to render the UI and to query any mouse or keyboard event. InGltfRendererElement
, we render the UI, but also the final image. The rendered image is consider a UI element, and that image covers the entireviewport
ImGui window. -
onUIMenu() <br> Will be modifying what we see in the the window title. It will also create the menu, like
File
,Help
and deal with some key combinations. -
onFileDrop() <br> Will receive the path of the file been dropped on. If it is a .gltf, .glb or .hdr, it will load that file.
Scene Graph
The GLTF scene is loaded using tinygltf and then converted to a Vulkan version. The Vulkan version is a simplified version of the scene, where the geometry is stored in buffers, and the textures are uploaded to the GPU. The Vulkan version is used for both raster and ray tracing.
The scene is composed of nodes, where each node can have children and each node can have a mesh. The mesh is composed of primitives, where each primitive has a material. The material is composed of textures and parameters. However, none of this is directly used in the rendering, as we are using a simplified version of the scene.
Once the scene has been loaded, we proceed to parse it in order to collect the RenderNodes and RenderPrimitives. The RenderNode represents the flattened version of the tree of nodes, where the world transformation matrix and the material are stored. The RenderPrimitive, in contrast, represents the unique version of the primitive, where the index and vertex buffers are stored.
RenderNodes represent the elements to be rendered, while RenderPrimitives serve as references to the data utilized for rendering.
Animation
If there is animation in the scene, a new section will appear under the Scene section. It allows to play/pause, step and reset the animation, as well as changing its speed.
Multiple Scene
If there are multiple scenes, a new section will appear under the Scene section. It will show all the scenes and their name. Clicking on a scene name will switch to the scene.
Material Variant
If there are multiple material variant, a new section will appear under the Scene section. It will show all the material variant and their name. Clicking on a variant name will apply it on the models.
Scene Graph UI
It is possible to visualize the scene hierarchy, to select node, to modify their transformation and their material, to some level.
Here's a shorter version of the text, tailored for developers on GitHub:
Recompiling Shaders
For quick shader testing, use the Recompile Shaders
button to hot-reload shaders. Requirements:
- Shaders in the
shaders
folder shaderc_shared
andslang
libraries in the path (copied tobin
by default)
To force using the external shaders:
vk_gltf_reenderer.exe -forceExternalShaders
Note: Hot-reloading won't work without the shared libraries and shaders, but the app will still run.
Tools
The application comes with a few tools to help debug and visualize the scene.
Profiler
The profiler is a tool that allows to measure the time spent on the GPU. It is possible to measure the time spent on the different stages of the rendering, like the path tracing, the rasterization, the tonemapping, etc.
Logger
The logger is a tool that allows to see the log information. It is possible to filter the log information by selecting the level of the log.
Nvml
The Nvml is a tool that allows to see the status of the GPU. It is possible to see the temperature, the power, the memory usage, etc.
Tangent Space
There is a tangent space tool that allows to fix or to recreate the tangent space of the model. This is useful when the normal map is not looking right or there are errors with the tangents in the scene.
Utilities
gltf-material-modifier.py
Modify materials in a GLTF file and optionally reorient the scene from Z-up to Y-up.
usage: gltf-material-modifier.py [-h] [--metallic METALLIC] [--roughness ROUGHNESS] [--override] [--reorient]
input_file output_file
positional arguments:
input_file Path to the input GLTF file.
output_file Path to save the modified GLTF file.
options:
-h, --help show this help message and exit
--metallic METALLIC Set the metallic factor (default: 0.1).
--roughness ROUGHNESS Set the roughness factor (default: 0.1).
--override Override existing material values if set.
--reorient Reorient the scene from Z-up to Y-up.