Home

Awesome

Introduction

The Dirtchamber, inspired by the homonymous Prodigy album, is a mixed reality (MR) testing environment for real-time global illumination algorithms. It evolved while I was writing papers to figure out if a global illumination algorithm is suitable (with adaptations) to be used as a relighting solution for an MR scenario. Two main products came out of this: Delta Light Propagation Volumes and Delta Voxel Cone Tracing.

XYZRGB Dragon next to a 3D printed one

This suite features four samples:

Both, MR and non-MR samples are rendered with a deferred renderer and a postprocessing pipeline which includes FXAA, SSAO, Depth-of-Field, crepuscular Rays, Bloom, a CRT monitor effect, TV grain, exposure adaptation and tonemapping. The model loader uses Assimp, which means you should be able to load a good range of different model formats. The deferred shader uses a physically-based BRDF model with GGX as its normal distribution function. Both MR samples as well as both non-MR renderers share a common file and switch between the volumetric GI method with a define.

The Dirtchamber uses Dune, a Direct3D helper library which includes many classes that simply wrap and manage the absurd amount of pointers and structs needed to upload stuff to the GPU. Hopefully, you'll find something useful in there too.

Tracking is realized with OpenCV and textures representing Kinect cameras or simple webcams. If you have access to neither, you can still use the tracking code with still images, since the tracking code only cares for cached textures which contain an image without regard for the source. This functionality is also useful to exchange scenes with other researchers to compare different rendering solutions for mixed reality.

The Delta Radiance Field

The Dirtchamber is the product of 4 publications:

You can download preprints at http://tobias-franke.eu/?pub.

The initial poster, presented at SIGGRAPH 2013, pitched the idea of the Delta Radiance Field (DRF). Up to this point, most if not all real-time relighting solutions were based on a highly influential paper by Paul Debevec which introduced a method called Differential Rendering. A synthetic copy of a real scene is rendered into one buffer \( \mu \). The same scene is rendered again, but this time with an additional augmenting object \( \rho \). By subtracting the former from the latter, one is left with the differential which can be added on top of a background image.

The intriguing aspect of Differential Rendering is its simplicity. However, when dealing with GI solutions, the cost of rendering the scene twice will sooner rather than later introduce a performance bottleneck. Another issue with this approach is inherent in its screen-space nature: volumetric information along the a light path between real and synthetic particles is lost.

To combat the latter problem, volumes instead of two screen-space renderings can be used to identify exchange between real and synthetic particles in mid air. However, calculating the differential of two volumes introduces even higher costs than Differential Rendering ever had. In this sense, Differential Radiance Fields aren't feasible. Instead, we can figure out which portion of light is left over to simulate inside a volume by calculating the difference between Reflective Shadow Maps.

It is this difference which, when used within a GI solution, produces a radiance field \( L_\Delta \) representing the delta between the synthetic and real space.

$$ L_\Delta = L_\rho - L_\mu = \sum_{i=0}^\infty \mathbf{T}{\rho}^{i} L_e - \sum{i=0}^\infty \mathbf{T}{\mu}^{i} L_e = \sum{i=0}^\infty \mathbf{T}_{\Delta_i} L_e $$

Delta Light Propagation Volumes encode this DRF with the help of Light Propagation Volumes, whereas Delta Voxel Cone Tracing cone-traces volumes with the left-over injection of real bounces on synthetic surfaces, while tracing a separate structure encoding occlusions.

Compiling & Running

Prerequisites

You will need to download the following tools:

Furthermore, you will need to download the following dependencies:

To ensure binary compatibility, it is recommended to compile DXUT and Assimp yourself. Dune only uses header-only libraries from Boost, which therefore doesn't need to be built. OpenCV is provided as binary release for all major Microsoft compilers and works out of the box.

Here are the steps you should take:

Get the source

You can find the project at http://github.com/thefranke/dirtchamber.

$ git clone https://github.com/thefranke/dirtchamber.git

Please post any issues to the bugtracker on GitHub.

Running CMake

Open the CMake GUI and point the source to the root directory of the Dirtchamber. Select your Visual Studio distribution with either 32 or 64 bit. Create a separate directory somewhere else and point "Where to build the binaries" to that directory, then click on Configure.

CMake should be able to identify all libraries automatically except for DXUT, Boost and Assimp. You will need to point the following variables to your build of these libraries:

After configuration, click on generate and you should have a working solution in your build folder.

Running the samples

The solution has either two or four projects. This depends on whether or not OpenCV and the Kinect SDK are available, because both of them are needed to run the mixed reality samples.

The first two projects (lpv and vct) are sample implementations of regular Light Propagation Volumes and Voxel Cone Tracing. Both projects can be run with command line arguments which can be either files or path patterns (something like C:/foo/*.obj) to the geometry which should be loaded.

The other two projects (dlpv and dvct) are the mixed reality applications which implement Delta Light Propagation Volumes and Delta Voxel Cone Tracing respectively. Similarly to the first two projects, both executables will try to load all command line arguments as files or file patterns. The important bit is that the last parameter is the synthetic object, while all other parameters are assumed to be real reconstructed scene geometry.

Once running, you can manipulate rendering settings and the geometric attributes of a main light source. If you repeatedly need to access the same configuration with the same camera position and orientation, you can save these settings with by pressing p on your keyboard. Pressing l opens a dialog to open the settings again. You can find a sample configuration in data/demo_gi.xml.

Understanding the code

First of all, you can read the documentation either by browsing the source code directly, or generate a formatted help by running Doxygen in the doc subfolder.

$ cd doc
$ doxygen

A precompiled version is available online. Most of the code is located in the helper library called Dune. There are several classes which mostly wrap up D3D11 state code and manage resources. Some files have a suffix tools(.h|.cpp) which usually contain free functions either doing trivial things such as making sure a file path is absolute or compiling shaders, or they handle one of the Dune containers. You can trivially include everything important with the header dune.h.

Several classes such as dune::tracker can contain code necessary to set up a few things or implementation details which should not be visible to a class user. Similarly to the Boost library, the code in these cases is wrapped up in a subnamespace called detail. This makes it easier to collapse and hide functionality when browsing the code which isn't necessary to understand the rest.

All objects in Dune are explicitly managed using member functions create(...) and destroy(). Because of this the state of an object is deemed invalid before calling create(...) or after calling destroy(). Member functions generally do not guard against calls with an invalid state.

All four samples together share a lot of code, most of it being GUI and GI rendering related. There are three file pairs:

Shaders are explicitely managed from the outside. This means that each object which requires one or more shaders has a member set_shader_x() or simply set_shader() which receives precompiled shaders. Each application therefore has one centralized function load_shaders() which will load every shader necessary for the application, compile all of them and pass them to their respective objects. When reloading shaders, this function is called again and will reload everything. Slots at which an object can expect which Direct3D buffer is handled the same way.

Objects which feature shader parameters which may be manipulated from the outside come with a member function parameters() which returns the internal cbuffer.

When loading files, each object will make sure that the URI entered is always in absolute relation to the executable path on the disk, not on the execution path.

Debugging the code

Dune uses a four-class system to log during runtime. These are:

All logging is routed through a special logbuf class which, apart from displaying nice popups, will write each line to a log file. All samples write to data/log.txt.

Future Improvements

Real-time relighting is still a largely unexplored area. Although many lessons can be learned from the "offline relighting community" (i.e. augmenting legacy photographs or movies with path-traced solutions), many issues cannot simply be transferred to the real-time domain. If you want to research in this area or contribute to this project, here are a few things on my list:

License

The entire source code of The Dirtchamber and Dune is licensed as MIT. If you have any specific questions about the license please let me know at tobias.franke@siggraph.org. See LICENSE for more information.