Awesome
Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition
Project Page | Video | Paper
Implementation for Neural-PIL. A novel method which decomposes multiple images into shape, BRDF and illumination with a split-sum preintegrated illumination network.
<br><br>
Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition<br>
Mark Boss<sup>1</sup>, Varun Jampani<sup>2</sup>, Raphael Braun<sup>1</sup>, Ce Liu<sup>3</sup>, Jonathan T. Barron<sup>2</sup>, Hendrik P. A. Lensch<sup>1</sup><br>
<sup>1</sup>University of Tübingen, <sup>2</sup>Google Research, <sup>3</sup>Microsoft Azure AI (work done at Google)
<br><br>
Setup
A conda environment is used for dependency management
conda env create -f environment.yml
conda activate neuralpil
Running
python train_neural_pil.py --datadir [DIR_TO_DATASET_FOLDER] --basedir [TRAIN_DIR] --expname [EXPERIMENT_NAME] --gpu [COMMA_SEPARATED_GPU_LIST]
Specific Arguments per Dataset
Most setup is handled by configurations files located in configs/neural_pil/.
Our Synthethic Scenes
--config configs/neural_pil/blender.txt
NeRF Synthethic Scenes
--config configs/neural_pil/nerf_blender.txt
Real-World
--config configs/neural_pil/real_world.txt
Often objects are captured in a spherical manner and the flag --spherify
should be applied for those scenes.
Datasets
All datasets are taken from NeRD.
Run Your Own Data
Mainly camera poses and segmentation masks are required. For the poses the scripts from NeRF are used to prepare the scenes. The dataset then needs to be put in the following file structure:
images/
[IMG_NAME_1].jpg
[IMG_NAME_2].jpg
...
masks/
[IMG_NAME_1].jpg
[IMG_NAME_2].jpg
...
poses_bounds.npy
The poses_bounds.npy
is generated from the LLFF script.
Evaluation
The train_neural_pil.py can be called with a --render_only
flag and the --config
flag pointing to the args.txt
of the experiments folder.
Citation
@inproceedings{boss2021neuralpil,
title = {Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition},
author = {Boss, Mark and Jampani, Varun and Braun, Raphael and Liu, Ce and Barron, Jonathan T. and Lensch, Hendrik P.A.},
booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
year = {2021},
}