Awesome
Neural Density-Distance Fields
Project Page | arXiv
PyTorch implementation of Neural Distance-Density Field (NeDDF), a 3D representation that reciprocally constrains the distance and density fields.
Neural Density-Distance Fields
Itsuki Ueda<sup>1</sup>,
Yoshihiro Fukuhara<sup>2</sup>,
Hirokatsu Kataoka<sup>3</sup>,
Hiroaki Aizawa<sup>4</sup>,
Hidehiko Shishido<sup>1</sup>,
Itaru Kitahara<sup>1</sup> <br>
<sup>1</sup>University of Tsukuba, <sup>2</sup>Waseda University, <sup>3</sup>National Institute of Advanced Industrial Science and Technology (AIST), <sup>4</sup>Hiroshima University
<br>
in ECCV 2022(poster)
Prerequisite
- docker
- docker-compose
- nvidia-docker2
How to setup docker container
This repository is based on Ascender project. Please refer to Ascender for detailed instructions on how to set up the host environment.
## Move to the directory where docker-compose.yaml exists.
$ cd /path/to/neddf/environments/gpu
## build and start docker container
$ docker compose up -d core
## Enter docker container with bash
$ docker compose exec core bash
How to run code
You can download the nerf_synthetic_dataset and nerf_llff_dataset proposed in original NeRF paper here.
Please place downloaded data to data/
Train NeDDF
In the default configuration, the bunny_smoke dataset is set to learn.
Run
$ poetry run python neddf/scripts/run.py
To use other dataset with data format of nerf_synthetic, please select other dataset directory like follow:
$ poetry run python neddf/scripts/run.py dataset.dataset_dir="data/nerf_synthetic/drums/"
Logs are saved in outputs/{datatime}/.
Model parameters are saved to outputs/{datatime}/models/.
Rendering results are saved to outputs/{datatime}/render/, images rendered from a camera is in iteration number directory, and visualization of field slices are in fields directory.
Evaluate NeDDF
Run
$ poetry run python neddf/scripts/run_eval.py {pretrained files directory}
For example, use following commands to evaluate neddf model in bunny_smoke scene.
$ poetry run python neddf/scripts/run_eval.py pretrained/bunny_smoke/
You can download pretrained models of nerf_synthetic_dataset from this link
Visualize pretrained fields
To visualize trained distance, density, aux.gradient and color fields, please run following command.
$ poetry run python neddf/scripts/fields_visualizer.py {pretrained files directory}
This visualizer draw fields by 2D slices. Following video is visualization example in lego scene(pretrained/lego)
Visualize for check dataset
To visualize dataset, please run following command (override of dataset config is optional):
$ poetry run python neddf/scripts/dataset_visualizer.py dataset.dataset_dir=data/bunny_smoke/