Home

Awesome

InfoNorm

This is the official repo for the implementation of [ECCV 2024] InfoNorm: Mutual Information Shaping of Normals for Sparse-View Reconstruction.

ProjectPage

MainFigure

Environment

conda env create -f environment.yml

This script is only for reference.

For fair competition, we use a thirdparty Raytracing to cull the unseen mesh from training views. We slightly modify the script to support the newer cuda version in third_pary/raytracing.

For some hash-grid based methods which depend tiny-cuda-cnn, you also need follow the install intructions from tiny-cuda-cnn.

You can feel free to raise issues if you encounter any problems.

Data

The indoor scene dataset are borrowed from ScanNet++. You can apply and download the dataset according to the instructions on the website.

Preprocess

We provide lots of scripts to process the various datasets.

You can follow the instruction in dataset/scannetpp_convert.sh to generate the data from raw ScanNet++.

The main steps are as follows:

  1. Undistort the scenes you want to evaluate.

  2. Convert the colmap to NeuS-like npz file of camera poses

  3. Predict the normal and the vision feature. We use the Surface Normal Uncertainty and dino-vits-features to generate cues for most of methods.

  4. (Optional) For some methods(MonoSDF, GeoNeuS), they need extra cues, you can follow their instructions to generate.

Run

We provide a shell script run.sh to run the code for various methods.

The Key Idea

If you really don't like reading other people's code and are constrained by the tedious preprocessing process,we also provide core scripts to help you to add our ideas to your framework.

Please refer to core_idea.py to quickly understand our ideas and add this cool idea to your fancy framework.

Acknowledgement

Some code snippets are borrowed from these repository. Thanks for all these pretty great works.

NeuS, NeuRIS, GeoNeuS, i2-SDF, MonoSDF, NeuralAngelo, VolSDF, JacobiNeRF, raytracing.

Thanks for the indoor scenes dataset Scannet++, Replica. We use the Pre-rendered Replica Data from Semantic-NeRF

Citation

We will appreciate cite as below if you find this repository is helpful to your project

@misc{wang2024infonormmutualinformationshaping,
    title={InfoNorm: Mutual Information Shaping of Normals for Sparse-View Reconstruction}, 
    author={Xulong Wang and Siyan Dong and Youyi Zheng and Yanchao Yang},
    year={2024},
    eprint={2407.12661},
    archivePrefix={arXiv},
    primaryClass={cs.CV},
    url={https://arxiv.org/abs/2407.12661}, 
}