Home

Awesome

2D Gaussian Splatting for Geometrically Accurate Radiance Fields

Project page | Paper | Video | Surfel Rasterizer (CUDA) | Surfel Rasterizer (Python) | DTU+COLMAP (3.5GB) | Viewer Pre-built for Windows<br>

Teaser image

This repo contains the official implementation for the paper "2D Gaussian Splatting for Geometrically Accurate Radiance Fields". Our work represents a scene with a set of 2D oriented disks (surface elements) and rasterizes the surfels with perspective correct differentiable raseterization. Our work also develops regularizations that enhance the reconstruction quality. We also devise meshing approaches for Gaussian splatting.

⭐ New Features

Add SIBR viewer for 2DGS.

https://github.com/RongLiu-Leo/2d-gaussian-splatting/assets/102014841/b75dd9a7-e3ee-4666-99ff-8c9121ff66dc

The Pre-built Viewer for Windows can be found here. If you use Ubuntu or you want to check the viewer usage, please refer to GS Monitor.

How to use

Firstly open the viewer,

<path to downloaded/compiled viewer>/bin/SIBR_remoteGaussian_app_rwdi.exe

and then

# Monitor the training process
python train.py -s <path to COLMAP or NeRF Synthetic dataset> 
# View the trained model
python view.py -s <path to COLMAP or NeRF Synthetic dataset> -m <path to trained model>

visualization

Installation

# download
git clone https://github.com/hbb1/2d-gaussian-splatting.git --recursive

# if you have an environment used for 3dgs, use it
# if not, create a new environment
conda env create --file environment.yml
conda activate surfel_splatting

Training

To train a scene, simply use

python train.py -s <path to COLMAP or NeRF Synthetic dataset>

Commandline arguments for regularizations

--lambda_normal  # hyperparameter for normal consistency
--lambda_distortion # hyperparameter for depth distortion
--depth_ratio # 0 for mean depth and 1 for median depth, 0 works for most cases

Tips for adjusting the parameters on your own dataset:

Testing

Bounded Mesh Extraction

To export a mesh within a bounded volume, simply use

python render.py -m <path to pre-trained model> -s <path to COLMAP dataset>

Commandline arguments you should adjust accordingly for meshing for bounded TSDF fusion, use

--depth_ratio # 0 for mean depth and 1 for median depth
--voxel_size # voxel size
--depth_trunc # depth truncation

If these arguments are not specified, the script will automatically estimate them using the camera information.

Unbounded Mesh Extraction

To export a mesh with an arbitrary size, we devised an unbounded TSDF fusion with space contraction and adaptive truncation.

python render.py -m <path to pre-trained model> -s <path to COLMAP dataset> --mesh_res 1024

Quick Examples

Assuming you have downloaded MipNeRF360, simply use

python train.py -s <path to m360>/<garden> -m output/m360/garden
# use our unbounded mesh extraction!!
python render.py -s <path to m360>/<garden> -m output/m360/garden --unbounded --skip_test --skip_train --mesh_res 1024
# or use the bounded mesh extraction if you focus on foreground
python render.py -s <path to m360>/<garden> -m output/m360/garden --skip_test --skip_train --mesh_res 1024

If you have downloaded the DTU dataset, you can use

python train.py -s <path to dtu>/<scan105> -m output/date/scan105 -r 2 --depth_ratio 1
python render.py -r 2 --depth_ratio 1 --skip_test --skip_train

Custom Dataset: We use the same COLMAP loader as 3DGS, you can prepare your data following here.

Full evaluation

We provide two scripts to evaluate our method of novel view synthesis and geometric reconstruction. For novel view synthesis on MipNeRF360 (which also works for other colmap datasets), use

python scripts/mipnerf_eval.py -m60 <path to the MipNeRF360 dataset>

For geometry reconstruction on DTU dataset, please download the preprocessed data. You also need to download the ground truth DTU point cloud.

python scripts/dtu_eval.py --dtu <path to the preprocessed DTU dataset>   \
     --DTU_Official <path to the official DTU dataset>

FAQ

Acknowledgements

This project is built upon 3DGS. The TSDF fusion for extracting mesh is based on Open3D. The rendering script for MipNeRF360 is adopted from Multinerf, while the evaluation scripts for DTU and Tanks and Temples dataset are taken from DTUeval-python and TanksAndTemples, respectively. The fusing operation for accelerating the renderer is inspired by Han's repodcue. We thank all the authors for their great repos.

Citation

If you find our code or paper helps, please consider citing:

@inproceedings{Huang2DGS2024,
    title={2D Gaussian Splatting for Geometrically Accurate Radiance Fields},
    author={Huang, Binbin and Yu, Zehao and Chen, Anpei and Geiger, Andreas and Gao, Shenghua},
    publisher = {Association for Computing Machinery},
    booktitle = {SIGGRAPH 2024 Conference Papers},
    year      = {2024},
    doi       = {10.1145/3641519.3657428}
}