Awesome

3d gaussian splatting

This branch contains another unofficial implementation (?) of the paper 3D Gaussian Splatting for Real-Time Radiance Field Rendering. Currently Work In Process. Welcome to PR and discussions. For a better implementation, you can refers to Feng Wang's repo.

This is currently a Work In Process

I roughly implement the functions required in the original paper, all of which reside in the src/include folder. I have no idea on how to speed up renderering and backward. So if you have any idea or suggestions, welcome to post a issue or email me, I will be grateful and honored to discuss with you ! (Please concat Feng Wang or Zilong Chen)

Performance

command : python main_sh.py --config-name=garden_full pos_grad_thresh=2e-4 adaptive_control_iteration=100 alpha_reset_period=3000 remove_low_alpha_period=3000 alpha_scheduler=nothing svec_scheduler=nothing warmup_steps=300 use_train_sample=True split_scale_thresh=0.004 alpha_lr=0.1 Ours 7K: psnr: 24.17 ssim: 0.7611 ssim2: 0.8422 lpips: 0.1645 Ours 8K: psnr: 24.14 ssim: 0.7604 ssim2: 0.8414 lpips: 0.1548 12K: psnr: 24.59 ssim: 0.7792 ssim2: 0.8532 lpips: 0.1347

command: python main_sh.py --config-name=garden_full pos_grad_thresh=2e-4 adaptive_control_iteration=100 alpha_reset_period=3000 remove_low_alpha_period=3000 alpha_scheduler=nothing svec_scheduler=nothing warmup_steps=300 use_train_sample=True split_scale_thresh=0.004 alpha_lr=0.1 sh_coeffs_lr=0.1 sh_coeffs_scheduler=nothing 8K: psnr: 24.37 ssim: 0.7764 ssim2: 0.8508 lpips: 0.139

To run the code

1. first install all the requirements: pip install -e requirements.txt and addtional cuda extension: cd gs && ./build.sh

2. prepare your data in {workspace}/data, make sure you have the outputs of colmap.

3. run the training script python main_sh.py using viewer=True to enable a Viser based viewer.

The configurations are managed with hydra and some of the configurations I have tested are in conf.

TODOs:

• correct adaptive control
• train eval split and do complete evaluation
• profiling
• faster rendering !!

Updates:

See update log here