Home

Awesome

<div align="center"> <h3 align="center">Compact 3D Scene Representation via Self-Organizing Gaussian Grids</h3> <br /> <p align="center"> <img src="https://fraunhoferhhi.github.io/Self-Organizing-Gaussians/static/images/teaser.png" alt="Teaser of the publication. Millions of Gaussians at 174 MB with a PSNR of 24.90 are sorted into 2D attribute grids, stored at 17 MB with the same PSNR"> <br /> <br /> <a href="https://fraunhoferhhi.github.io/Self-Organizing-Gaussians/"><strong>Project Page</strong></a> · <a href="https://arxiv.org/abs/2312.13299" target="_blank"><strong>arXiv</strong></a> </p> </div>

Code

This repository is a fork of the official authors implementation associated with the paper "3D Gaussian Splatting for Real-Time Radiance Field Rendering".

The code for "Compact 3D Scene Representation via Self-Organizing Gaussian Grids" consists of multiple parts. The multi-dimensional sorting algorithm, PLAS, is available under the Apache License at fraunhoferhhi/PLAS.

The integration of the sorting, the smoothness regularization and the compression code for training and compressing 3D scenes is available in this repository.

Cloning the Repository

The repository contains submodules, thus please check it out with

# SSH
git clone git@github.com:fraunhoferhhi/Self-Organizing-Gaussians.git --recursive

or

# HTTPS
git clone https://github.com/fraunhoferhhi/Self-Organizing-Gaussians.git --recursive

Python Environment

The code is using a few additional Python packages on top of graphdeco-inria/gaussian-splatting. We provide an extended environment.yml:

Installation with micromamba:

micromamba env create --file environment.yml --channel-priority flexible -y
micromamba activate sogs

Example training

Download a dataset, e.g. T&T.

The train.py script expects a name to a .yaml config file in the config/ folder. All parameters for the run are by default loaded from the yaml file. An example launch file can be found in .vscode/launch.json, for launching from Visual Studio Code.

Example:

python train.py \
  --config-name ours_q_sh_local_test \
  hydra.run.dir=/data/output/${now:%Y-%m-%d}/${now:%H-%M-%S}-${run.name} \
  dataset.source_path=/data/gaussian_splatting/tandt_db/tandt/truck \
  run.no_progress_bar=false \
  run.name=vs-code-debug

The parameter configurations can be overriden in the launch as shown (using Hydra).

Pre-Trained Models & Evaluation

Trained and compressed scenes are available for download in the ECCV 2024 release.

The script at eval/download_eval.sh will automatically:

The evaluation results can be found in results/.

Differences with graphdeco-inria/gaussian-splatting

Code differences can be found in this diff: https://github.com/fraunhoferhhi/Self-Organizing-Gaussians/pull/1/files

Usage

Code extensions

Updates