Awesome

SSCBench: Monocular 3D Semantic Scene Completion Benchmark in Street Views

Yiming Li*, Sihang Li*, Xinhao Liu*, Moonjun Gong*, Kenan Li, Nuo Chen, Zijun Wang, Zhiheng Li, Tao Jiang, Fisher Yu, Yue Wang, Hang Zhao, Zhiding Yu, Chen Feng

[PDF]

News

• [2024/09]: SSCBench is now available on Hugging Face
• [2024/08]: We release SSCBench-Waymo for academic usage.
• [2024/06]: SSCBench is accepted at IROS 2024!
• [2023/10]: We release OCFBench, a large-scale dataset for OCF, derived from nuScenes, Lyft, Argoverse, and ApolloScape (Waymo is coming soon).
• [2023/08]: We add demo for SSCBench-PandaSet. We are working on incoporating the dataset
• [2023/06]: We release SSCBench-KITTI-360 and SSCBench-nuScenes for academic usage.
• [2023/06]: The preprint version is available on arXiv.

Abstract

Semantic scene completion (SSC) is crucial for holistic 3D scene understanding by jointly estimating semantics and geometry from sparse observations. However, progress in SSC, particularly in autonomous driving scenarios, is hindered by the scarcity of high-quality datasets. To overcome this challenge, we introduce SSCBench, a comprehensive benchmark that integrates scenes from widely-used automotive datasets (e.g., KITTI-360, nuScenes, and Waymo). SSCBench follows an established setup and format in the community, facilitating the easy exploration of the camera- and LiDAR-based SSC across various real-world scenarios. We present quantitative and qualitative evaluations of state-of-the-art algorithms on SSCBench and commit to continuously incorporating novel automotive datasets and SSC algorithms to drive further advancements in this field.

SSCBench Dataset

SSCBench consists of three carefully designed datasets, all based on existing data sources. For more details, please refer to the dataset folder.

Model Checkpoints

We provide the model checkpoints of the experiments reported in the paper. The checkpoints can be accessed on Hugging Face.

Note that the provided checkpoints are trained with the unified class labels. Details of class mappings can be found in the configs folder.

Related SSC Projects

• Semantic Scene Completion from a Single Depth Image, CVPR 2017
• LMSCNet: Lightweight Multiscale 3D Semantic Completion, 3DV 2020
• MonoScene: Monocular 3D Semantic Scene Completion, CVPR 2022
• VoxFormer: a Cutting-edge Baseline for 3D Semantic Occupancy Prediction, CVPR 2023
• TPVFormer: An academic alternative to Tesla's Occupancy Network, CVPR2023
• OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Prediction, ICCV 2023
• SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving, ICCV 2023
• S4C: Self-Supervised Semantic Scene Completion with Neural Fields, arXiv 2023

Related Dataset/Benchmark

• Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving, arXiv 2023
• OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perception, ICCV 2023
• Occ4cast: LiDAR-based 4D Occupancy Completion and Forecasting, arXiv 2023.

License

Due to the license of the different original datasets, we release SSCBench under the following licenses:

• SSCBench-KITTI-360: CC BY-NC-SA 3.0
• SSCBench-nuScenes: CC BY-NC-SA 4.0
• SSCBench-Waymo: Waymo Dataset License Agreement for Non-Commercial Use (August 2019)

For more details, please refer to the dataset folder file.

Bibtex

If this work is helpful for your research, please cite the following BibTeX entry.

@inproceedings{li2024sscbench,
      title={SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Driving}, 
      author={Li, Yiming and Li, Sihang and Liu, Xinhao and Gong, Moonjun and Li, Kenan and Chen, Nuo and Wang, Zijun and Li, Zhiheng and Jiang, Tao and Yu, Fisher and Wang, Yue and Zhao, Hang and Yu, Zhiding and Feng, Chen},
      booktitle={2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
      year={2024}
}

Star History