Awesome

About

<strong>LaserMix</strong> is a semi-supervised learning (SSL) framework designed for LiDAR semantic segmentation. It leverages the strong <strong>spatial prior</strong> of driving scenes to construct <strong>low-variation areas</strong> via <strong>laser beam mixing</strong>, and encourages segmentation models to make <strong>confident</strong> and <strong>consistent</strong> predictions before and after mixing.

Visit our <a href="https://ldkong.com/LaserMix" target='_blank'>project page</a> to explore more details. :red_car:

Updates

• [2024.05] - Our improved framework, LaserMix++ :rocket:, is avaliable on arXiv.
• [2024.01] - The toolkit tailored for The RoboDrive Challenge has been released. :hammer_and_wrench:
• [2023.12] - We are hosting The RoboDrive Challenge at ICRA 2024. :blue_car:
• [2023.12] - Introducing FRNet, an efficient and effective real-time LiDAR segmentation model that achieves promising semi-supervised learning results on SemanticKITTI and nuScenes. Code and checkpoints are available for downloading.
• [2023.03] - Intend to test the robustness of your LiDAR semantic segmentation models? Check our recent work, :robot: Robo3D, a comprehensive suite that enables OoD robustness evaluation of 3D segmentors on our newly established datasets: SemanticKITTI-C, nuScenes-C, and WOD-C.
• [2023.03] - LaserMix was selected as a :sparkles: highlight :sparkles: at CVPR 2023 (top 10% of accepted papers).
• [2023.02] - LaserMix was accepted to CVPR 2023! :tada:
• [2023.02] - LaserMix has been integrated into the MMDetection3D codebase! Check this PR in the dev-1.x branch to know more details. :beers:
• [2023.01] - As suggested, we will establish a sequential track taking into account the LiDAR data collection nature in our semi-supervised LiDAR semantic segmentation benchmark. The results will be gradually updated in RESULT.md.
• [2022.12] - We support a wider range of LiDAR segmentation backbones, including RangeNet++, SalsaNext, FIDNet, CENet, MinkowskiUNet, Cylinder3D, and SPVCNN, under both fully- and semi-supervised settings. The checkpoints will be available soon!
• [2022.12] - The derivation of spatial-prior-based SSL is available here. Take a look! :memo:
• [2022.08] - LaserMix achieves 1st place among the semi-supervised semantic segmentation leaderboards of nuScenes, SemanticKITTI, and ScribbleKITTI, based on Paper-with-Code. :bar_chart:
• [2022.08] - We provide a video demo for visual comparisons on the SemanticKITTI val set. Take a look!
• [2022.07] - Our paper is available on arXiv, click <a href="https://arxiv.org/abs/2207.00026" target='_blank'>here</a> to check it out. Code will be available soon!

Outline

• Installation
• Data Preparation
• Getting Started
• Video Demo
• Main Results
• TODO List
• License
• Acknowledgement
• Citation

Installation

Please refer to INSTALL.md for the installation details.

Data Preparation

Please refer to DATA_PREPARE.md for the details to prepare the ¹nuScenes, ²SemanticKITTI, and ³ScribbleKITTI datasets.

Getting Started

Please refer to GET_STARTED.md to learn more usage about this codebase.

Video Demo

Main Result

Framework Overview

Range View

Voxel

Ablation Studies

Qualitative Examples

Checkpoints & More Results

For more experimental results and pretrained weights, please refer to RESULT.md.

TODO List

• Initial release. :rocket:
• Add license. See here for more details.
• Add video demos :movie_camera:
• Add installation details.
• Add data preparation details.
• Add evaluation details.
• Add training details.

Citation

If you find this work helpful, please kindly consider citing our paper:

@inproceedings{kong2023lasermix,
  title = {LaserMix for Semi-Supervised LiDAR Semantic Segmentation},
  author = {Kong, Lingdong and Ren, Jiawei and Pan, Liang and Liu, Ziwei},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages = {21705--21715},
  year = {2023},
}

License

<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/80x15.png" /></a> <br /> This work is under the <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.

Acknowledgement

This work is developed based on the MMDetection3D codebase.

<img src="https://github.com/open-mmlab/mmdetection3d/blob/main/resources/mmdet3d-logo.png" width="30%"/><br> MMDetection3D is an open-source toolbox based on PyTorch, towards the next-generation platform for general 3D perception. It is a part of the OpenMMLab project developed by MMLab.

We acknowledge the use of the following public resources during the course of this work: ¹nuScenes, ²nuScenes-devkit, ³SemanticKITTI, ⁴SemanticKITTI-API, ⁵ScribbleKITTI, ⁶FIDNet, ⁷CENet, ⁸SPVNAS, ⁹Cylinder3D, ¹⁰TorchSemiSeg, ¹¹MixUp, ¹²CutMix, ¹³CutMix-Seg, ¹⁴CBST, ¹⁵MeanTeacher, and ¹⁶Cityscapes.

We would like to thank <a href="https://hongfz16.github.io/" target='_blank'>Fangzhou Hong</a> for the insightful discussions and feedback. ❤️