Awesome

CVPR 2024 | 3DSFLabelling: Boosting 3D Scene Flow Estimation by Pseudo Auto-labelling

Check out the project demo here: 3DSFLabelling-Page

The code is gradually being released, please be patient.

[poster coming soon] [video coming soon]

Description	Simple Example of the Auto-Labelling
The proposed 3D scene flow pseudo-auto-labelling framework. Given point clouds and initial bounding boxes, both global and local motion parameters are iteratively optimized. Diverse motion patterns are augmented by randomly adjusting these motion parameters, thereby creating a diverse and realistic set of motion labels for the training of flow estimation models.

Highlights <a name="highlights"></a>

:fire: GENERATE FULLY ALIGNED INTER-FRAME POINT CLOUDS

We propose a new framework for the automatic labelling of 3D scene flow pseudo-labels, significantly enhancing the accuracy of current scene flow estimation models, and effectively addressing the scarcity of 3D flow labels in autonomous driving.

:fire

:star2: Plug-and-play & Novel motion augmentation

We propose a universal 3D box optimization method with multiple motion attributes. Building upon this, we further introduce a plug-and-play 3D scene flow augmentation module with global-local motions and motion status. This allows for flexible motion adjustment of ego-motion and dynamic environments, setting a new benchmark for scene flow data augmentation.

News <a name="news"></a>

• [2024/4] :fire: Open source example model (GMSF, MSBRN and FLOT), Training and evaluation code.
• [2024/4] :fire: Open sourced a new data set with 3D scene flow GT, The data comes from KITTI Odometry Dataset, Argoverse Dataset and nuScenes Dataset respectively.
• [2024/4] :fire: Open sourced the code for the 3D Scene Flow Label Generation
• [2024/03] 3DSFLabelling code and models initially released.
• [2024/02] 3DSFLabelling is accepted by CVPR 2024.
• [2024/02] 3DSFLabelling paper released.

TODO List <a name="TODO List"></a>

Still in progress:

• Datasets are easier to use.
• The validity of the generated labels is verified on motion segmentation and LiDAR odometry.
• Readability optimization of configuration files and data reading code sections.

Table of Contents

1. Results and Model Zoo
2. License and Citation
3. Comparative Results

Results and Model Zoo <a name="models"></a>

Comparative results <a name="Comparative"></a>

The comparative results between our method and baseline. "↑" signifies accuracy enhancement. In real-world LiDAR scenarios, our method markedly improves the 3D flow estimation accuracy across three datasets on the three baselines. This demonstrates that the proposed pseudo-auto-labelling framework can substantially boost the accuracy of existing methods, even without the need for ground truth.

<a id="1">[1]</a> Puy G, Boulch A, Marlet R. Flot: Scene flow on point clouds guided by optimal transport[C]//European conference on computer vision. Cham: Springer International Publishing, 2020: 527-544.
<a id="2">[2]</a> Cheng W, Ko J H. Multi-scale bidirectional recurrent network with hybrid correlation for point cloud based scene flow estimation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023: 10041-10050.
<a id="3">[3]</a> Zhang Y, Edstedt J, Wandt B, et al. Gmsf: Global matching scene flow[J]. Advances in Neural Information Processing Systems, 2024, 36.

License and Citation <a name="license-and-citation"></a>

All assets and code are under the Apache 2.0 license unless specified otherwise.

If this work is helpful for your research, please consider citing the following BibTeX entry.

@inproceedings{yang2023vidar,
  title={3DSFLabelling: Boosting 3D Scene Flow Estimation by Pseudo Auto-labelling},
  author={Jiang, Chaokang and Wang, Guangming and Liu, Jiuming and Wang, Hesheng and Ma, Zhuang and Liu, Zhenqiang and Liang, Zhujin and Shan, Yi and Du, Dalong},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2024}
}