Home

Awesome

ST3D & ST3D++

Code release for the paper ST3D: Self-training for Unsupervised Domain Adaptation on 3D Object Detection, CVPR 2021 and ST3D++: Denoised Self-training for Unsupervised Domain Adaptation on 3D Object Detection, T-PAMI 2022.

News

[2023-05-17] Support Waymo -> Lyft setting.

[2022-09-26] ST3D++ (The extension of ST3D) has been integrated in this repo for Waymo->KITTI and nuScenes->KITTI.

Introduction

Our code is based on OpenPCDet v0.3.0. More updates on OpenPCDet are supposed to be compatible with our code.

Model Zoo

Waymo -> KITTI TASK

methodCar@R40Ped@R40Cyc@R40
SECOND-IoUST3D62.1948.3346.09
SECOND-IoUST3D (w/ sn)73.6251.9253.00
SECOND-IoUST3D++65.1053.8753.43
SECOND-IoUST3D++ (w/ sn)74.7359.2160.76
PVRCNNST3D64.05--
PVRCNNST3D (w/ sn)77.33--

We could not provide the above pretrained models due to Waymo Dataset License Agreement, but you should achieve similar performance by training with the default configs. To access these pretrained models, please send us an email with your name, institute, a screenshot of the Waymo dataset registration confirmation mail, and your intended usage. Please send a second email if we don't get back to you in two days. Please note that Waymo open dataset is under strict non-commercial license, so we are not allowed to share the model with you if it will use for any profit-oriented activities.

Also, the training Waymo data used in our work is version 1.0, but the version now available is version 1.2. The pretrained model on these two version data should be similar when adapted to KITTI.

nuScenes -> KITTI TASK

methodCar@R40Ped@R40Cyc@R40
SECOND-IoUSource Only17.92--
SECOND-IoUROS25.37--
SECOND-IoUSN21.2334.365.67
SECOND-IoUST3D55.19--
SECOND-IoUST3D (w/ SN)62.27--
SECOND-IoUST3D++66.0145.2325.98
SECOND-IoUST3D++ (w/ SN)66.2446.7522.66
PV-RCNNSource Only37.17--
PV-RCNNROS38.84--
PV-RCNNSN49.47--
PV-RCNNST3D71.11--
PV-RCNNST3D (w/ SN)73.16--
PointRCNNROS55.92--
PointRCNNST3D++67.51--

We provide pretrained models here for nuScenes > KITTI task in models.

Waymo -> nuScenes TASK

methodCar@R11Car@R40download
SECOND-IoUST3D23.2420.19model
PVRCNNST3D27.1822.99model
SECOND-IoUST3D (w/ sn)23.5220.38model
PVRCNNST3D (w/ sn)28.0623.67model

We could not provide the above pretrained models due to Waymo Dataset License Agreement, but you should achieve similar performance by training with the default configs.

Waymo -> Lyft TASK

methodCar@R11Car@R40download
SECOND-IoUSource Only-54.34-
SECOND-IoUSN-54.34-
SECOND-IoUST3D-59.24model
SECOND-IoUST3D (w/ sn)-57.99model
PVRCNNSource Only-58.53-
PVRCNNSN-56.64-
PVRCNNST3D-60.53model
PVRCNNST3D (w/ sn)-58.54model

We could not provide the above pretrained models due to Waymo Dataset License Agreement, but you should achieve similar performance by training with the default configs.

Installation

Please refer to INSTALL.md for the installation.

Getting Started

Please refer to GETTING_STARTED.md to learn more usage about this project.

Supported features and ToDo List

License

Our code is released under the Apache 2.0 license.

Acknowledgement

Our code is heavily based on OpenPCDet v0.3. Thanks OpenPCDet Development Team for their awesome codebase.

Citation

If you find this project useful in your research, please consider cite:

@inproceedings{yang2021st3d,
    title={ST3D: Self-training for Unsupervised Domain Adaptation on 3D Object Detection},
    author={Yang, Jihan and Shi, Shaoshuai and Wang, Zhe and Li, Hongsheng and Qi, Xiaojuan},
    booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
    year={2021}
}
@article{yang2021st3d++,
  title={ST3D++: Denoised Self-training for Unsupervised Domain Adaptation on 3D Object Detection},
  author={Yang, Jihan and Shi, Shaoshuai and Wang, Zhe and Li, Hongsheng and Qi, Xiaojuan},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2022}
}
@misc{openpcdet2020,
    title={OpenPCDet: An Open-source Toolbox for 3D Object Detection from Point Clouds},
    author={OpenPCDet Development Team},
    howpublished = {\url{https://github.com/open-mmlab/OpenPCDet}},
    year={2020}
}