Awesome

HEDNet & SAFDNet

It is the official code release of HEDNet (NeurIPS 2023) and SAFDNet (CVPR 2024). We unify the codebase for HEDNet and SAFDNet on all datasets based on OpenPCDet. Please note that, since we rebuilt the code, the results are slightly different from those in the original paper.

Results on Waymo Open

Validation set

Test set

We could not provide the above pretrained models due to Waymo Dataset License Agreement.

Results on NuScenes

Validation set

Test set

Note: We originally implemented HEDNet on the nuScenes dataset using MMDetection3D. The TransFusion-L in OpenPCDet which uses the hierarchical 3D SECOND as its backbone achieves worse results than that in MMDetection3D. To unify the codebase, we release a single-stride 2D version for both HEDNet and SAFDNet on the nuScenes dataset. This version differs from the hierarchical 3D version described in the original paper but achieves similar results.

Results on Argoverse2

Validation set

Installation and usage

Please refer to INSTALL.md and GETTING_STARTED.md for the installation and usage, respectively. We used python 3.8, pytorch 1.10, cuda11.3, spconv-cu113 2.3.3. We provide a list of Python packages output from pip freeze here, to help configure the environment.

You can create an experiment folder in any location, and organize it like this:

FOLDER_NAME:
├── ${PATH_TO_HEDNet_ROOT}/HEDNet/tools/cfgs
├── ${PATH_TO_HEDNet_ROOT}/HEDNet/data
├── xxx.yaml (copy the yaml file here)
├── dist_train.sh (copy the training script from tools/scripts here)
├── dist_test.sh (copy the testing script from tools/scripts here)

For faster evaluation on the Waymo Open dataset, please download the compute_detection_metrics_main and gt.bin, and then put them under the HEDNet/data/waymo. You may need to excute chmod +x compute_detection_metrics_main to modify the file permission to make it an executable file. If you want to generate these two files on your own, please refer to the this repo.

Then you can train and test models like this:

# Train with 8 gpus
./dist_train.sh xxx.yaml 8

# Test with 8 gpus
./dist_test.sh xxx.yaml 8 output/ckpt/xxx.pth

TODO

• A cross-modal fusion method based on HEDNet and SAFDNet is on the way.

Citation

@inproceedings{zhang2023hednet,
  title={{HEDNet}: A Hierarchical Encoder-Decoder Network for 3D Object Detection in Point Clouds},
  author={Zhang, Gang and Chen, Junnan and Gao, Guohuan and Li, Jianmin and Hu, Xiaolin},
  booktitle={Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS)},
  year={2023},
}

@inproceedings{zhang2024cvpr,
    title     = {{SAFDNet}: A Simple and Effective Network for Fully Sparse 3D Object Detection},
    author    = {Zhang, Gang and Chen, Junnan and Gao, Guohuan and Li, Jianmin and Liu, Si and Hu, Xiaolin},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2024},
    pages     = {14477-14486}
}

Acknowleadgement

This two works were supported by the National Key Research and Development Program of China (No. 2021ZD0200301) and the National Natural Science Foundation of China (Nos. U19B2034, 61836014) and THU-Bosch JCML center.