Awesome

HEDNet & SAFDNet

It is the official code release of HEDNet (NeurIPS 2023) and SAFDNet (CVPR 2024). We unify the codebase for HEDNet and SAFDNet on all datasets based on OpenPCDet.

Results on Waymo Open

Validation set

Model	mAP/mAPH_L1	mAP/mAPH_L2	Vec_L1	Vec_L2	Ped_L1	Ped_L2	Cyc_L1	Cyc_L2
HEDNet-1x	81.1/79.1	75.0/73.0	80.2/79.7	72.3/71.9	79.3/76.4	76.4/71.9	79.1/78.1	76.2/75.3
SAFDNet-1x	81.2/79.2	75.1/73.2	80.2/79.7	72.2/71.8	79.9/76.9	76.8/72.6	79.1/78.1	76.2/75.2
HEDNet-2x	81.4/79.5	75.3/73.4	81.1/80.6	73.2/72.7	84.4/80.0	76.8/72.6	78.7/77.7	75.8/74.9
SAFDNet-2x	81.8/79.9	75.7/73.9	80.6/80.1	72.7/72.3	84.7/80.4	77.3/73.1	80.0/79.0	77.2/76.2

Test set

Model	mAP/mAPH_L1	mAP/mAPH_L2	Vec_L1	Vec_L2	Ped_L1	Ped_L2	Cyc_L1	Cyc_L2	Submission
HEDNet-2x	82.2/80.2	76.9/75.0	84.2/83.8	77.0/76.6	84.1/79.7	78.3/74.0	78.2/77.0	75.4/74.3	link
SAFDNet-2x	81.9/79.8	76.5/74.6	83.9/83.5	76.6/76.2	84.3/79.8	78.3/74.1	77.5/76.3	74.6/73.4	link

We could not provide the above pretrained models due to Waymo Dataset License Agreement.

Results on NuScenes

Validation set

Model	mATE	mASE	mAOE	mAVE	mAAE	mAP	NDS	Checkpoint
HEDNet	27.2	25.1	26.5	25.7	18.0	67.1	71.3	ckpt & log
SAFDNet	27.3	25.3	27.1	25.2	18.3	66.6	71.0	ckpt & log

Test set

Model	mATE	mASE	mAOE	mAVE	mAAE	mAP	NDS	Submission
HEDNet	25.0	23.8	31.7	24.0	13.0	67.5	72.0	json
SAFDNet	25.1	24.2	31.1	25.8	12.7	68.3	72.3	json

Results on Argoverse2

Validation set

Model	mAP	Checkpoint
SAFDNet-1x	39.4	ckpt & log
SAFDNet-2x	39.9	ckpt & log

Installation and usage

Please refer to INSTALL.md and GETTING_STARTED.md for the installation and usage, respectively. We used python 3.8, pytorch 1.10, cuda11.3, spconv-cu113 2.3.3. We provide a list of Python packages output from pip freeze here, to help configure the environment.

You can create an experiment folder in any location, and organize it like this:

FOLDER_NAME:
├── ${PATH_TO_HEDNet_ROOT}/HEDNet/tools/cfgs
├── ${PATH_TO_HEDNet_ROOT}/HEDNet/data
├── xxx.yaml (copy the yaml file here)
├── dist_train.sh (copy the training script from tools/scripts here)
├── dist_test.sh (copy the testing script from tools/scripts here)

For faster evaluation on the Waymo Open dataset, please download the compute_detection_metrics_main and gt.bin, and then put them under the HEDNet/data/waymo. You may need to excute chmod +x compute_detection_metrics_main to modify the file permission to make it an executable file. If you want to generate these two files on your own, please refer to the this repo.

Then you can train and test models like this:

# Train with 8 gpus
./dist_train.sh xxx.yaml 8

# Test with 8 gpus
./dist_test.sh xxx.yaml 8 output/ckpt/xxx.pth

TODO

A cross-modal fusion method based on HEDNet and SAFDNet is on the way.

Citation

@inproceedings{zhang2023hednet,
  title={{HEDNet}: A Hierarchical Encoder-Decoder Network for 3D Object Detection in Point Clouds},
  author={Zhang, Gang and Chen, Junnan and Gao, Guohuan and Li, Jianmin and Hu, Xiaolin},
  booktitle={Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS)},
  year={2023},
}

@inproceedings{zhang2024cvpr,
    title     = {{SAFDNet}: A Simple and Effective Network for Fully Sparse 3D Object Detection},
    author    = {Zhang, Gang and Chen, Junnan and Gao, Guohuan and Li, Jianmin and Liu, Si and Hu, Xiaolin},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2024},
    pages     = {14477-14486}
}

Acknowleadgement

This two works were supported by the National Key Research and Development Program of China (No. 2021ZD0200301) and the National Natural Science Foundation of China (Nos. U19B2034, 61836014) and THU-Bosch JCML center.