Home

Awesome

<div align="center"> <h1>StreamPETR with 3dppe Extension</h1> </div>

Introduction

This repository is an implementation of StreamPETR with 3dppe.


Getting Started

  1. Prepare nuScenes dataset and generate 2D annotations and temporal information for training & evaluation. (see streamPETR)

  2. Conda env

conda create -n xxx python=3.8 -y
conda activate xxx
pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html

pip install flash-attn==0.2.2  # (Tesla v100 is not compatible)

pip install mmcv-full==1.6.0
pip install mmdet==2.28.2
pip install mmsegmentation==0.30.0
git clone https://github.com/open-mmlab/mmdetection3d.git
cd mmdetection3d
git checkout v1.0.0rc6 
pip install -v -e .

Note : make sure that numba 0.53.0 numpy 1.23.5
(if not, reinstall numba==0.53.0).

Catalogue: tree -d -L 1

.
├── ckpts
├── data
├── mmdetection3d
├── projects
└── tools
  1. Train & Infer
tools/dist_train.sh [-config] [-num_gpus]
tools/dist_test.sh [-config] [-model] [-num_gpus] --eval bbox

Results on NuScenes Val Set

ModelSettingPretrainLr SchdTraining TimeNDSmAPConfigDownload
StreamPETRV2-99-900q-800x320FCOS3D24ep13h57.148.3configmodel/log
Stream3dppeV2-99-900q-800x320FCOS3D24ep16h58.45/58.4549.95/50.04configmodel1,model2)/(log1,log2)
Stream3dppe_gt_detphV2-99-900q-800x320FCOS3D24ep22h61.755.3configmodel/log
StreamPETRV2-99-900q-1600x640FCOS3D24ep
Stream3DPPEV2-99-900q-1600x640FCOS3D24ep

Note : Stream3dppe is trained on 4 x RTX 3090 with bs4 ,while Stream3dppe_gt_detph is trained on 4 x RTX 2080Ti with bs2 .

More result please refer to https://github.com/drilistbox/3DPPE.


Acknowledgement

Many thanks to the authors of PETR and StreamPETR.


Citation

If you find this project useful for your research, please consider citing:

@article{shu20233DPPE,
  title={3DPPE: 3D Point Positional Encoding for Multi-Camera 3D Object Detection Transformers},
  author={Shu, Changyong and Deng, Jiajun and Yu, Fisher and Liu, Yifan},
  journal={arXiv preprint arXiv:2211.14710},
  year={2023}
}