Home

Awesome

SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimation

<img align="center" src="figures/animation.gif" width="750">

Video

This repository is the official implementation of our paper SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimation. For more details, please see our paper.

Introduction

SMOKE is a real-time monocular 3D object detector for autonomous driving. The runtime on a single NVIDIA TITAN XP GPU is ~30ms. Part of the code comes from CenterNet, maskrcnn-benchmark, and Detectron2.

The performance on KITTI 3D detection (3D/BEV) is as follows:

EasyModerateHard
Car14.17 / 21.089.88 / 15.138.63 / 12.91
Pedestrian5.16 / 6.223.24 / 4.052.53 / 3.38
Cyclist1.11 / 1.620.60 / 0.980.47 / 0.74

The pretrained weights can be downloaded here.

Requirements

All codes are tested under the following environment:

Dataset

We train and test our model on official KITTI 3D Object Dataset. Please first download the dataset and organize it as following structure:

kitti
│──training
│    ├──calib 
│    ├──label_2 
│    ├──image_2
│    └──ImageSets
└──testing
     ├──calib 
     ├──image_2
     └──ImageSets

Setup

  1. We use conda to manage the environment:
conda create -n SMOKE python=3.7
  1. Clone this repo:
git clone https://github.com/lzccccc/SMOKE
  1. Build codes:
python setup.py build develop
  1. Link to dataset directory:
mkdir datasets
ln -s /path_to_kitti_dataset datasets/kitti

Getting started

First check the config file under configs/.

We train the model on 4 GPUs with 32 batch size:

python tools/plain_train_net.py --num-gpus 4 --config-file "configs/smoke_gn_vector.yaml"

For single GPU training, simply run:

python tools/plain_train_net.py --config-file "configs/smoke_gn_vector.yaml"

We currently only support single GPU testing:

python tools/plain_train_net.py --eval-only --config-file "configs/smoke_gn_vector.yaml"

Acknowledgement

CenterNet

maskrcnn-benchmark

Detectron2

Citations

Please cite our paper if you find SMOKE is helpful for your research.

@article{liu2020SMOKE,
  title={{SMOKE}: Single-Stage Monocular 3D Object Detection via Keypoint Estimation},
  author={Zechen Liu and Zizhang Wu and Roland T\'oth},
  journal={arXiv preprint arXiv:2002.10111},
  year={2020}
}