Home

Awesome

EPro-PnP v2

This repository contains the upgraded code for the CVPR 2022 paper EPro-PnP, featuring improved models for both the 6DoF and 3D detection benchmarks.

A new updated preprint can be found on arXiv: EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimation.

<img src="overview.png" width="800" alt=""/>

Models

EPro-PnP-Det v2: state-of-the-art monocular 3D object detector

Main differences to v1b:

At the time of submission (Aug 30, 2022), EPro-PnP-Det v2 ranks 1st among all camera-based single-frame object detection models on the official nuScenes benchmark (test split, without extra data).

MethodTTABackboneNDSmAPmATEmASEmAOEmAVEmAAESchedule
EPro-PnP-Det v2 (ours)YR1010.4900.4230.5470.2360.3021.0710.12312 ep
PETRNSwin-B0.4830.4450.6270.2490.4490.9270.14124 ep
BEVDet-BaseYSwin-B0.4820.4220.5290.2360.3950.9790.15220 ep
EPro-PnP-Det v2 (ours)NR1010.4810.4090.5590.2390.3251.0900.11512 ep
PolarFormerNR1010.4700.4150.6570.2630.4050.9110.13924 ep
BEVFormer-SNR1010.4620.4090.6500.2610.4390.9250.14724 ep
PETRNR1010.4550.3910.6470.2510.4330.9330.14324 ep
EPro-PnP-Det v1YR1010.4530.3730.6050.2430.3591.0670.12412 ep
PGDYR1010.4480.3860.6260.2450.4511.5090.12724+24 ep
FCOS3DYR1010.4280.3580.6900.2490.4521.4340.124-

EPro-PnP-6DoF v2 for 6DoF pose estimation<br>

Main differences to v1b:

With these updates the v2 model can be trained without 3D models to achieve better performance (ADD 0.1d = 93.83) than GDRNet (ADD 0.1d = 93.6), unleashing the full potential of simple end-to-end training.

Citation

If you find this project useful in your research, please consider citing:

@inproceedings{epropnp, 
  author = {Hansheng Chen and Pichao Wang and Fan Wang and Wei Tian and Lu Xiong and Hao Li, 
  title = {EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimation}, 
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, 
  year = {2022}
}