Awesome

Deep Learning-Based Object Pose Estimation: A Comprehensive Survey

Jian Liu, Wei Sun, Hui Yang, Zhiwen Zeng, Chongpei Liu,

Jin Zheng, Xingyu Liu, Hossein Rahmani, Nicu Sebe, Ajmal Mian

Introduction | Datasets | Instance-Level | Category-Level | Unseen | Applications

</div>

Note: For any missing or recently published papers, feel free to pull a request, we will add them asap :)

Introduction

This is the official repository of ''Deep Learning-Based Object Pose Estimation: A Comprehensive Survey''. Specifically, we first introduce the datasets used for object pose estimation. Then, we review the instance-level, category-level, and unseen methods, respectively. Finally, we summarize the common applications of this task. The taxonomy of this survey is shown as follows

A comparison of instance-level, category-level, and unseen methods is shown as follows. Instance-level methods can only estimate the pose of specific object instances on which they are trained. Category-level methods can infer intra-class unseen instances rather than being limited to specific instances in the training data. In contrast, unseen object pose estimation methods have stronger generalization ability and can handle object categories not encountered during training.

Datasets

Chronological overview of the datasets for object pose estimation evaluation. Notably, the pink arrows represent the BOP Challenge datasets, which can be used to evaluate both instance-level and unseen object methods. The red references represent the datasets of articulated objects.

2.1 Datasets for Instance-Level Methods

<details> <summary>All Datasets</summary>

BOP Challenge Datasets [Paper] [Data]
YCBInEOAT Dataset [Paper] [Data]
ClearPose Dataset [Paper] [Data]
MP6D Dataset [Paper] [Data]

</details>

2.2 Datasets for Category-Level Methods

<details> <summary>2.2.1 Rigid Objects Datasets</summary>

CAMERA25 Dataset [Paper] [Data]
REAL275 Dataset [Paper] [Data]
kPAM Dataset [Paper] [Data]
TOD Dataset [Paper] [Data]
Objectron Dataset [Paper] [Data]
Wild6D Dataset [Paper] [Data]
PhoCaL Dataset [Paper] [Data]
HouseCat6D Dataset [Paper] [Data]

</details> <details> <summary>2.2.2 Articulated Objects Datasets</summary>

BMVC Dataset [Paper] [Data]
RBO Dataset [Paper] [Data]
HOI4D Dataset [Paper] [Data]
ReArtMix/ReArtVal Dataset [Paper]
ContactArt Dataset [Paper] [Data]

</details>

2.3 Datasets for Unseen Methods

<details> <summary>All Datasets</summary>

BOP Challenge Datasets [Paper] [Data]
MOPED Dataset [Paper] [Data]
GenMOP Dataset [Paper] [Data]
OnePose Dataset [Paper] [Data]
OnePose-LowTexture Dataset [Paper] [Data]

</details>

Instance-Level

Instance-level object pose estimation describes the task of estimating the pose of the objects that have been seen during the training of the model. We classify existing instance-level methods into four categories: correspondence-based, template-based, voting-based, and regression-based methods. The illustration of these four categories is shown as follows

3.1 Correspondence-Based Methods

<details> <summary>3.1.1 Sparse Correspondence Methods</summary>

2017

BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects Without Using Depth [Paper]
6-dof Object Pose from Semantic Keypoints [Paper] [Code]
Robust 3D Object Tracking from Monocular Images Using Stable Parts [Paper]

2018

Real-Time Seamless Single Shot 6D Object Pose Prediction [Paper]
Making Deep Heatmaps Robust to Partial Occlusions for 3D Object Pose Estimation [Paper]

2019

Segmentation-Driven 6D Object Pose Estimation [Paper]
Learning Local RGB-to-CAD Correspondences for Object Pose Estimation [Paper]

2020

HOPE-Net: A Graph-Based Model for Hand-Object Pose Estimation [Paper]
HybridPose: 6D Object Pose Estimation Under Hybrid Representations [Paper]
Learning Deep Network for Detecting 3D Object Keypoints and 6D Poses [Paper]
Introducing Pose Consistency and Warp-Alignment for Self-Supervised 6D Object Pose Estimation in Color Images [Paper]

2021

MFPN-6D : Real-time One-stage Pose Estimation of Objects on RGB Images [Paper]
Wide-Depth-Range 6D Object Pose Estimation in Space [Paper]
Confidence-Based 6D Object Pose Estimation [Paper]
DSC-PoseNet: Learning 6DoF Object Pose Estimation via Dual-Scale Consistency [Paper]
Semi-Supervised 3D Hand-Object Poses Estimation With Interactions in Time [Paper] [Code]
Keypoint-Graph-Driven Learning Framework for Object Pose Estimation [Paper]
PyraPose: Feature Pyramids for Fast and Accurate Object Pose Estimation under Domain Shift [Paper]
GhostPose: Multi-view Pose Estimation of Transparent Objects for Robot Hand Grasping [Paper]

2023

Knowledge Distillation for 6D Pose Estimation by Aligning Distributions of Local Predictions [Paper]
Linear-Covariance Loss for End-to-End Learning of 6D Pose Estimation [Paper]
CheckerPose: Progressive Dense Keypoint Localization for Object Pose Estimation with Graph Neural Network [Paper] [Code]

</details> <details> <summary>3.1.2 Dense Correspondence Methods</summary>

2019

CDPN: Coordinates-based Disentangled Pose Network for Real-time RGB-Based 6-DoF Object Pose Estimation [Paper] [Code]
DPOD: 6D Pose Object Detector and Refiner [Paper] [Code]
Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation [Paper] [Code]

2020

Reconstruct Locally, Localize Globally: A Model Free Method for Object Pose Estimation [Paper]
EPOS: Estimating 6D Pose of Objects with Symmetries [Paper] [Code]

2021

GeoPose: Dense Reconstruction Guided 6D Object Pose Estimation With Geometric Consistency [Paper]
DPODv2: Dense Correspondence-Based 6 DoF Pose Estimation [Paper]
Pseudo-Siamese Graph Matching Network for Textureless Objects’ 6-D Pose Estimation [Paper] [Code]
SD-Pose: Semantic Decomposition for Cross-Domain 6D Object Pose Estimation [Paper]

2022

ZebraPose: Coarse to Fine Surface Encoding for 6DoF Object Pose Estimation [Paper] [Code]
EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimation [Paper] [Code]
BiCo-Net: Regress Globally, Match Locally for Robust 6D Pose Estimation [Paper] [Code]
Neural Correspondence Field for Object Pose Estimation [Paper] [Code]
Perspective Flow Aggregation for Data-Limited 6D Object Pose Estimation [Paper] [Code]
SurfEmb: Dense and Continuous Correspondence Distributions for Object Pose Estimation with Learnt Surface Embeddings [Paper] [Code]
Coupled Iterative Refinement for 6D Multi-Object Pose Estimation [Paper] [Code]

2023

Center-Based Decoupled Point Cloud Registration for 6D Object Pose Estimation [Paper] [Code]
Geometric-aware dense matching network for 6D pose estimation of objects from RGB-D images [Paper] [Code]
NeRF-Pose: A First-Reconstruct-Then-Regress Approach for Weakly-supervised 6D Object Pose Estimation [Paper]
Pseudo Flow Consistency for Self-Supervised 6D Object Pose Estimation [Paper] [Code]

2024

HiPose: Hierarchical Binary Surface Encoding and Correspondence Pruning for RGB-D 6DoF Object Pose Estimation [Paper] [Code]
RNNPose: 6-DoF Object Pose Estimation Via Recurrent Correspondence Field Estimation and Pose Optimization [Paper] [Code]

</details>

3.2 Template-Based Methods

<details> <summary>3.2.1 RGB-Based Template Methods</summary>

2018

Implicit 3D Orientation Learning for 6D Object Detection from RGB Images [Paper] [Code]

2019

6DoF Pose Estimation with Object Cutout Based on a Deep Autoencoder [Paper]

2020

6D Object Pose Estimation Algorithm Using Preprocessing of Segmentation and Keypoint Extraction [Paper]
Domain-Translated 3D Object Pose Estimation [Paper]
Pose-guided Auto-Encoder and Feature-Based Refinement for 6-DoF Object Pose Regression [Paper]
Spatial Attention Improves Iterative 6D Object Pose Estimation [Paper] [Code]

2021

PoseRBPF: A Rao–Blackwellized Particle Filter for 6-D Object Pose Tracking [Paper] [Code]

</details> <details> <summary>3.2.2 Point Cloud-Based Template Methods</summary>

2022

DCL-Net: Deep Correspondence Learning Network for 6D Pose Estimation [Paper] [Code]

2023

SE(3) Diffusion Model-based Point Cloud Registration for Robust 6D Object Pose Estimation [Paper] [Code]

2024

Match Normalization: Learning-Based Point Cloud Registration for 6D Object Pose Estimation in the Real World [Paper]

</details>

3.3 Voting-Based Methods

<details> <summary>3.3.1 Indirect Voting Methods</summary>

2019

PVNet: Pixel-Wise Voting Network for 6DoF Pose Estimation [Paper] [Code]

2020

PVN3D: A Deep Point-Wise 3D Keypoints Voting Network for 6DoF Pose Estimation [Paper] [Code]
6DoF Object Pose Estimation via Differentiable Proxy Voting Loss [Paper]

2021

KDFNet: Learning Keypoint Distance Field for 6D Object Pose Estimation [Paper]
FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation [Paper] [Code]
ParametricNet: 6DoF Pose Estimation Network for Parametric Shapes in Stacked Scenarios [Paper] [Code]

2022

DGECN: A Depth-Guided Edge Convolutional Network for End-to-End 6D Pose Estimation [Paper]
Spatial Feature Mapping for 6DoF Object Pose Estimation [Paper]
Vote from the Center: 6 DoF Pose Estimation in RGB-D Images by Radial Keypoint Voting [Paper] [Code]
MV6D: Multi-View 6D Pose Estimation on RGB-D Frames Using a Deep Point-wise Voting Network [Paper]
Learning to Estimate Object Poses without Real Image Annotations [Paper] [Code]
Sim2Real Instance-Level Style Transfer for 6D Pose Estimation [Paper]

2023

BDR6D: Bidirectional Deep Residual Fusion Network for 6D Pose Estimation [Paper]
KVNet: An iterative 3D keypoints voting network for real-time 6-DoF object pose estimation [Paper]
Deep Fusion Transformer Network with Weighted Vector-Wise Keypoints Voting for Robust 6D Object Pose Estimation [Paper] [Code]

2024

6D-Diff: A Keypoint Diffusion Framework for 6D Object Pose Estimation [Paper]

</details> <details> <summary>3.3.2 Direct Voting Methods</summary>

2019

DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion [Paper] [Code]

2020

A Novel Depth and Color Feature Fusion Framework for 6D Object Pose Estimation [Paper]
PPR-Net++: Accurate 6-D Pose Estimation in Stacked Scenarios [Paper] [Code]

2021

Robust 6D Object Pose Estimation by Learning RGB-D Features [Paper] [Code]
PR-GCN: A Deep Graph Convolutional Network With Point Refinement for 6D Pose Estimation [Paper] [Code]
TemporalFusion: Temporal Motion Reasoning with Multi-Frame Fusion for 6D Object Pose Estimation [Paper] [Code]

2022

ES6D: A Computation Efficient and Symmetry-Aware 6D Pose Regression Framework [Paper] [Code]
SC6D: Symmetry-agnostic and Correspondence-free 6D Object Pose Estimation [Paper] [Code]

2023

A Depth Adaptive Feature Extraction and Dense Prediction Network for 6-D Pose Estimation in Robotic Grasping [Paper]

</details>

3.4 Regression-Based Methods

<details> <summary>3.4.1 Geometry-Guided Regression Methods</summary>

2020

6D Object Pose Regression via Supervised Learning on Point Clouds [Paper]
G2L-Net: Global to Local Network for Real-Time 6D Pose Estimation With Embedding Vector Features [Paper] [Code]
Single-Stage 6D Object Pose Estimation [Paper]

2021

6D Object Pose Estimation with Pairwise Compatible Geometric Features [Paper]
CloudAAE: Learning 6D Object Pose Regression with On-Line Data Synthesis on Point Clouds [Paper] [Code]
Semi-Supervised 6D Object Pose Estimation Without Using Real Annotations [Paper]
StablePose: Learning 6D Object Poses From Geometrically Stable Patches [Paper] [Code]
GCCN: Geometric Constraint Co-attention Network for 6D Object Pose Estimation [Paper]
SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation [Paper] [Code]
GDR-Net: Geometry-Guided Direct Regression Network for Monocular 6D Object Pose Estimation [Paper] [Code]
Occlusion-Aware Self-Supervised Monocular 6D Object Pose Estimation [Paper] [Code]

2022

Trans6D: Transformer-Based 6D Object Pose Estimation and Refinement [Paper]

2023

SMOC-Net: Leveraging Camera Pose for Self-Supervised Monocular Object Pose Estimation [Paper]
NVR-Net: Normal Vector Guided Regression Network for Disentangled 6D Pose Estimation [Paper]

2024

PA-Pose: Partial Point Cloud Fusion Based on Reliable Alignment for 6D Pose Tracking [Paper]
HFT6D: Multimodal 6D Object Pose Estimation Based on Hierarchical Feature Transformer [Paper]

</details> <details> <summary>3.4.2 Direct Regression Methods</summary>

2017

SSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again [Paper] [Code]
Deep 6-DOF Tracking [Paper] [Code]
PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes [Paper] [Code]

2018

Learning 6DoF Object Poses from Synthetic Single Channel Images [Paper]
Real-Time Object Pose Estimation with Pose Interpreter Networks [Paper] [Code]
Deep-6DPose: Recovering 6D Object Pose from a Single RGB Image [Paper] [Code]
A Unified Framework for Multi-View Multi-Class Object Pose Estimation [Paper]
DeepIM: Deep Iterative Matching for 6D Pose Estimation [Paper] [Code]
Deep Model-Based 6D Pose Refinement in RGB [Paper] [Code]
Object Detection and Pose Estimation Based on Convolutional Neural Networks Trained with Synthetic Data [Paper]

2019

PCRNet: Point Cloud Registration Network using PointNet Encoding [Paper] [Code]
Explaining the Ambiguity of Object Detection and 6D Pose From Visual Data [Paper]
3D Object Pose Estimation Using Multi-Objective Quaternion Learning [Paper]
Regression-Based Three-Dimensional Pose Estimation for Texture-Less Objects [Paper] [Code]

2020

Single Shot 6D Object Pose Estimation [Paper]
EfficientPose: An Efficient, Accurate and Scalable End-to-End 6D Multi Object Pose Estimation Approach [Paper] [Code]
Self6D: Self-Supervised Monocular 6D Object Pose Estimation [Paper] [Code]
CosyPose: Consistent Multi-view Multi-object 6D Pose Estimation [Paper] [Code]
se(3)-TrackNet: Data-Driven 6D Pose Tracking by Calibrating Image Residuals in Synthetic Domains [Paper] [Code]

2021

Pose Estimation from RGB Images of Highly Symmetric Objects using a Novel Multi-Pose Loss and Differential Rendering [Paper] [Code]
VIPose: Real-Time Visual-Inertial 6D Object Pose Tracking [[Paper]](VIPose: Real-time Visual-Inertial 6D Object Pose Tracking)
RePOSE: Fast 6D Object Pose Refinement via Deep Texture Rendering [Paper] [Code]

2022

DProST: Dynamic Projective Spatial Transformer Network for 6D Pose Estimation [Paper] [Code]
Uni6D: A Unified CNN Framework Without Projection Breakdown for 6D Pose Estimation [Paper]
HFF6D: Hierarchical Feature Fusion Network for Robust 6D Object Pose Tracking [Paper]

2023

Shape-Constraint Recurrent Flow for 6D Object Pose Estimation [Paper] [Code]

2024

Confronting Ambiguity in 6D Object Pose Estimation via Score-Based Diffusion on SE(3) [Paper]
MRC-Net: 6-DoF Pose Estimation with MultiScale Residual Correlation [Paper] [Code]

</details>

Category-Level

Research on category-level methods has garnered significant attention due to their potential for generalizing to unseen objects within established categories. We classify them into two categories: shape prior-based and shape prior-free methods. The illustration of these two categories is shown as follows

4.1 Shape Prior-Based Methods

<details> <summary>4.1.1 NOCS Shape Alignment Methods</summary>

2020

Shape Prior Deformation for Categorical 6D Object Pose and Size Estimation [Paper] [Code]

2021

Category-Level 6D Object Pose Estimation via Cascaded Relation and Recurrent Reconstruction Networks [Paper]
SGPA: Structure-Guided Prior Adaptation for Category-Level 6D Object Pose Estimation [Paper] [Code]
ACR-Pose: Adversarial Canonical Representation Reconstruction Network for Category Level 6D Object Pose Estimation [Paper]

2022

6D-ViT: Category-Level 6D Object Pose Estimation via Transformer-Based Instance Representation Learning [Paper] [Code]
Towards Self-Supervised Category-Level Object Pose and Size Estimation [Paper]
Object Level Depth Reconstruction for Category Level 6D Object Pose Estimation from Monocular RGB Image [Paper]

2023

Category-Level 6D Pose Estimation Using Geometry-Guided Instance-Aware Prior and Multi-Stage Reconstruction [Paper]
StereoPose: Category-Level 6D Transparent Object Pose Estimation from Stereo Images via Back-View NOCS [Paper] [Code]
DR-Pose: A Two-Stage Deformation-and-Registration Pipeline for Category-Level 6D Object Pose Estimation [Paper] [Code]
GPT-COPE: A Graph-Guided Point Transformer for Category-Level Object Pose Estimation [Paper]
SD-Pose: Structural Discrepancy Aware Category-Level 6D Object Pose Estimation [Paper]
Generative Category-Level Shape and Pose Estimation with Semantic Primitives [Paper] [Code]
RGB-based Category-level Object Pose Estimation via Decoupled Metric Scale Recovery [Paper] [Code]

2024

CatFormer: Category-Level 6D Object Pose Estimation with Transformer [Paper] [Code]

</details> <details> <summary>4.1.2 Direct Regress Pose Methods</summary>

2022

SAR-Net: Shape Alignment and Recovery Network for Category-Level 6D Object Pose and Size Estimation [Paper] [Code]
CenterSnap: Single-Shot Multi-Object 3D Shape Reconstruction and Categorical 6D Pose and Size Estimation [Project Page] [Paper] [Code]
ShAPO: Implicit Representations for Multi-Object Shape Appearance and Pose Optimization [Project Page] [Paper] [Code]
SSP-Pose: Symmetry-Aware Shape Prior Deformation for Direct Category-Level Object Pose Estimation [Paper]
RBP-Pose: Residual Bounding Box Projection for Category-Level Pose Estimation [Paper] [Code]
CATRE: Iterative Point Clouds Alignment for Category-Level Object Pose Refinement [Paper] [Code]
Category-Level 6D Object Pose and Size Estimation Using Self-supervised Deep Prior Deformation Networks [Paper] [Code]
Category-Level 6D Object Pose Estimation in the Wild: A Semi-Supervised Learning Approach and A New Dataset [Paper] [Code]

2023

Self-Supervised Geometric Correspondence for Category-Level 6D Object Pose Estimation in the Wild [Paper] [Code]
i2c-net: Using Instance-Level Neural Networks for Monocular Category-Level 6D Pose Estimation [Paper]
DTF-Net: Category-Level Pose Estimation and Shape Reconstruction via Deformable Template Field [Paper]

2024

FSD: Fast Self-Supervised Single RGB-D to Categorical 3D Objects [Project Page] [Paper] [Video]
DiffusionNOCS: Managing Symmetry and Uncertainty in Sim2Real Multi-Modal Category-level Pose Estimation [Project Page] [Paper] [Code]
MH6D: Multi-Hypothesis Consistency Learning for Category-Level 6-D Object Pose Estimation [Paper] [Code]
GeoReF: Geometric Alignment Across Shape Variation for Category-level Object Pose Refinement [Paper] [Code]

</details>

4.2 Shape Prior-Free Methods

<details> <summary>4.2.1 Depth-Guided Geometry-Aware Methods</summary>

2020

Category-Level Articulated Object Pose Estimation [Paper] [Code]

2021

FS-Net: Fast Shape-Based Network for Category-Level 6D Object Pose Estimation With Decoupled Rotation Mechanism [Paper] [Code]
CAPTRA: CAtegory-Level Pose Tracking for Rigid and Articulated Objects From Point Clouds [Paper] [Code]
GarmentNets: Category-Level Pose Estimation for Garments via Canonical Space Shape Completion [Paper] [Code]
Leveraging SE(3) Equivariance for Self-Supervised Category-Level Object Pose Estimation from Point Clouds [Paper] [Code]

2022

GPV-Pose: Category-Level Object Pose Estimation via Geometry-Guided Point-Wise Voting [Paper] [Code]
CPPF: Towards Robust Category-Level 9D Pose Estimation in the Wild [Paper] [Code]
iCaps: Iterative Category-Level Object Pose and Shape Estimation [Paper]
Category-Level 6D Object Pose Estimation With Structure Encoder and Reasoning Attention [Paper]

2023

Self-Supervised Category-Level Articulated Object Pose Estimation with Part-Level SE(3) Equivariance [Paper] [Code]
HS-Pose: Hybrid Scope Feature Extraction for Category-Level Object Pose Estimation [Paper] [Code]
Query6DoF: Learning Sparse Queries as Implicit Shape Prior for Category-Level 6DoF Pose Estimation [Paper] [Code]
SOCS: Semantically-Aware Object Coordinate Space for Category-Level 6D Object Pose Estimation under Large Shape Variations [Paper] [Code]
Generative Category-level Object Pose Estimation via Diffusion Models [Paper] [Code]
Category-Level Articulated Object 9D Pose Estimation via Reinforcement Learning [Paper]

2024

Instance-Adaptive and Geometric-Aware Keypoint Learning for Category-Level 6D Object Pose Estimation [Paper] [Code]
Category-Level Multi-Part Multi-Joint 3D Shape Assembly [Paper]

</details> <details> <summary>4.2.2 RGBD-Guided Semantic and Geometry Fusion Methods</summary>

2019

Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation [Paper] [Code]

2020

Learning Canonical Shape Space for Category-Level 6D Object Pose and Size Estimation [Paper]
6-PACK: Category-Level 6D Pose Tracker with Anchor-Based Keypoints [Paper] [Code]

2021

DualPoseNet: Category-Level 6D Object Pose and Size Estimation Using Dual Pose Network With Refined Learning of Pose Consistency [Paper] [Code]
BundleTrack: 6D Pose Tracking for Novel Objects without Instance or Category-Level 3D Models [Paper] [Code]
Sparse Steerable Convolutions: An Efficient Learning of SE(3)-Equivariant Features for Estimation and Tracking of Object Poses in 3D Space [Paper] [Code]

2022

Self-Supervised Category-Level 6D Object Pose Estimation with Deep Implicit Shape Representation [Paper] [Code]
UDA-COPE: Unsupervised Domain Adaptation for Category-Level Object Pose Estimation [Paper]
Attention-guided RGB-D Fusion Network for Category-level 6D Object Pose Estimation [Paper]

2023

TTA-COPE: Test-Time Adaptation for Category-Level Object Pose Estimation [Paper]
IST-Net: Prior-Free Category-Level Pose Estimation with Implicit Space Transformation [Paper] [Code]
VI-Net: Boosting Category-Level 6D Object Pose Estimation via Learning Decoupled Rotations on the Spherical Representations [Paper] [Code]

2024

SecondPose: SE(3)-Consistent Dual-Stream Feature Fusion for Category-Level Pose Estimation [Paper] [Code]

</details> <details> <summary>4.2.3 Others</summary>

2020

Category Level Object Pose Estimation via Neural Analysis-by-Synthesis [Paper] [Code]

2021

Category-Level Metric Scale Object Shape and Pose Estimation [Paper]
iNeRF: Inverting Neural Radiance Fields for Pose Estimation [Paper]

2022

Single-Stage Keypoint-Based Category-Level Object Pose Estimation from an RGB Image [Paper]
Keypoint-Based Category-Level Object Pose Tracking from an RGB Sequence with Uncertainty Estimation [Paper]
A Visual Navigation Perspective for Category-Level Object Pose Estimation [Paper] [Code]
Robust Category-Level 6D Pose Estimation with Coarse-to-Fine Rendering of Neural Features [Paper]
TransNet: Category-Level Transparent Object Pose Estimation [Paper]
Zero-Shot Category-Level Object Pose Estimation [Paper] [Code]

2023

CatTrack: Single-Stage Category-Level 6D Object Pose Tracking via Convolution and Vision Transformer [Paper]
Self-Supervised Category-Level 6D Object Pose Estimation With Optical Flow Consistency [Paper]

2024

OV9D: Open-Vocabulary Category-Level 9D Object Pose and Size Estimation [Paper] [Code]
Zero123-6D: Zero-shot Novel View Synthesis for RGB Category-level 6D Pose Estimation [Paper]
CLIPose: Category-Level Object Pose Estimation with Pre-trained Vision-Language Knowledge [Paper]

</details>

Unseen

Unseen object pose estimation methods can generalize to unseen objects without the need for retraining. We classify existing unseen methods into two categories: CAD model-based and manual reference view-based methods. The illustration of these two categories is shown as follows

5.1 CAD Model-Based Methods

<details> <summary>5.1.1 Feature Matching-Based Methods</summary>

2019

CorNet: Generic 3D Corners for 6D Pose Estimation of New Objects without Retraining [Paper]

2020

3D Object Detection and Pose Estimation of Unseen Objects in Color Images with Local Surface Embeddings [Paper]

2022

Unseen Object 6D Pose Estimation: A Benchmark and Baselines [Paper] [Code]

2023

Learning Symmetry-Aware Geometry Correspondences for 6D Object Pose Estimation [Paper] [Code]
KeyMatchNet: Zero-Shot Pose Estimation in 3D Point Clouds by Generalized Keypoint Matching [Paper]
ZeroPose: CAD-Model-based Zero-Shot Pose Estimation [Paper]

2024

FreeZe: Training-Free Zero-Shot 6D Pose Estimation with Geometric and Vision Foundation Models [Paper] [Code]
MatchU: Matching Unseen Objects for 6D Pose Estimation from RGB-D Images [Paper]
SAM-6D: Segment Anything Model Meets Zero-Shot 6D Object Pose Estimation [Paper] [Code]

</details> <details> <summary>5.1.2 Template Matching-Based Methods</summary>

2015-2019

Learning Descriptors for Object Recognition and 3D Pose Estimation [Paper]
Pose Guided RGBD Feature Learning for 3D Object Pose Estimation [Paper]
Pose Estimation for Objects with Rotational Symmetry [Paper] [Code]

2020

Multi-path Learning for Object Pose Estimation Across Domains [Paper] [Code]

2021

ZePHyR: Zero-Shot Pose Hypothesis Rating [Paper] [Code]

2022

I Like to Move It: 6D Pose Estimation as an Action Decision Process [Paper]
DISP6D: Disentangled Implicit Shape and Pose Learning for Scalable 6D Pose Estimation [Paper] [Code]
Templates for 3D Object Pose Estimation Revisited: Generalization to New objects and Robustness to Occlusions [Paper] [Code]
Fusing Local Similarities for Retrieval-based 3D Orientation Estimation of Unseen Objects [Paper] [Code]
Self-Supervised Vision Transformers for 3D Pose Estimation of Novel Objects [Paper]
OVE6D: Object Viewpoint Encoding for Depth-Based 6D Object Pose Estimation [Paper] [Code]
OSOP: A Multi-Stage One Shot Object Pose Estimation Framework [Paper]
MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare [Paper] [Code]

2023

Diff-DOPE: Differentiable Deep Object Pose Estimation [Paper] [Code]
ZS6D: Zero-Shot 6D Object Pose Estimation using Vision Transformers [Paper] [Code]
FoundPose: Unseen Object Pose Estimation with Foundation Features [Paper] [Code]

2024

FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects [Paper] [Code]
Object Pose Estimation via the Aggregation of Diffusion Features [Paper] [Code]
GigaPose: Fast and Robust Novel Object Pose Estimation via One Correspondence [Paper] [Code]
GenFlow: Generalizable Recurrent Flow for 6D Pose Refinement of Novel Objects [Paper]

</details>

5.2 Manual Reference View-Based Methods

<details> <summary>5.2.1 Feature Matching-Based Methods</summary>

2021

Unseen Object Pose Estimation via Registration [Paper]

2022

FS6D: Few-Shot 6D Pose Estimation of Novel Objects [Paper] [Code]
OnePose: One-Shot Object Pose Estimation without CAD Models [Paper] [Code]
OnePose++: Keypoint-Free One-Shot Object Pose Estimation without CAD Models [Paper] [Code]

2023

POPE: 6-DoF Promptable Pose Estimation of Any Object, in Any Scene, with One Reference [Paper] [Code]
PoseMatcher: One-shot 6D Object Pose Estimation by Deep Feature Matching [Paper] [Code]

2024

Open-Vocabulary Object 6D Pose Estimation [Paper] [Code]
MFOS: Model-Free & One-Shot Object Pose Estimation [Paper]

</details> <details> <summary>5.2.2 Template Matching-Based Methods</summary>

2020

LatentFusion: End-to-End Differentiable Reconstruction and Rendering for Unseen Object Pose Estimation[Paper] [Code]

2022

PIZZA: A Powerful Image-only Zero-Shot Zero-CAD Approach to 6 DoF Tracking [Paper] [Code]
Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Images [Paper] [Code]

2023

SA6D: Self-Adaptive Few-Shot 6D Pose Estimator for Novel and Occluded Objects [Paper]
BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects [Paper] [Code]

2024

NOPE: Novel Object Pose Estimation from a Single Image [Paper] [Code]
LocPoseNet: Robust Location Prior for Unseen Object Pose Estimation [Paper] [Code]
Learning to Estimate 6DoF Pose from Limited Data: A Few-Shot, Generalizable Approach using RGB Images [Paper] [Code]
GS-Pose: Cascaded Framework for Generalizable Segmentation-based 6D Object Pose Estimation [Paper] [Code]
FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects [Paper] [Code]

</details>

Applications

Chronological overview of some representative applications of object pose estimation methods. The black references, red references, and orange references represent the application of instance-level, category-level, and unseen methods, respectively.

6.1 Robotic Manipulation

<details> <summary>6.1.1 Instance-Level Manipulation</summary>

Weakly supervised 6D pose estimation for robotic grasping [Paper]
Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects [Paper] [Code]
Detect in RGB, Optimize in Edge: Accurate 6D Pose Estimation for Texture-less Industrial Parts [Paper]
PPR-Net:Point-wise Pose Regression Network for Instance Segmentation and 6D Pose Estimation in Bin-picking Scenarios [Paper]
MoreFusion: Multi-object Reasoning for 6D Pose Estimation from Volumetric Fusion [Paper] [Code]
GhostPose: Multi-view Pose Estimation of Transparent Objects for Robot Hand Grasping [Paper]
Tomato harvesting robotic system based on Deep-ToMaToS: Deep learning network using transformation loss for 6D pose estimation of maturity classified tomatoes with side-stem [Paper]
6D Robotic Assembly Based on RGB-only Object Pose Estimation [Paper]
Sim-to-Real 6D Object Pose Estimation via Iterative Self-training for Robotic Bin Picking [Paper]
AttentionVote: A coarse-to-fine voting network of anchor-free 6D pose estimation on point cloud for robotic bin-picking application [Paper]

</details> <details> <summary>6.1.2 Category-Level Manipulation</summary>

Robotic Continuous Grasping System by Shape Transformer-Guided Multiobject Category-Level 6-D Pose Estimation [Paper] [Code]
Fine Segmentation and Difference-Aware Shape Adjustment for Category-Level 6DoF Object Pose Estimation [Paper]
Category-Level 6-D Object Pose Estimation With Shape Deformation for Robotic Grasp Detection [Paper]
ICK-Track: A Category-Level 6-DoF Pose Tracker Using Inter-Frame Consistent Keypoints for Aerial Manipulation [Paper] [Code]
Robotic Grasp Detection Based on Category-Level Object Pose Estimation With Self-Supervised Learning [Paper]
Domain-Generalized Robotic Picking via Contrastive Learning-Based 6-D Pose Estimation [Paper]

</details> <details> <summary>6.1.3 Unseen Object Manipulation</summary>

ZePHyR: Zero-shot Pose Hypothesis Rating [Paper] [Code]
MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare [Paper] [Code]
FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects [Paper] [Code]

</details>

6.2 Augmented Reality/Virtual Reality

<details> <summary>All Methods</summary>

Deep Multi-state Object Pose Estimation for Augmented Reality Assembly [Paper]
Efficient 6-DoF Tracking of Handheld Objects from an Egocentric Viewpoint [Paper]
Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Images [Paper] [Code]
OnePose++: Keypoint-Free One-Shot Object Pose Estimation without CAD Models [Paper] [Code]
FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects [Paper] [Code]

</details>

6.3 Aerospace

<details> <summary>All Methods</summary>

Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering [Paper]
Wide-Depth-Range 6D Object Pose Estimation in Space [Paper] [Code]
CA-SpaceNet: Counterfactual Analysis for 6D Pose Estimation in Space [Paper] [Code]
6D Object Pose Estimation from Approximate 3D Models for Orbital Robotics [Paper]

</details>

6.4 Hand-Object Interaction

<details> <summary>All Methods</summary>

Object Learning for 6D Pose Estimation and Grasping from RGB-D Videos of In-hand Manipulation [Paper]
Harmonious Feature Learning for Interactive Hand-Object Pose Estimation [Paper] [Code]
Hierarchical Graph Neural Networks for Proprioceptive 6D Pose Estimation of In-hand Objects [Paper]
HOISDF: Constraining 3D Hand-Object Pose Estimation with Global Signed Distance Fields [Paper] [Code]

</details>

6.5 Autonomous Driving

<details> <summary>All Methods</summary>

Deep Learning for 6D Pose Estimation of Objects — A Case Study for Autonomous Driving [Paper]
PanelPose: A 6D Pose Estimation of Highly-Variable Panel Object for Robotic Robust Cockpit Panel Inspection [Paper]

</details>

Citation

If you find the paper useful, please cite our paper.

@article{liu2024survey,
  title={Deep Learning-Based Object Pose Estimation: A Comprehensive Survey},
  author={Liu, Jian and Sun, Wei and Yang, Hui and Zeng, Zhiwen and Liu, Chongpei and Zheng, Jin and Liu, Xingyu and Rahmani, Hossein and Sebe, Nicu and Mian, Ajmal},  
  journal={arXiv preprint arXiv:2405.07801},
  year={2024}
}

Contact

Due to the one-sided nature of our knowledge, if you find any issues or have any suggestions, please feel free to post an issue or contact us via email