Awesome

Self-Supervised Deep Prior Deformation Network

Code for "Category-Level 6D Object Pose and Size Estimation using Self-Supervised Deep Prior Deformation Networks". ECCV2022.

[Paper] [Arxiv]

Created by Jiehong Lin, Zewei Wei, Changxing Ding, and Kui Jia.

News

• 2023/08: Our another work named VI-Net is accepted by ICCV2023.
• 2023/07: Codes tested with Pytorch 1.9.0 and CUDA 11.2 are also provided.

Requirements

The code has been tested with

• python 3.6.5
• pytorch 1.3.0
• CUDA 10.2

Some dependent packages：

• gorilla

pip install gorilla-core==0.2.5.6

• pointnet2

cd model/pointnet2
python setup.py install

Data Processing

Download the data provided by NOCS (camera_train, camera_test, camera_composed_depths, real_train, real_test, ground truths, and mesh models) and segmentation results (Link), and unzip them in data folder as follows:

data
├── CAMERA
│   ├── train
│   └── val
├── camera_full_depths
│   ├── train
│   └── val
├── Real
│   ├── train
│   └── test
├── gts
│   ├── val
│   └── real_test
├── obj_models
│   ├── train
│   ├── val
│   ├── real_train
│   └── real_test
├── segmentation_results
│   ├── train_trainedwoMask
│   ├── test_trainedwoMask
│   └── test_trainedwithMask
└── mean_shapes.npy

Run the following scripts to prepare the dataset:

python data_processing.py

Training DPDN under Different Settings

Train DPDN under unsupervised setting:

python train.py --gpus 0,1 --config config/unsupervised.yaml

Train DPDN under supervised setting:

python train.py --gpus 0,1 --config config/supervised.yaml

Evaluation

Download trained models and test results [Link]. Evaluate our models under different settings:

python test.py --config config/unsupervised.yaml
python test.py --config config/supervised.yaml

or directly evaluate our results on REAL275 test set:

python test.py --config config/unsupervised.yaml --only_eval
python test.py --config config/supervised.yaml --only_eval

One can also evaluate our models under the easier unsupervised setting with mask labels for segmentation (still without pose annotations):

python test.py --config config/unsupervised.yaml --mask_label

or

python test.py --config config/unsupervised.yaml --mask_label --only_eval

Results

Qualitative results on REAL275 test set:

Citation

If you find our work useful in your research, please consider citing:

@inproceedings{lin2022category,
  title={Category-level 6D object pose and size estimation using self-supervised deep prior deformation networks},
  author={Lin, Jiehong and Wei, Zewei and Ding, Changxing and Jia, Kui},
  booktitle={European Conference on Computer Vision},
  pages={19--34},
  year={2022},
  organization={Springer}
}

Acknowledgements

Our implementation leverages the code from NOCS, DualPoseNet, and SPD.

License

Our code is released under MIT License (see LICENSE file for details).

Contact

lin.jiehong@mail.scut.edu.cn