Awesome
<p align="center"> <font color=#008000>SAM-6D</font>: Segment Anything Model Meets Zero-Shot 6D Object Pose Estimation </p>
<p align="center"> Jiehong Lin, Lihua Liu, Dekun Lu, Kui Jia</p>
<p align="center">CVPR 2024 </p>
<p align="center">[Paper] </p>
<p align="center"> <img width="100%" src="https://github.com/JiehongLin/SAM-6D/blob/main/pics/vis.gif"/> </p>News
- [2024/03/07] We publish an updated version of our paper on ArXiv.
- [2024/02/29] Our paper is accepted by CVPR2024!
Update Log
- [2024/03/05] We update the demo to support FastSAM, you can do this by specifying
SEGMENTOR_MODEL=fastsam
in demo.sh. - [2024/03/03] We upload a docker image for running custom data.
- [2024/03/01] We update the released model of PEM. For the new model, a larger batchsize of 32 is set, while that of the old is 12.
Overview
In this work, we employ Segment Anything Model as an advanced starting point for zero-shot 6D object pose estimation from RGB-D images, and propose a novel framework, named SAM-6D, which utilizes the following two dedicated sub-networks to realize the focused task:
<p align="center"> <img width="50%" src="https://github.com/JiehongLin/SAM-6D/blob/main/pics/overview_sam_6d.png"/> </p>Getting Started
1. Preparation
Please clone the repository locally:
git clone https://github.com/JiehongLin/SAM-6D.git
Install the environment and download the model checkpoints:
cd SAM-6D
sh prepare.sh
We also provide a docker image for convenience.
2. Evaluation on the custom data
# set the paths
export CAD_PATH=Data/Example/obj_000005.ply # path to a given cad model(mm)
export RGB_PATH=Data/Example/rgb.png # path to a given RGB image
export DEPTH_PATH=Data/Example/depth.png # path to a given depth map(mm)
export CAMERA_PATH=Data/Example/camera.json # path to given camera intrinsics
export OUTPUT_DIR=Data/Example/outputs # path to a pre-defined file for saving results
# run inference
cd SAM-6D
sh demo.sh
Citation
If you find our work useful in your research, please consider citing:
@article{lin2023sam,
title={SAM-6D: Segment Anything Model Meets Zero-Shot 6D Object Pose Estimation},
author={Lin, Jiehong and Liu, Lihua and Lu, Dekun and Jia, Kui},
journal={arXiv preprint arXiv:2311.15707},
year={2023}
}
Contact
If you have any questions, please feel free to contact the authors.
Jiehong Lin: mortimer.jh.lin@gmail.com
Lihua Liu: lihualiu.scut@gmail.com
Dekun Lu: derkunlu@gmail.com
Kui Jia: kuijia@gmail.com