Awesome
Generalizing-Grasp
Generalizing 6-DoF Grasp Detection via Domain Prior Knowledge<br>
Haoxiang Ma, Modishi, Boyang Gao, Di Huang<br> In CVPR'2024
Paper Video
Introduction
This repository is official PyTorch implementation for our CVPR2024 paper. The code is based on GraspNet-baseline
Note: The repo is still updating
Environments
- Anaconda3
- Python == 3.7.9
- PyTorch >= 1.8.0
- Open3D >= 0.8
Installation
Follow the installation of graspnet-baseline.
Get the code.
git clone https://github.com/mahaoxiang822/Generalizing-Grasp.git
cd Generalizing-Grasp
Install packages via Pip.
pip install -r requirements.txt
Compile and install pointnet2 operators (code adapted from votenet).
cd pointnet2
python setup.py install
Compile and install knn operator (code adapted from pytorch_knn_cuda).
cd knn
python setup.py install
Install graspnetAPI for evaluation.
git clone https://github.com/graspnet/graspnetAPI.git
cd graspnetAPI
pip install .
Prepare Datasets
For GraspNet dataset, you can download from GraspNet
Full scene data generation
You can generate fusion scene data by yourself by running:
cd scripts
python TSDFreconstruction_dataset.py
Or you can download the pre-generated data from Google Drive and unzip it under the dataset root:
Object SDF generation
You can generate object SDF by running:
pip install mesh-to-sdf
python dataset/grid_sample.py
Tolerance Label Generation(Follow graspnet-baseline)
Tolerance labels are not included in the original dataset, and need additional generation. Make sure you have downloaded the orginal dataset from GraspNet. The generation code is in dataset/generate_tolerance_label.py. You can simply generate tolerance label by running the script: (--dataset_root
and --num_workers
should be specified according to your settings)
cd dataset
sh command_generate_tolerance_label.sh
Or you can download the tolerance labels from Google Drive/Baidu Pan and run:
mv tolerance.tar dataset/
cd dataset
tar -xvf tolerance.tar
Train&Test
Train with physical constrained regularization
sh command_train.sh
Test
- We offer our checkpoints for inference and evaluation, you can download from Google Drive
sh command_test.sh
- For contact-score joint optimization, first download the pretrained contactnet & scorenet and unzip under the logs folder then run:
sh optimization.sh
note: In current version, only the optimization with gt mask is uploaded and we will update the 3d segmentation version in the future.
Evaluation
python evaluate.py
Citation
If any part of our paper and repository is helpful to your work, please generously cite with:
@InProceedings{Ma_2024_cvpr,
author = {Haoxiang, Ma and Modi, Shi and Boyang Gao and Di, Huang},
title = {Generalizing 6-DoF Grasp Detection via Domain Prior Knowledge},
booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2024}