Awesome
CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration (NeurIPS 2021)
PyTorch implementation of the paper:
CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration by:
Hao Yu, Fu Li, Mahdi Saleh, Benjamin Busam and Slobodan Ilic.
Introduction
We study the problem of extracting correspondences between a pair of point clouds for registration. For correspondence retrieval, existing works benefit from matching sparse keypoints detected from dense points but usually struggle to guarantee their repeatability. To address this issue, we present CoFiNet - Coarse-to-Fine Network which extracts hierarchical correspondences from coarse to fine without keypoint detection. On a coarse scale and guided by a weighting scheme, our model firstly learns to match down-sampled nodes whose vicinity points share more overlap, which significantly shrinks the search space of a consecutive stage. On a finer scale, node proposals are consecutively expanded to patches that consist of groups of points together with associated descriptors. Point correspondences are then refined from the overlap areas of corresponding patches, by a density-adaptive matching module capable to deal with varying point density. Extensive evaluation of CoFiNet on both indoor and outdoor standard benchmarks shows our superiority over existing methods. Especially on 3DLoMatch where point clouds share less overlap, CoFiNet significantly outperforms state-of-the-art approaches by at least 5% on Registration Recall, with at most two-third of their parameters.
News
-
09.03.2022: Add support for training and testing on KITTI.
-
28.10.2021: Paper available on arxiv.
-
27.10.2021: Release training and testing code of 3DMatch and 3DLoMatch.
Installation
-
Clone the repository:
git clone https://github.com/haoyu94/Coarse-to-fine-correspondences.git cd Coarse-to-fine-correspondences
-
Create conda environment and install requirements:
conda create -n {environment name} python=3.8 pip install -r requirements.txt
-
Compile C++ and CUDA scripts:
cd cpp_wrappers sh compile_wrappers.sh cd ..
Demo
TBD
3DMatch & 3DLoMatch
Pretrained model
Pretrained model is given in weights/
.
Prepare datasets
sh scripts/download_data.sh
Train
sh scripts/train_3dmatch.sh
Test
- Point correspondences are first extracted by running:
sh scripts/test_3dmatch.sh
and stored on snapshot/tdmatch_enc_dec_test/3DMatch/
.
-
To evaluate on 3DLoMatch, please change the
benchmark
keyword inconfigs/tdmatch/tdmatch_test.yaml
from3DMatch
to3DLoMatch
. -
The evaluation of extracted correspondences and relative poses estimated by RANSAC can be done by running:
sh scripts/run_ransac.sh
-
The final results are stored in
est_traj/3DMatch/{number of correspondences}/result
and the results evaluated on our computer have been provided inest_traj/
. -
To evaluate on 3DLoMatch, please change
3DMatch
inscripts/run_ransac.sh
to3DLoMatch
.
KITTI
Pretrained model
Pretrained model is given in weights/
.
Prepare datasets
Please follow PREDATOR for data preperation.
There should be two folders poses
and sequences
under ./data/kitti/dataset/
.
Train
sh scripts/train_kitti.sh
Test
sh scripts/test_kitti.sh
Acknowledgments
-
The code is heavily borrowed from PREDATOR.
-
Our backbone network is from KPConv.
-
We use the Transformer implementation in SuperGlue.
We thank the authors for their excellent work!
Citiation
If you find this repository helpful, please cite:
@article{yu2021cofinet,
title={CoFiNet: Reliable Coarse-to-fine Correspondences for Robust PointCloud Registration},
author={Yu, Hao and Li, Fu and Saleh, Mahdi and Busam, Benjamin and Ilic, Slobodan},
journal={Advances in Neural Information Processing Systems},
volume={34},
year={2021}
}