Awesome

PointNetLK Revisited

Xueqian Li, Jhony Kaesemodel Pontes, Simon Lucey

Will appear on 2021 Conference on Computer Vision and Pattern Recognition (CVPR) (oral)

arXiv link: https://arxiv.org/pdf/2008.09527.pdf

Prerequisites

This code is based on PyTorch implementation, and tested on 1.0.0<=torch<=1.6.0. You may go to the PyTorch official site (https://pytorch.org/) to decide which torch/torchvision version is suitable for your system. You may also need to go to the tensorflow website (https://www.tensorflow.org/install) to download tensorboard. Other packages can be installed through,

pip install -r requirements.txt

NOTE: we use open3d interactive visualization window in the jupyter notebook, which is only available for open3d>=0.13.0. You may need to wait for a few seconds for the interactive visualization to show.

Demo Notebook

We provide a jupyter notebook (you may need to install the jupyter notebook) of toy example in the demo folder. You can run ./demo/test_toysample.ipynb to play with a point cloud pairs from 3DMatch dataset.

Here is a short video showing how to run the demo.

Dataset

You may download dataset used in the paper from these websites.

We used 3DMatch dataset prepared by authors of Deep Global Registration, you may download using this script provided by them, https://github.com/chrischoy/DeepGlobalRegistration/blob/master/scripts/download_3dmatch.sh.

After you download the dataset, you can create a symbolic link in the ./dataset folder as ./dataset/ModelNet and ./dataset/ThreeDMatch.

Notation on voxelization

In the current setting, the voxelization for overlapped source and target point clouds still relies on "registered point clouds" (such as results from a global registration). The code for voxelization after "transformation" is updated by setting argument --voxel_after_transf to True. In the real-world application, our algorithm would be better suited for local registration problem.

Training

python train.py

Evaluation

python test.py

Pre-Trained Models

You can find the pre-trained model in logs/model_trained_on_ModelNet40_model_best.pth. This model is trained on ModelNet40 dataset, and is used to produce most results (except results of trained/tested on 3DMatch dataset) in the paper.

Acknowledgement

This code is mostly adapted from the original PointNetLK, https://github.com/hmgoforth/PointNetLK.

The 3DMatch data loader code was adapted from the Deep Global Registration, https://github.com/chrischoy/DeepGlobalRegistration.

Part of the data voxelization code was adapted from the SECOND, https://github.com/traveller59/second.pytorch.

Part of the metric computaion code was adapted from the Deep Closest Point, https://github.com/WangYueFt/dcp.

Contributing

If you find the project useful for your research, you may cite,

@InProceedings{Li_2021_CVPR,
    author    = {Li, Xueqian and Pontes, Jhony Kaesemodel and Lucey, Simon},
    title     = {PointNetLK Revisited},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2021},
    pages     = {12763-12772}
}