Awesome
BUFFER: Balancing Accuracy, Efficiency, and Generalizability in Point Cloud Registration (CVPR 2023)
This is the official repository of BUFFER, a point cloud registration method for balancing accuracy, efficiency, and generalizability. For technical details, please refer to:
BUFFER: Balancing Accuracy, Efficiency, and Generalizability in Point Cloud Registration <br /> Sheng Ao, Qingyong Hu, Hanyun Wang, Kai Xu, Yulan Guo. <br />
[Paper] [Video] [Project page] <br />
(1) Overview
(2) Setup
This code has been tested with Python 3.8, Pytorch 1.9.1, CUDA 11.1 on Ubuntu 20.04.
- Clone the repository
git clone https://github.com/aosheng1996/BUFFER && cd BUFFER
- Setup conda virtual environment
conda create -n buffer python=3.8
source activate buffer
pip install torch==1.9.1+cu111 torchvision==0.10.1+cu111 torchaudio==0.9.1 -f https://download.pytorch.org/whl/torch_stable.html
pip install open3d==0.13.0
export CUDA_HOME=/your/cuda/home/directory/
pip install "git+git://github.com/erikwijmans/Pointnet2_PyTorch.git#egg=pointnet2_ops&subdirectory=pointnet2_ops_lib"
pip install --upgrade https://github.com/unlimblue/KNN_CUDA/releases/download/0.2/KNN_CUDA-0.2-py3-none-any.whl
pip install ninja kornia einops easydict tensorboard tensorboardX
pip install nibabel -i http://pypi.douban.com/simple --trusted-host pypi.douban.com
cd cpp_wrappers && sh compile_wrappers.sh && cd ..
git clone https://github.com/KinglittleQ/torch-batch-svd.git && cd torch-batch-svd && python setup.py install && cd .. && sudo rm -rf torch-batch-svd/
(3) 3DMatch
Following Predator, we provide the processed 3DMatch training set (subsampled fragments with voxel size of 1.5cm and their ground truth transformation matrices).
Download the processed dataset from Google Drive and put the folder into data
.
Then the structure should be as follows:
data
ThreeDMatch
train
7-scenes-chess
- ...
3DMatch_train_overlap.pkl
train_3dmatch.txt
val_3dmatch.txt
test
3DLoMatch
3DMatch
Training
Training BUFFER on the 3DMatch dataset:
cd ./ThreeDMatch
python train.py
Testing
Evaluate the performance of the trained models on the 3DMatch dataset:
cd ./ThreeDMatch
python test.py
To evaluate the performance of BUFFER on the 3DLoMatch dataset, you only need to modify the _C.data.dataset = '3DMatch'
in config.py
to _C.data.dataset = '3DLoMatch'
and performs:
python test.py
(4) KITTI
Download the data from the KITTI Odometry website into data
.
Then the structure is as follows:
data
KITTI
dataset
pose
00.txt
- ...
sequences
00
- ...
Training
Training BUFFER on the KITTI dataset:
cd ./KITTI
python train.py
Testing
Evaluate the performance of the trained models on the KITTI dataset:
cd ./KITTI
python test.py
(5) ETH
The test set can be downloaded from here, and put the folder into data
, then the structure is as follows:
data
ETH
gazebo_summer
gazebo_winter
wood_autmn
wood_summer
(6) Generalizing to Unseen Datasets
3DMatch to ETH
Generalization from 3DMatch dataset to ETH dataset:
cd ./generalization/ThreeD2ETH
python test.py
3DMatch to KITTI
Generalization from 3DMatch dataset to KITTI dataset:
cd ./generalization/ThreeD2KITTI
python test.py
KITTI to 3DMatch
Generalization from KITTI dataset to 3DMatch dataset:
cd ./generalization/KITTI2ThreeD
python test.py
KITTI to ETH
Generalization from KITTI dataset to ETH dataset:
cd ./generalization/KITTI2ETH
python test.py
Acknowledgement
In this project, we use (parts of) the implementations of the following works:
Citation
If you find our work useful in your research, please consider citing:
@inproceedings{ao2023buffer,
title={BUFFER: Balancing Accuracy, Efficiency, and Generalizability in Point Cloud Registration},
author={Ao, Sheng and Hu, Qingyong and Wang, Hanyun and Xu, Kai and Guo, Yulan},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={1255--1264},
year={2023}
}
Updates
- 07/06/2023: The code is released!
- 28/02/2023: This paper has been accepted by CVPR 2023!
Related Repos
- RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds
- SoTA-Point-Cloud: Deep Learning for 3D Point Clouds: A Survey
- 3D-BoNet: Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds
- SensatUrban: Learning Semantics from Urban-Scale Photogrammetric Point Clouds
- SpinNet: Learning a General Surface Descriptor for 3D Point Cloud Registration