Awesome

Deep Global Registration

Introduction

This repository contains python scripts for training and testing Deep Global Registration, CVPR 2020 Oral. Deep Global Registration (DGR) proposes a differentiable framework for pairwise registration of real-world 3D scans. DGR consists of the following three modules:

• a 6-dimensional convolutional network for correspondence confidence prediction
• a differentiable Weighted Procrustes algorithm for closed-form pose estimation
• a robust gradient-based SE(3) optimizer for pose refinement.

For more details, please check out

• CVPR 2020 oral paper
• 1min oral video
• Full CVPR oral presentation

Quick Pipleine Visualization

Indoor 3DMatch Registration	Outdoor KITTI Lidar Registration

Related Works

Recent end-to-end frameworks combine feature learning and pose optimization. PointNetLK combines PointNet global features with an iterative pose optimization method. Wang et al. in Deep Closest Point train graph neural network features by backpropagating through pose optimization. We further advance this line of work. In particular, our Weighted Procrustes method reduces the complexity of optimization from quadratic to linear and enables the use of dense correspondences for highly accurate registration of real-world scans.

Deep Global Registration

The first component is a 6-dimensional convolutional network that analyzes the geometry of 3D correspondences and estimates their accuracy. Please refer to High-dim ConvNets, CVPR'20 for more details.

The second component we develop is a differentiable Weighted Procrustes solver. The Procrustes method provides a closed-form solution for rigid registration in SE(3). A differentiable version of the Procrustes method used for end-to-end registration passes gradients through coordinates, which requires O(N^2) time and memory for N keypoints. Instead, the Weighted Procrustes method passes gradients through the weights associated with correspondences rather than correspondence coordinates. The computational complexity of the Weighted Procrustes method is linear to the number of correspondences, allowing the registration pipeline to use dense correspondence sets rather than sparse keypoints. This substantially increases registration accuracy.

Our third component is a robust optimization module that fine-tunes the alignment produced by the Weighted Procrustes solver and the failure detection module. This optimization module minimizes a differentiable loss via gradient descent on the continuous SE(3) representation space. The optimization is fast since it does not require neighbor search in the inner loop such as ICP.

Configuration

Our network is built on the MinkowskiEngine and the system requirements are:

• Ubuntu 14.04 or higher
• <b>CUDA 10.1.243 or higher</b>
• pytorch 1.5 or higher
• python 3.6 or higher
• GCC 7

You can install the MinkowskiEngine and the python requirements on your system with:

# Install MinkowskiEngine
sudo apt install libopenblas-dev g++-7
pip install torch
export CXX=g++-7; pip install -U MinkowskiEngine --install-option="--blas=openblas" -v

# Download and setup DeepGlobalRegistration
git clone https://github.com/chrischoy/DeepGlobalRegistration.git
cd DeepGlobalRegistration
pip install -r requirements.txt

Demo

You may register your own data with relevant pretrained DGR models. 3DMatch is suitable for indoor RGB-D scans; KITTI is for outdoor LiDAR scans.

python demo.py

Input PointClouds	Output Prediction

Experiments

Comparison	Speed vs. Recall Pareto Frontier

Training

The entire network depends on pretrained FCGF models. Please download corresponding models before training.

3DMatch

You may download preprocessed data and train via these commands:

./scripts/download_3dmatch.sh /path/to/3dmatch
export THREED_MATCH_DIR=/path/to/3dmatch; FCGF_WEIGHTS=/path/to/fcgf_3dmatch.pth ./scripts/train_3dmatch.sh

KITTI

Follow the instruction on KITTI Odometry website to download the KITTI odometry train set. Then train with

export KITTI_PATH=/path/to/kitti; FCGF_WEIGHTS=/path/to/fcgf_kitti.pth ./scripts/train_kitti.sh

Testing

3DMatch test set is different from train set and is available at the download section of the official website. You may download and decompress these scenes to a new folder.

To evaluate trained model on 3DMatch or KITTI, you may use

python -m scripts.test_3dmatch --threed_match_dir /path/to/3dmatch_test/ --weights /path/to/dgr_3dmatch.pth

and

python -m scripts.test_kitti --kitti_dir /path/to/kitti/ --weights /path/to/dgr_kitti.pth

Generate figures

We also provide experimental results of 3DMatch comparisons in results.npz. To reproduce figures we presented in the paper, you may use

python scripts/analyze_stats.py assets/results.npz

Citing our work

Please cite the following papers if you use our code:

@inproceedings{choy2020deep,
  title={Deep Global Registration},
  author={Choy, Christopher and Dong, Wei and Koltun, Vladlen},
  booktitle={CVPR},
  year={2020}
}

@inproceedings{choy2019fully,
  title = {Fully Convolutional Geometric Features},
  author = {Choy, Christopher and Park, Jaesik and Koltun, Vladlen},
  booktitle = {ICCV},
  year = {2019}
}

@inproceedings{choy20194d,
  title={4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks},
  author={Choy, Christopher and Gwak, JunYoung and Savarese, Silvio},
  booktitle={CVPR},
  year={2019}
}

Concurrent Works

There have a number of 3D registration works published concurrently.

• Gojcic et al., Learning Multiview 3D Point Cloud Registration, CVPR'20
• Wang et al., PRNet: Self-Supervised Learning for Partial-to-Partial Registration, NeurIPS'19
• Yang et al., TEASER: Fast and Certifiable Point Cloud Registration, arXiv'20