Awesome

FastMAC: Stochastic Spectral Sampling of Correspondence Graph (CVPR 2024)

Source code of FastMAC: Stochastic Spectral Sampling of Correspondence Graph

Introduction

3D correspondence, i.e., a pair of 3D points, is a fundamental concept in computer vision. A set of 3D correspondences, when equipped with compatibility edges, forms a correspondence graph. This graph is a critical component in several state-of-the-art 3D point cloud registration approaches, e.g., the one based on maximal cliques (MAC). However, its properties have not been well understood. So we present the first study that introduces graph signal processing into the domain of correspondence graph. We exploit the generalized degree signal on correspondence graph and pursue sampling strategies that preserve high-frequency components of this signal. To address time-consuming singular value decomposition in deterministic sampling, we resort to a stochastic approximate sampling strategy. As such, the core of our method is the stochastic spectral sampling of correspondence graph. As an application, we build a complete 3D registration algorithm termed as FastMAC, that reaches real-time speed while leading to little to none performance drop. Through extensive experiments, we validate that FastMAC works for both indoor and outdoor benchmarks. For example, FastMAC can accelerate MAC by 80 times while maintaining high registration success rate on KITTI.

News

• [2024/2/27] Paper is accepted by CVPR 2024.
• [2023/12/4] Code is released.

Installation

Please install PyTorch first, and then install other dependencies by the following command. Code has been tested with Python 3.8.10, PyTorch 1.12.0, CUDA 11.3 and cuDNN 8302 on Ubuntu 22.04.

pip install -r requirements.txt

Finally, install MAC(3D Registration with Maximal Cliques) as instructed.

NOTE: As our PCR method is based on MAC, please install and run MAC first. Or If you only want the output sampled correspondences, then it's OK to only install our code.

Datasets

The test datasets include KITTI, 3DMatch, 3DLoMatch. Please download them from MAC(3D Registration with Maximal Cliques).

Usage

To demonstrate the reliability of our method's boosting performance for MAC, we use the original MAC as the registration module. Therefore, to run the complete pipeline, use the code we present here to downsample the input correspondences and then feed them into MAC, using the code in MAC repository. In the future we would integrate the two parts into one codebase to form a complete pipeline for practical usage.

KITTI

To run FastMAC on KITTI, please use the following command:

python sota.py

In function Config(), modify "data_dir", "filename", "gtname", "labelname" and "outpath" as the actual path you set. "ratio" refers to the downsampling ratio from 0 to 1.

NOTE: set 'thresh' to 0.999 if using FCGF descriptor, 0.9 if using FPFH descriptor.

3DMatch

To run FastMAC on 3DMatch, please use the following command:

python 3dmatch.py

In function Config(), modify "data_dir", "descriptor" as the actual path you set. The output path will be the original dataset direction for convenience to apply MAC. "ratio" refers to the downsampling ratio from 0 to 1. Set "name" to "3dmatch" to run on 3DMatch.

3DLoMatch

To run FastMAC on 3DMatch, please use the following command:

python 3dmatch.py

In function Config(), modify "data_dir", "descriptor" as the actual path you set. The output path will be the original dataset direction for convenience to apply MAC. "ratio" refers to the downsampling ratio from 0 to 1. Set "name" to "3dlomatch" to run on 3DLoMatch.

Results

KITTI

3DMatch

3DLoMatch