Awesome
SGP: Self-supervised Geometric Perception
[CVPR 2021 Oral] Self-supervised Geometric Perception https://arxiv.org/abs/2103.03114
Introduction
In short, SGP is, to the best of our knowledge, the first general framework for feature learning in geometric perception without any supervision from ground-truth geometric labels.
SGP runs in an EM fashion. It iteratively performs robust estimation of the geometric models to generate pseudo-labels, and feature learning under the supervision of the noisy pseudo-labels.
<img src="assets/overview.png" alt="overview" width="600"/>We applied SGP to camera pose estimation and point cloud registration, demonstrating performance that is on par or even superior to supervised oracles in large-scale real datasets.
Camera pose estimation
Deep image features like CAPS can be trained with relative pose labels generated by 5pt-RANSAC, bootstraped with the handcrafted SIFT feature. They can be later used in robust relative camera pose estimation.
<div float="left"> <img src="assets/caps-megadepth.png" width="350" /> <img src="assets/caps-scannet.png" width="350" /> </div>Point cloud registration
Deep 3D features like FCGF can be trained with relative pose labels generated by 3pt-RANSAC, bootstraped by the handcrafted FPFH feature. They can be later used in robust point cloud registration.
<div float="left"> <img src="assets/fpfh-3dmatch.png" width="350" /> <img src="assets/fcgf-3dmatch.png" width="350" /> </div>Code
Please see code/ for detailed intructions about how to use the code base.
Citation
@inproceedings{yang2021sgp,
title={Self-supervised Geometric Perception},
author={Yang, Heng and Dong, Wei and Carlone, Luca and Koltun, Vladlen},
booktitle={CVPR},
year={2021}
}