Awesome

ICP-Flow: LiDAR Scene Flow Estimation with ICP

Official implementation: ICP-Flow: LiDAR Scene Flow Estimation with ICP (CVPR 2024)

Yancong Lin and Holger Caesar.

Delft University of Technology, The Netherlands.

Winner of the Argoverse 2 Scene Flow challenge (unsupervised track, CVPR 2024).

Introduction

Scene flow characterizes the 3D motion between two LiDAR scans captured by an autonomous vehicle at nearby timesteps. Prevalent methods consider scene flow as point-wise unconstrained flow vectors that can be learned by either large-scale training beforehand or time-consuming optimization at inference. However, these methods do not take into account that objects in autonomous driving often move rigidly. We incorporate this rigid-motion assumption into our design, where the goal is to associate objects over scans and then estimate the locally rigid transformations. We propose ICP-Flow, a learning-free flow estimator. The core of our design is the conventional Iterative Closest Point (ICP) algorithm, which aligns the objects over time and outputs the corresponding rigid transformations. Crucially, to aid ICP, we propose a histogram-based initialization that discovers the most likely translation, thus providing a good starting point for ICP. The complete scene flow is then recovered from the rigid transformations. We outperform state-of-the-art baselines, including supervised models, on the Waymo dataset and perform competitively on Argoverse-v2 and nuScenes. Further, we train a feedforward neural network, supervised by the pseudo labels from our model, and achieve top performance among all models capable of real-time inference. We validate the advantage of our model on scene flow estimation with longer temporal gaps, up to 0.4 seconds where other models fail to deliver meaningful results.

Main feature

Simple as it is: no need for annotation; no need for big data; no need for deep learning;

Reproduction

Bug fix:

A bug fixed on 01-05-2024; tested locally.

Installation

1. Conda env setup.

conda create -f environment.yml
conda activate icp_flow

2. Install Patchwork++ for ground segmentation.

There are several modifications on top of the original Patchwork++ to output point indices.

# To install Eigen
$ sudo apt-get install libeigen3-dev

# To install Patchwork++
$ cd patchwork-plusplus
$ mkdir build && cd build
$ cmake ..
$ make

Note: This version requires a GPU. The CPU-only version has not been released yet.

Dataset

1. Waymo and nuScenes: see Dynamic 3D Scene Analysis by Point Cloud Accumulation for details.

wget --no-check-certificate --show-progress https://share.phys.ethz.ch/~gsg/PCAccumulation/data.zip
unzip data.zip

2. Argoverse-v2: see ZeroFlow: Scalable Scene Flow via Distillation for details.

   For simplicity, I took a different approach to load data: run ZeroFlow on the val/test set, and then save the points/labels in a .npz file per sample. See dataset_argo.py for details.

   Alternatively, you may also load pretrained checkpoints (see below) to ZeroFlow and test on Argoverse-v2 using the ZeroFlow codebase.

Test ICP-Flow

Configure all variables beforehand within the *.sh files. You may find all results at SURFdrive.

bash main.sh

ICP-Flow + FNN

ICP-Flow + FNN has an identical design to ZeroFlow. You may download pre-trained weights from SURFdrive.

Demo

bash demo.sh

Argoverse-v2 Scene Flow Challenge

Please refer to the leaderboard.

Acknowledge

Many thanks to the authors of ZeroFlow and Point Cloud Accumulation.

Citation

@article{lin2024icp,
  title={ICP-Flow: LiDAR Scene Flow Estimation with ICP},
  author={Lin, Yancong and Caesar, Holger},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2024}
}