Awesome

<h2 align="center">Let It Flow: Simultaneous Optimization of 3D Flow and Object Clustering</h3> <img src="docs/comparison_v3.png" align="center" width="95%"> <a href="https://github.com/vacany">Patrik Vacek</a>1    <a href="https://scholar.google.com/citations?user=XY1PVwYAAAAJ">David Hurych</a>2    <a href="https://sites.google.com/view/karelzimermann/">Karel Zimmermann</a>1     <a href="https://cmp.felk.cvut.cz/~svoboda/">Tomas Svoboda</a>1    1CTU Deparment of Cybernetics    2Valeo AI       <a href="https://arxiv.org/abs/2404.08363" target='_blank'> <img src="https://img.shields.io/badge/arXiv-2404.08363-b31b1b.svg">

:movie_camera: 1-Minute Summary
<img width="80%" src="docs/intro_page.png">
>>>>>>> Click Here <<<<<<<

:sparkles: Performance

:rocket: Scalability: Let It Flow does not require training or supervision as an optimization-based method and works on out-of-distribution data.
:rainbow: Real Scene Flow: Let It Flow performs consistently accross real world benchmark datasets and not only on StereoKITTI. Improvements are observable even on new dynamic/object class error metrics.
:balance_scale: Generalizability: Let It Flow does not have to adjust the parameter config for accurate results on the tested datasets.

Mechanism

Installation

Tested with Python==3.10, PyTorch3d, PyTorch Scatter See install.sh for installation of libraries or run it directly:

conda create -n let-it-flow python=3.10
conda activate let-it-flow

conda install pytorch=1.13.0 torchvision pytorch-cuda=11.6 -c pytorch -c nvidia
conda install -c fvcore -c iopath -c conda-forge fvcore iopath

git clone https://github.com/facebookresearch/pytorch3d.git
cd pytorch3d && python3 -m pip install -e .
cd ..

conda install pytorch-scatter -c pyg

Dataset

Download and unpack the Argoverse2 preprocessed sequence samples into the dataset/argoverse2/ path.

tar -xvf argoverse2.tgz datasets/argoverse2/

Set up the config.yaml to include path to the argoverse2 folder and path to store the results.

Inference

To infer the model on Argoverse2 dataset sequences, please run the following:

python run_optimization.py

The script will store the outputs (and inputs with Ground truth) into the store_path for metric calculation.

Generation of EPE metrics

After the model inference (takes about 2 hours on GPU), generate metrics of Let It Flow (lif) by running:

python compute_metrics.py lif

Result will print the End-Point-Error per class annotated in Argoverse2 and also overall ThreewayEPE.

Citation

If you find our work helpful, please consider citing our paper:

@ARTICLE{Vacek-IEEE-TIV2024,
  author={Vacek, Patrik and Hurych, David and Zimmermann, Karel and Svoboda, Tomáš},
  journal={IEEE Transactions on Intelligent Vehicles}, 
  title={Let-It-Flow: Simultaneous Optimization of 3D Flow and Object Clustering}, 
  year={2024},
  volume={},
  number={},
  pages={1-10},
  keywords={Springs;Rigidity;Point cloud compression;Estimation;Three-dimensional displays;Intelligent vehicles;Europe;Autonomous Driving;LiDAR;3D Scene Flow;Object Clustering},
  doi={10.1109/TIV.2024.3443316}}

License

<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/80x15.png" /></a> This work is under the <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.

Acknowledgement

This work is developed based on the open-source repositories and baselines: KISS-ICP, NSFP, MBNSFP, Chodosh

:heart: We thank to the contributors from the above open-source repositories.