Home

Awesome

DVLO

The official codes for ECCV 2024 Oral paper: 'DVLO: Deep Visual-LiDAR Odometry with Local-to-Global Feature Fusion and Bi-Directional Structure Alignment'

Jiuming Liu, Dong Zhuo, Zhiheng Feng, Siting Zhu, Chensheng Peng, Zhe Liu, and Hesheng Wang

šŸ“£ News

Pipeline

<img src="fuser.png">

Installation

Our model only depends on the following commonly used packages.

PackageVersion
CUDA1.11.3
Python3.8.10
PyTorch1.12.0
h5pynot specified
tqdmnot specified
numpynot specified
openpyxlnot specified

Device: NVIDIA RTX 3090

Install the pointnet2 library

Compile the furthest point sampling, grouping and gathering operation for PyTorch with following commands.

cd pointnet2
python setup.py install

Install the CUDA-based KNN searching and random searching

We leverage CUDA-based operator for parallel neighbor searching [Reference: [EfficientLONet] (https://github.com/IRMVLab/EfficientLO-Net)]. You can compile them with following commands.

cd ops_pytorch
cd fused_conv_random_k
python setup.py install
cd ../
cd fused_conv_select_k
python setup.py install
cd ../

Datasets

KITTI Odometry

Datasets are available at KITTI Odometry benchmark website: https://drive.google.com/drive/folders/1Su0hCuGFo1AGrNb_VMNnlF7qeQwKjfhZ The data of the KITTI odometry dataset should be organized as follows:

data_root
ā”œā”€ā”€ 00
ā”‚   ā”œā”€ā”€ velodyne
ā”‚   ā”œā”€ā”€ calib.txt
ā”œā”€ā”€ 01
ā”œā”€ā”€ ...

Training

Train the network by running :

python train.py 

Please reminder to specify the GPU, data_root,log_dir, train_list(sequences for training), val_list(sequences for validation). You may specify the value of arguments. Please find the available arguments in the configs.py.

Testing

Our network is evaluated every 2 epoph during training. If you only want the evaluation results, you can set the parameter 'eval_before' as 'True' in file config.py, then evaluate the network by running :

python train.py

Please reminder to specify the GPU, data_root,log_dir, test_list(sequences for testing) in the scripts. You can also get the pretrined wieghts in the pretrain_weights file.

Clustering Visualization

<img src="visual.png">

Citation

@article{liu2024dvlo,
  title={DVLO: Deep Visual-LiDAR Odometry with Local-to-Global Feature Fusion and Bi-Directional Structure Alignment},
  author={Liu, Jiuming and Zhuo, Dong and Feng, Zhiheng and Zhu, Siting and Peng, Chensheng and Liu, Zhe and Wang, Hesheng},
  journal={arXiv preprint arXiv:2403.18274},
  year={2024}
}

Acknowledgments

We thank the following open-source project for the help of the implementations: