Awesome

LidarStereoNet

This repository contains the code for our CVPR 2019 paper Noise-Aware Unsupervised Deep Lidar-Stereo Fusion [CVPR] [arxiv] [video]

The implementation of our model part is heavily borrowed from PSMNet and Sparsity Invariant CNNs.

Citing

If you find this code useful, please consider to cite our work.

@inproceedings{cheng2019cvpr,
title={Noise-Aware Unsupervised Deep Lidar-Stereo Fusion},
author={Cheng, Xuelian and Zhong, Yiran and Dai, Yuchao and Ji, Pan and Li, Hongdong},
booktitle={Proc. IEEE Conf. Comp. Vis. Patt. Recogn.},
year={2019}
}

Installation

Environment

Python 2.7.*
CUDA 10.0
PyTorch 1.0.0
TorchVision 0.4.0

Install

Create a virtual environment and activate it.

pip install -r requirements.txt

If the requirements.txt doesn't work, please try the following commands directly:

pip install torch==1.0.0 torchvision==0.4.0
pip install scikit-image path.py

Dataset

Training Dataset

KITTI VO dataset: contains 42104 images with a typical image resolution of 1241 × 376.

We sorted all 22 KITTI VO sequences and found 7 frames from sequence 17 and 20 having corresponding frames in the KITTI 2015 training set. Therefore we excluded these two sequences and used the remaining 20 stereo sequences as our training dataset.

Validation Dataset

We proveide the selected KITTI 141 dataset. It contains 141 frames from KITTI raw dataset that have corresponding frames in the KITTI 2015 dataset.

Inference

Download the model-pth provided in Best Model, and put them in ./checkpoint/

sh inference.sh

To evaluate our best results on the selected KITTI 141 dataset, please directly use this link Inference Images to download the inference disparity images and use the matlab code provided in inference_matlab to compute errors. You need to change the path of stereo_disp_name_gt and stereo_disp_name_est in Inference_kitti141.m.

run Inference_kitti141.m