Awesome

SEDNet

This is the Python implementation of the SEDNet with GwcNet backbone. (Learning the Distribution of Errors in Stereo Matching for Joint Disparity and Uncertainty Estimation. Published at CVPR 2023) [Paper]

Environment

• python 3.7
• Pytorch >= 0.10.2
• Cuda >= 11.0
• Anaconda
• Create environment by conda env create -f sednet.yml or conda create --name myenv --file sednet.txt

Data Preparation

Download datasets at:

• Scene Flow Datasets
• KITTI 2015
• Virtual KITTI 2
• DrivingStereo

Training

Training List

• Training/Test split used in the paper.
• You can also use files in generate_datas to generate your own split.
• Please save the split files to ./filenames.

Training Scripts

• main.py is used to training the SEDNet.
• Training scripts are saved in ./scripts
• For --losstype, smooth_l1 is the smooth L1 loss in Guo et al., KG is the log-likelihood loss in Kendall and Gal., UC is our novel divergence loss with the log-likelihood loss.
• To train the LAF baseline, you need to run ./generate_datas/generate_laf_data.py to save the cost volumn of stereo network at first.

Example of Scene Flow Datasets

• run the scripts ./scripts/sceneflow.sh to traing on Scene Flow datasets
• Please update DATAPATH and SAVEPATH as your train data path and the log/checkpoints save path.
• You can use --loadckpt to specific the pre-trained checkpoint file.

Evaluation

• Files in post_process are used to evaluate the models.
• generate_statistic.py is to compute the evaluation metrics.
• generate_conf_and_depth.py can covert the disparity maps and uncertainty maps to depth maps and the confidence maps via gaussian error function.
• Run ./scripts/sceneflow_analysis.sh to generate the evaluation metric of models trained with Scene Flow datasets.

Save Outputs

• Run ./scripts/kitti15_save.sh to save the disparity maps of the model is fine-tunned on KIITI 2015 dataset. Please update the --loadckpt as your checkpoint file to generate the disparity maps.

Pretrained Models

• SceneFlow: SEDNet with a soft inlier threshold of 3 sigma and 11 bins in logspace.
• VKITT2: SEDNet with a soft inlier threshold of 3 sigma and 11 bins in logspace.

Citation

@inproceedings{chen2023learning,
  title={Learning the Distribution of Errors in Stereo Matching for Joint Disparity and Uncertainty Estimation},
  author={Chen, Liyan and Wang, Weihan and Mordohai, Philippos},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={17235--17244},
  year={2023}
}