Awesome
LGCNet-Tensorflow
Tensorflow implementation of a local-global framework for confidence estimation.
Beyond local reasoning for stereo confidence estimation with deep learning
Fabio Tosi, Matteo Poggi, Antonio Benincasa and Stefano Mattoccia
ECCV 2018
Qualitative results on KITTI
Example of confidence estimation. (a) Reference image from KITTI 2015 dataset, (b) disparity map obtained with MC-CNN, (c) confidence estimated with a local approach (CCNN) and (d) the proposed local-global framework, highlighting regions on which the latter method provides more reliable predictions (red bounding boxes).
For more details: pdf
Training
For training, the KITTI input images have been padded to 384x1280.
The training file should be a .txt file in which each line contains: [path_image_left];[path_image_disparity];[path_image_groundtruth]. Disparity maps and groundtruth as 16 bit images.
You can train the local network as follows:
python ./model/main.py --is_training True --epoch 14 --batch_size 128 --patch_size 9 --dataset [path_training_file] --initial_learning_rate 0.003 --log_directory [path_log] --model_name model --model [CCNN, EFN, LFN]
Use the --model argument to choose the architecture:
- CCNN (Poggi et al.)
- EFN (Early Fusion Network)
- LFN (Late Fusion Network)
Similarly, you can train the global network (ConfNet):
python ./model/main.py --is_training True --epoch 1600 --batch_size 1 --crop_height 256 --crop_width 512 --dataset [path_training_file] --initial_learning_rate 0.003 --log_directory [path_log] --model_name model --model ConfNet
Finally, you can load weights from the local and global networks and train LGCNet thereafter:
python ./model/main.py --is_training True --epoch 14 --batch_size 128 --patch_size 9 --dataset [path_training_file] --initial_learning_rate 0.003 --log_directory [path_log] --model_name model --model LGC --checkpoint_path [path_checkpoint_ConfNet] [path_checkpoint_CCNN/LFN] --late_fusion
Use --late_fusion flag to set LFN as local network. Otherwise, disable it to use CCNN.
Warning: set checkpoint_CCNN/LFN accordingly.
Testing
The testing file should be a .txt file in which each line contains: [path_image_left];[path_image_disparity]. Disparity maps as 16 bit images.
If you want to test the local network or the global network indipendently:
python ./model/main.py --is_training False --batch_size 1 --dataset [path_testing_file] --checkpoint_path [path_checkpoint] --output_path [path_output] --model [CCNN, EFN, LFN, ConfNet]
For testing LGCNet, instead, you can run:
python ./model/main.py --is_training False --batch_size 1 --dataset [path_testing_file] --checkpoint_path [path_checkpoint_ConfNet] [path_checkpoint_CCNN/LFN] [path_checkpoint_LGC] --output_path [path_output] --model LGC --late_fusion
Pretrained models
You can download the pre-trained models, trained on 20 images of KITTI 12 dataset for AD-CENSUS, SMG and MC-CNN stereo algorithms, here:
Requirements
This code was tested with Tensorflow 1.4, CUDA 8.0 and Ubuntu 16.04.