Home

Awesome

NConv-CNN on NYU-Depth-v2

The repo provides an implementation to train/test our method "Confidence Propagation through CNNs for Guided Sparse Depth Regression" on the "NYU-Depth-v2 dataset"

This repo is forked from the PyTorch implementation for "Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image" by Fangchang Ma and Sertac Karaman.

We provide training for both networks Enc-Dec-Net[EF] and MS-Net[LF] on the RGB-D input of the dataset. as they were described in the paper.

Contents

  1. Requirements
  2. Training
  3. Testing
  4. Citation

Requirements

This code was tested with Python 3 and PyTorch 1.0.

Training

The training scripts come with several options, which can be listed with the --help flag.

python main.py --help

For instance, run the following command to train the network Enc-Dec-Net[EF], and both RGB and 100 random sparse depth samples as the input to the network.

python main.py -a guided_enc_dec -m rgbd -s 100 --data nyudepthv2 --optimizer adam --lr 0.001 --lr-decay 10

Training results will be saved under the results folder. To resume a previous training, run

python main.py --resume [path_to_previous_model]

Testing

To test the performance of a trained model without training, simply run main.py with the -e option. For instance,

python main.py --evaluate [path_to_trained_model]

Citation

If you use the code or method in your work, please consider citing the original authors of the code:

@article{Ma2017SparseToDense,
	title={Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image},
	author={Ma, Fangchang and Karaman, Sertac},
	booktitle={ICRA},
	year={2018}
}
@article{ma2018self,
	title={Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera},
	author={Ma, Fangchang and Cavalheiro, Guilherme Venturelli and Karaman, Sertac},
	journal={arXiv preprint arXiv:1807.00275},
	year={2018}
}

And our paper:

@article{eldesokey2018confidence,
  title={Confidence Propagation through CNNs for Guided Sparse Depth Regression},
  author={Eldesokey, Abdelrahman and Felsberg, Michael and Khan, Fahad Shahbaz},
  journal={arXiv preprint arXiv:1811.01791},
  year={2018}
}