Awesome
Enforcing geometric constraints of virtual normal for depth prediction.
[NEW] Training codes have been uploaded!
This repository contains the source code of our paper: Yin Wei, Yifan Liu, Chunhua Shen, Youliang Yan, Enforcing geometric constraints of virtual normal for depth prediction (accepted for publication in ICCV' 2019).
Some Results
Framework
Hightlights
- State-of-the-art performance: The comparisons with other SOTA methods on NYU and KITTI are illustrated as follow. The published model trained on NYU can achieve 10.5% on absrel.
Installation
- Please refer to Installation.
Datasets
-
NYUDV2 The details of datasets can be found here. The Eigen split of labeled images can be downloaded here. Please extract it to ./datasets. Our SOTA model is trained on the around 20K unlabled images.
-
KITTI The details of KITTI benchmark for monocular depth prediction is here. We use both the official split and Eigen split. You can find the filenames here.
Model Zoo
-
ResNext101_32x4d backbone, trained on NYU dataset, download here
-
ResNext101_32x4d backbone, trained on KITTI dataset (Eigen split), download here
-
ResNext101_32x4d backbone, trained on KITTI dataset (Official split), download here
-
ImageNet pretrained weight, download here.
Inference
# Run the inferece on NYUDV2 dataset
python ./tools/test_nyu_metric.py \
--dataroot ./datasets/NYUDV2 \
--dataset nyudv2 \
--cfg_file lib/configs/resnext101_32x4d_nyudv2_class \
--load_ckpt ./nyu_rawdata.pth
# Test depth predictions on any images, please replace the data dir in test_any_images.py
python ./tools/test_any_images.py \
--dataroot ./ \
--dataset any \
--cfg_file lib/configs/resnext101_32x4d_nyudv2_class \
--load_ckpt ./nyu_rawdata.pth
If you want to test the kitti dataset, please see here
Training
# Run the inferece on NYUDV2 dataset
python ./tools/train_nyu_metric.py \
--dataroot ./datasets/NYUDV2 \
--dataset nyudv2 \
--cfg_file lib/configs/resnext101_32x4d_nyudv2_class
#--load_ckpt *.pth
#--resume
Citation
@inproceedings{Yin2019enforcing,
title={Enforcing geometric constraints of virtual normal for depth prediction},
author={Yin, Wei and Liu, Yifan and Shen, Chunhua and Yan, Youliang},
booktitle= {The IEEE International Conference on Computer Vision (ICCV)},
year={2019}
}
@article{yin2021virtual,
title={Virtual Normal: Enforcing Geometric Constraints for Accurate and Robust Depth Prediction},
author={Yin, Wei and Liu, Yifan and Shen, Chunhua},
journal={IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
year={2021}
}
Other works :)
We have done a new work to aim to solve the generalization problem of monocular depth prediction. Please refer to following link. :)
- Paper: DiverseDepth: Affine-invariant Depth Prediction Using Diverse Data
- Code: https://github.com/YvanYin/DiverseDepth
Contact
Wei Yin: wei.yin@adelaide.edu.au