Awesome
StructDepth
PyTorch implementation of our ICCV2021 paper:
StructDepth: Leveraging the structural regularities for self-supervised indoor depth estimation
Boying Li*, Yuan Huang*, Zeyu Liu, Danping Zou, Wenxian Yu
(* Equal Contribution) Please consider citing our paper in your publications if the project helps your research.
@inproceedings{structdepth,
title={StructDepth: Leveraging the structural regularities for self-supervised indoor depth estimation},
author={Li, Boying and Huang, Yuan and Liu, Zeyu and Zou, Danping and Yu, Wenxian},
booktitle={Proceedings of the IEEE International Conference on Computer Vision},
year={2021}
}
Getting Started
Installation
The Python and PyTorch versions we use:
python=3.6
pytorch=1.7.1=py3.6_cuda10.1.243_cudnn7.6.3_0
Step1: Creating a virtual environment
conda create -n struct_depth python=3.6
conda activate struct_depth
conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=10.1 -c pytorch
Step2: Download the modified scikit_image package , in which the input parameters of the Felzenswalb algorithm have been changed to accommodate our method.
unzip scikit-image-0.17.2.zip
cd scikit-image-0.17.2
python setup.py build_ext -i
pip install -e .
Step3: Installing other packages
pip install -r requirements.txt
Download pretrained model
Please download pretrained models and unzip them to MODEL_PATH
Inference single image
python inference_single_image.py --image_path=/path/to/image --load_weights_folder=MODEL_PATH
Evaluation
Download test dataset
Please download test dataset
It is recommended to unpack all test data and training data into the same data path and then modify the DATA_PATH when running a training or evaluation script.
Evaluate NYUv2/InteriorNet/ScanNet depth or norm
Modify the evaluation script in eval.sh to evaluate NYUv2/InteriorNet/ScanNet depth and norm separately
python evaluation/nyuv2_eval_norm.py \
--data_path DATA_PATH \
--load_weights_folder MODEL_PATH \
Trainning
Download NYU V2 dataset
The raw NYU dataset is about 400G and has 590 videos. You can download the raw datasets from there
Extract Main directions
python extract_vps_nyu.py --data_path DATA_PATH --output_dir VPS_PATH --failed_list TMP_LIST -- thresh 60
If you need to train with a random flip, run the main direction extraction script on the images before and after the flip(add --flip) in advance, and note the failure examples, which can be skipped by referring to the code in datasets/nyu_datases.py.
Training
Modify the training script train.sh for PATH or different trainning settings.
python train.py \
--data_path DATA_PATH \
--val_path DATA_PATH \
--train_split ./splits/nyu_train_0_10_20_30_40_21483-exceptfailed-21465.txt \
--vps_path VPS_PATH \
--log_dir LOG_PATH \
--model_name 1 \
--batch_size 32 \
--num_epochs 50 \
--start_epoch 0 \
--using_disp2seg \
--using_normloss \
--load_weights_folder PRETRAIN_MODEL_PATH \
--lambda_planar_reg 0.1 \
--lambda_norm_reg 0.05 \
--planar_thresh 200 \
Acknowledgement
We borrowed a lot of codes from scikit-image, monodepth2, P<sup>2</sup>Net, and LEGO. Thanks for their excellent works!