Home

Awesome

Transformer-based Monocular Depth Estimation with Attention Supervision

Pretrained models

You can download pre-trained model

Evaluation

Evaluate on KITTI

python3 eval.py --model_dir "PATH/KITTI.pkl" --evaluate --batch_size 1 --dataset KITTI --data_path "PATH" --gpu_num 0 --encoder AST

Evaluate on NYU2

python3 eval.py --model_dir "PATH/NYU2.pkl" --evaluate --batch_size 1 --dataset NYU --data_path "PATH" --gpu_num 0 --encoder AST

DPT-Large trained by us

python3 eval.py --model_dir "MODEL_PATH" --evaluate --batch_size 1 --dataset "DATASET" --data_path "PATH" --gpu_num 0 --other_method DPT-Large

Adabins trained by us

python3 eval.py --model_dir "MODEL_PATH" --evaluate --batch_size 1 --dataset "DATASET" --data_path "PATH" --gpu_num 0 --other_method Adabins

Dataset Preparation

We referred to LapDepth in the data preparation process. Thanks for sharing the processed dataset.

KITTI

1. Official ground truth

    $ cd ./datasets
    $ mkdir KITTI && cd KITTI
    $ mv ~/Downloads/data_depth_annotated.zip ./datasets/KITTI
    $ unzip data_depth_annotated.zip

2. Raw dataset

    $ cd ./datasets/KITTI
    $ aria2c -x 16 -i ./datasets/kitti_archives_to_download.txt
    $ parallel unzip ::: *.zip

3. Dense g.t dataset
We take an inpainting method from DenseDepth to get dense g.t for gradient loss.
(You can train our model using only data loss without gradient loss, then you don't need dense g.t)
Corresponding inpainted results from './datasets/KITTI/data_depth_annotated/2011_xx_xx_drive_xxxx_sync/proj_depth/groundtruth/image_02' are should be saved in './datasets/KITTI/data_depth_annotated/2011_xx_xx_drive_xxxx_sync/dense_gt/image_02'.
KITTI data structures are should be organized as below:

|-- datasets
  |-- KITTI
     |-- data_depth_annotated  
        |-- 2011_xx_xx_drive_xxxx_sync
           |-- proj_depth  
              |-- groundtruth            # official G.T folder
        |-- ... (all drives of all days in the raw KITTI)  
     |-- 2011_09_26                      # raw RGB data folder  
        |-- 2011_09_26_drive_xxxx_sync
     |-- 2011_09_29
     |-- ... (all days in the raw KITTI)  

NYU Depth V2

1. Training set
Make NYU dataset directory

    $ cd ./datasets
    $ mkdir NYU_Depth_V2 && cd NYU_Depth_V2

2. Testing set
Download official nyu_depth_v2_labeled.mat and extract image files from the mat file.

    $ cd ./datasets
    ## Download official labled NYU_Depth_V2 mat file
    $ wget http://horatio.cs.nyu.edu/mit/silberman/nyu_depth_v2/nyu_depth_v2_labeled.mat
    ## Extract image files from the mat file
    $ python extract_official_train_test_set_from_mat.py nyu_depth_v2_labeled.mat splits.mat ./NYU_Depth_V2/official_splits/