Home

Awesome

<div id="top" align="center">

Lite-Mono

A Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimation [paper link]

Ning Zhang*, Francesco Nex, George Vosselman, Norman Kerle

<a href="#license"> <img alt="License: MIT" src="https://img.shields.io/badge/license-MIT-blue.svg"/> </a> <img src="./img/teaser_m.gif" width="100%" alt="teaser" align=center />

(Lite-Mono-8m 1024x320)

</div>

Table of Contents

Overview

<img src="./img/overview.png" width="100%" alt="overview" align=center />

Results

KITTI

You can download the trained models using the links below.

--modelParamsImageNet PretrainedInput sizeAbs RelSq RelRMSERMSE logdelta < 1.25delta < 1.25^2delta < 1.25^3
lite-mono3.1Myes640x1920.1070.7654.5610.1830.8860.9630.983
lite-mono-small2.5Myes640x1920.1100.8024.6710.1860.8790.9610.982
lite-mono-tiny2.2Myes640x1920.1100.8374.7100.1870.8800.9600.982
lite-mono-8m8.7Myes640x1920.1010.7294.4540.1780.8970.9650.983
lite-mono3.1Myes1024x3200.1020.7464.4440.1790.8960.9650.983
lite-mono-small2.5Myes1024x3200.1030.7574.4490.1800.8940.9640.983
lite-mono-tiny2.2Myes1024x3200.1040.7644.4870.1800.8920.9640.983
lite-mono-8m8.7Myes1024x3200.0970.7104.3090.1740.9050.9670.984

Speed Evaluation

<img src="./img/speed.png" width="100%" alt="speed evaluation" align=center />

Robustness

<img src="./img/robustness.png" width="100%" alt="robustness" align=center />

The RoboDepth Challenge Team is evaluating the robustness of different depth estimation algorithms. Lite-Mono has achieved the best robustness to date.

Data Preparation

Please refer to Monodepth2 to prepare your KITTI data.

Single Image Test

preparing trained model

From this table you can download trained models (depth encoder and depth decoder).

Click on the links in the '--model' column to download a trained model.

start testing

python test_simple.py --load_weights_folder path/to/your/weights/folder --image_path path/to/your/test/image

Evaluation

python evaluate_depth.py --load_weights_folder path/to/your/weights/folder --data_path path/to/kitti_data/ --model lite-mono

Training

dependency installation

pip install 'git+https://github.com/saadnaeem-dev/pytorch-linear-warmup-cosine-annealing-warm-restarts-weight-decay'

preparing pre-trained weights

From this table you can also download weights of backbone (depth encoder) pre-trained on ImageNet.

Click 'yes' on a row to download specific pre-trained weights. The weights are agnostic to image resolutions.

start training

python train.py --data_path path/to/your/data --model_name mytrain --num_epochs 30 --batch_size 12 --mypretrain path/to/your/pretrained/weights  --lr 0.0001 5e-6 31 0.0001 1e-5 31

tensorboard visualization

tensorboard --log_dir ./tmp/mytrain

Make Your Own Pre-training Weights On ImageNet

Since a lot of people are interested in training their own backbone on ImageNet, I also upload my pre-training scripts to this repo.

Citation

@InProceedings{Zhang_2023_CVPR,
author    = {Zhang, Ning and Nex, Francesco and Vosselman, George and Kerle, Norman},
title     = {Lite-Mono: A Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimation},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month     = {June},
year      = {2023},
pages     = {18537-18546}
}