Awesome
Monocular-Depth-Estimation-Toolbox
Introduction
Monocular-Depth-Estimation-Toolbox is an open source monocular depth estimation toolbox based on PyTorch and MMSegmentation v0.16.0.
It aims to benchmark MonoDepth methods and provides effective supports for evaluating and visualizing results.
<div align=center><img width="550" height="150" src="resources/teaser.gif"/></div>Major features
-
Unified benchmark
Provide a unified benchmark toolbox for various depth estimation methods.
-
Modular design
Depth estimation frameworks are decomposed into different components. One can easily construct a customized framework by combining different modules.
-
Support of multiple methods out of box
I would like to reproduce some of the most excellent depth estimation methods based on this toolbox.
-
High efficiency
It seems that there are few depth estimation benchmarks, so I start this project and hope it is helpful for research.
Thanks to MMSeg, we own these major features. :blush:
Benchmark and model zoo
Results and models are available in the model zoo.
Supported backbones (partially release):
- ResNet (CVPR'2016)
- EfficientNet (ICML'2019)
- Vision Transformer (ICLR'2021)
- Swin Transformer (ICCV'2021)
- I recommend cross-package import in config, so that you can utilize other backbone in MMcls, MMseg, etc. Refer to introduction. I will add more backbones in the future.
Supported methods:
- BTS (Arxiv'2019)
- Adabins (CVPR'2021)
- SimIPU (AAAI'2022)
- DPT (ICCV'2021) (TBD)
- NeWCRFs (CVPR'2022) (TBD)
- DepthFormer (Arxiv'2022)
- BinsFormer (Arxiv'2022) (Early release. It may contain bugs or potential performance drop)
Supported datasets:
Installation
Please refer to get_started.md for installation and dataset_prepare.md for dataset preparation.
Get Started
We provide train.md and inference.md for the usage of this toolbox.
<!-- In the future, there will be tutorials for [customizing dataset (TODO)](docs/tutorials/customize_datasets.md), [designing data pipeline (TODO)](docs/tutorials/data_pipeline.md), [customizing modules (TODO)](docs/tutorials/customize_models.md), and [customizing runtime (TODO)](docs/tutorials/customize_runtime.md). We also provide [training tricks (TODO)](docs/tutorials/training_tricks.md). -->License
This project is released under the Apache 2.0 license.
Acknowledgement
This repo benefits from awesome works of mmsegmentation, Adabins, BTS. Please also consider citing them.
Cite
@misc{lidepthtoolbox2022,
title={Monocular Depth Estimation Toolbox},
author={Zhenyu Li},
howpublished = {\url{https://github.com/zhyever/Monocular-Depth-Estimation-Toolbox}},
year={2022}
}
Changelog
- Nov. 29, 2022: Release the config for BinsFormer on KITTI. Release configs for cross-dataset inference (nyu2sunrgbd). Fix NYUv2 download bugs as discussed in Issue #66 and #36. I add several codes for NeWCRFs but have no plan to conduct detailed experiments for it due to the current limited resource and time.
- Jul. 23, 2022: Release codes for BinsFormer. Potential bugs or performance drops can exist, waiting for completion.
- Jun. 5, 2022: Add support for custom dataset training. Add a warmup interface for sigloss to help convergence as discussed in Issue #20. Enhance the DPT support and fix bugs in provided pre-trained models as reported in Issue #23.
- Apr. 16, 2022: Finish most of docs and provide all pre-trained parameters. Release codes about BTS, Adabins, DPT, SimIPU, and DepthFormer. Support KITTI, NYU-v2, SUN RGB-D(eval), and CityScapes.
TODO
- I will complete the release of BinsFormer models on KITTI dataset (currently limited by the GPU resource).
- I would like to include self-supervised depth estimation methods, such as MonoDepth2.