Awesome
droid_metric
This repo is for project combind DROID-SLAM and Metric3D, taking metric depth to improve the performance of DROID-SLAM in monocular mode.
Installation
# clone the repo with '--recursive' to get the submodules
# or run 'git submodule update --init --recursive' after cloning
cd droid_metric
# create conda env
conda create -n droid_metric python=3.9
conda activate droid_metric
# install requirements (other torch/cuda versions may also work)
pip install -r requirements.txt
# install droid-slam-backend
cd modules/droid_slam
python setup.py install
cd ../..
If you want to install specific version of pytorch
and cuda
, check this link.
If you want to install mmcv
under specific cuda version, check this link.
Usage
1. Setup pretrained models
Download DROID-SLAM and Metric3D pretrained model running
python download_models.py
(Optional) Download ADVIO dataset running
python download_dataset.py
2. Utils
For camera calibration, check scripts/calib.py
For video sampling, check scripts/sample.py
3. Run reconstruction
python reconstruct.py --input ${RGB-images dir or video file} --output ${ouptut dir} --intr ${intrinsic file} --viz
# for more options, check `reconstruct.py`
# if you do not provide intrinsic, it will be estimated as:
# - fx = fy = max{image_width, image_height} * 1.2 (follow COLMAP)
# - cx = image_width / 2
# - cy = image_height / 2
3*. Run reconstruction stepwise
## depth estimate
python depth.py --images ${RGB-images dir} --out ${ouptut dir} --intr ${intrinsic file}
# for more options, check `depth.py`
## droid-slam
python slam.py --images ${RGB-images dir} --depth ${depth data dir} --intr ${intrinsic file} --out-poses ${output poses dir} --viz
# for more options, check `slam.py`. You should run depth estimation first.
## mesh recon
python mesh.py --images ${RGB-images dir} --depth ${depth data dir} --poses ${poses dir} --intr ${intrinsic file} --save ${output mesh path}
# for more options, check `mesh.py`. You should run droid-slam first.
!Note
The format of intrinsic file should be as follows (4 elements only):
# intrinsic.txt
${fx}
${fy}
${cx}
${cy}
Experiment
Tested on part of ICL-NUIM and ADVIO dataset. droid_D
refers to DROID-SLAM with Metric3D, droid
refers to the oroginal DROID-SLAM and vslam
refers to the OpenVSLAM framework. Notice that vslam method get lost on ICL-OfficeRoom-1 and all sequences of ADVIO.
1. Trajectory
(some of the trajectories seem not aligned correctly, sorry for that.)