Awesome

Installation

The code is tested with Python == 3.9, PyTorch == 1.13.0 and CUDA == 11.6 on NVIDIA GeForce RTX 3090. We recommend using anaconda to manage dependencies. You may need to change the torch and cuda version according to your computer. For troubleshooting the PyTorch3D install, see the install documentation.

conda create -n unsupmvs python=3.9
conda activate unsupmvs
conda install pytorch=1.13.0 torchvision pytorch-cuda=11.6 -c pytorch -c nvidia

# pytorch3d install (https://github.com/facebookresearch/pytorch3d/blob/main/INSTALL.md)
conda install -c fvcore -c iopath -c conda-forge fvcore iopath
conda install -c bottler nvidiacub
conda install pytorch3d -c pytorch3d

# remaining dependencies
pip install -r requirements.txt

Datasets

All provided scripts assume the dataset folders are placed in the home folder.

DTU

Training

Download the DTU dataset pre-processed by MVSNet and extract the archive. See MVSNet for file formats. Download the original resolution depth maps provided by YaoYao. Extract it and rename the folder to Depths_raw. Merge the folders together and you should get a dataset folder like below:

dtu
├── Cameras
├── Depths
├── Depths_raw
└── Rectified

Testing

Download the DTU testing dataset pre-processed by MVSNet and extract the archive. See MVSNet for file formats. We rename the dtu test set to dtu_eval so it can be placed in the same parent folder as the training set.

dtu_eval
├── scan1
├── scan4
├── scan9
...
├── scan114
└── scan118

Tanks and Temples (testing only)

Download the Tanks and Temples testing set pre-processed by UniMVSNet and unzip. We use the standard short range camera parameters for the intermediate subset and Temple (see this GitHub issue). The folder structure should look like this:

tanksandtemples
├── advanced
│   ├── Auditorium
│   │   ├── cams
│   │   ├── images
│   │   ├── Auditorium.log
│   │   └── pair.txt
│   ├── Ballroom
│   └── ...
└── intermediate
    ├── Family
    │   ├── cams
    │   ├── images
    │   ├── Family.log
    │   └── pair.txt
    ├── Francis
    └── ...

ScanNet++ (testing only)

Download the ScanNet++ dataset from the official website and place in your home folder. Run data_preprocess/preprocess_scannetpp.py to pre-process the ScanNet++ validation set for use with MVS. The pre-processed dataset should now be stored at $HOME/scannetpp_mvs. We also include our evaluation script, run_scannetpp_eval.py.

Re-producing Results

We include code for re-producing the DIV-MVS results. Scripts for re-producing all DIV-MVS reconstructions can be found in the scripts folder and are named appropriately. All scripts assume the datasets have been downloaded and placed in your home directory. If you wish to place them elsewhere, modify the --testpath argument in the script. The scripts folder also contains a training script for re-training DIV-MVS. These are set up for multi-GPU training. If changing the dataset location to somewhere other than the home folder, set the DATASET_DIR variable in the script.

The default depth-smoothness weight and clip value for DIV-MVS are 0.36 and 2.0 respectively, which is different from values reported in the paper. To reproduce the exact paper results, this is correct; however, DIV-MVS achieves comparable results with paper hyperparameters. There was a small discrepancy between DIV-MVS and DIV-RC/CL depth-smoothness implementations, resulting in the hyperparameters reported in the paper being off for DIV-MVS. For DIV-RC/CL, the reported hyperparameters of 0.18 and 4.0 are correct.

Acknowledgments

We made use of the following code-bases:

• RC-MVSNet
• MVSNet
• CasMVSNet
• KD-MVS
• UniMVSNet