Home

Awesome

SDPS-Net

SDPS-Net: Self-calibrating Deep Photometric Stereo Networks, CVPR 2019 (Oral). <br> Guanying Chen, Kai Han, Boxin Shi, Yasuyuki Matsushita, Kwan-Yee K. Wong <br>

This paper addresses the problem of learning based uncalibrated photometric stereo for non-Lambertian surface. <br>

<p align="center"> <img src='data/images/buddha.gif' height="250" > <img src='data/images/GT.png' height="250" > </p>

Changelog

Dependencies

SDPS-Net is implemented in PyTorch and tested with Ubuntu (14.04 and 16.04), please install PyTorch first following the official instruction.

You are highly recommended to use Anaconda and create a new environment to run this code.

# Create a new python3.7 environment named py3.7
conda create -n py3.7 python=3.7

# Activate the created environment
source activate py3.7

# Example commands for installing the dependencies 
conda install pytorch torchvision cudatoolkit=9.0 -c pytorch
conda install -c anaconda scipy 
conda install -c anaconda scikit-image 

# Download this code
git clone https://github.com/guanyingc/SDPS-Net.git
cd SDPS-Net

Overview

We provide:

Testing

Download the trained models

sh scripts/download_pretrained_models.sh

If the above command is not working, please manually download the trained models from BaiduYun (LCNet and NENet) and put them in ./data/models/.

Test SDPS-Net on the DiLiGenT main dataset

# Prepare the DiLiGenT main dataset
sh scripts/prepare_diligent_dataset.sh
# This command will first download and unzip the DiLiGenT dataset, and then centered crop 
# the original images based on the object mask with a margin size of 15 pixels.

# Test SDPS-Net on DiLiGenT main dataset using all of the 96 image
CUDA_VISIBLE_DEVICES=0 python eval/run_stage2.py --retrain data/models/LCNet_CVPR2019.pth.tar --retrain_s2 data/models/NENet_CVPR2019.pth.tar
# Please check the outputs in data/models/

# If you only have CPUs, please add the argument "--cuda" to disable the usage of GPU
python eval/run_stage2.py --cuda --retrain data/models/LCNet_CVPR2019.pth.tar --retrain_s2 data/models/NENet_CVPR2019.pth.tar

Test SDPS-Net on your own dataset

You have two options to test our method on your dataset. In the first option, you have to implement a customized Dataset class to load your data, which should not be difficult. Please refer to datasets/UPS_DiLiGenT_main.py for an example that loads the DiLiGenT main dataset.

If you don't want to implement your own Dataset class, you may try our datasets/UPS_Custom_Dataset.py. However, you have to first arrange your dataset in the same format as the data/ToyPSDataset/. Then you can call the following commands.

CUDA_VISIBLE_DEVICES=0 python eval/run_stage2.py --retrain data/models/LCNet_CVPR2019.pth.tar --retrain_s2 data/models/NENet_CVPR2019.pth.tar --benchmark UPS_Custom_Dataset --bm_dir /path/to/your/dataset

# To test SDPS-Net on the ToyPSDataset, simply run
CUDA_VISIBLE_DEVICES=0 python eval/run_stage2.py --retrain data/models/LCNet_CVPR2019.pth.tar --retrain_s2 data/models/NENet_CVPR2019.pth.tar --benchmark UPS_Custom_Dataset --bm_dir data/ToyPSDataset/
# Please check the outputs in data/models/

You may find input arguments in run_model_opts.py (particularly --have_l_dirs, --have_l_ints, and --have_gt_n) useful when testing your own dataset.

Training

We adopted the publicly available synthetic PS Blobby and Sculpture datasets for training. To train a new SDPS-Net model, please follow the following steps:

Download the training data

# The total size of the zipped synthetic datasets is 4.7+19=23.7 GB 
# and it takes some times to download and unzip the datasets.
sh scripts/download_synthetic_datasets.sh

If the above command is not working, please manually download the training datasets from BaiduYun (PS Sculpture Dataset and PS Blobby Dataset and put them in ./data/datasets/.

First stage: train Lighting Calibration Network (LCNet)

# Train LCNet on synthetic datasets using 32 input images
CUDA_VISIBLE_DEVICES=0 python main_stage1.py --in_img_num 32
# Please refer to options/base_opt.py and options/stage1_opt.py for more options

# You can find checkpoints and results in data/logdir/
# It takes about 20 hours to train LCNet on a single Titan X Pascal GPU.

Second stage: train Normal Estimation Network (NENet)

# Train NENet on synthetic datasets using 32 input images
CUDA_VISIBLE_DEVICES=0 python main_stage2.py --in_img_num 32 --retrain data/logdir/path/to/checkpointDirOfLCNet/checkpoint20.pth.tar
# Please refer to options/base_opt.py and options/stage2_opt.py for more options

# You can find checkpoints and results in data/logdir/
# It takes about 26 hours to train NENet on a single Titan X Pascal GPU.

FAQ

Q1: How to test SDPS-Net on other datasets?

Q2: What should I do if I have problem in running your code?

Q3: Could I run your code only using CPUs?

Citation

If you find this code or the provided models useful in your research, please consider cite:

@inproceedings{chen2019SDPS_Net,
  title={SDPS-Net: Self-calibrating Deep Photometric Stereo Networks},
  author={Chen, Guanying and Han, Kai and Shi, Boxin and Matsushita, Yasuyuki and Wong, Kwan-Yee~K.},
  booktitle={CVPR},
  year={2019}
}