Home

Awesome

CNN-PS

Satoshi Ikeahta. CNN-PS: CNN-based Photometric Stereo for General Non-Convex Surfaces, ECCV2018.

paper

supplementary

Getting Started

This is a Keras implementation of a CNN for estimating surface normals from images captured under different illumination.

Prerequisites

Tested on:

Running the tests

For testing network (with DiLiGenT dataset), please download DiLiGenT dataset (DiLiGenT.zip) by Boxin Shi [1] and extract it anywhere. Then, specify the path of DiLiGenT/pmsData in test.py as

diligent = 'USER_PATH/DiLiGenT/pmsData'

The pretrained model for TensorFlow backend is included (weight_and_model.hdf5). You can simply try to run test.py as

python test.py

If the program properly works, you will get average angular errors (in degrees) for each dataset.

<img src="webimage/img000.png" width="300">

The final result [Mean] is the error about the averaged surface normal over normals predicted from K (K=10 in this case) differently rotated observation maps (See details in my paper). Finally, you will get the predicted surface normal map and the error map.

<img src="webimage/img001.png" width="600">

Important notice about DiLiGenT datasets

As mentioned in the paper, I found that DiLiGenT dataset has some problems.

# [Sv, Nv, Rv, IDv, Szv] = dio.prep_data_2d_from_images_test(dirlist, 1, w, 10, index = range(20, 96)) # for bearPNG
# nml = np.flipud(nml) # Uncomment when test on Harvest, the surface noraml needs to be flipped upside down

Running the test on othter datasets (Unsupported)

If you want to run this code on ohter dataset, please allocate your own data just in the same manner with DiLiGenT. The required files are

Running the training

If you want the training dataset, please send a request to sikehata@nii.ac.jp

License

This project is licensed under the MIT License - see the LICENSE.md file for details

Acknowledgments

This work was supported by JSPS KAKENHI Grant Number JP17H07324.

References

[1] Boxin Shi, Zhipeng Mo, Zhe Wu, Dinglong Duan, Sai-Kit Yeung, and Ping Tan, "A Benchmark Dataset and Evaluation for Non-Lambertian and Uncalibrated Photometric Stereo", In IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2018.