Home

Awesome

Virtual Object Insertion

This repository contains the code to generate the object insertion results in paper Inverse rendering for complex indoor scenes: Shape, spatially-varying lighting and svbrdf from a single image, CVPR 2020. Please visit our project webpage for more information.

We hope that the code can help reproduce the photorealistic object insertion results in our paper and compare future lighting estimation methods more easily. Please consider citing the paper if you find the code useful. Please contact Zhengqin Li if you have any questions.

Results and Comparisons

The original models were trained by extending the SUNCG dataset with an SVBRDF-mapping. Since SUNCG is not available now due to copyright issues, we are not able to release the original models. Instead, we rebuilt a new high-quality synthetic indoor scene dataset and trained our models on it. We will release the new dataset in the near future. Some object insertion results on real images generated using our network trained on the new dataset are shown below. The quantitative and qualitative comparisons with prior works on object insertion are shown below. Compared with our networks trained on SUNCG-based dataset, our networks trained on new dataset can predict more consistent lighting color and intensity. Therefore, it achieves better performance in user study.

Prerequisites

To run our code, you will need

Data

Please download the data from the link. It includes the inverse rendering predictions of the 20 examples from Garon et al. 2019 dataset (Example 1-20) and 4 examples (demo0-3) downloaded from the Internet.

Instructions

To reproduce the 20 object insertion results from Garon et al. 2019 dataset.

The program will read the im_*.mat files in each Example* directory to generate the object insertion results.

We also provide the code to generate the object insertion code from scratch. The instructions are as follows:

Limitations

The current code only supports inserting an object on a planar surface. It has not been extended to handle objects in arbitrary 3D geometry yet.