Awesome
CVPR2020-Polarized-Reflection-Removal-with-Perfect-Alignment
Code for CVPR2020 paper "Polarized Reflection Removal with Perfect Alignment in the Wild"
paper | project website | Other datasets
<img src="example/IMG_0058_input.jpg" height="140px"/> <img src="example/IMG_0058_ours.jpg" height="140px"/>
Fast Demo (TL;DR)
git clone https://github.com/ChenyangLEI/polarization-reflection-removal
cd polarization-reflection-removal
conda env create -f environment.yml
conda activate PolarRR
bash download.sh
mkdir result
mv Submission_ckpt result
python test.py --task Submission_ckpt --test_dir demo
The results are placed in ./test_result
PolarRR Dataset
Please download the dataset on OneDrive, there are 807 pairs of polarization images. In each pair of images, the first one is the mixed image, the second one is the reflection images.
bash download_data.sh
Setup
Environment
This code is based on tensorflow. It has been tested on Ubuntu 18.04 LTS.
Anaconda is recommended: Ubuntu 18.04 | Ubuntu 16.04
After installing Anaconda, you can setup the environment simply by
conda env create -f environment.yml
Download checkpoint and VGG model
Download the ckpt and VGG model by
bash download.sh
Inference
python test.py --task Submission_ckpt --test_dir demo
The results are placed in ./test_result
Key designs
1. Polarization information
To be able to relax the assumptions about the appearance of reflection, we leverage polarization that inherently exists in almost all reflected light.
2. Perfect-alignment real-world data
We first identify the misalignment issues of existing reflection removal datasets where the collected reflection-free images are not perfectly aligned with input mixed images due to glass refraction. To address this issue, we design a new data collection pipeline called M-R, which helps us collect diverse real-world data with perfect alignment by utilizing glass in the real world.
3. Two-stage framework for reflection removal
We propose a two-stage framework for reflection removal from polarized images. Our approach firstly estimates reflection, with which it infers the transmission image secondly. This framework helps us improve 2dB PSNR on our real-world dataset.
Citation
If you find this work useful for your research, please cite:
@InProceedings{Lei_2020_CVPR,
author = {Lei, Chenyang and Huang, Xuhua and Zhang, Mengdi and Yan, Qiong and Sun, Wenxiu and Chen, Qifeng},
title = {Polarized Reflection Removal With Perfect Alignment in the Wild},
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2020}
}
Contact
Please contact me if there is any question (Chenyang Lei, leichenyang7@gmail.com)