Awesome
Deep Generative Prior (DGP)
Paper
Xingang Pan, Xiaohang Zhan, Bo Dai, Dahua Lin, Chen Change Loy, Ping Luo, "Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation", ECCV2020 (Oral)
Video: https://youtu.be/p7ToqtwfVko<br>
Demos
DGP exploits the image prior of an off-the-shelf GAN for various image restoration and manipulation.
Image restoration:
<p align="center"> <img src="data/restoration.gif", width="900"> </p>Image manipulation:
<p align="center"> <img src="data/manipulation.gif", width="800"> </p>A learned prior helps internal learning:
<p align="center"> <img src="data/windows.png", width="900"> </p>Requirements
-
python>=3.6
-
pytorch>=1.0.1
-
others
pip install -r requirements.txt
Get Started
Before start, please download the pretrained BigGAN at Google drive or Baidu cloud (password: uqtw), and put them to pretrained
folder.
Example1: run image colorization example:
sh experiments/examples/run_colorization.sh
The results will be saved in experiments/examples/images
and experiments/examples/image_sheet
.
Example2: process images with an image list:
sh experiments/examples/run_inpainting_list.sh
Example3: evaluate on 1k ImageNet validation images via distributed training based on slurm:
# need to specifiy the root path of imagenet validate set in --root_dir
sh experiments/imagenet1k_128/colorization/train_slurm.sh
Note:
- BigGAN needs a class condition as input. If no class condition is provided, it would be chosen from a set of random samples.
- The hyperparameters provided may not be optimal, feel free to tune them.
Acknowledgement
The code of BigGAN is borrowed from https://github.com/ajbrock/BigGAN-PyTorch.
Citation
@inproceedings{pan2020dgp,
author = {Pan, Xingang and Zhan, Xiaohang and Dai, Bo and Lin, Dahua and Loy, Chen Change and Luo, Ping},
title = {Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation},
booktitle = {European Conference on Computer Vision (ECCV)},
year = {2020}
}
@ARTICLE{pan2020dgp_pami,
author={Pan, Xingang and Zhan, Xiaohang and Dai, Bo and Lin, Dahua and Loy, Chen Change and Luo, Ping},
journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
title={Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation},
year={2021},
volume={},
number={},
pages={1-1},
doi={10.1109/TPAMI.2021.3115428}
}