Awesome
CTSDG
<img src='assets/teaser.png'/>Paper | Pre-trained Models | BibTex
Image Inpainting via Conditional Texture and Structure Dual Generation<br>
Xiefan Guo, Hongyu Yang, Di Huang<br> In ICCV'2021
Introduction
Generator. Image inpainting is cast into two subtasks, i.e., structure-constrained texture synthesis (left, blue) and texture-guided structure reconstruction (right, red), and the two parallel-coupled streams borrow encoded deep features from each other. The Bi-GFF module and CFA module are stacked at the end of the generator to further refine the results.
Discriminator. The texture branch estimates the generated texture, while the structure branch guides structure reconstruction.
<img src='assets/framework.png'/>Prerequisites
- Python >= 3.6
- PyTorch >= 1.0
- NVIDIA GPU + CUDA cuDNN
Getting Started
Installation
- Clone this repo:
git clone https://github.com/Xiefan-Guo/CTSDG.git
cd CTSDG
- Install PyTorch and dependencies from http://pytorch.org
- Install python requirements:
pip install -r requirements.txt
Datasets
Image Dataset. We evaluate the proposed method on the CelebA, Paris StreetView, and Places2 datasets, which are widely adopted in the literature.
Mask Dataset. Irregular masks are obtained from Irregular Masks and classified based on their hole sizes relative to the entire image with an increment of 10%.
Training
Analogous to PConv by Liu et.al, initial training followed by finetuning are performed.
python train.py \
--image_root [path to image directory] \
--mask_root [path mask directory]
python train.py \
--image_root [path to image directory] \
--mask_root [path to mask directory] \
--pre_trained [path to checkpoints] \
--finetune True
Distributed training support. You can train model in distributed settings.
python -m torch.distributed.launch --nproc_per_node=N_GPU train.py
Testing
To test the model, you run the following code.
python test.py \
--pre_trained [path to checkpoints] \
--image_root [path to image directory] \
--mask_root [path to mask directory] \
--result_root [path to output directory] \
--number_eval [number of images to test]
Citation
If any part of our paper and repository is helpful to your work, please generously cite with:
@InProceedings{Guo_2021_ICCV,
author = {Guo, Xiefan and Yang, Hongyu and Huang, Di},
title = {Image Inpainting via Conditional Texture and Structure Dual Generation},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2021},
pages = {14134-14143}
}