Awesome
GatedConvolution_pytorch
A modified reimplemented in pytorch of inpainting model in Free-Form Image Inpainting with Gated Convolution [http://jiahuiyu.com/deepfill2/] This repo is transfered from the https://github.com/avalonstrel/GatedConvolution and https://github.com/JiahuiYu/generative_inpainting.
It is a model for image inpainting task. I implement the network structure and gated convolution in Free-Form Image Inpainting with Gated Convolution, but a little difference about the original structure described in Free-Form Image Inpainting with Gated Convolution.
- In refine network, I do not employ the contextual attention but a self-attention layer instead.
- I add batch norm to each layer.
Some results
BenchMark data and Mask data can be found in Google Drive
How to test images by pre-trained model?
I provide a pre-trained Baidu, Google model on Places2 256x256 dataset, (but unfortunately only the coarse network can be loaded since I change the network structure after the pre-train process, in fact the coarse network also work).
Run bash scripts/test_inpaint.sh
You should provide a file containing file paths you want to test following the form of
test1.png
test2.png
... ...
Change the parameters in config/test_places2_sagan.yml About the image
places2:
[
'flist_file_for_train',
'flist_file_for_test'
]
About the mask
val:
[
'mask_flist_file_for_train',
'mask_flist_file_for_test'
]
The mask file should be a pkl file containing a numpy.array.
The MODEL_RESTORE should be set to the path of the pre-trained model. After successfully running, you can find the results in result_logs/MODEL_RESTORE
How to train your own model?
To train your own model with some other dataset you can
Run bash scripts/run_inpaint_sa.sh
By providing the
places2:
[
'flist_file_for_train',
'flist_file_for_test'
]
About the mask
val:
[
'mask_flist_file_for_train',
'mask_flist_file_for_test'
]
And in training you can use random free-form mask or random rectangular mask. I use random free-form mask. If you want use random rectangular mask you need to change the process in train_sagan.py(line 163) and set MASK_TYPES: ['random_bbox'].
Some detials about the training parameters is easy to understand as shown in config file.
Tensorboard
Run tensorboard --logdir model_logs --port 6006
to view training progress.
Some tips about mask generation?
We provide two random mask generation function.
-
random free form masks
The parameters about this function are
RANDOM_FF_SETTING:
img_shape: [256,256] mv: 5 ma: 4.0 ml: 40 mbw: 10
Following the meaning in http://jiahuiyu.com/deepfill2/.
-
random rectangular masks
RANDOM_BBOX_SHAPE: [32, 32]
RANDOM_BBOX_MARGIN: [64, 64]
means the shape of the random bbox and the margin between the boarder. (The number of rectangulars can be set in inpaint_dataset.py random_bbox_number=5)
LICENSE
CC 4.0 Attribution-NonCommercial International
The software is for educational and academic research purposes only.
Acknowledgments
My project acknowledge the official code DeepFillv1 and SNGAN. Especially, thanks for the authors of this amazing algorithm.