Awesome
GANILLA
We provide PyTorch implementation for:
GANILLA: Generative Adversarial Networks for Image to Illustration Translation.
Updates
- (October, 2023) We released code for our recent work on animation to illustration video translation WAIT: Feature Warping for Animation to Illustration video Translation using GANs.
- (February, 2021) We released code for our recent work on sketch colorization Adversarial Segmentation Loss for Sketch Colorization.
Dataset Stats:
Sample Images:
GANILLA:
GANILLA results on the illustration dataset:
Comparison with other methods:
Style transfer using Miyazaki's anime images:
Ablation Experiments:
Prerequisites
- Linux, macOS or Windows
- Python 2 or 3
- CPU or NVIDIA GPU + CUDA CuDNN
Getting Started
Downloading Datasets
Please refer to datasets.md for details.
Installation
- Clone this repo:
git clone https://github.com/giddyyupp/ganilla.git
cd ganilla
- Install PyTorch 0.4+ and torchvision from http://pytorch.org and other dependencies (e.g., visdom and dominate). You can install all the dependencies by
pip install -r requirements.txt
- For Conda users, we include a script
./scripts/conda_deps.sh
to install PyTorch and other libraries.
GANILLA train/test
- Download a GANILLA/CycleGAN dataset (e.g. maps):
bash ./datasets/download_cyclegan_dataset.sh maps
- Train a model:
#!./scripts/train_ganilla.sh
python train.py --dataroot ./datasets/maps --name maps_cyclegan --model cycle_gan --netG resnet_fpn
- To view training results and loss plots, run
python -m visdom.server
and click the URL http://localhost:8097. To see more intermediate results, check out./checkpoints/maps_cyclegan/web/index.html
- Test the model:
#!./scripts/test_cyclegan.sh
python test.py --dataroot ./datasets/maps --name maps_cyclegan --model cycle_gan --netG resnet_fpn
The test results will be saved to a html file here: ./results/maps_cyclegan/latest_test/index.html
.
You can find more scripts at scripts
directory.
Apply a pre-trained model (GANILLA)
- You can download pretrained models using following link
Put a pretrained model under ./checkpoints/{name}_pretrained/100_net_G.pth
.
- To test the model, you also need to download the monet2photo dataset and use trainB images as source:
bash ./datasets/download_cyclegan_dataset.sh monet2photo
- Then generate the results using
python test.py --dataroot datasets/monet2photo/testB --name {name}_pretrained --model test
The option --model test
is used for generating results of GANILLA only for one side. python test.py --model cycle_gan
will require loading and generating results in both directions, which is sometimes unnecessary. The results will be saved at ./results/
. Use --results_dir {directory_path_to_save_result}
to specify the results directory.
- If you would like to apply a pre-trained model to a collection of input images (rather than image pairs), please use
--dataset_mode single
and--model test
options. Here is a script to apply a model to Facade label maps (stored in the directoryfacades/testB
).
#!./scripts/test_single.sh
python test.py --dataroot ./datasets/monet2photo/testB/ --name {your_trained_model_name} --model test
You might want to specify --netG
to match the generator architecture of the trained model.
Style & Content CNN
We shared style & content CNNs in this repo. It contains train/test procedure as well as pretrained weights for both cnns.
Training/Test Tips
Best practice for training and testing your models.
Frequently Asked Questions
Before you post a new question, please first look at the above Q & A and existing GitHub issues.
Citation
If you use this code for your research, please cite our papers.
@article{hicsonmez2020ganilla,
title={GANILLA: Generative adversarial networks for image to illustration translation},
author={Hicsonmez, Samet and Samet, Nermin and Akbas, Emre and Duygulu, Pinar},
journal={Image and Vision Computing},
pages={103886},
year={2020},
publisher={Elsevier}
}
@inproceedings{Hicsonmez:2017:DDN:3078971.3078982,
author = {Hicsonmez, Samet and Samet, Nermin and Sener, Fadime and Duygulu, Pinar},
title = {DRAW: Deep Networks for Recognizing Styles of Artists Who Illustrate Children's Books},
booktitle = {Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval},
year = {2017}
}
Acknowledgments
Our code is heavily inspired by CycleGAN.
The numerical calculations reported in this work were fully performed at TUBITAK ULAKBIM, High Performance and Grid Computing Center (TRUBA resources).