Awesome

StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation

📝 This repository contains the official PyTorch implementation of the following paper:

StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation<br> Wonjong Jang, Gwangjin Ju, Yucheol Jung, Jiaolong Yang, Xin Tong, Seungyong Lee, SIGGRAPH 2021

🚀 >> Project page
🚀 >> Fast-forward video

Overview

method

</div> <details> <summary><b>Explanation</b></summary> <div markdown="1"> The key component of our method is <b>shape exaggeration blocks</b> that are used for modulating coarse layer feature maps of StyleGAN to produce desirable caricature shape exaggerations. We first build a layer-mixed StyleGAN for photo-to-caricature style conversion by swapping fine layers of the StyleGAN for photos to the corresponding layers of the StyleGAN trained to generate caricatures. Given an input photo, the layer-mixed model produces detailed color stylization for a caricature but without shape exaggerations. We then append shape exaggeration blocks to the coarse layers of the layer-mixed model and train the blocks to create shape exaggerations while preserving the characteristic appearances of the input. </div> </details>

Requirements

✔️ PyTorch 1.3.1
✔️ torchvision 0.4.2
✔️ CUDA 10.1/10.2
✔️ dlib 19.22.0
✔️ requests 2.23.0
✔️ tqdm 4.46.2

⚠️ If you are using Anaconda environment and get errors regarding compiler version mismatch, check issue #1.

Usage

First download pre-trained model weights:

bash ./download.sh

Train

python -m torch.distributed.launch --nproc_per_node=N_GPU train.py --name EXPERIMENT_NAME --freeze_D

Test

Test on user's input images:

python test.py --ckpt CHECKPOINT_PATH --input_dir INPUT_IMAGE_PATH --output_dir OUTPUT_CARICATURE_PATH --invert_images

We provide some sample images. Test on sample images:

python test.py --ckpt CHECKPOINT_PATH --input_dir examples/samples --output_dir examples/results --invert_images

It inverts latent codes from input photos and generates caricatures from latent codes.

Examples

Input image	Output caricature

</div>

Citation

If you find this code useful, please consider citing:

@article{Jang2021StyleCari,
  author    = {Wonjong Jang and Gwangjin Ju and Yucheol Jung and Jiaolong Yang and Xin Tong and Seungyong Lee},
  title     = {StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation},
  booktitle = {ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH)},
  publisher = {ACM},
  volume = {40},
  number = {4},
  year = {2021}
}

Download pre-trained models

🏷️ StyleCariGAN
🏷️ Photo-StyleGAN (generator_ffhq.pt)
🏷️ Caricature-StyleGAN (generator_cari.pt)
🏷️ Photo-Attribute-Classifier (photo_resnet.pth)
🏷️ Cari-Attribute-Classifier (cari_resnet.pth)

Contact

📫 You can have contact with wonjong@postech.ac.kr or ycjung@postech.ac.kr

License

This software is being made available under the terms in the LICENSE file.

Any exemptions to these terms require a license from the Pohang University of Science and Technology.

Credits

❤️ Our code is based on the official StyleGAN2 implementation and rosinality's StyleGAN2-pytorch code
❤️ Specially thanks to CJWBW who ported our project to Replicate.