Awesome
Naver Webtoon Faces
Face cartoonization code is added. See here.
This repo heavily uses codes from [this repo]
Dataset [link]
256*256 cartoon face images collected from on-going NAVER Webtoon series.
num titles 58
total images: 17662
StyleGAN2
Training detail: batch 12, transfer from FFHQ, non-leaking augmentation
FID (10k samples)
| Iteration | FID |
|---|---|
| FFHQ | 256.54 |
| 50k | 11.73 |
| 100k | 9.29 |
| 150k | 8.87 |
| 200k | 8.11 |
| 250k | 7.41 |
Samples (FID 8.87)
4-Way Linear interpolation in w-space
Swapping codes at different layers
Swapping Autoencoder for Deep Image Manipulation
[paper] [model]
SwapAE is a fully unsupervised generative model that embeds images into structure and style codes (similar to MUNIT). In SwapAE, the style encoder is forced to extract the global texture of the image by explicitly matching the patch statistics of the original image and swap-generated image (patch co-occurrence loss).
Training detail: batch 10, iteration 500k, single co-occurrence patch per sample
Samples
Style code interpolation
Structure code interpolation
Simply interpolating the structure code didn't work well. The regional structure code editing method introduced in the original paper might work.
Swapping codes at different layers
Injecting target style codes from the very first layer of the decoder often changes the whole identity of the original character. Detailed structures can be better preserved by applying the source style in the first few layers.
swap location = 2
swap location = 3
For the webcomics data domain, injecting target style codes from the 2nd~3rd layer gives pleasing style-transfer results.
swap location = 7
The last few layers of the generator control the overall color.
Test image
From Slam Dunk
Face2Webtoon
Finetuning SwapAE
Finetuning the webtoon-trained SwapAE model with real human faces for small amount of steps improves the robustness. Using the StyleGAN as a style image generator, exploration of styles without real webtoon data is also possible. See train_aux.py and face2webtoon.ipynb. An unconditional lightweight model can be found here.
Facial cartoonizer + Style Swapping
Without finetuning, EdgeExtraction [code] + FacialCartoonization [code] → SwapAE also produces nice results.
Failures
JTBC Samples
