Awesome
GENRE
Face sketch synthesis, GANs, SPADE, image-to-image translation
We provide PyTorch
implementations for our ICME2021 paper GENRE
:
@InProceedings{Li2021GENRE,
author = {Xiang Li and Fei Gao and Fei Huang},
title = {High-quality Face Sketch Synthesis via Geometric Normalization and Regularization},
booktitle = {IEEE International Conference on Multimedia and Expo (ICME) 2021},
month = {July 5-9},
year = {2021}
}
This project generates artistic portrait drawings (e.g. pencil-drawing) from face photos using a GAN-based model. You may find useful information in preprocessing steps.
Paper@IEEE
Code@Github
Project
Framework
Sample Results
Synthesized face sketches on the CUFS dataset: (a) input face photo, (b) FSS-NST, (c) CycleGAN, (d) Pix2Pix, (e) pGAN, (f) SS-FSS, (g) MDAL, (h) KT-FSS, (i) GauGAN, (j) SCA-GAN, (k) GENRE (ours), (l) target sketch, and (m) parsing mask predicted by BiSeNet.
Synthesised sketches for faces in-the-wild: (a) input photo, (b) FSS-NST, (c) CycleGAN, (d) Pix2Pix, (e) SSL-FSS, (f) GauGAN, (g) SCA-GAN, and (h) GENRE.
Synthesised sketches for faces in the CelebA-HQ dataset: (a) input photo, (b) FSS-NST, (c) CycleGAN, (d) Pix2Pix, (e) SSL-FSS, (f) GauGAN, (g) SCA-GAN, and (h) GENRE.
Prerequisites
- Linux or macOS
- Python 3.7.3
- Pytorch-lightning 0.7.5
- CPU or NVIDIA GPU + CUDA CuDNN
Getting Started
Installation
-
Clone this repo:
git clone https://github.com/fei-hdu/genre cd genre
-
Install PyTorch 0.4+ and torchvision from http://pytorch.org and other dependencies (e.g., visdom and dominate). You can install all the dependencies by
pip install -r requirements.txt
train/test
-
Download our CUFS datasetand copy content to
dataset
folder -
Train a model
python train.py --dataset_train_list list_train.txt --dataset_test_list list_test.txt --use_en_feature --use_gmsd --img_w 200 --img_h 250 --input_size 256
-
Test the model
python test.py --use_en_feature --img_w 200 --img_h 250 --input_size 256
-
If you want to train on your own data, please first align your pictures and prepare your data's facial parsing according to tutorial in preprocessing steps.
-
The test results are available at [Google Drive]
Apply a pre-trained model
- A face $photo \mapsto sketch$ model pre-trained on CUHK/CUFS
- The pre-trained model need to be save at
./checkpoint
- Then you can test the model
Preprocessing steps
Face photos (and paired drawings) need to be aligned and have facial parsing. And facial parsing after alignment are needed in our code in training.
In our work,facial parsing is segmented by method in [1]
- First, we need to align, resize and crop face photos (and corresponding drawings) to 256x256
- Then,we use code in https://github.com/zllrunning/face-parsing.PyTorch to detect facial parsing for face photos and drawings.
[1] Yu, Changqian, et al. "Bisenet: Bilateral segmentation network for real-time semantic segmentation." Proceedings of the European conference on computer vision (ECCV). 2018.
Demo
<video width="480" height="270" controls> <source src="demo.mp4" type="video/mp4"> </video>Our portrait drawing demos: (a) QR code of the applet of WeChat, (b) QR code of the Web API, (c)-(e) layouts of the applet of WeChat, and (f) picture of the drawing robot. Readers can try our demos by scanning the QR codes.
Citation
If you use this code for your research, please cite our paper.
Xiang Li, Fei Gao*, and Fei Huang, High-quality Face Sketch Synthesis via Geometric Normalization and Regularization, IEEE International Conference on Multimedia and Expo (ICME) 2021, July 5-9, 2021, Virtual. (Accepted)
bibtex:
@InProceedings{Li2021GENRE,
author = {Xiang Li and Fei Gao and Fei Huang},
title = {High-quality Face Sketch Synthesis via Geometric Normalization and Regularization},
booktitle = {IEEE International Conference on Multimedia and Expo (ICME) 2021},
month = {July 5-9},
year = {2021}
}
Acknowledgments
Our code is inspired by pytorch-CycleGAN-and-pix2pix and SPADE/GauGAN.