Home

Awesome

GENRE

Face sketch synthesis, GANs, SPADE, image-to-image translation

We provide PyTorch implementations for our ICME2021 paper GENRE:

@InProceedings{Li2021GENRE,
author = {Xiang Li and Fei Gao and Fei Huang},
title = {High-quality Face Sketch Synthesis via Geometric Normalization and Regularization},
booktitle = {IEEE International Conference on Multimedia and Expo (ICME) 2021},
month = {July 5-9},
year = {2021}
}

This project generates artistic portrait drawings (e.g. pencil-drawing) from face photos using a GAN-based model. You may find useful information in preprocessing steps.

Paper@IEEE Code@Github Project

Framework

Sample Results

Synthesized face sketches on the CUFS dataset: (a) input face photo, (b) FSS-NST, (c) CycleGAN, (d) Pix2Pix, (e) pGAN, (f) SS-FSS, (g) MDAL, (h) KT-FSS, (i) GauGAN, (j) SCA-GAN, (k) GENRE (ours), (l) target sketch, and (m) parsing mask predicted by BiSeNet.

Synthesised sketches for faces in-the-wild: (a) input photo, (b) FSS-NST, (c) CycleGAN, (d) Pix2Pix, (e) SSL-FSS, (f) GauGAN, (g) SCA-GAN, and (h) GENRE.

Synthesised sketches for faces in the CelebA-HQ dataset: (a) input photo, (b) FSS-NST, (c) CycleGAN, (d) Pix2Pix, (e) SSL-FSS, (f) GauGAN, (g) SCA-GAN, and (h) GENRE.

Prerequisites

Getting Started

Installation

train/test

Apply a pre-trained model

Preprocessing steps

Face photos (and paired drawings) need to be aligned and have facial parsing. And facial parsing after alignment are needed in our code in training.

In our work,facial parsing is segmented by method in [1]

[1] Yu, Changqian, et al. "Bisenet: Bilateral segmentation network for real-time semantic segmentation." Proceedings of the European conference on computer vision (ECCV). 2018.

Demo

<video width="480" height="270" controls> <source src="demo.mp4" type="video/mp4"> </video>

Our portrait drawing demos: (a) QR code of the applet of WeChat, (b) QR code of the Web API, (c)-(e) layouts of the applet of WeChat, and (f) picture of the drawing robot. Readers can try our demos by scanning the QR codes.

Citation

If you use this code for your research, please cite our paper.

Xiang Li, Fei Gao*, and Fei Huang, High-quality Face Sketch Synthesis via Geometric Normalization and Regularization, IEEE International Conference on Multimedia and Expo (ICME) 2021, July 5-9, 2021, Virtual. (Accepted)

bibtex:

@InProceedings{Li2021GENRE,
author = {Xiang Li and Fei Gao and Fei Huang},
title = {High-quality Face Sketch Synthesis via Geometric Normalization and Regularization},
booktitle = {IEEE International Conference on Multimedia and Expo (ICME) 2021},
month = {July 5-9},
year = {2021}
}

Acknowledgments

Our code is inspired by pytorch-CycleGAN-and-pix2pix and SPADE/GauGAN.