Awesome
Exp-GAN: 3D-Aware Facial Image Generation with Expression Control
This repository is the official implementation of the ACCV 2022 paper Exp-GAN: 3D-Aware Facial Image Generation with Expression Control.
Yeonkyeong Lee, Taeho Choi, Hyunsung Go, Hyunjoon Lee, Sunghyun Cho, and Junho Kim.
Installation
Requirements for using pytorch3d
- Python >= 3.7
- PyTorch >= 1.12.0
pip install -r requirements.txt
git clone https://github.com/facebookresearch/pytorch3d.git
cd pytorch3d
git checkout v0.7.0
pip install -e .
cd -
Dataset and model
Download the aligned FFHQ dataset images from the official repository,
and place them under data/FFHQ/img.
Annotations for DECA parameters from FFHQ dataset (head pose, shape and expression) can be downloaded below(place the files under data/FFHQ/annots)
DECA is used to generate facial texture, download required assets by running
cd data
sh download_deca.sh
cd -
Below we present the dataset folder tree:
data/
└── DECA/
├── data/
└── indices_ear_noeye.pkl
└── demo/
└── meta_smooth.json
└── FFHQ/
├── annots/
├── ffhq_deca_ear_ortho_flipped.json
└── ffhq_deca_ear_ortho.pkl
└── img/
Please refer experiments/config/config.yaml to see how the data is used.
Training
Run the following script to train our model:
sh ./experiments/ffhq/train.sh
evaluation
A pretrained model can be downloaded here. Place the model file under pretrained_model/model_checkpoint.ckpt.
Run the following script to generate images for the FID evaluation:
python eval.py --cfg <cfg> --ckpt <ckpt> --savedir <savedir>
Then run the following to measure the FID between generated and real images:
python fid.py --root_real <root_real> --root_fake <root_fake> --batch_size 50
where <root_real> contains downsampled FFHQ images and <root_fake> contains images generated by eval.py.
Demo
Please check demo.ipynb to see how to generate some examples by using a pretrained model.
Pose interpolation
Expression interpolation
Pose and expression interpolation
Low- and high-resolution results before and after StyleGAN upsampling
Shape interpolation
Latent vector (w space) interpolation
Contact
This project is maintained by
- Taeho Choi(major.1965@kakaobrain.com)
- Yeonkyeong Lee(mag.i@kakaobrain.com)
- Hyunjoon Lee(malfo.lee@kakaobrain.com)
License
Copyright (c) 2022 POSTECH, Kookmin University, Kakao Brain Corp. All Rights Reserved. Licensed under the Apache License, Version 2.0 (see LICENSE for details)