Home

Awesome

HFGI: High-Fidelity GAN Inversion for Image Attribute Editing (CVPR 2022)

High-Fidelity GAN Inversion for Image Attribute Editing
https://tengfei-wang.github.io/HFGI/

Update: We released the inference code and the pre-trained model on Oct. 31. The training code is coming soon.
Update: We provided a Colab notebook for play.
Update: We released the training code.

paper | supp. | project website | demo video | Colab | online demo

<img src="pics/1.jpg" width="130px"/> <img src="pics/4.jpg" width="130px"/> <img src="pics/5.jpg" width="130px"/> <img src="pics/6.jpg" width="130px"/>

<img src="pics/1.gif" width="130"/> <img src="pics/4.gif" width="130"/> <img src="pics/5.gif" width="130"/> <img src="pics/6.gif" width="130"/>

Introduction

We present a novel high-fidelity GAN inversion framework that enables attribute editing with image-specific details well-preserved (e.g., background, appearance and illumination).

<img src="pics/method.jpg" width="800px"/>

To Do

Set up

Installation

git clone https://github.com/Tengfei-Wang/HFGI.git
cd HFGI

Environment

The environment can be simply set up by Anaconda (only tested for inference):

conda create -n HFGI python=3.7
conda activate HFGI
pip install torch==1.6.0+cu101 torchvision==0.7.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html
pip install matplotlib
conda install ninja
conda install -c 3dhubs gcc-5

Or, you can also set up the environment from the provided environment.yml:

conda env create -f environment.yml

Quick Start

Pretrained Models

Please download our pre-trained model and put it in ./checkpoint.

ModelDescription
Face EditingTrained on FFHQ.

Prepare Images

We put some images from CelebA-HQ in ./test_imgs, and you can quickly try them (and other images from CelebA-HQ or FFHQ).
For customized images, it is encouraged to first pre-process (align & crop) them, and then edit with our model. See FFHQ for alignment details.

Inference

Modify inference.sh according to the follwing instructions, and run:
(It is possibly slow for the first-time running.)

bash inference.sh
ArgsDescription
--images_dirthe path of images.
--n_samplenumber of images that you want to infer.
--edit_attributeWe provide options of 'inversion', 'age', 'smile', 'eyes', 'lip' and 'beard' in the script.
--edit_degreecontrol the degree of editing (works for 'age' and 'smile').

Training

Preparation

  1. Download datasets and modify the dataset path in ./configs/paths_config.py accordingly.
  2. Download some pretrained models and put them in ./pretrained.
ModelDescription
StyleGAN2 (FFHQ)Pretrained face generator on FFHQ from rosinality.
e4e (FFHQ)Pretrained initial encoder on FFHQ from omertov.
Feature extractor (for face)Pretrained IR-SE50 model taken from TreB1eN for ID loss calculation.
Feature extractor (for car)Pretrained ResNet-50 model taken from omertov for ID loss calculation.

Start Training

Modify option and training.sh and run:

bash train.sh

Video Editing

The source videos and edited results in our paper can be found in this link.
For video editing, we first pre-process (align & crop) each frame, and then perform editing with the pre-trained model.

More Results

<img src="pics/real_edit.jpg" width="700px"/>

Citation

If you find this work useful for your research, please cite:

@inproceedings{wang2021HFGI,
  title={High-Fidelity GAN Inversion for Image Attribute Editing},
  author={Wang, Tengfei and Zhang, Yong and Fan, Yanbo and Wang, Jue and Chen, Qifeng},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2022}
}

Acknowledgement

Thanks to omertov for sharing their code.