Awesome
Low-Rank Subspaces in GANs
Figure: Image editing results using LowRankGAN on StyleGAN2 (first three columns) and BigGAN (last column).
Low-Rank Subspaces in GANs <br> Jiapeng Zhu, Ruili Feng, Yujun Shen, Deli Zhao, Zhengjun Zha, Jingren Zhou, Qifeng Chen <br> Conference on Neural Information Processing Systems (NeurIPS)
In the repository, we propose LowRankGAN to locally control the image synthesis from GANs with the novel low-rank subspaces. Concretely, we first relate the image regions with the latent space with the help of Jacobian. We then perform low-rank factorization on the Jacobian to get the principal and null spaces. We finally project the principal space w.r.t. the region of interest onto the null space w.r.t. the rest region. In this way, by altering the latent codes along the directions within the projected space, which we call low-rank subspaces, we manage to precisely control the region of interest yet barely affect the rest region.
[Paper] [Project Page] [Demo]
NEWS: Please also find this repo for the Pytorch code!
Examples of Local Editing | |||
---|---|---|---|
Eyes | Mouth | Nose | Hair |
![]() | ![]() | ![]() | ![]() |
Manipulate with Provided Directions
We have already provided some directions under the directory directions/
. Users can easily use these directions for image local editing.
MODEL_PATH='stylegan2-ffhq-config-f-1024x1024.pkl'
DIRECTION='directions/ffhq1024/eyes_size.npy'
python manipulate.py $MODEL_PATH $DIRECTION
Find More Directions
We also provide the code for users to find customized directions. Please follow the steps below.
Step-0: Prepare the pre-trained generator
Here, we use the FFHQ model officially released in StyleGAN2 as an example. Please download it first.
Step-1: Compute Jacobian with random syntheses
MODEL_PATH='stylegan2-ffhq-config-f-1024x1024.pkl'
python compute_jacobian.py $MODEL_PATH
Step-2: Compute the directions from the Jacobian
JACOBIAN_PATH='outputs/jacobian_seed_4/w_dataset_ffhq.npy'
python compute_directions.py $JACOBIAN_PATH
Step-3: Verify the directions through image manipulation
MODEL_PATH='stylegan2-ffhq-config-f-1024x1024.pkl'
DIRECTION_PATH='outputs/directions/${DIRECTION_NAME}'
python manipulate.py $MODEL_PATH $DIRECTION
BibTeX
@inproceedings{zhu2021lowrankgan,
title = {Low-Rank Subspaces in {GAN}s},
author = {Zhu, Jiapeng and Feng, Ruili and Shen, Yujun and Zhao, Deli and Zha, Zhengjun and Zhou, Jingren and Chen, Qifeng},
booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
year = {2021}
}