Awesome
Make Encoder Great Again in 3D GAN Inversion through Geometry and Occlusion-Aware Encoding (ICCV 2023)
Paper | Project page | Demo video
Official implementation of "Make Encoder Great Again in 3D GAN Inversion through Geometry and Occlusion-Aware Encoding" ICCV 2023
<div align="center"> <img src="assets/teaser.gif" > </div>:fire: Introduction
We present a encoder-based 3D generative adversarial network (GAN) inversion framework that can efficiently synthesize photo-realistic novel views while preserving geometry and details of the input image.
<div align="center"> <img src="assets/framework.png" width="600px"/> </div>:desktop_computer: Requirements
- 64-bit Python 3.8 and PyTorch 1.11.0 (or later).
- CUDA toolkit 11.3 or later.
- Python libraries: see requirements.txt
cd goae
conda create --name goae python=3.8
conda activate goae
pip install -r requirements.txt
:running_woman: Inference
Prepare Dataset
Dataset preparation can refer to EG3D or these codes
Download Models
The pretrained model checkpoint can be downloaded from google drive, Put those checkpoint into the directory GOAE/pretrained
. Note that current pretrained AFA only modifies the triplane on 32*32 resolution, more higher resolution modify can achieve better result.
Commands
You can use the command below to test the example.
python infer.py --multi_view --video
You can use the command below to edit the example.
python infer.py --multi_view --video --edit --edit_attr glass --alpha 1.0
Training codes
Training codes can be downloaded from here.
:handshake: Citation
If you find this work useful for your research, please cite:
@article{yuan2023make,
title={Make Encoder Great Again in 3D GAN Inversion through Geometry and Occlusion-Aware Encoding},
author={Yuan, Ziyang and Zhu, Yiming and Li, Yu and Liu, Hongyu and Yuan, Chun},
journal={arXiv preprint arXiv:2303.12326},
year={2023}
}
:mailbox: Contact
If you have any comments or questions, please open a new issue or feel free to contact Ziyang Yuan (yuanzy22@mails.tsinghua.edu).