Awesome

<h1 align="center">GAN Inversion: A Survey</h1> TPAMI 2022 <a href="https://weihaox.github.io/">Weihao Xia</a> · <a href="https://yulunzhang.com/">Yulun Zhang</a> · <a href="https://sites.google.com/view/iigroup-thu/about">Yujiu Yang</a> · <a href="http://www.homepages.ucl.ac.uk/~ucakjxu/">Jing-Hao Xue</a> · <a href="https://boleizhou.github.io/">Bolei Zhou</a> · <a href="https://faculty.ucmerced.edu/mhyang/">Ming-Hsuan Yang</a> <a href='https://arxiv.org/abs/2101.05278'> <img src='https://img.shields.io/badge/Paper-PDF-green?style=flat&logo=arxiv&logoColor=green' alt='arxiv PDF'> </a> <a href='https://github.com/weihaox/awesome-gan-inversion' style='padding-left: 0.5rem;'> <img src='https://img.shields.io/badge/Project-Page-blue?style=flat&logo=Google%20chrome&logoColor=blue' alt='Project Page'> </a> <a href='https://ieeexplore.ieee.org/document/9792208' style='padding-left: 0.5rem;'> <img src='https://img.shields.io/badge/TPAMI-PDF-red?style=flat&logoColor=red' alt='TPAMI PDF'> </a>

This repo is a collection of resources on GAN inversion, as a supplement for our survey. If you find any work missing or have any suggestions (papers, implementations and other resources), feel free to pull requests. You could manually edit items or use the script to produce them in the markdown format.

<details style="margin-left:3%;"> <summary>citation</summary> <pre><code class="language-bib" style="font-size: 0.9rem;" id="citation">@article{xia2022gan, author = {Xia, Weihao and Zhang, Yulun and Yang, Yujiu and Xue, Jing-Hao and Zhou, Bolei and Yang, Ming-Hsuan}, title = {GAN Inversion: A Survey}, journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)}, year={2022} } </code></pre> </details> <details><summary>Table of Contents</summary>

Inverted Pretrained Models
- 2D GANs
- 3D GANs
GAN Inversion Methods
- 3D GAN Inversion
- 2D GAN Inversion
Diffusion Inversion
GAN Latent Space Editing
Diffusion Latent Space Editing
Applications
Acknowledgement

</details>

Inverted Pretrained Models

2D GANs

Scaling up GANs for Text-to-Image Synthesis. Minguk Kang, Jun-Yan Zhu, Richard Zhang, Jaesik Park, Eli Shechtman, Sylvain Paris, Taesung Park. CVPR 2023 (Highlight). [PDF] [Project]

StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesis. Axel Sauer, Tero Karras, Samuli Laine, Andreas Geiger, Timo Aila. ICML 2023. [Project] [PDF] [Code]

StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets. Axel Sauer, Katja Schwarz, Andreas Geiger. SIGGRAPH 2022. [PDF] [Project] [Code]

Self-Distilled StyleGAN: Towards Generation from Internet Photos. Ron Mokady, Michal Yarom, Omer Tov, Oran Lang, Daniel Cohen-Or, Tali Dekel, Michal Irani, Inbar Mosseri. SIGGRAPH 2022. [PDF] [Project] [Code]

Ensembling Off-the-shelf Models for GAN Training. Nupur Kumari, Richard Zhang, Eli Shechtman, Jun-Yan Zhu CVPR 2022. [PDF] [Project] [Code]

StyleGAN3: Alias-Free Generative Adversarial Networks. Tero Karras, Miika Aittala, Samuli Laine, Erik Härkönen, Janne Hellsten, Jaakko Lehtinen, Timo Aila. NeurIPS 2021. [PDF] [Project] [Code] [Rosinality]

StyleGAN2-Ada: Training Generative Adversarial Networks with Limited Data. Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, Timo Aila. NeurIPS 2020. [PDF] [Code] [Steam StyleGAN2-ADA]

StyleGAN2: Analyzing and Improving the Image Quality of StyleGAN. Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, Timo Aila. CVPR 2020. [PDF] [PyTorch] [Offical TF] [Unoffical Tensorflow 2.0]

StyleGAN: A Style-Based Generator Architecture for Generative Adversarial Networks. Tero Karras, Samuli Laine, Timo Aila. CVPR 2019. [PDF] [Offical TF]

ProGAN: Progressive Growing of GANs for Improved Quality, Stability, and Variation. Tero Karras, Timo Aila, Samuli Laine, Jaakko Lehtinen. ICLR 2018. [PDF] [Offical TF]

3D-aware GANs

Please check our 3D-aware image synthesis survey, paper list, and project for more details.

EG3D: Efficient Geometry-aware 3D Generative Adversarial Networks. Eric R. Chan, Connor Z. Lin, Matthew A. Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas Guibas, Jonathan Tremblay, Sameh Khamis, Tero Karras, Gordon Wetzstein. CVPR 2022. [PDF] [Project] [Code]

StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation. Roy Or-El, Xuan Luo, Mengyi Shan, Eli Shechtman, Jeong Joon Park, Ira Kemelmacher-Shlizerman. CVPR 2022. [PDF] [Project] [Code]

StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis. Jiatao Gu, Lingjie Liu, Peng Wang, Christian Theobalt. ICLR 2022. [PDF] [Project]

pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis. Eric R. Chan, Marco Monteiro, Petr Kellnhofer, Jiajun Wu, Gordon Wetzstein. CVPR 2021. [PDF] [Project] [Code]

GAN Inversion Methods

The section primarily encompasses general-purpose 2D or 3D inversion techniques, whereas the methods presented in the following section cater to particular applications.

3D GAN Inversion

TriPlaneNet: An Encoder for EG3D Inversion. Ananta R. Bhattarai, Matthias Nießner, Artem Sevastopolsky. WACV 2024. [PDF] [Project]

In-N-Out: Faithful 3D GAN Inversion with Volumetric Decomposition for Face Editing. Yiran Xu, Zhixin Shu, Cameron Smith, Jia-Bin Huang, Seoung Wug Oh. CVPR 2024. [PDF] [Project]

Make Encoder Great Again in 3D GAN Inversion through Geometry and Occlusion-Aware Encoding. Ziyang Yuan, Yiming Zhu, Yu Li, Hongyu Liu, Chun Yuan. ICCV 2023. [PDF] [Project] [Code]

LatentSwap3D: Semantic Edits on 3D Image GANs. Enis Simsar, Alessio Tonioni, Evin Pınar Örnek, Federico Tombari. ICCV 2023 Workshops on AI3DCC. [PDF] [Code]

High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimization. Jiaxin Xie, Hao Ouyang, Jingtan Piao, Chenyang Lei, Qifeng Chen. CVPR 2023. [PDF] [Project] [Code]

Learning Detailed Radiance Manifolds for High-Fidelity and 3D-Consistent Portrait Synthesis from Monocular Image. Yu Deng, Baoyuan Wang, Heung-Yeung Shum. CVPR 2023. [PDF] [Project]

E3DGE: Self-Supervised Geometry-Aware Encoder for Style-based 3D GAN Inversion. Yushi Lan, Xuyi Meng, Shuai Yang, Chen Change Loy, Bo Dai. CVPR 2023. [PDF] [Project] [Code]

3D GAN Inversion with Pose Optimization. Jaehoon Ko, Kyusun Cho, Daewon Choi, Kwangrok Ryoo, Seungryong Kim. WACV 2023. [PDF] [Project] [Code]

3D GAN Inversion for Controllable Portrait Image Animation. Connor Z. Lin, David B. Lindell, Eric R. Chan, Gordon Wetzstein. ECCV 2022 Workshop on Learn3DG. [PDF] [Project]

Pix2NeRF: Unsupervised Conditional π-GAN for Single Image to Neural Radiance Fields Translation. Shengqu Cai, Anton Obukhov, Dengxin Dai, Luc Van Gool. CVPR 2022. [PDF]

2D GAN Inversion

StyleRes: Transforming the Residuals for Real Image Editing with StyleGAN. Hamza Pehlivan, Yusuf Dalva, Aysegul Dundar. CVPR 2023. [PDF] [Code]

ReGANIE: Rectifying GAN Inversion Errors for Accurate Real Image Editing. Bingchuan Li, Tianxiang Ma, Peng Zhang, Miao Hua, Wei Liu, Qian He, Zili Yi. AAAI 2023. [PDF]

Intra-Source Style Augmentation for Improved Domain Generalization. Yumeng Li, Dan Zhang, Margret Keuper, Anna Khoreva. WACV 2023. [PDF] [Code]

Pivotal Tuning for Latent-based Editing of Real Images. Daniel Roich, Ron Mokady, Amit H. Bermano, Daniel Cohen-Or. TOG 2022. [PDF] [Code]

E2Style: Improve the Efficiency and Effectiveness of StyleGAN Inversion. Tianyi Wei, Dongdong Chen, Wenbo Zhou, Jing Liao, Weiming Zhang, Lu Yuan, Gang Hua, Nenghai Yu. TIP 2022. [PDF] [Project] [Code]

High-fidelity GAN Inversion with Padding Space. Qingyan Bai, Yinghao Xu, Jiapeng Zhu, Weihao Xia, Yujiu Yang, Yujun Shen. ECCV 2022. [PDF] [Project] [Code]

Editing Out-of-Domain GAN Inversion via Differential Activations. Haorui Song, Yong Du, Tianyi Xiang, Junyu Dong, Jing Qin, Shengfeng He. ECCV 2022. [PDF] [Code]

IntereStyle: Encoding an Interest Region for Robust StyleGAN Inversion. Seungjun Moon, GyeongMoon Park. ECCV 2022. [PDF]

Chunkmogrify: Real image inversion via Segments. David Futschik, Michal Lukáč, Eli Shechtman, Daniel Sýkora. ECCV 2022. [PDF] [Code]

A Style-Based GAN Encoder for High Fidelity Reconstruction of Images and Videos. Xu Yao, Alasdair Newson, Yann Gousseau, Pierre Hellier. ECCV 2022. [PDF] [Code]

Cycle Encoding of a StyleGAN Encoder for Improved Reconstruction and Editability. Xudong Mao, Liujuan Cao, Aurele Tohokantche Gnanha, Zhenguo Yang, Qing Li, Rongrong Ji. ACM MM 2022. [PDF] [Code]

Spatially-Adaptive Multilayer Selection for GAN Inversion and Editing. Gaurav Parmar, Yijun Li, Jingwan Lu, Richard Zhang, Jun-Yan Zhu, Krishna Kumar Singh. CVPR 2022. [PDF] [Project] [Code]

Style Transformer for Image Inversion and Editing. Xueqi Hu, Qiusheng Huang, Zhengyi Shi, Siyuan Li, Changxin Gao, Li Sun, Qingli Li. CVPR 2022. [PDF] [Code]

High-Fidelity GAN Inversion for Image Attribute Editing. Tengfei Wang, Yong Zhang, Yanbo Fan, Jue Wang, Qifeng Chen. CVPR 2022. [PDF] [Project] [Code]

HyperInverter: Improving StyleGAN Inversion via Hypernetwork. Tan M. Dinh, Anh Tuan Tran, Rang Nguyen, Binh-Son Hua. CVPR 2022. [PDF] [Project]

HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing. Yuval Alaluf, Omer Tov, Ron Mokady, Rinon Gal, Amit H. Bermano. CVPR 2022. [PDF] [Project] [Code]

Overparameterization Improves StyleGAN Inversion. Yohan Poirier-Ginter, Alexandre Lessard, Ryan Smith, Jean-François Lalonde. CVPR 2022 Workshop on AI for Content Creation. [PDF] [Code]

StyleAlign: Analysis and Applications of Aligned StyleGAN Models. Zongze Wu, Yotam Nitzan, Eli Shechtman, Dani Lischinski. ICLR 2022. [PDF]

GAN-Control: Explicitly Controllable GANs. Alon Shoshan, Nadav Bhonker, Igor Kviatkovsky, Gerard Medioni. ICCV 2021. [PDF]

From Continuity to Editability: Inverting GANs with Consecutive Images. Yangyang Xu, Yong Du, Wenpeng Xiao, Xuemiao Xu and Shengfeng He. ICCV 2021. [PDF] [Code]

Explaining in Style: Training a GAN to explain a classifier in StyleSpace. Oran Lang, Yossi Gandelsman, Michal Yarom, Yoav Wald, Gal Elidan, Avinatan Hassidim, William T. Freeman, Phillip Isola, Amir Globerson, Michal Irani, Inbar Mosseri. ICCV 2021. [PDF] [Project]

BDInvert: GAN Inversion for Out-of-Range Images with Geometric Transformations. Kyoungkook Kang, Seongtae Kim, Sunghyun Cho. ICCV 2021. [PDF] [Project]

ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement. Yuval Alaluf, Or Patashnik, Daniel Cohen-Or. ICCV 2021. [PDF] [Project] [Code]

LatentCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions. Oğuz Kaan Yüksel, Enis Simsar, Ezgi Gülperi Er, Pinar Yanardag. ICCV 2021. [PDF] [Code]

Lifting 2D StyleGAN for 3D-Aware Face Generation. Yichun Shi, Divyansh Aggarwal, Anil K. Jain. CVPR 2021. [PDF]

Ensembling with Deep Generative Views. Lucy Chai, Jun-Yan Zhu, Eli Shechtman, Phillip Isola, Richard Zhang. CVPR 2021. [PDF] [Code] [Project]

Navigating the GAN Parameter Space for Semantic Image Editing. Anton Cherepkov, Andrey Voynov, Artem Babenko. CVPR 2021. [PDF] [Code]

StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation. Zongze Wu, Dani Lischinski, Eli Shechtman. CVPR 2021 (oral). [PDF] [Code]

Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation. Elad Richardson, Yuval Alaluf, Or Patashnik, Yotam Nitzan, Yaniv Azar, Stav Shapiro, Daniel Cohen-Or. CVPR 2021. [PDF] [Code] [Project]

GHFeat: Generative Hierarchical Features from Synthesizing Images. Yinghao Xu, Yujun Shen, Jiapeng Zhu, Ceyuan Yang, Bolei Zhou. CVPR 2021. [PDF] [Code] [Project]

Hijack-GAN: Unintended-Use of Pretrained, Black-Box GANs. Hui-Po Wang, Ning Yu, Mario Fritz. CVPR 2021. [PDF]

Prior Image-Constrained Reconstruction using Style-Based Generative Models. Varun A Kelkar, Mark Anastasio. ICML 2021. [PDF]

Intermediate Layer Optimization for Inverse Problems using Deep Generative Models. Giannis Daras, Joseph Dean, Ajil Jalal, Alexandros G. Dimakis. ICML 2021. [PDF] [Code]

Using Latent Space Regression to Analyze and Leverage Compositionality in GANs. Lucy Chai, Jonas Wulff, Phillip Isola. ICLR 2021. [PDF] [Code] [Project] [Colab]

Enjoy Your Editing: Controllable GANs for Image Editing via Latent Space Navigation. Peiye Zhuang, Oluwasanmi Koyejo, Alexander G. Schwing. ICLR 2021. [PDF]

Disentangled Face Attribute Editing via Instance-Aware Latent Space Search. Yuxuan Han, Jiaolong Yang, Ying Fu. IJCAI 2021. [PDF] [Code]

High Fidelity GAN Inversion via Prior Multi-Subspace Feature Composition. Guanyue Li, Qianfen Jiao, Sheng Qian, Si Wu, Hau-San Wong. AAAI 2021. [PDF]

e4e: Designing an Encoder for StyleGAN Image Manipulation. Omer Tov, Yuval Alaluf, Yotam Nitzan, Or Patashnik, Daniel Cohen-Or. TOG 2021. [PDF] [Code]

StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows. Rameen Abdal, Peihao Zhu, Niloy Mitra, Peter Wonka. TOG 2021. [PDF] [Code]

PIE: Portrait Image Embedding for Semantic Control. A. Tewari, M. Elgharib, M. BR, F. Bernard, H-P. Seidel, P. P‌érez, M. Zollhöfer, C.Theobalt. TOG 2020. [PDF] [Project]

Face Identity Disentanglement via Latent Space Mapping. Yotam Nitzan, Amit Bermano, Yangyan Li, Daniel Cohen-Or. SIGGRAPH Asia 2020. [PDF] [Project] [Code]

Understanding the Role of Individual Units in a Deep Neural Network. David Bau, Jun-Yan Zhu, Hendrik Strobelt, Agata Lapedriza, Bolei Zhou, Antonio Torralba. National Academy of Sciences 2020. [PDF] [Code] [Project]

Face Identity Disentanglement via Latent Space Mapping. Yotam Nitzan, Amit Bermano, Yangyan Li, Daniel Cohen-Or. TOG 2020. [PDF] [Code]

Transforming and Projecting Images into Class-conditional Generative Networks. Minyoung Huh, Richard Zhang, Jun-Yan Zhu, Sylvain Paris, Aaron Hertzmann. ECCV 2020. [PDF] [Code] [Project]

MimicGAN: Robust Projection onto Image Manifolds with Corruption Mimicking. Rushil Anirudh, Jayaraman J. Thiagarajan, Bhavya Kailkhura, Timo Bremer. IJCV 2020. [PDF]

Rewriting a Deep Generative Model. David Bau, Steven Liu, Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba. ECCV 2020. [PDF] [Code]

StyleGAN2 Distillation for Feed-forward Image Manipulation. Yuri Viazovetskyi, Vladimir Ivashkin, Evgeny Kashin. ECCV 2020. [PDF] [Code]

In-Domain GAN Inversion for Real Image Editing. Jiapeng Zhu, Yujun Shen, Deli Zhao, Bolei Zhou. ECCV 2020. [PDF] [Project] [Code]

Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation. Xingang Pan, Xiaohang Zhan, Bo Dai, Dahua Lin, Chen Change Loy, Ping Luo. ECCV 2020. [PDF] [Code]

Your Local GAN: Designing Two Dimensional Local Attention Mechanisms for Generative Models. Giannis Daras, Augustus Odena, Han Zhang, Alexandros G. Dimakis. CVPR 2020. [PDF]

A Disentangling Invertible Interpretation Network for Explaining Latent Representations. Patrick Esser, Robin Rombach, Björn Ommer. CVPR 2020. [PDF] [Project] [Code]

Editing in Style: Uncovering the Local Semantics of GANs. Edo Collins, Raja Bala, Bob Price, Sabine Süsstrunk. CVPR 2020. [PDF] [Code]

Image Processing Using Multi-Code GAN Prior. Jinjin Gu, Yujun Shen, Bolei Zhou. CVPR 2020. [PDF] [Project] [Code]

Image2StyleGAN++: How to Edit the Embedded Images? Rameen Abdal, Yipeng Qin, Peter Wonka. CVPR 2020. [PDF]

Semantic Photo Manipulation with a Generative Image Prior. David Bau, Hendrik Strobelt, William Peebles, Jonas, Bolei Zhou, Jun-Yan Zhu, Antonio Torralba. TOG 2019. [PDF]

Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space? Rameen Abdal, Yipeng Qin, Peter Wonka. ICCV 2019. [PDF] [Code]

GAN-based Projector for Faster Recovery with Convergence Guarantees in Linear Inverse Problems. Ankit Raj, Yuqi Li, Yoram Bresler. ICCV 2019. [PDF]

Inverting Layers of a Large Generator. David Bau, Jun-Yan Zhu, Jonas Wulff, William Peebles, Hendrik Strobelt, Bolei Zhou, Antonio Torralba. ICCV 2019. [PDF]

Detecting Overfitting in Deep Generators via Latent Recovery. Ryan Webster, Julien Rabin, Loic Simon, Frederic Jurie. CVPR 2019. [PDF][Colab]

Inverting The Generator Of A Generative Adversarial Network (II). Antonia Creswell, Anil A Bharath. TNNLS 2018. [PDF] [Code]

Invertibility of Convolutional Generative Networks from Partial Measurements. Fangchang Ma, Ulas Ayaz, Sertac Karaman. NeurIPS 2018. [PDF] [Code]

Metrics for Deep Generative Models. Nutan Chen, Alexej Klushyn, Richard Kurle, Xueyan Jiang, Justin Bayer, Patrick van der Smagt. AISTATS 2018. [PDF]

Towards Understanding the Invertibility of Convolutional Neural Networks. Anna C. Gilbert, Yi Zhang, Kibok Lee, Yuting Zhang, Honglak Lee. IJCAI 2017. [PDF]

One Network to Solve Them All - Solving Linear Inverse Problems using Deep Projection Models. J. H. Rick Chang, Chun-Liang Li, Barnabas Poczos, B. V. K. Vijaya Kumar, Aswin C. Sankaranarayanan. ICCV 2017. [PDF]

Precise Recovery of Latent Vectors from Generative Adversarial Networks. Zachary C. Lipton, Subarna Tripathi. ICLR 2017 workshop. [PDF] [Code]

Inverting The Generator Of A Generative Adversarial Network. Antonia Creswell, Anil Anthony Bharath. NeurIPS 2016 Workshop. [PDF]

Generative Visual Manipulation on the Natural Image Manifold. Jun-Yan Zhu, Philipp Krähenbühl, Eli Shechtman, Alexei A. Efros. ECCV 2016. [PDF]

Improved StyleGAN Embedding: Where Are The Good Latents?. Peihao Zhu, Rameen Abdal, Yipeng Qin, John Femiani, Peter Wonka. arxiv 2020. [PDF] [Code]

Improving Inversion and Generation Diversity in StyleGAN Using A Gaussianized Latent Space. Jonas Wulff, Antonio Torralba. arxiv 2020. [PDF]

GAN Latent Space Editing

Inversion isn't the ultimate goal; it's a means to enable real image or video editing within a GAN's latent space. Commonly referred to as GAN latent space editing, navigation, traversal, steerability, or other names in the literature, this task, although sometimes seen as a standalone research domain, acts as an indispensable component of GAN inversion based editing. This section is about GAN latent space editing. Recent studies reveal diffusion model can be used for real image and video editing in a similar way. Please refer to Diffusion Inversion and Diffusion Latent Space Editing for more details.

Deep Curvilinear Editing: Commutative and Nonlinear Image Manipulation for Pretrained Deep Generative Model. Takehiro Aoshima, Takashi Matsubara. CVPR 2023. [PDF]

Fantastic Style Channels and Where to Find Them: A Submodular Framework for Discovering Diverse Directions in GANs. Enis Simsar, Umut Kocasari, Ezgi Gülperi Er, Pinar Yanardag. WACV 2023. [PDF] [Project] [Demo]

Latent Traversals in Generative Models as Potential Flows. Yue Song, Andy Keller, Nicu Sebe, Max Welling. ICML 2023. [PDF] [Code]

PandA: Unsupervised Learning of Parts and Appearances in the Feature Maps of GANs. James Oldfield, Christos Tzelepis, Yannis Panagakis, Mihalis A. Nicolaou, Ioannis Patras. ICLR 2023. [PDF] [Project] [Code]

Rayleigh EigenDirections (REDs): GAN Latent Space Traversals for Multidimensional Features. Guha Balakrishnan, Raghudeep Gadde, Aleix Martinez, Pietro Perona. ECCV 2022. [PDF]

Rayleigh EigenDirections (REDs): Nonlinear GAN Latent Space Traversals for Multidimensional Features. Guha Balakrishnan, Raghudeep Gadde, Aleix Martinez, Pietro Perona. ECCV 2022. [PDF]

Exploring Gradient-based Multi-directional Controls in GANs. Zikun Chen, Ruowei Jiang, Brendan Duke, Han Zhao, Parham Aarabi. ECCV 2022 (oral). [PDF] [Project]

Hierarchical Semantic Regularization of Latent Spaces in StyleGANs. Tejan Karmali, Rishubh Parihar, Susmit Agrawal, Harsh Rangwani, Varun Jampani, Maneesh Singh, R. Venkatesh Babu. ECCV 2022. [PDF] [Project]

CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directions. Rameen Abdal, Peihao Zhu, John Femiani, Niloy J. Mitra, Peter Wonka. SIGGRAPH 2022. [PDF] [Code]

Region-Based Semantic Factorization in GANs. Jiapeng Zhu, Yujun Shen, Yinghao Xu, Deli Zhao, Qifeng Chen. ICML 2022. [PDF] [Code]

Latent Image Animator: Learning to Animate Image via Latent Space Navigation. Yaohui Wang, Di Yang, Francois Bremond, Antitza Dantcheva. ICLR 2022. [PDF] [Project] [Code]

Do Not Escape From the Manifold: Discovering the Local Coordinates on the Latent Space of GANs. Jaewoong Choi, Changyeon Yoon, Junho Lee, Jung Ho Park, Geonho Hwang, Myungjoo Kang. ICLR 2022. [PDF]

Tensor-based Emotion Editing in the StyleGAN Latent Space. René Haas, Stella Graßhof, Sami S. Brandt. CVPR 2022 Workshop on AI for Content Creation. [PDF]

PaintInStyle: One-Shot Discovery of Interpretable Directions by Painting. Berkay Doner, Elif Sema Balcioglu, Merve Rabia Barin, Umut Kocasari, Mert Tiftikci, Pinar Yanardag. CVPR 2022 Workshop. [PDF]

Rank in Style: A Ranking-Based Approach To Find Interpretable Directions. Umut Kocasari, Kerem Zaman, Mert Tiftikci, Enis Simsar, Pinar Yanardag. CVPR 2022 Workshop. [PDF]

LARGE: Latent-Based Regression through GAN Semantics. Yotam Nitzan, Rinon Gal, Ofir Brenner, Daniel Cohen-Or. CVPR 2022. [PDF] [Code] [Project]

StyleFusion: Disentangling Spatial Segments in StyleGAN-Generated Images. Omer Kafri, Or Patashnik, Yuval Alaluf, Daniel Cohen-Or. TOG 2022. [PDF] [Code]

Optimizing Latent Space Directions For GAN-based Local Image Editing. Ehsan Pajouheshgar, Tong Zhang, Sabine Süsstrunk. ICASSP 2022. [PDF]

Tensor Component Analysis for Interpreting the Latent Space of GANs. James Oldfield, Markos Georgopoulos, Yannis Panagakis, Mihalis A. Nicolaou, Ioannis Patras. BMVC 2021. [PDF] [Project] [Code]

Tensor-based Subspace Factorization for StyleGAN. Rene Haas, Stella Graßhof and Sami S. Brandt. FG 2021. [PDF]

Exploratory Search of GANs with Contextual Bandits. Ivan Kropotov, Alan Medlar, Dorota Glowacka. CIKM 2021. [PDF]

LowRankGAN: Low-Rank Subspaces in GANs. Jiapeng Zhu, Ruili Feng, Yujun Shen, Deli Zhao, Zhengjun Zha, Jingren Zhou, Qifeng Chen. NeurIPS 2021. [PDF] [Code]

Controllable and Compositional Generation with Latent-Space Energy-Based Models. Weili Nie, Arash Vahdat, Anima Anandkumar. NeurIPS 2021. [PDF]

A Latent Transformer for Disentangled Face Editing in Images and Videos. Xu Yao, Alasdair Newson, Yann Gousseau, Pierre Hellier. ICCV 2021. [PDF] [arxiv] [Code]

Toward a Visual Concept Vocabulary for GAN Latent Space. Sarah Schwettmann, Evan Hernandez, David Bau, Samuel Klein, Jacob Andreas, Antonio Torralba. ICCV 2021. [PDF] [Project]

WarpedGANSpace: Finding Non-linear RBF Paths in GAN Latent Space. Christos Tzelepis, Georgios Tzimiropoulos, Ioannis Patras. ICCV 2021. [PDF] [Code]

Latent Transformations via NeuralODEs for GAN-based Image Editing. Valentin Khrulkov, Leyla Mirvakhabova, Ivan Oseledets, Artem Babenko. ICCV 2021. [PDF] [Code]

OroJaR: Orthogonal Jacobian Regularization for Unsupervised Disentanglement in Image Generation. Yuxiang Wei, Yupeng Shi, Xiao Liu, Zhilong Ji, Yuan Gao, Zhongqin Wu, Wangmeng Zuo. ICCV 2021. [PDF] [Code]

EigenGAN: Layer-Wise Eigen-Learning for GANs. Zhenliang He, Meina Kan, Shiguang Shan. ICCV 2021. [PDF] [Code]

SSFlow: Style-guided Neural Spline Flows for Face Image Manipulation. Hanbang Liang, Xianxu Hou, Linlin Shen. ACM MM 2021. [PDF]

SalS-GAN: Spatially-Adaptive Latent Space in StyleGAN for Real Image Embedding. Lingyun Zhang, Xiuxiu Bai, Yao Gao. ACM MM 2021. [PDF]

Discovering Density-Preserving Latent Space Walks in GANs for Semantic Image Transformations. Guanyue Li, Yi Liu, Xiwen Wei, Yang Zhang, Si Wu, Yong Xu, Hau San Wong. ACM MM 2021. [PDF]

Discovering Interpretable Latent Space Directions of GANs Beyond Binary Attributes. Huiting Yang, Liangyu Chai, Qiang Wen, Shuang Zhao, Zixun Sun, Shengfeng He. CVPR 2021. [PDF] [Code]

Surrogate Gradient Field for Latent Space Manipulation. Minjun Li, Yanghua Jin, Huachun Zhu. CVPR 2021. [PDF]

SeFa: Closed-Form Factorization of Latent Semantics in GANs. Yujun Shen, Bolei Zhou. CVPR 2021. [PDF] [Code] [Project]

L2M-GAN: Learning To Manipulate Latent Space Semantics for Facial Attribute Editing. Guoxing Yang, Nanyi Fei, Mingyu Ding, Guangzhen Liu, Zhiwu Lu, Tao Xiang. CVPR 2021. [PDF] [Unofficial Pytorch]

MoCoGAN-HD: A Good Image Generator Is What You Need for High-Resolution Video Synthesis. Yu Tian, Jian Ren, Menglei Chai, Kyle Olszewski, Xi Peng, Dimitris N. Metaxas, Sergey Tulyakov. ICLR 2021. [PDF] [Code]

GAN Steerability without optimization. Nurit Spingarn-Eliezer, Ron Banner, Tomer Michaeli. ICLR 2021. [OpenReview] [PDF]

On the "steerability" of generative adversarial networks. Ali Jahanian, Lucy Chai, Phillip Isola. ICLR 2020. [PDF] [Project]

GANSpace: Discovering Interpretable GAN Controls. Erik Härkönen, Aaron Hertzmann, Jaakko Lehtinen, Sylvain Paris. NeurIPS 2020. [PDF] [Code]

Interpreting the Latent Space of GANs for Semantic Face Editing. Yujun Shen, Jinjin Gu, Xiaoou Tang, Bolei Zhou. CVPR 2020. [PDF] [Project] [Code]

Seeing What a GAN Cannot Generate. David Bau, Jun-Yan Zhu, Jonas Wulff, William Peebles, Hendrik Strobelt, Bolei Zhou, Antonio Torralba. ICCV 2019. [PDF] [PDF]

Unsupervised Discovery of Interpretable Directions in the GAN Latent Space. Andrey Voynov, Artem Babenko. ICML 2020. [PDF] [Code]

Multi-level Latent Space Structuring for Generative Control. Oren Katzir, Vicky Perepelook, Dani Lischinski, Daniel Cohen-Or. arxiv 2022. [PDF]

Diffusion Inversion

An Edit Friendly DDPM Noise Space: Inversion and Manipulations. Inbar Huberman-Spiegelglas, Vladimir Kulikov, Tomer Michaeli. CVPR 2024. [PDF] [Project] [Code]

Direct Inversion: Boosting Diffusion-based Editing with 3 Lines of Code. Xuan Ju, Ailing Zeng, Yuxuan Bian, Shaoteng Liu, Qiang Xu. ICLR 2024. [PDF] [Code]

NULL-text Inversion for Editing Real Images using Guided Diffusion Models. Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, Daniel Cohen-Or. CVPR 2023. [PDF] [Project] [Code]

EDICT: Exact Diffusion Inversion via Coupled Transformations. Bram Wallace, Akash Gokul, Nikhil Naik. CVPR 2023. [PDF] [Code]

An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion. Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H. Bermano, Gal Chechik, Daniel Cohen-Or. ICLR 2023 (Oral). [PDF] [Project] [Code]

Prompt-to-Prompt Image Editing with Cross Attention Control. Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, Daniel Cohen-Or. ICLR 2023. [PDF] [Project] [Code]

A Neural Space-Time Representation for Text-to-Image Personalization. Yuval Alaluf, Elad Richardson, Gal Metzer, Daniel Cohen-Or. SIGGRAPH Asia 2023. [PDF] [Project]

Discovering Interpretable Directions in the Semantic Latent Space of Diffusion Models. René Haas, Inbar Huberman-Spiegelglas, Rotem Mulayoff, and Tomer Michaeli. arxiv 2023. [PDF]

Diffusion Latent Space Editing

Semantic Editing in Diffusion Latent Spaces.

Self-Discovering Interpretable Diffusion Latent Directions for Responsible Text-to-Image Generation. Hang Li, Chengzhi Shen, Philip Torr, Volker Tresp, Jindong Gu. CVPR 2024. [PDF] [Project] [Code]

NoiseCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions in Diffusion Models. Yusuf Dalva, Pinar Yanardag. CVPR 2024 (Oral). [PDF] [Project]

An Edit Friendly DDPM Noise Space: Inversion and Manipulations. Inbar Huberman-Spiegelglas, Vladimir Kulikov, Tomer Michaeli. CVPR 2024. [PDF] [Code]

Diffusion Models Already Have A Semantic Latent Space. Mingi Kwon, Jaeseok Jeong, Youngjung Uh. ICLR 2023. [PDF]

Unifying Diffusion Models' Latent Space, with Applications to CycleDiffusion and Guidance. Chen Henry Wu, Fernando De la Torre. ICCV 2023. [PDF] [Code]

Generative Visual Prompt: Unifying Distributional Control of Pre-Trained Generative Models. Chen Henry Wu, Saman Motamed, Shaunak Srivastava, Fernando De la Torre. NeurIPS 2022. [PDF]

$P+$: Extended Textual Conditioning in Text-to-Image Generation. Andrey Voynov, Qinghao Chu, Daniel Cohen-Or, Kfir Aberman. arxiv 2023. [PDF] [Project]

Applications

Image and Video Generation and Manipulation

Diffusion-driven GAN Inversion for Multi-Modal Face Image Generation. Jihyun Kim, Changjae Oh, Hoseok Do, Soohyun Kim, Kwanghoon Sohn. CVPR 2024. [PDF] [Project]

The Devil is in the Details: StyleFeatureEditor for Detail-Rich StyleGAN Inversion and High Quality Image Editing. Denis Bobkov, Vadim Titov, Aibek Alanov, Dmitry Vetrov. CVPR 2024. [PDF] [Project]

Robust One-Shot Face Video Re-enactment using Hybrid Latent Spaces of StyleGAN2. Trevine Oorloff, Yaser Yacoob. ICCV 2023. [PDF] [Project]

Expressive Talking Head Video Encoding in StyleGAN2 Latent-Space. Trevine Oorloff, Yaser Yacoob. ICCVW 2023. [PDF] [Project] [Code] [Data]

CLIP-Guided StyleGAN Inversion for Text-Driven Real Image Editing. Ahmet Canberk Baykal, Abdul Basit Anees, Duygu Ceylan, Erkut Erdem, Aykut Erdem, Deniz Yuret. TOG 2023. [PDF]

NeRFInvertor: High Fidelity NeRF-GAN Inversion for Single-shot Real Image Animation. Yu Yin, Kamran Ghasedi, HsiangTao Wu, Jiaolong Yang, Xin Tong, Yun Fu. CVPR 2023. [PDF]

Fine-Grained Face Swapping via Regional GAN Inversion. Zhian Liu, Maomao Li, Yong Zhang, Cairong Wang, Qi Zhang, Jue Wang, Yongwei Nie. CVPR 2023. [PDF] [Project]

VIVE3D: Viewpoint-Independent Video Editing using 3D-Aware GANs. Anna Frühstück, Nikolaos Sarafianos, Yuanlu Xu, Peter Wonka, Tony Tung. CVPR 2023. [PDF] [Project] [Code]

Delving StyleGAN Inversion for Image Editing: A Foundation Latent Space Viewpoint. Hongyu Liu, Yibing Song, Qifeng Chen. CVPR 2023. [PDF] [Project] [Code]

Balancing Reconstruction and Editing Quality of GAN Inversion for Real Image Editing with StyleGAN Prior Latent Space. Kai Katsumata, Duc Minh Vo, Bei Liu, Hideki Nakayama. CVPR 2023 Workshop on AI4CC. [PDF]

Modeling the Latent Dynamics of StyleGAN using Neural ODEs. Weihao Xia, Yujiu Yang, Jing-Hao Xue. NeurIPSW 2023. [PDF] [Code]

Dr.3D: Adapting 3D GANs to Artistic Drawings. Wonjoon Jin, Nuri Ryu, Geonung Kim, Seung-Hwan Baek, Sunghyun Cho. SIGGRAPH Asia 2022. [PDF] [Project]

Stitch it in Time: GAN-Based Facial Editing of Real Videos. Rotem Tzaban, Ron Mokady, Rinon Gal, Amit H. Bermano, Daniel Cohen-Or. SIGGRAPH Asia 2022. [PDF] [Project] [Code]

DyStyle: Dynamic Neural Network for Multi-Attribute-Conditioned Style Editing. Bingchuan Li, Shaofei Cai, Wei Liu, Peng Zhang, Miao Hua, Qian He, Zili Yi. WACV 2023. [PDF] [Code]

Generative Visual Prompt: Unifying Distributional Control of Pre-Trained Generative Models. Chen Henry Wu, Saman Motamed, Shaunak Srivastava, Fernando De la Torre. NeurIPS 2022. [PDF] [Code]

Generalized One-shot Domain Adaption of Generative Adversarial Networks. Zicheng Zhang, Yinglu Liu, Congying Han, Tiande Guo, Ting Yao, Tao Mei. NeurIPS 2022. [PDF] [Code]

3D-FM GAN: Towards 3D-Controllable Face Manipulation. Yuchen Liu, Zhixin Shu, Yijun Li, Zhe Lin, Richard Zhang, and Sun-Yuan Kung. ECCV 2022. [PDF] [Project]

JoJoGAN: One Shot Face Stylization. Min Jin Chong, David Forsyth. ECCV 2022. [PDF] [Code]

Generative Multiplane Images: Making a 2D GAN 3D-Aware. Xiaoming Zhao, Fangchang Ma, David Güera, Zhile Ren, Alexander G. Schwing, Alex Colburn. ECCV 2022. [PDF] [Project] [Code]

Injecting 3D Perception of Controllable NeRF-GAN into StyleGAN for Editable Portrait Image Synthesis. Jeong-gi Kwak, Yuanming Li, Dongsik Yoon, Donghyeon Kim, David Han, Hanseok Ko. ECCV 2022. [PDF] [Project]

Temporally Consistent Semantic Video Editing. Yiran Xu, Badour AlBahar, Jia-Bin Huang. ECCV 2022. [PDF] [Project]

Sound-Guided Semantic Video Generation. Seung Hyun Lee, Gyeongrok Oh, Wonmin Byeon, Jihyun Bae, Chanyoung Kim, Won Jeong Ryoo, Sang Ho Yoon, Jinkyu Kim, Sangpil Kim. ECCV 2022. [PDF] [Project] [Code]

Third Time's the Charm? Image and Video Editing with StyleGAN3. Yuval Alaluf, Or Patashnik, Zongze Wu, Asif Zamir, Eli Shechtman, Dani Lischinski, Daniel Cohen-Or. ECCV 2022 Workshop on Advances in Image Manipulation. [PDF] [Project] [Code]

Everything is There in Latent Space: Attribute Editing and Attribute Style Manipulation by StyleGAN Latent Space Exploration. Rishubh Parihar, Ankit Dhiman, Tejan Karmali, R. Venkatesh Babu. ACM MM 2022. [PDF] [Project]

SD-GAN: Semantic Decomposition for Face Image Synthesis with Discrete Attribute. Zhou Kangneng, Zhu Xiaobin, Gao Daiheng, Lee Kai, Li Xinjie, Yin Xu-Cheng. ACM MM 2022. [PDF]

Identity-Guided Face Generation with Multi-modal Contour Conditions. Qingyan Bai, Weihao Xia, Fei Yin, Yujiu Yang. ICIP 2022. [PDF]

Self-Conditioned Generative Adversarial Networks for Image Editing. Yunzhe Liu, Rinon Gal, Amit H. Bermano, Baoquan Chen, Daniel Cohen-Or. SIGGRAPH 2022. [PDF] [Project]

StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generators. Rinon Gal, Or Patashnik, Haggai Maron, Gal Chechik, Daniel Cohen-Or. SIGGRAPH 2022. [PDF] [Project] [Code]

SphericGAN: Semi-Supervised Hyper-Spherical Generative Adversarial Networks for Fine-Grained Image Synthesis. Tianyi Chen, Yunfei Zhang, Xiaoyang Huo, Si Wu, Yong Xu, Hau San Wong. CVPR 2022. [PDF]

Sound-Guided Semantic Image Manipulation. Seung Hyun Lee, Wonseok Roh, Wonmin Byeon, Sang Ho Yoon, Chan Young Kim, Jinkyu Kim, Sangpil Kim. CVPR 2022. [PDF]

HairCLIP: Design Your Hair by Text and Reference Image. Tianyi Wei, Dongdong Chen, Wenbo Zhou, Jing Liao, Zhentao Tan, Lu Yuan, Weiming Zhang, Nenghai Yu. CVPR 2022. [PDF] [Code]

HairMapper: Removing Hair from Portraits Using GANs. Yiqian Wu, Yong-Liang Yang, Xiaogang Jin. CVPR 2022. [PDF] [Project] [Code] [Non-hair-FFHQ Data]

Attribute Group Editing for Reliable Few-shot Image Generation. Guanqi Ding, Xinzhe Han, Shuhui Wang, Shuzhe Wu, Xin Jin, Dandan Tu, Qingming Huang. CVPR 2022. [PDF] [Code]

InsetGAN for Full-Body Image Generation. Anna Frühstück, Krishna Kumar Singh, Eli Shechtman, Niloy J. Mitra, Peter Wonka, Jingwan Lu. CVPR 2022. [PDF] [Project] [Unofficial]

SpaceEdit: Learning a Unified Editing Space for Open-Domain Image Editing. Jing Shi, Ning Xu, Haitian Zheng, Alex Smith, Jiebo Luo, Chenliang Xu. CVPR 2022. [PDF]

In&Out: Diverse Image Outpainting via GAN Inversion. Yen-Chi Cheng, Chieh Hubert Lin, Hsin-Ying Lee, Jian Ren, Sergey Tulyakov, Ming-Hsuan Yang. CVPR 2022. [PDF] [Code]

InfinityGAN: Towards Infinite-Resolution Image Synthesis. Chieh Hubert Lin, Hsin-Ying Lee, Yen-Chi Cheng, Sergey Tulyakov, Ming-Hsuan Yang. ICLR 2022. [PDF] [Project]

Latent to Latent: A Learned Mapper for Identity Preserving Editing of Multiple. Siavash Khodadadeh, Shabnam Ghadar, Saeid Motiian, Wei-An Lin, Ladislau Bölöni, Ratheesh Kalarot. WACV 2022. [PDF]

StyleVideoGAN: A Temporal Generative Model using a Pretrained StyleGAN. Gereon Fox, Ayush Tewari, Mohamed Elgharib, Christian Theobalt. BMVC 2021 (Oral). [PDF]

Face Image Retrieval With Attribute Manipulation. Alireza Zaeemzadeh, Shabnam Ghadar, Baldo Faieta, Zhe Lin, Nazanin Rahnavard, Mubarak Shah, Ratheesh Kalarot. ICCV 2021. [PDF]

StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation. Wongjong Jang, Gwangjin Ju, Yucheol Jung, Jiaolong Yang, Xin Tong, Seungyong Lee. TOG 2021. [PDF] [Code]

Coarse-to-Fine: Facial Structure Editing of Portrait Images via Latent Space Classifications. Yiqian Wu, Yongliang Yang, Qinjie Xiao, Xiaogang Ji. TOG 2021. [PDF] [Project]

SAM: Only a Matter of Style-Age Transformation Using a Style-Based Regression Model. Yuval Alaluf, Or Patashnik, Daniel Cohen-Or. TOG 2021. [PDF] [Code]

Barbershop: GAN-based Image Compositing using Segmentation Masks. Peihao Zhu, Rameen Abdal, John Femiani, Peter Wonka. SIGGRAPH Asia 2021. [PDF] [Project] [Code]

Constrained Graphic Layout Generation via Latent Optimization. Kotaro Kikuchi, Edgar Simo-Serra, Mayu Otani, Kota Yamaguchi. ACM MM 2021. [PDF] [Code]

CI-GAN: Cycle-Consistent Inverse GAN for Text-to-Image Synthesis. Hao Wang, Guosheng Lin, Steven C. H. Hoi, Chunyan Miao. ACM MM 2021. [PDF]

Exploring Adversarial Fake Images on Face Manifold. Dongze Li, Wei Wang, Hongxing Fan, Jing Dong. CVPR 2021. [PDF]

HistoGAN: Controlling Colors of GAN-Generated and Real Images via Color Histograms. Mahmoud Afifi, Marcus A. Brubaker, Michael S. Brown. CVPR 2021. [PDF] [Code] [4K Landscape]

One Shot Face Swapping on Megapixels. Yuhao Zhu, Qi Li, Jian Wang, Chengzhong Xu, Zhenan Sun. CVPR 2021. [PDF] [Code]

LOHO: Latent Optimization of Hairstyles via Orthogonalization. Rohit Saha, Brendan Duke, Florian Shkurti, Graham W. Taylor, Parham Aarabi. CVPR 2021. [PDF] [Code]

StyleMapGAN: Exploiting Spatial Dimensions of Latent in GAN for Real-time Image Editing. Hyunsu Kim, Yunjey Choi, Junho Kim, Sungjoo Yoo, Youngjung Uh. CVPR 2021. [PDF] [Code]

TediGAN: Text-Guided Diverse Image Generation and Manipulation. Weihao Xia, Yujiu Yang, Jing-Hao Xue, Baoyuan Wu. CVPR 2021. [PDF] [Data] [Code]

DeepI2I: Enabling Deep Hierarchical Image-to-Image Translation by Transferring from GANs. yaxing wang, Lu Yu, Joost van de Weijer. NeurIPS 2020. [PDF] [Code]

DeepLandscape: Adversarial Modeling of Landscape Videos. E. Logacheva, R. Suvorov, O. Khomenko, A. Mashikhin, and V. Lempitsky. ECCV 2020. [PDF] [Code] [Project]

Few-shot Semantic Image Synthesis Using StyleGAN Prior. Yuki Endo, Yoshihiro Kanamori. arxiv 2021. [PDF] [Code]

Paint by Word. David Bau, Alex Andonian, Audrey Cui, YeonHwan Park, Ali Jahanian, Aude Oliva, Antonio Torralba. arxiv 2021. [PDF] [Project] [Project]

Image Restoration

LTT-GAN: Looking Through Turbulence by Inverting GANs. Kangfu Mei, Vishal M. Patel. J-STSP 2023. [PDF] [Project]

Semantic Uncertainty Intervals for Disentangled Latent Spaces. Swami Sankaranarayanan, Anastasios N. Angelopoulos, Stephen Bates, Yaniv Romano, Phillip Isola. NeurIPS 2022. [PDF] [Code]

High-Fidelity Image Inpainting with GAN Inversion. Yongsheng Yu, Libo Zhang, Heng Fan, Tiejian Luo. ECCV 2022. [PDF]

Escaping Data Scarcity for High-Resolution Heterogeneous Face Hallucination. Yiqun Mei, Pengfei Guo, Vishal M. Patel. CVPR 2022. [PDF]

Towards High-Fidelity Face Self-Occlusion Recovery via Multi-View Residual-Based GAN Inversion. Jinsong Chen, Hu Han, Shiguang Shan. AAAI 2022. [PDF]

Time-Travel Rephotography. Xuan Luo, Xuaner Zhang, Paul Yoo, Ricardo Martin-Brualla, Jason Lawrence, Steven M. Seitz. SIGGRAPH Asia 2021 (TOG). [PDF] [Project] [Code]

GPEN: GAN Prior Embedded Network for Blind Face Restoration in the Wild. Tao Yang, Peiran Ren, Xuansong Xie, Lei Zhang. CVPR 2021. [PDF] [Code]

GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution. Kelvin C.K. Chan, Xintao Wang, Xiangyu Xu, Jinwei Gu, Chen Change Loy. CVPR 2021. [PDF] [Project] [Code]

GFP-GAN: Towards Real-World Blind Face Restoration with Generative Facial Prior. Xintao Wang, Yu Li, Honglun Zhang, Ying Shan. CVPR 2021. [PDF] [Project]

PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models. Sachit Menon, Alexandru Damian, Shijia Hu, Nikhil Ravi, Cynthia Rudin. CVPR 2020. [PDF] [Code]

Style Generator Inversion for Image Enhancement and Animation. Aviv Gabbay, Yedid Hoshen. arxiv 2019. [PDF] [Project] [Code]

Image Understanding

Finding an Unsupervised Image Segmenter in each of your Deep Generative Models. Luke Melas-Kyriazi, Christian Rupprecht, Iro Laina, Andrea Vedaldi. ICLR 2022. [PDF]

Labels4Free: Unsupervised Segmentation using StyleGAN. Rameen Abdal, Peihao Zhu, Niloy Mitra, Peter Wonka. ICCV 2021. [PDF] [Project] [Code]

DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort. Yuxuan Zhang, Huan Ling, Jun Gao, Kangxue Yin, Jean-Francois Lafleche, Adela Barriuso, Antonio Torralba, Sanja Fidler. CVPR 2021. [PDF] [Code] [Project]

Repurposing GANs for One-shot Semantic Part Segmentation. Nontawat Tritrong, Pitchaporn Rewatbowornwong, Supasorn Suwajanakorn. CVPR 2021 (oral). [PDF] [Project] [Code]

Segmentation in Style: Unsupervised Semantic Image Segmentation with Stylegan and CLIP. Daniil Pakhomov, Sanchit Hira, Narayani Wagle, Kemar E. Green, Nassir Navab. arxiv 2021. [PDF] [Project] [Code]

Face Recognition

How to Boost Face Recognition with StyleGAN? Artem Sevastopolsky, Yury Malkov, Nikita Durasov, Luisa Verdoliva, Matthias Nießner. ICCV 2023. [PDF] [Code]

3D Reconstruction

StyleGAN Knows Normal, Depth, Albedo, and More. Anand Bhattad, Daniel McKee, Derek Hoiem, D. A. Forsyth. NeurIPS 2023. [PDF]

Shape, Pose, and Appearance from a Single Image via Bootstrapped Radiance Field Inversion. Dario Pavllo, David Joseph Tan, Marie-Julie Rakotosaona, Federico Tombari. CVPR 2023. [PDF] [Code]

Monocular 3D Object Reconstruction with GAN Inversion. Junzhe Zhang, Daxuan Ren, Zhongang Cai, Chai Kiat Yeo, Bo Dai, Chen Change Loy. ECCV 2022. [PDF] [Project] [Code]

CoordGAN: Self-Supervised Dense Correspondences Emerge from GANs. Jiteng Mu, Shalini De Mello, Zhiding Yu, Nuno Vasconcelos, Xiaolong Wang, Jan Kautz, Sifei Liu. CVPR 2022. [PDF] [Project]

Normalized Avatar Synthesis Using StyleGAN and Perceptual Refinement. Huiwen Luo, Koki Nagano, Han-Wei Kung, Mclean Goldwhite, Qingguo Xu, Zejian Wang, Lingyu Wei, Liwen Hu, Hao Li. CVPR 2021. [PDF]

Unsupervised 3D Shape Completion through GAN-Inversion. Junzhe Zhang, Xinyi Chen, Zhongang Cai, Liang Pan, Haiyu Zhao, Shuai Yi, Chai Kiat Yeo, Bo Dai, Chen Change Loy. CVPR 2021. [PDF] [Project]

OSTeC: One-Shot Texture Completion. Baris Gecer, Jiankang Deng, Stefanos Zafeiriou. CVPR 2021. [PDF] [Code]

GAN2Shape: Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANs. Xingang Pan, Bo Dai, Ziwei Liu, Chen Change Loy, Ping Luo. ICLR 2021 (oral). [PDF] [Code] [Project]

Other Applications

Compressed Sensing

Generator Surgery for Compressed Sensing. Niklas Smedemark-Margulies, Jung Yeon Park, Max Daniels, Rose Yu, Jan-Willem van de Meent, Paul Hand. NeurIPS 2020 Workshop on Deep Inverse. [PDF] [Code]

Task-Aware Compressed Sensing with Generative Adversarial Networks. Maya Kabkab, Pouya Samangouei, Rama Chellappa. AAAI 2018. [PDF]

Medical Imaging

Controllable Medical Image Generation via Generative Adversarial Networks. Zhihang Ren, Stella X. Yu, David Whitney. Human Vision and Electronic Imaging 2021. [PDF]

High-resolution Controllable Prostatic Histology Synthesis using StyleGAN. Gagandeep B. Daroach, Josiah A. Yoder, Kenneth A. Iczkowski, Peter S. LaViolette. Bioimaging 2021. [PDF]

Compression, Fairness, and Security

FairStyle: Debiasing StyleGAN2 with Style Channel Manipulations. Cemre Karakas, Alara Dirik, Eylul Yalcinkaya, Pinar Yanardag. ECCV 2022. [PDF] [Project] [Code]

Video Coding Using Learned Latent GAN Compression. Mustafa Shukor, Bharath Bhushan Damodaran, Xu Yao, Pierre Hellier. ACM MM 2022. [PDF]

Differentially Private Imaging via Latent Space Manipulation. Tao Li, Chris Clifton. IEEE Symposium on Security & Privacy (S&P) 2021. [PDF]

Acknowledgement

Thanks for the constructive comments from anonymous reviewers and feedback from Jun-Yan Zhu, Andrey Voynov, and Rushil Anirudh.

If you find this repo or our paper is helpful for your research, please consider to cite:

@article{xia2022gan,
    author  = {Xia, Weihao and Zhang, Yulun and Yang, Yujiu and Xue, Jing-Hao and Zhou, Bolei and Yang, Ming-Hsuan},
    title   = {GAN Inversion: A Survey},
    journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
    year={2022}
  }