Awesome

Few-shot Image Generation with Mixup-based Distance Learning [ECCV 2022]

Chaerin Kong, Jeesoo Kim, Donghoon Han, Nojun Kwak <br> <br> Producing diverse and realistic images with generative models such as GANs typically requires large scale training with vast amount of images. GANs trained with limited data can easily memorize few training samples and display undesirable properties like "stairlike" latent space where interpolation in the latent space yields discontinuous transitions in the output space. In this work, we consider a challenging task of pretraining-free few-shot image synthesis, and seek to train existing generative models with minimal overfitting and mode collapse. We propose mixup-based distance regularization on the feature space of both a generator and the counterpart discriminator that encourages the two players to reason not only about the scarce observed data points but the relative distances in the feature space they reside. Qualitative and quantitative evaluation on diverse datasets demonstrates that our method is generally applicable to existing models to enhance both fidelity and diversity under few-shot setting. Codes are available.

<a href="https://arxiv.org/abs/2111.11672"><img src="https://img.shields.io/badge/arXiv-2111.11672-b31b1b.svg"></a> <a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-yellow.svg"></a> <a href="https://pytorch.org/"><img src="https://img.shields.io/badge/PyTorch->=1.9.0-Red?logo=pytorch"></a>

Getting Started

Requirements

1. This code base heavily relies on pytorch implementation of StyleGAN2. We have mainly tested on

- python=3.6.8
- torch=1.9.0
- CUDA=10.2

but we found it robust to minor version mismatches. (e.g., CUDA version upto 11.2)

2. For FastGAN implementation and dependencies, refer to their official code base. We used the same aforementioned environment settings.

Datasets

1. The datasets used in our paper is accessible through the following links. (Big thanks to the authors of FastGAN and CDC).

2. Prepare your dataset as lmdb with prepare_data.py.

$ python3 prepare_data.py --out PATH/TO/LMDB PATH/TO/IMAGE_FOLDER

3. You can freely try custom datasets. Gather a handful of images (perhaps from the web), put them in a folder (i.e., IMAGE_FOLDER), and run the above code.

Training

$ python3 train_mixdl.py

Sampling

$ python3 generate.py

Evaluation

We use pytorch implementation of Frechet Inception Score.

$ python3 -m pytorch_fid PATH/TO/REAL_IMAGES PATH/TO/FAKE_IMAGES