Home

Awesome

InsGen - Data-Efficient Instance Generation from Instance Discrimination

image

Data-Efficient Instance Generation from Instance Discrimination <br> Ceyuan Yang, Yujun Shen, Yinghao Xu, Bolei Zhou <br> arXiv preprint arXiv: 2106.04566

[Paper] [Project Page]

In this work, we develop a novel data-efficient Instance Generation (InsGen) method for training GANs with limited data. With the instance discrimination as an auxiliary task, our method makes the best use of both real and fake images to train the discriminator. The discriminator in turn guides the generator to synthesize as many diverse images as possible. Experiments under different data regimes show that InsGen brings a substantial improvement over the baseline in terms of both image quality and image diversity, and outperforms previous data augmentation algorithms by a large margin.

Qualitative results

Here we provide some synthesized samples with different numbers of training images and correspoding FID. Full codebase and weights are coming soon. image

Inference

Here, all pretrained models can be downloaded from Google Drive:

ModelFIDLink
AFHQ512-CAT2.60link
AFHQ512-DOG5.44link
AFHQ512-WILD1.77link
ModelFIDLink
FFHQ256-2K11.92link
FFHQ256-10K4.90link
FFHQ256-140K3.31link

You can download one of them and put it under MODEL_ZOO directory, then synthesize images via

# Generate AFHQ512-CAT with truncation.
python generate.py --network=${MODEL_ZOO}/afhqcat.pkl \
                   --outdir=${TARGET_DIR} \
                   --trunc=0.7 \
                   --seeds=0-10

Training

This repository is built based on styleGAN2-ada-pytorch. Therefore, please prepare datasets first use dataset_tool.py. On top of Generative Adversarial Networks (GANs), we introduce contrastive loss into the training of discriminator, following MoCo. Concretely, the discriminator is used to extract features from images (either real or synthesized) and then trained with an auxiliary task by distinguishing every individual image.

As described in training/contrastive_head.py, we add two addition heads on top of the original discriminator. These two heads are used to project features extracted from real and fake data onto a unit ball respectively. More details can be found in paper. Note that InsGen can be easily applied to any GAN model by merely introducing two contrastive heads. According to MoCo, the feature extractor should be updated in a momentum manner. Here, in InsGen, the contrastive heads are updated in the forward() function, while the discriminator is updated in training/training_loop.py (see D_ema).

Please use the following command to start your own training:

python train.py --gpus=8 \
                --data=${DATA_PATH} \
                --cfg=paper256 \
                --outdir=training_example

In this example, the results are saved to a created director training_example. --cfg specifies the training configuration, which can be further customized with additional options:

More functions would be supported after this projest is merged into our genforce. Please stay tuned!

License

This work is made available under the Nvidia Source Code License.

Acknowledgements

We thank Janne Hellsten and Tero Karras for the pytorch version codebase of their styleGAN2-ada-pytorch.

BibTeX

@article{yang2021insgen,
  title   = {Data-Efficient Instance Generation from Instance Discrimination},
  author  = {Yang, Ceyuan and Shen, Yujun and Xu, Yinghao and Zhou, Bolei},
  journal = {arXiv preprint arXiv:2106.04566},
  year    = {2021}
}