Home

Awesome

DAE-GAN

Pytorch implementation for reproducing DAE-GAN results in the paper [DAE-GAN: Dynamic Aspect-aware GAN for Text-to-Image Synthesis] by Shulan Ruan, Yong Zhang, Kun Zhang, Yanbo Fan, Fan Tang, Qi Liu, Enhong Chen. (This work was performed when Ruan was an intern with Tencent AI Lab).

<img src="framework.png" width="800px" height="250px"/>

Dependencies

python 3.6

Pytorch

In addition, please add the project folder to PYTHONPATH and pip install the following packages:

Data

  1. Download our preprocessed metadata for birds coco, name them as captions.pickle and save them to data/birds and data/coco
  2. Download the birds image data. Extract them to data/birds/
  3. Download coco dataset and extract the images to data/coco/

Training

Pretrained Model

Validation

Examples generated by DAE-GAN

<!-- bird example | coco example :-------------------------:|:-------------------------: ![] --> <img src="comparison.png" width="800px" height="300px"/>

Citing DAE-GAN

If you find DAE-GAN useful in your research, please consider citing:

@inproceedings{ruan2021dae,
  title={DAE-GAN: Dynamic Aspect-aware GAN for Text-to-Image Synthesis},
  author={Ruan, Shulan and Zhang, Yong and Zhang, Kun and Fan, Yanbo and Tang, Fan and Liu, Qi and Chen, Enhong},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={13960--13969},
  year={2021}
}
<!-- ``` @article{Tao18attngan, author = {Tao Xu, Pengchuan Zhang, Qiuyuan Huang, Han Zhang, Zhe Gan, Xiaolei Huang, Xiaodong He}, title = {AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks}, Year = {2018}, booktitle = {{CVPR}} } ``` -->

Reference