Home

Awesome

License CC BY-NC-SA 4.0 Python 3.6 Packagist Last Commit Maintenance Contributing Ask Me Anything !

AttentionGAN-v2 for Unpaired Image-to-Image Translation

AttentionGAN-v2 Framework

The proposed generator learns both foreground and background attentions. It uses the foreground attention to select from the generated output for the foreground regions, while uses the background attention to maintain the background information from the input image. Please refer to our papers for more details.

Framework

Comparsion with State-of-the-Art Methods

Selfie To Anime Translation

Result

Horse to Zebra Translation

Result <br> Result

Zebra to Horse Translation

Result

Apple to Orange Translation

Result

Orange to Apple Translation

Result

Map to Aerial Photo Translation

Result

Aerial Photo to Map Translation

Result

Style Transfer

Result

Visualization of Learned Attention Masks

Selfie to Anime Translation

Result

Horse to Zebra Translation

Attention

Zebra to Horse Translation

Attention

Apple to Orange Translation

Attention

Orange to Apple Translation

Attention

Map to Aerial Photo Translation

Attention

Aerial Photo to Map Translation

Attention

Extended Paper | Conference Paper

AttentionGAN: Unpaired Image-to-Image Translation using Attention-Guided Generative Adversarial Networks.<br> Hao Tang<sup>1</sup>, Hong Liu<sup>2</sup>, Dan Xu<sup>3</sup>, Philip H.S. Torr<sup>3</sup> and Nicu Sebe<sup>1</sup>. <br> <sup>1</sup>University of Trento, Italy, <sup>2</sup>Peking University, China, <sup>3</sup>University of Oxford, UK.<br> In TNNLS 2021 & IJCNN 2019 Oral. <br> The repository offers the official implementation of our paper in PyTorch.

Are you looking for AttentionGAN-v1 for Unpaired Image-to-Image Translation?

Paper | Code

Are you looking for AttentionGAN-v1 for Multi-Domain Image-to-Image Translation?

Paper | Code

Facial Expression-to-Expression Translation

Result Order: The Learned Attention Masks, The Learned Content Masks, Final Results

Facial Attribute Transfer

Attention Order: The Learned Attention Masks, The Learned Content Masks, Final Results

Result Order: The Learned Attention Masks, AttentionGAN, StarGAN

License

<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /> Copyright (C) 2019 University of Trento, Italy.

All rights reserved. Licensed under the CC BY-NC-SA 4.0 (Attribution-NonCommercial-ShareAlike 4.0 International)

The code is released for academic research use only. For commercial use, please contact bjdxtanghao@gmail.com.

Installation

Clone this repo.

git clone https://github.com/Ha0Tang/AttentionGAN
cd AttentionGAN/

This code requires PyTorch 0.4.1+ and python 3.6.9+. Please install dependencies by

pip install -r requirements.txt (for pip users)

or

./scripts/conda_deps.sh (for Conda users)

To reproduce the results reported in the paper, you would need an NVIDIA Tesla V100 with 16G memory.

Dataset Preparation

Download the datasets using the following script. Please cite their paper if you use the data. Try twice if it fails the first time!

sh ./datasets/download_cyclegan_dataset.sh dataset_name

The selfie2anime dataset can be download here.

AttentionGAN Training/Testing

sh ./scripts/train_attentiongan.sh
sh ./scripts/test_attentiongan.sh

Generating Images Using Pretrained Model

sh ./scripts/download_attentiongan_model.sh horse2zebra
python test.py --dataroot ./datasets/horse2zebra --name horse2zebra_pretrained --model attention_gan --dataset_mode unaligned --norm instance --phase test --no_dropout --load_size 256 --crop_size 256 --batch_size 1 --gpu_ids 0 --num_test 5000 --epoch latest --saveDisk

The results will be saved at ./results/. Use --results_dir {directory_path_to_save_result} to specify the results directory. Note that if you want to save the intermediate results and have enough disk space, remove --saveDisk on the command line.

Image Translation with Geometric Changes Between Source and Target Domains

For instance, if you want to run experiments of Selfie to Anime Translation. Usage: replace attention_gan_model.py and networks with the ones in the AttentionGAN-geo folder.

Test the Pretrained Model

Download data and pretrained model according above instructions.

python test.py --dataroot ./datasets/selfie2anime/ --name selfie2anime_pretrained --model attention_gan --dataset_mode unaligned --norm instance --phase test --no_dropout --load_size 256 --crop_size 256 --batch_size 1 --gpu_ids 0 --num_test 5000 --epoch latest

Train a New Model

python train.py --dataroot ./datasets/selfie2anime/ --name selfie2anime_attentiongan --model attention_gan --dataset_mode unaligned --pool_size 50 --no_dropout --norm instance --lambda_A 10 --lambda_B 10 --lambda_identity 0.5 --load_size 286 --crop_size 256 --batch_size 4 --niter 100 --niter_decay 100 --gpu_ids 0 --display_id 0 --display_freq 100 --print_freq 100

Test the Trained Model

python test.py --dataroot ./datasets/selfie2anime/ --name selfie2anime_attentiongan --model attention_gan --dataset_mode unaligned --norm instance --phase test --no_dropout --load_size 256 --crop_size 256 --batch_size 1 --gpu_ids 0 --num_test 5000 --epoch latest

Evaluation Code

Citation

If you use this code for your research, please cite our papers.

@article{tang2021attentiongan,
  title={AttentionGAN: Unpaired Image-to-Image Translation using Attention-Guided Generative Adversarial Networks},
  author={Tang, Hao and Liu, Hong and Xu, Dan and Torr, Philip HS and Sebe, Nicu},
  journal={IEEE Transactions on Neural Networks and Learning Systems (TNNLS)},
  year={2021} 
}

@inproceedings{tang2019attention,
  title={Attention-Guided Generative Adversarial Networks for Unsupervised Image-to-Image Translation},
  author={Tang, Hao and Xu, Dan and Sebe, Nicu and Yan, Yan},
  booktitle={International Joint Conference on Neural Networks (IJCNN)},
  year={2019}
}

Acknowledgments

This source code is inspired by CycleGAN, GestureGAN, and SelectionGAN.

Contributions

If you have any questions/comments/bug reports, feel free to open a github issue or pull a request or e-mail to the author Hao Tang (bjdxtanghao@gmail.com).

Collaborations

I'm always interested in meeting new people and hearing about potential collaborations. If you'd like to work together or get in contact with me, please email bjdxtanghao@gmail.com. Some of our projects are listed here.


Figure out what you like. Try to become the best in the world of it.