Home

Awesome

Adaptive Affine Transformation: A Simple and Effective Operation for Spatial Misaligned Image Generation. (accepted in MM2022)

在这里插入图片描述

Paper Demo video Supplementary materials

Face Reenactment & Talking Face

The 3DMM model can not be released, we will replace it with FLAME in the future.

Face visually dubbing

The source code is in here.

Person image generation

Download resources (pretrained model etc.)

Download resources in Google drive, including:

Train on deep fashion dataset

  1. Download deep fashion dataset from here. We use the dataset as same as in SelectionGAN.
  2. Unzip the dataset.
  3. run
python train_person_image.py --train_data=./assert/fasion_train_data.json --train_img_dir=./deepFashion/fashion_data/train

Inference

To inference one person image from one source person image, source key points and target key points, run

python inference_person_image.py --inference_model_path=./assert/person_epoch_30.pth --source_img_path=./assert/example_person_source_img.jpg --source_kp_path=./assert/example_person_souce_kp.txt --target_kp_path=./assert/example_person_target_kp.txt --res_person_path=./assert/example_person_inference_img.jpg

Compute metrics

To compute the metrics of SSIM and LIPIS on deep fashion test data, run

python compute_metrics.py --inference_img_dir --real_img_dir=./deepFashion/fashion_data/test --task_type=person

Citation

If you use AdaAT operator in your work, please cite

@inproceedings{zhang2022adaptive,
  title={Adaptive Affine Transformation: A Simple and Effective Operation for Spatial Misaligned Image Generation},
  author={Zhang, Zhimeng and Ding, Yu},
  booktitle={Proceedings of the 30th ACM International Conference on Multimedia},
  pages={1167--1176},
  year={2022}
}

Acknowledgement

The basic modules are borrowed from first-order-model, thanks for their contributions.