Awesome

Alice4Alice

ALICE algorithms for painting the cartoon "Alice's Adventures in Wonderland"

Ideas: Leveraging weakly-supervised information can significantly alleviate the non-identifiablility issues of CycleGAN/DiscoGAN/DualGAN in image-to-image translation.

If interested, please check out the main code repo for our NIPS 2017 paper ALICE:

Note: These are results based on noisy edge inputs

Experiments

For 52 images (21 alice image and 31 rabbit images) in each domain (cartoon and edge), we provide 1 pairwise correspondence for each character, which significantly improves the generation quality.

Results on sample correspondence

ALICE (Explicit variant)
CycleGAN

As references:

Real Cartoon (Groundtruth)
Edges (Input)

Detailed Comparison:

Rabbit

Alice

Cat (remotely related to the training set)

How to use

The scripts to train and test with various algorithms are in alice_tesorflow/script.sh. For example, training with explicit ALICE:

 $ CUDA_VISIBLE_DEVICES=0 python main.py --checkpoint_dir ./alice_exp_checkpoint --sample_dir=./alice_exp_sample --test_dir=./alice_exp_test_dir --L1_lambda_sup 10.0 --cgan_lambda 0.0

Creat a dataset with two domains

Convert video to images.

For example: https://image.online-convert.com/convert-to-jpg

You might be ineterested in youtube video: https://keepvid.com/sites/download-youtube-video.html
Resize your images, and extract edges from the images.

For example: https://github.com/ppwwyyxx/tensorpack/tree/master/examples/HED

Our development is largely based on the code (greatly appreciated!): https://github.com/xhujoy/CycleGAN-tensorflow

Note: these are only for academic use, please ask the providers for commercial purposes.

Citation

If you use this code for your research, please cite our paper:

@article{li2017alice,
  title={ALICE: Towards Understanding Adversarial Learning for Joint Distribution Matching},
  author={Li, Chunyuan and Liu, Hao and Chen, Changyou and Pu, Yunchen and Chen, Liqun and Henao, Ricardo and Carin, Lawrence},
  journal={Neural Information Processing Systems (NIPS)},
  year={2017}
}