Home

Awesome

MC-GAN in PyTorch

<img src="https://people.eecs.berkeley.edu/~sazadi/MCGAN/datasets/ft51_1_fake_B.gif" width="90%"/>

This is the implementation of the Multi-Content GAN for Few-Shot Font Style Transfer. The code was written by Samaneh Azadi. If you use this code or our collected font dataset for your research, please cite:

Multi-Content GAN for Few-Shot Font Style Transfer; Samaneh Azadi, Matthew Fisher, Vladimir Kim, Zhaowen Wang, Eli Shechtman, Trevor Darrell, in arXiv, 2017.

Prerequisites:

Getting Started

Installation

git clone https://github.com/pytorch/vision
cd vision
python setup.py install
pip install visdom
pip install dominate
pip install scikit-image
mkdir FontTransfer
cd FontTransfer
git clone https://github.com/azadis/MC-GAN
cd MC-GAN

MC-GAN train/test

./datasets/download_font_dataset.sh Capitals64

../datasets/Capitals64/test_dict/dict.pkl makes observed random glyphs be similar at different test runs on Capitals64 dataset. It is a dictionary with font names as keys and random arrays containing indices from 0 to 26 as their values. Lengths of the arrays are equal to the number of non-observed glyphs in each font.

../datasets/Capitals64/BASE/Code New Roman.0.0.png is a fixed simple font used for training the conditional GAN in the End-to-End model.

./datasets/download_font_dataset.sh public_web_fonts

Given a few letters of font ${DATA} for examples 5 letters {T,O,W,E,R}, training directory ${DATA}/A should contain 5 images each with dimension 64x(64x26)x3 where 5 - 1 = 4 letters are given and the rest are zeroed out. Each image should be saved as ${DATA}_${IND}.png where ${IND} is the index (in [0,26) ) of the letter omitted from the observed set. Training directory ${DATA}/B contains images each with dimension 64x64x3 where only the omitted letter is given. Image names are similar to the ones in ${DATA}/A though. ${DATA}/A/test/${DATA}.png contains all 5 given letters as a 64x(64x26)x3-dimensional image. Structure of the directories for above real-world fonts (including only a few observed letters) is as follows. One can refer to the examples in ../datasets/public_web_fonts for more information.

../datasets/public_web_fonts
                      └── ${DATA}/
                          ├── A/
                          │  ├──train/${DATA}_${IND}.png
                          │  └──test/${DATA}.png
                          └── B/
                             ├──train/${DATA}_${IND}.png
                             └──test/${DATA}.png
./datasets/download_font_dataset.sh Capitals_colorGrad64
./scripts/train_cGAN.sh Capitals64

Model parameters will be saved under ./checkpoints/GlyphNet_pretrain.

./scripts/test_cGAN.sh Capitals64
cd ./results/GlyphNet_pretrain/test_400/

If you are running the code in your local machine, open index.html. If you are running remotely via ssh, on your remote machine run:

python -m SimpleHTTPServer 8881

Then on your local machine, start an SSH tunnel: ssh -N -f -L localhost:8881:localhost:8881 remote_user@remote_host Now open your browser on the local machine and type in the address bar:

localhost:8881
python util/plot_loss.py --logRoot ./checkpoints/GlyphNet_pretrain/
./pretrained_models/download_cGAN_models.sh

Now, you can train the full model:

./scripts/train_StackGAN.sh ${DATA}
./scripts/test_StackGAN.sh ${DATA}

results will be saved under ./results/${DATA}_MCGAN_train.

First, train your model and save model weights in every epoch by setting opt.save_epoch_freq=1 in scripts/train_StackGAN.sh. Then test in different epochs and make the video by:

./scripts/make_video.sh ${DATA}

Follow the previous steps to visualize generated images and training curves where you replace GlyphNet_train with ${DATA}_StackGAN_train.

Training/test Details

Citation

If you use this code or the provided dataset for your research, please cite our paper:

@inproceedings{azadi2018multi,
  title={Multi-content gan for few-shot font style transfer},
  author={Azadi, Samaneh and Fisher, Matthew and Kim, Vladimir and Wang, Zhaowen and Shechtman, Eli and Darrell, Trevor},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  volume={11},
  pages={13},
  year={2018}
}

Acknowledgements

We thank Elena Sizikova for downloading all fonts used in the 10K font data set.

Code is inspired by pytorch-CycleGAN-and-pix2pix.