Awesome
Anime Sketch Coloring with Swish-Gated Residual U-Net
Paper authors: Gang Liu, Xin Chen, Yanzhong Hu
Implementation authors: Alexander Koumis, Amlesh Sivanantham, Pradeep Lam, Georgio Pizzorni
This is an implementation of the paper Anime Sketch Coloring with Swish-Gated Residual U-Net, which uses uses deep learning to colorize anime line-art (sketches).
BIG shout out to the paper authors Xin Chen and Gang Liu for helping us with implementation details that we initially got wrong.
Setup
Use the requirements.txt
file to install the necessary depedencies for this
project.
$ pip install -r requirements.txt
Training the SGRU Model
Before proceeding, make sure the data folder has the following structure.
data/
├── images/
│ ├── images_bw/
│ └── images_rgb/
└── vgg_19.ckpt
You can populate the image directories using using your own dataset (see utility script scripts/process_dir.py
for creating sketch/RGB pairs).
You can find the pretrained vgg_19.ckpt
checkpoint file here.
Once you have placed the dataset and VGG checkpoint in your data directory, start the training procedure using the src/train.py
script:
$ ./train.py ${DATA_DIR} ${OUTPUT_DIR}
Run ./train.py --help
to see the options available for changing hyperparamenters from the command line.
The output directory will have the following structure
${OUTPUT_DIR}/
└── ${EXP_NAME:-TIME_STAMP}/
├── images/
├── events.out.tfevents (tensorboard)
└── model.ckpt
Evaluting pre-trained model
To evaluate a trained model, you must train the weights yourself using the above procedure, or use our pretrained weights available here. Place your self-trained model.ckpt.*
files inside of a directory $CKPT_DIR
or unzip our provided checkpoint.zip
file into a directory $CKPT_DIR
.
After you have placed the weights checkpoint files in $CKPT_DIR
, you can use the script src/evaluate.py
to see the results on your own image (written as sketch_image.jpg
below). The best results occur when the input image is 256x256 (similar to the dataset the model was trained on). To display the results, run the script like this:
./evaluate.py sketch_image.jpg $CKPT_DIR/model.ckpt --show
To save the results of into a directory, run the script with the --output-dir
argument:
./evaluate.py sketch_image.jpg $CKPT_DIR/model.ckpt --output-dir $OUTPUT_DIR
To show and save the results, use both the --show
and --output-dir
arguments.
Results
If your model trains correctly it should generate results like this: In the above image, the first column is the input sketch, and the rest are generated by the network given the sketch.