Awesome
Stylenet
This is a Tensorflow project of neural network with the style synthesis algorithm modified from 2 major methods Gram Matrix and Markov Random Fields. By given a content and style image, the style and pattern can be synthesised into the content. This project also support region mapping. We have added several modifications in the Markov Random Fields cost functions. See belows for the detail.
The visual network is make use of the Tensorflow VGG19 network (Original Caffe implementation is in here and here).
Here are some sample result generated by this algorithm.
Basic synthesis
<table> <tr> <td><img src="https://github.com/machrisaa/stylenet/blob/master/images/cat-water-colour.jpg"/></td> <td><img src="https://github.com/machrisaa/stylenet/blob/master/images/cat_h.jpg"/></td> <td><img src="https://github.com/machrisaa/stylenet/blob/master/images/cat-result.jpeg"/></td> </tr> <tr> <td align='center'>Content</td> <td align='center'>Style</td> <td align='center'>Result</td> </tr> </table> >See the intermediate results in [this video](https://youtu.be/4ssJyLivbBM) <br/> Synthesis with region mapping <table> <tr> <td><img src="https://github.com/machrisaa/stylenet/blob/master/images/husky_paint.jpg"/></td> <td><img src="https://github.com/machrisaa/stylenet/blob/master/images/husky_real.jpg"/></td> <td rowspan=3><img src="https://github.com/machrisaa/stylenet/blob/master/images/husky-result.jpg"/></td> </tr> <tr> <td align='center'>Content</td> <td align='center'>Style</td> </tr> <tr> <td><img src="https://github.com/machrisaa/stylenet/blob/master/images/husky_paint_region.jpg"/></td> <td><img src="https://github.com/machrisaa/stylenet/blob/master/images/husky_real_region.jpg"/></td> </tr> <tr> <td align='center'>Content Region Map</td> <td align='center'>Style Region Map</td> <td align='center'>Result</td> </tr> </table> <br/> <br/>##Modification of Algorithm <img src="https://github.com/machrisaa/stylenet/blob/master/images/stylenet_patch_diagram.png"/> There are 2 modifications of the algorithm from the original Markov Random Field in the paper.
-
We added the blur filtering in the convariant matrix before calculate the max argument in order to reduce the different of the convariant score in a local neighbouthood. This can make the result image look more natural. (Yellow part in the diagram)
-
The second modification is to replace the cost function of the piecwise square difference between all patches and the slices of the image. We replaced the function to a single averaged patch tensor in order greatly improve the training process. (Red part in the diagram)
<br/>
##Requirement
- Tensorflow
- Tensorflow-VGG <br/>
##Basic Usage
stylenet_patch.render_gen( <content image path> , <style image path>, height=<output height>)
See the smaple main function in stylenet_patch for more detail.