Home

Awesome

Semantic Image Synthesis with SPADE (GauGAN) - Tensorflow

Simple Tensorflow implementation of "Semantic Image Synthesis with Spatially-Adaptive Normalization" (CVPR 2019 Oral)

<div align="center"> <img src="https://nvlabs.github.io/SPADE/images/treepond.gif"> <img src="https://nvlabs.github.io/SPADE/images/ocean.gif"> </div>

Project page | Paper | Pytorch code

Requirements

Preparation

Pretrained model

Usage

├── dataset
   └── YOUR_DATASET_NAME
       ├── image
           ├── 000001.jpg 
           ├── 000002.png
           └── ...
       ├── segmap
           ├── 000001.jpg
           ├── 000002.png
           └── ...
       ├── segmap_test
           ├── a.jpg 
           ├── b.png
           └── ...
       ├── segmap_label.txt (Automatically created) 
       
├── guide.jpg (example for guided image translation task)

Train

> python main.py --dataset spade_celebA --img_ch 3 --segmap_ch 3 --phase train 

Random test

> python main.py --dataset spade_celebA --segmap_ch 3 --phase random

Guide test

> python main.py --dataset spade_celebA --img_ch 3 --segmap_ch 3 --phase guide --guide_img ./guide_img.png

Our Results

Loss grpah

<div align="center"> <img src="./assets/loss.png"> </div>

CityScape

<div align="center"> <img src="./assets/result_img/cityscape_hinge.png"> </div>

CelebA-HQ (Style Manipulation)

<div align="center"> <img src="./assets/result_img/women_hinge.png"> </div>
<div align="center"> <img src="./assets/result_img/men_hinge.png"> </div>

CelebA-HQ (Random Manipulation)

<div align="center"> <img src="./assets/result_img/women_random_hinge.png"> </div>
<div align="center"> <img src="./assets/result_img/men_random_hinge.png"> </div>

How about the Least-Square loss ?

CelebA-HQ (Style Manipulation)

<div align="center"> <img src="./assets/result_img/women_lsgan.png"> </div>
<div align="center"> <img src="./assets/result_img/men_lsgan.png"> </div>

CelebA-HQ (Random Manipulation)

<div align="center"> <img src="./assets/result_img/women_random_lsgan.png"> </div>
<div align="center"> <img src="./assets/result_img/men_random_lsgan.png"> </div>

Architecture

GeneratorImage EncoderDiscriminatorAll-in-one
<img src = './assets/generator.png' width = '400px' height = '400px'><img src = './assets/image_encoder.png' width = '400px' height = '400px'><img src = './assets/discriminator.png' width = '350px' height = '350px'><img src = './assets/architecture.png' width = '400px' height = '400px'>

SPADE architecture

SPADESPADE Residual Block
<img src = './assets/spade.png' width = '850px' height = '300px'><img src = './assets/spade_resblock.png' width = '500px' height = '400px'>

Author

Junho Kim