Home

Awesome

Caffe_SRGAN

A caffe implementation of Christian et al's "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network" paper.

Dependencis

Train

Test

1. obtain the SR imgs:

2. compute_psnr_ssim:

Usage

  1. Overload or add the caffe_gan/caffe.proto, solver.cpp and caffe_gan/include/_layer.hpp and caffe_gan/src/_layer.cpp *_layer.cu to your caffe

 cd caffe && make clean && make all && make pycaffe 
 cp -r SRGAN caffe/examples/ 
  1. Preparing training data: randomly crop images of the ImageNet dataset into 216000 sub-imgs and save as .h5 format(my finally training set: 216000x3x74x74, 21600x3x296x296)

    • My implementation is in the win10 matlab : run utils/Generate_data/mygenerate_sr_trainx4.m
  2. Training SRResNet-MSE:

     cd yourpath/caffe && sh examples/SRGAN/train_srres_75s.sh
    
  3. Testing SRResNet-MSE:

    cd yourpath/caffe/examples/SRGAN/ && python srres-deploy.py 
    
  4. Training SRGAN-MSE:

     cd yourpath/caffe && sh examples/SRGAN/train_srgan_is2.sh 
    
  5. Testing SRGAN-MSE:

     cd yourpath/caffe/examples/SRGAN/ && python srres-deploy.py 
    

Benchmarks

After fixing the SRResNet-MSE architecture(Add Scale_layer) and the deploy.prototxt(For test, use global stats in the BN_layer) , the SRResNet-MSE worked well and its results are close to the offical results. My results of SRGAN-MSE are weird, and I don't know what's going wrong with it...

Set5 [4x upscaling]:

Model/BenchmarksPSNRSSIM
SRResNet-MSE(official)32.050.9019
SRResNet-MSE(mine)32.010.8914
SRGAN-MSE(official)29.400.8472
SRGAN-MSE(mine)32.010.8916

Set14 [4x upscaling]:

Model/BenchmarksPSNRSSIM
SRResNet-MSE(official)28.490.8184
SRResNet-MSE(mine)28.550.7786
SRGAN-MSE(official)26.020.7397
SRGAN-MSE(mine)28.520.7794

Results

BicubicSRResNet-MSE(mine)Ground Truth
Alt textAlt textAlt text
Alt textAlt textAlt text
Alt textAlt textAlt text
Alt textAlt textAlt text
Alt textAlt textAlt text

Notes

  1. Before you train the networks , maybe you should change the directory path of training and testing data.
  2. Here offer a model named SRResNet-MSE_74s_sb_iter_180000.caffemodel which you can finetuning .
  3. Currently, the SRGAN-MSE doesn't work well , and it is still training and tuning.

Implementation Details

  1. According the paper "Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network" , I implementated the pixelshffuler_layer, which is added to reshape_layer.hpp && .cpp . If you want to use pixelshuffler_layer by yourself , just following this:
layer{
    name: "psx2_1"
    type: "Reshape"
    bottom: "conv_g35"
    top: "psx2_1"
    reshape_param {
       pixelshuffler: 2
    }
}
  1. For the GAN parts, the code borrows heavily from "在caffe 中实现Generative Adversarial Nets(二)" However , I add some new features to GAN parts. For examples , make gan_mode support the parameter "iter_size". When your gan_networks out of memory ,you can set the iter_sieze : 2 . For more details ,you can refference my srgan_is2_solver.prototxt and train_srgan_is2.prototxt
  2. For the utils of compute_psnr_ssim , the code borrows from this link https://github.com/ShenghaiRong/caffe-vdsr . But I modified the codes a lot.
  3. Because of the different caffe version, you may have problem when compiling the caffe. If so ,you can just modify the step function in your solver.cpp instead of directly using mine.

Plans