Awesome

OISR-PyTorch

PyTorch implementation of "ODE-inspired Network Design for Single Image Super-Resolution". This code is built on EDSR (PyTorch). We would like to thank the authors for sharing their codes!

Dependencies :

• Python 3.7
• PyTorch 0.4.1
• numpy
• skimage
• imageio
• matplotlib
• tqdm

How to use ?

The code structure of this project is the same as EDSR, except that we change the definition of residual block (src/model/common.py) to Leapfrog / Heun / 3-stage Runge-Kutta blocks and reset the number of residual blocks. The training and testing scripts remain relatively unchanged. All datasets used in this work can be found here [1,2].

Pretrained models :

We report the 4x PSNR(dB) results on benckmark datasets.

Model Structures :

Representative images (4x) :

• LR

  <img src="./img/LR_1.png" width="341" height="255" />

• OISR-RK3

  <img src="./img/RK3_1.png" width="341" height="255" />

• LR

  <img src="./img/LR_2.png" width="341" height="255" />

• OISR-RK3

  <img src="./img/RK3_2.png" width="341" height="255" />

• LR

  <img src="./img/LR_3.png" width="341" height="255" />

• OISR-RK3

  <img src="./img/RK3_3.png" width="341" height="255" />

• Urban100

• B100

You can download some results from here. This link contains RK3-x3/x4 Urban100 and B100 results. You can easily generate result images with demo.sh scripts.

Q&A :

1. The improvement of OISR is limited, compared with NTIRE2018 top rankers.

We admit that the ODE-inspired network design remains experimental, so far it just gives us a different point of view to understand SR problems. It could be a new direction and deserves more attentions of the community (e.g., Neural Ordinary Differential Equations in NeurIPS2018 is encouraging).

2. More diverse comparative experiments should be involved, since there are many numerical ODE methods.

Due to the limited computing resources, we only evaluate a few network structures at the present time. It is possible that there exists another better architecture, or, more likely, many attempts may not even converge. Note that we only focus on the order of numerical ODE methods intuitively. It is only one of the central concepts in numerical analysis (also includes convergence and stability). To the best of our knowledge, there is no correspondence strictly between numerical ODEs and deep neural networks in numerical analysis, but utlizing convergence, consistency and order, and stability and stiffness may further explain the emprical success of the existing methods and the potential failures.

References :

@InProceedings{Lim_2017_CVPR_Workshops,
  author    = {Lim, Bee and Son, Sanghyun and Kim, Heewon and Nah, Seungjun and Lee, Kyoung Mu},
  title     = {Enhanced Deep Residual Networks for Single Image Super-Resolution},
  booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
  month     = {July},
  year      = {2017}
}

@InProceedings{OISR,
  author    = {Xiangyu He and Zitao Mo and Peisong Wang and Yang Liu and Mingyuan Yang and Jian Cheng},
  title     = {ODE-inspired Network Design for Single Image Super-Resolution},
  booktitle = {2019 {IEEE} Conference on Computer Vision and Pattern Recognition},
  month     = {July},
  year      = {2019}
}