Home

Awesome

PatchScaler: An Efficient Patch-independent Diffusion Model for Image Super-Resolution

Yong Liu, Hang Dong, Jinshan Pan, Qingji Dong, Kai Chen, Rongxiang Zhang, Lean Fu, Fei Wang<br/>

:sparkling_heart: If our PatchScaler is helpful to your researches or projects, please help star this repository. Thanks! :hugs:

visitors <img alt="GitHub" src="https://img.shields.io/badge/license-Apache_2.0-brightgreen"> GitHub Stars

While diffusion models significantly improve the perceptual quality of super-resolved images, they usually require a large number of sampling steps, resulting in high computational costs and long inference times. Recent efforts have explored reasonable acceleration schemes by reducing the number of sampling steps. However, these approaches treat all regions of the image equally, overlooking the fact that regions with varying levels of reconstruction difficulty require different sampling steps. To address this limitation, we propose PatchScaler, an efficient patch-independent diffusion pipeline for single image super-resolution. Specifically, PatchScaler introduces a Patch-adaptive Group Sampling (PGS) strategy that groups feature patches by quantifying their reconstruction difficulty and establishes shortcut paths with different sampling configurations for each group. To further optimize the patch-level reconstruction process of PGS, we propose a texture prompt that provides rich texture conditional information to the diffusion model. The texture prompt adaptively retrieves texture priors for the target patch from a common reference texture memory. Extensive experiments show that our PatchScaler achieves superior performance in both quantitative and qualitative evaluations, while significantly speeding up inference.

<p align="center"> <img src=assets/method.png width="1000px"/> </p>

:snowboarder: Update

:whale: License

This project is released under the Apache 2.0 license. Redistribution and use should follow this license.

:trophy: Results on Real-world Image SR

<p align="center"> <img src=assets/results.png width="1000px"/> </p>

:star: BibTeX

If you find this project useful for your research, please use the following BibTeX entry.

@inproceedings{liu2024patchscaler,
  title={PatchScaler: An Efficient Patch-independent Diffusion Model for Image Super-Resolution},
  author={Yong Liu, Hang Dong, Jinshan Pan, Qingji Dong, Kai Chen, Rongxiang Zhang, Lean Fu, and Fei Wang},
  year={2024}
}