Awesome

Boosting Flow-based Generative Super-Resolution Models via Learned Prior [CVPR 2024]

This is the official repository of "Boosting Flow-based Generative Super-Resolution Models via Learned Prior". <br><br>

Li-Yuan Tsao, Yi-Chen Lo, Chia-Che Chang, Hao-Wei Chen, Roy Tseng, Chien Feng, Chun-Yi Lee

Overview

In this work, we identify several challenges in flow-based SR methods, including grid artifacts, exploding inverses, and suboptimal results due to a fixed sampling temperature. To tackle these issues, we introduce a learned prior, which is predicted by the proposed latent module, to the inference phase of flow-based SR models. <br><br> <img width="65%" height="65%" alt="image" src="https://github.com/liyuantsao/FlowSR-LP/assets/73187544/e0900fdf-b11b-44d0-912d-ca502c2628b3"> <br>

This framework not only addresses the inherent issues in flow-based SR models but also enhances the quality of synthesized images without modifying the original design or pre-trained weights of these models. Our proposed framework is effective, flexible in design, and able to generalize to both fixed-scale and arbitrary-scale SR frameworks without requiring customized components. <br><br> <img width="1050" alt="image" src="https://github.com/liyuantsao/FlowSR-LP/assets/73187544/647a7011-cf67-461b-b170-079b151432b2">

Launch your experiments

This repository includes the training/evaluation code for LINF-LP, along with the evaluation code for SRFlow-LP (the training code will be released soon), which are the implementations after integrating the proposed latent module with LINF and SRFlow.

To run your experiments on LINF-LP and SRFlow-LP, please refer to LINF-LP and SRFlow-LP, respectively.

Results

Arbitrary-scale SR results

• The arbitrary-scale SR results on SR benchmark datasets. “In-scales” and “OOD-scales” refer to in- and out-of-training-distribution scales. LPIPS scores are reported (lower is better), with the best and second-best highlighted in red and blue, respectively. <img width="926" alt="image" src="https://github.com/liyuantsao/FlowSR-LP/assets/73187544/43b9761a-188a-4d70-964e-d556912471a0">

Generative SR results

• The 4× SR results on the DIV2K validation set. The best results are highlighted in red.

Qualitative Results

• Our method tackles the grid artifacts and exploding inverse issue.

• A qualitative comparison between the 4× SR results of SRFlow and our SRFlow-LP.

• A qualitative comparison between the 4× SR results of LINF and our LINF-LP.

Citation

If you find our work helpful for your research, we would greatly appreciate your assistance in sharing it with the community and citing it using the following BibTex. Thank you for supporting our research.

@InProceedings{Tsao_2024_CVPR,
    author    = {Tsao, Li-Yuan and Lo, Yi-Chen and Chang, Chia-Che and Chen, Hao-Wei and Tseng, Roy and Feng, Chien and Lee, Chun-Yi},
    title     = {Boosting Flow-based Generative Super-Resolution Models via Learned Prior},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2024},
    pages     = {26005-26015}
}

Acknowledgements

Our codes are built on LINF (Paper, code) and SRFlow (Paper, Code), we appriciate their amazing works that advance this community.