Awesome
<div align="center"> <i>CAMixerSR</i>: Only Details Need More “Attention” </div>
<div align="center">Yan Wang<sup>1,2</sup>, Yi Liu<sup>1*</sup>, Shijie Zhao<sup>1*†</sup>, Junlin Li<sup>1</sup>, Li Zhang<sup>1</sup>
</div> <p align="center"> <sup>1</sup>ByteDance, <sup>2</sup>Nankai University </p> <p align="center"> <a href="https://arxiv.org/abs/2402.19289" alt="arXiv"> <img src="https://img.shields.io/badge/arXiv-2402.19289-b31b1b.svg?style=flat" /></a> <a href="https://github.com/icandle/CAMixerSR/blob/main/LICENSE" alt="license"> <img src="https://img.shields.io/badge/license-Apache--2.0-%23B7A800" /></a> <a href="https://colab.research.google.com/gist/icandle/404a89adbc264294dd77edacfd80f3b2/camixersr.ipynb" alt="Colab"> <img src="https://colab.research.google.com/assets/colab-badge.svg" /></a> <a href="https://drive.google.com/drive/folders/1wgSshZn8FZwuLtQcN9IoUY8XI4gBBxjs?usp=sharing"> <img src="https://img.shields.io/badge/Docs-Slide&Poster-8A2BE2" /></a> <a href="https://www.youtube.com/watch?v=WHEKNHPhZuc"> <img src="https://img.shields.io/badge/Video-Presentation-FF8C00" /></a> </p>Overview: We propose CAMixerSR, a new approach integrating content-aware accelerating framework and token mixer design, to pursue more efficient SR inference via assigning convolution for simple regions but window-attention for complex textures. It exhibits excellent generality and attains competitive results among state-of-the-art models with better complexity-performance trade-offs on large-image SR, lightweight SR, and omnidirectional-image SR.
<p align="center"> <img src="./figures/CAMixer.png" width=100% height=100% class="center"> </p>This repository contains PyTorch implementation for CAMixerSR (CVPR 2024).
<details> <summary>Table of contents</summary> <p align="center"> </p> </details>⚙️ Requirements
Dependencies
Installation
pip install -r requirements.txt
🎈 Datasets
Large-Image SR
Training: DIV2K.
Testing: F2K, Test2K, Test4K, Test8K (Google Drive/Baidu Netdisk).
Lightweight SR
Testing: Set5, Set14, BSD100, Urban100, Manga109 (Google Drive/Baidu Netdisk).
Preparing: Please refer to the Dataset Preparation of BasicSR.
Omni-Directional-Image SR
Training/Testing: lau dataset (Google Drive/Baidu Netdisk).
Preparing: Please refer to the Step 1&2&3 of OSRT.
▶️ How to Test
Clone this repository and change the directory to ./codes.
git clone https://github.com/icandle/CAMixerSR.git
cd codes
Large-Image SR
Testing: Change the dataset path of example option to your datasets and test with the command:
# 2K
python basicsr/test.py -opt options/test/test_2K.yml
# 4K/8K
python basicsr/test.py -opt options/test/test_8K.yml
Note: We use TileModel with Tile 64x64 and Overlap 4 to constrain the calculations.
Lightweight SR
Testing: Change the dataset path of example option to your datasets and test with the command:
# x2
python basicsr/test.py -opt options/test/test_x2.yml
# x4
python basicsr/test.py -opt options/test/test_x4.yml
✨ Results
<details> <summary>Visual comparison (click me)</summary> <p align="left"><img src="./figures/visual.png" width=100% height=100% class="left">
</p> </details> <details> <summary>Large-Image SR </summary> <p align="left"><img src="./figures/LargeSR_table.png" width=100% height=100% class="left">
</p> </details> <details> <summary>Lightweight SR </summary> <p align="left"><img src="./figures/LightSR_table.png" width=100% height=100% class="left">
</p> </details> <details> <summary>Omni-Directional-Image SR </summary> <p align="left"><img src="./figures/ODISR_table.png" width=100% height=100% class="left">
</p> </details>💖 Acknowledgments
We would thank BasicSR, ClassSR, and OSRT for their enlightening work!
🎓 Citation
@article{wang2024camixersr,
title={CAMixerSR: Only Details Need More ``Attention"},
author={Wang, Yan and Liu, Yi and Zhao, Shijie and Li, Junlin and Zhang, Li},
journal={arXiv preprint arXiv:2402.19289},
year={2024}
}