Awesome

AdaptIR: Parameter Efficient Multi-task Adaptation for Pre-trained Image Restoration Models

[Paper] [Suppl] [Project Page][Zhihu(知乎)]

Hang Guo, Tao Dai, Yuanchao Bai, Bin Chen, Shu-Tao Xia, Zexuan Zhu

Abstract: Pre-training has shown promising results on various image restoration tasks, which is usually followed by full finetuning for each specific downstream task (e.g., image denoising). However, such full fine-tuning usually suffers from the problems of heavy computational cost in practice, due to the massive parameters of pre-trained restoration models, thus limiting its real-world applications. Recently, Parameter Efficient Transfer Learning (PETL) offers an efficient alternative solution to full fine-tuning, yet still faces great challenges for pre-trained image restoration models, due to the diversity of different degradations. To address these issues, we propose AdaptIR, a novel parameter efficient transfer learning method for adapting pre-trained restoration models. Specifically, the proposed method consists of a multi-branch inception structure to orthogonally capture local spatial, global spatial, and channel interactions. In this way, it allows powerful representations under a very low parameter budget. Extensive experiments demonstrate that the proposed method can achieve comparable or even better performance than full fine-tuning, while only using 0.6% parameters.

⭐If this work is helpful for you, please help star this repo. Thanks!🤗

📑 Contents

• Visual Results
• News
• TODO
• Results
• Citation

<a name="visual_results"></a>:eyes:Visual Results On Different Restoration Tasks

<img src="assets/imgsli1.png" height="153"/> <img src="assets/imgsli7.png" height="153"/> <img src="assets/imgsli5.png" height="153"/> <img src="assets/imgsli2.png" height="153"/>

<img src="assets/imgsli4.png" height="150"/> <img src="assets/imgsli3.png" height="150"/> <img src="assets/imgsli6.png" height="150"/>

<a name="news"></a> 🆕 News

• 2023-12-12: arXiv paper available.
• 2023-12-16: This repo is released.

<a name="todo"></a> ☑️ TODO

• arXiv version
• Supplementary matrial
• Project page
• Release code
• Pretrained weights

<a name="results"></a> 🥇 Results

We achieve state-of-the-art adaptation performance on various downstream image restoration tasks. Detailed results can be found in the paper.

<a name="cite"></a> 🥰 Citation

Please cite us if our work is useful for your research.

@article{guo2023adaptir,
  title={AdaptIR: Parameter Efficient Multi-task Adaptation for Pre-trained Image Restoration Models},
  author={Guo, Hang and Dai, Tao and Bai, Yuanchao and Chen, Bin and Xia, Shu-Tao and Zhu, Zexuan},
  journal={arXiv preprint arXiv:2312.08881},
  year={2023}
}

License

This project is released under the Apache 2.0 license.

Acknowledgement

This code is based on AirNet, IPT and EDT. Thanks for their awesome work.

Contact

If you have any questions, feel free to approach me at cshguo@gmail.com