Home

Awesome

DeltaPapers

Must-read papers on parameter-efficient tuning methods (Delta Tuning) for pre-trained models.

Content

Why Parameter Efficient?

Increasingly larger pre-trained models bring both the blessing of effectiveness on existing and unknown tasks, and the curse of prohibitive adaptation cost. In this context, parameter-efficient tuning methods (delta tuning) are developed and demonstrate a promising way to stimulate colossal models with only a small portion of tunable parameters, thereby dramatically reducing the computational and storage costs of model adaptation. In addition to the obvious practical value, delta tuning seems to imply that the particular adaptation of pre-trained models may be a very simple process, which may usher in intriguing theoretical issues that are worth exploring.

Keywords Convention

We follow the general idea of PromptPapers to label the papers.

The abbreviation of the work.

The main explored task of the work.

The main explored property of delta tuning methods in the work.

Other important information of the work.

Papers

Overview

Methodology

Analysis

Applications & Tools

Blogs

Contribution

Contributors

<!-- Copy-paste in your Readme.md file --> <a href="https://github.com/thunlp/DeltaPapers/graphs/contributors"> <img src="https://contrib.rocks/image?repo=thunlp/DeltaPapers" /> </a>

Contributing to this paper list