Awesome
LiST (Lite Self-Training)
This is the implementation of the paper LiST: Lite Self-training Makes Efficient Few-shot Learners. LiST is short for Lite Self-Training.
Overview
<img src="./figs/list.png" width="450"/>Setup Environment
Install via pip:
- create a conda environment running Python 3.7:
conda create --name LiST python=3.7
conda activate LiST
- install the required dependencies:
pip install -r requirements.txt
Use docker:
- Pull docker </br>
docker pull yaqing/pytorch-few-shot:v0.6
- Run docker </br>
docker run -it --rm --runtime nvidia yaqing/pytorch-few-shot:v0.6 bash
Please refer to the following link if you first use docker: https://docs.docker.com/
NOTE: Different versions of packages (like pytorch
, transformers
, etc.) may lead to different results from the paper. However, the trend should still hold no matter what versions of packages you use.
Prepare the data
Please run the following commands to prepare data for experiments:
cd data
bash prepare_dataset.sh
cd ..
Run the model
We prepare scripts to run tasks. Please use bash script under LiST directory.
Run LiST as:
bash run.sh
Note that we ran experiments on V100 GPU (32GB) for LiST models. You may need to reduce batch size for other GPUs.
Supported datasets:
MNLI, RTE, QQP, SST-2, subj and MPQA with shots of 10, 20, 30.
Notes and Acknowledgments
The implementation is based on https://github.com/huggingface/transformers <br> We also used some code from: https://github.com/princeton-nlp/LM-BFF
How do I cite LiST?
@article{wang2021list,
title={LiST: Lite Self-training Makes Efficient Few-shot Learners},
author={Wang, Yaqing and Mukherjee, Subhabrata and Liu, Xiaodong and Gao, Jing and Awadallah, Ahmed Hassan and Gao, Jianfeng},
journal={arXiv preprint arXiv:2110.06274},
year={2021}
}