Awesome
PromptMix
This is the repository for the EMNLP 2023 paper PromptMix: A Class Boundary Augmentation Method for Large Language Model Distillation.
@inproceedings{sahu-etal-2023-promptmix,
title = "PromptMix: A Class Boundary Augmentation Method for Large Language Model Distillation",
author = "Sahu, Gaurav and
Vechtomova, Olga and
Bahdanau, Dzmitry and
Laradji, Issam",
booktitle = "Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing",
month = dec,
year = "2023",
address = "Singapore",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2023.emnlp-main.323",
pages = "5316--5327",
}
Running experiments
1. Clone the repo and setup up a virtualenv with python>=3.9
Below instructions are for conda/miniconda but you can also use virtualenv
git clone git@github.com:ServiceNow/PromptMix.git
cd PromptMix
conda create -n promptmix python=3.9
conda activate promptmix
pip install -r requirements.txt
2. Get the Datasets
Prepare the datasets by running the following command
python -m scripts.compile_ds -d subj
Set -d
to "tc" for the twitter complaints dataset and "trec6" for the TREC6 dataset. To use the banking77 dataset, download the zip file here and unzip it into the data
directory.
3. Train promptmix pipeline & Validate
Run the following command to run the promptmix pipeline (on SUBJ dataset)
# Run the experiment w/o relabeling (to get A1)
$(which python) -m main -e poc_naive -sb /path/to/save/directory -j 0 --python_binary $(which python)
# perform relabeling (to get A2)
$(which python) -m main -e poc_rlbl -sb /path/to/save/directory -j 0 --python_binary $(which python)
Run the following to get the NN+GPT3.5 results
$(which python) -m main -e gpt_nn_baseline -j 0 --python_binary $(which python)
Argument Descriptions:
-e [Experiment group to run]
-sb [Directory where the experiments are saved]
-j [whether or not to use toolkit]
You can modify exp_configs/promptmix_exps.py to change the hyperparameters/use different datasets.
4. Results
Open results/active_nlp.ipynb
to visualize the results.