Home

Awesome

LAUG

LAUG is an open-source toolkit for Language understanding AUGmentation. It is an automatic method to approximate the natural perturbations to existing data. Augmented data could be used to conduct black-box robustness testing or enhancing training. [paper]

Installation

Require python 3.6.

Clone this repository:

git clone https://github.com/thu-coai/LAUG.git

Install via pip:

cd LAUG
pip install -e .

Download data and models:

The data used in our paper and model parameters pre-trained by us are available at Link. Please download and place them into corresponding dir. For model parameters released by others, please refer to README.md under dirs of each augmentation method such as LAUG/aug/Speech_Recognition/README.md.

Augmentation Methods

Here are the 4 augmentation methods described in our paper. They are placed under LAUG/aug dir.

Please see our paper and README.md in each augmentation method for detailed information.

See demo.py for the usage of these augmentation methods.

python demo.py

Noting that our augmentation methods contains several neural models, pre-trained parameters need to be downloaded before use. Parameters pre-trained by us are available at Link. For parameters which released by others, please follow the instructions of each method.

Supported Datasets

The data used in our paper is available at Link . Please download it and place it data/ dir.

Our data contains 2 datasets: MultiWOZ and Frames, along with their augmented copies.

NLU Models

We provide four base NLU models which are described in our paper:

These models are adapted from ConvLab-2. For more details, You can refer to README.md under LUAG/nlu/$model/$dataset dir such as LAUG/nlu/gpt/multiwoz/README.md.

Citing

If you use LAUG in your research, please cite:

@inproceedings{liu2021robustness,
    title={Robustness Testing of Language Understanding in Task-Oriented Dialog},
    author={Liu, Jiexi and Takanobu, Ryuichi and Wen, Jiaxin and Wan, Dazhen and Li, Hongguang and Nie, Weiran and Li, Cheng and Peng, Wei and Huang, Minlie},
    year={2021},
    booktitle={Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics},
}

License

Apache License 2.0