Home

Awesome

OUTFOX🦊

arXiv LICENSE

This is the official code and data for our AAAI 2024 paper, "OUTFOX: LLM-Generated Essay Detection Through In-Context Learning with Adversarially Generated Examples"

<p align="center"> <img src="https://github.com/user-attachments/assets/39d5d34e-779d-49f0-b335-e7e2528f49d2" width="500"/> <!-- <img src="https://github.com/ryuryukke/OUTFOX/assets/61570900/4626abf6-5c75-43c9-91c0-812804e79104" width="500"/> --> </p>

📖 Introduction

Current LLM-generated text detectors lack robustness against attacks: they degrade detection accuracy by simply paraphrasing LLM-generated texts. Furthermore, there is the unexplored risk where malicious users might exploit LLMs to create texts specifically designed to evade detection.

In this paper, we propose OUTFOX, a framework that improves the robustness of LLM detectors by allowing both the detector and the attacker to consider each other's output. In this framework, the attacker uses the detector's prediction labels as examples for in-context learning and adversarially generates essays that are harder to detect, while the detector uses the adversarially generated essays as examples for in-context learning to learn to detect essays from a strong attacker.

Experiments in the domain of student essays show that...

📢 Updates

🔨 Setup

$ python -m venv env
$ source env/bin/activate
$ pip install -r requirements.txt

Run any script in the scripts directory.

scripts/detection.sh is a script for our OUTFOX detection and baseline detections.

scripts/attacking.sh is a script for our OUTFOX attacking and baseline paraphrasing attack.

:page_facing_up: Dataset Info

We created our dataset based on Kaggle FeedBack Prize, and our dataset contains 15,400 triplets of essay problem statements, human(native-student)-written essays, and LLM-generated essays. The native students range from 6th to 12th grade in the U.S.

We instruct three LMs to generate essays: ChatGPT(gpt-3.5-turbo-0613), GPT-3.5(text-davinci-003), and FLAN-T5-XXL. We split the dataset into three parts: train/validation/test with 14400/500/500 examples, respectively.

This is supplemental information about the file names consisting of our dataset.

File nameContent
(train|valid|test)_problem_statements.pklEssay problem statements in each set.
(train|valid|test)_humans.pklHuman-written essays in each set.
(train|valid|test)_lms.pklLLM-generated essays in each set.

Additionally, (train\|valid\|test)_contexts.pkl includes the prompts used to generate essays in each set. We use these to compute the likelihood in statistical outlier detectors.

We also provide the attacked essays by our OUTFOX attacker in data/chatgpt/test/test_outfox_attacks.pkl and the attacked essays by DIPPER in data/dipper/(chatgpt|text_davinci_003|flan_t5_xxl)/test_attacks.pkl.

📚 Citation

If you find our code/data/models or ideas useful in your research, please consider citing our work as follows:

@InProceedings{Koike:OUTFOX:2024,
  author    = {Ryuto Koike and Masahiro Kaneko and Naoaki Okazaki},
  title     = {OUTFOX: LLM-Generated Essay Detection Through In-Context Learning with Adversarially Generated Examples},
  booktitle = {Proceedings of the 38th AAAI Conference on Artificial Intelligence},
  year      = {2024},
  month     = {February},
  address   = {Vancouver, Canada}
}