Awesome
Awesome LLM-generated Text Detection
<div align="center" style="border-radius:50%;">
<img src="log.png" alt="Logo" width="200" class="avatar">
</div>
<p align="center">
<img src="https://awesome.re/badge.svg" alt="Awesome">
<img src="https://img.shields.io/badge/License-MIT-green.svg" alt="License-MIT">
<img src="https://img.shields.io/badge/PRs-Welcome-red" alt="PRs-Welcome">
<img src="https://img.shields.io/github/last-commit/junchaoIU/LLM-generated-Text-Detection/main?logo=github&color=blue" alt="last-commit">
</p>
The powerful ability of large language models (LLMs) to understand, follow, and generate complex languages has enabled LLM-generated texts to flood many areas of our daily lives at an incredible rate, with potentially negative impacts and risks on society and academia. As LLMs continue to expand, how can we detect LLM-generated texts to help minimize the threat posed by the misuse of LLMs?
<div align="center">
<img src="cover.png" alt="cover" width="750">
</div>
<br>
<!-- **Authors:** -->
¹ Junchao Wu, ¹ Shu Yang, ¹ Runzhe Zhan, ¹ ² Yulin Yuan, ¹ Derek Fai Wong, ¹ Lidia Sam Chao
<!-- **Affiliations:** -->
¹ University of Macau, ² Peking University
📢 News
🔍 Table of Contents
📃 Papers
Overview
A survey and reflection on the latest research breakthroughs in LLM-generated Text detection, including data, detectors, metrics, current issues and future directions.
Please refer to our article/paper for more details.
Datasets
Benchmarks
Potential Datasets
| Tasks | Datasets |
|---|
| Questions Answering | PubMedQA, Children book corpus (CBT), ELI5, TruthfulQA, NarrativeQA |
| Scientific writing | Peer Read, arXiv, TOEFL11 |
| Story generation | WritingPrompts |
| News Article writing | XSum |
| Web Text | Wiki40b, WebText, Avax tweets dataset, Climate Change Tweets Ids |
| Opinion statements | r/ChangeMyView (CMV) Reddit subcommunity, Yelp , IMDB Dataset |
| Comprehension and Reasoning | SciGen, ROCStories Corpora, HellaSwag, SQuAD |
Detectors
<div align="left">
<img src="image.png" alt="Detector" width="800">
</div>
Watermark Technology
| Paper | Link |
|---|
| A watermark for large language models. |   |
| On the Reliability of Watermarks for Large Language Models | |
| A Private Watermark for Large Language Models | |
| Distillation-Resistant Watermarking for Model Protection in NLP | |
| Watermarking Pre-trained Language Models with Backdooring | |
Zero-shot Methods
| Paper | Link |
|---|
| DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature | |
| Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature | |
| Efficient Detection of LLM-generated Texts with a Bayesian Surrogate Model | |
| DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text | |
| GLTR: Statistical Detection and Visualization of Generated Text | |
| HowkGPT: Investigating the Detection of ChatGPT-generated University Student Homework through Context-Aware Perplexity Analysis | |
| Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts | |
Fine-tuning LMs Methods
| Paper | Link |
|---|
| How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection | |
| Multiscale Positive-Unlabeled Detection of AI-Generated Texts | |
| Real or fake? Learning to discriminate machine from human generated text | |
| Automatic Detection of Generated Text is Easiest when Humans are Fooled | |
| Stylometric Detection of AI-Generated Text in Twitter Timelines | |
| TweepFake: about Detecting Deepfake Tweets | |
| Towards a Robust Detection of Language Model Generated Text: Is ChatGPT that Easy to Detect? | |
| Deepfake Text Detection in the Wild | |
| ArguGPT: evaluating, understanding and identifying argumentative essays generated by GPT models | |
| Check Me If You Can: Detecting ChatGPT-Generated Academic Writing using CheckGPT | |
| GPT-Sentinel: Distinguishing Human and ChatGPT Generated Content | |
| Neural Deepfake Detection with Factual Structure of Text | |
| ConDA: Contrastive Domain Adaptation for AI-generated Text Detection | |
Adversarial Learning Methods
| Paper | Link |
|---|
| RADAR: Robust AI-Text Detection via Adversarial Learning | |
| OUTFOX: LLM-generated Essay Detection through In-context Learning with Adversarially Generated Examples | |
| Red Teaming Language Model Detectors with Language Models | |
| Is ChatGPT Involved in Texts? Measure the Polish Ratio to Detect ChatGPT-Generated Text | |
LLMs as Detector
| Paper | Link |
|---|
| Fighting fire with fire: Can chatgpt detect ai-generated text? | |
| OUTFOX: LLM-generated Essay Detection through In-context Learning with Adversarially Generated Examples | |
| GPT Paternity Test: GPT Generated Text Detection with GPT Genetic Inheritance | |
Related Works
Other Surveys
| Paper | Link |
|---|
| Automatic Detection of Machine Generated Text: A Critical Survey | |
| The Science of Detecting LLM-Generated Texts | |
| Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods | |
| Computer-Generated Text Detection Using Machine Learning: A Systematic Review | |
| Attribution and Obfuscation of Neural Text Authorship: A Data Mining Perspective | |
🚩 Citation
If our research helps you, please kindly cite our paper.
@article{wu2023survey,
title={A Survey on LLM-gernerated Text Detection: Necessity, Methods, and Future Directions},
author={Junchao Wu and Shu Yang and Runzhe Zhan and Yulin Yuan and Derek F. Wong and Lidia S. Chao},
journal = {CoRR},
volume = {abs/2310.14724},
year = {2023},
url = {https://arxiv.org/abs/2310.14724},
eprinttype = {arXiv},
eprint = {2310.14724},
Contributing
Contributions are welcome! If you have any ideas, suggestions, or bug reports, please open an issue or submit a pull request. We appreciate your contributions to making LLM-generated Text Detection work even better.