Home

Awesome

Awesome LLM-generated Text Detection

<div align="center" style="border-radius:50%;"> <img src="log.png" alt="Logo" width="200" class="avatar"> </div> <p align="center"> <img src="https://awesome.re/badge.svg" alt="Awesome"> <img src="https://img.shields.io/badge/License-MIT-green.svg" alt="License-MIT"> <img src="https://img.shields.io/badge/PRs-Welcome-red" alt="PRs-Welcome"> <img src="https://img.shields.io/github/last-commit/junchaoIU/LLM-generated-Text-Detection/main?logo=github&color=blue" alt="last-commit"> </p>

The powerful ability of large language models (LLMs) to understand, follow, and generate complex languages has enabled LLM-generated texts to flood many areas of our daily lives at an incredible rate, with potentially negative impacts and risks on society and academia. As LLMs continue to expand, how can we detect LLM-generated texts to help minimize the threat posed by the misuse of LLMs?

<div align="center"> <img src="cover.png" alt="cover" width="750"> </div> <br> <!-- **Authors:** -->

¹ Junchao Wu, ¹ Shu Yang, ¹ Runzhe Zhan, ¹ ² Yulin Yuan, ¹ Derek Fai Wong, ¹ Lidia Sam Chao

<!-- **Affiliations:** -->

¹ University of Macau, ² Peking University

📢 News

🔍 Table of Contents

📃 Papers

Overview

A survey and reflection on the latest research breakthroughs in LLM-generated Text detection, including data, detectors, metrics, current issues and future directions. Please refer to our article/paper for more details.

Datasets

Benchmarks

Benchmarks / DatasetsUseHumanLLMs
HC3train58k26k
HC3-Chinesetrain22k17k
CHEATtrain15k35k
GROVER Datasettrain valid test5k 2k 8k5k 1k 4k
TweepFaketrain12k12k
GPT-2 Output Datasettrain250k250k
TuringBenchtrain10k190k
MGTBenchtrain test2k 56313k 3k
ArguGPTtrain valid test3k 350 3503k 350 350
DeepfakeText-Datasettrain valid test95k 29k 29k236k, 29k 28k
M4train valid test122k 500 500122k 500 500
GPABenchmarktrain600k600k
Scientific-articles Benchmarktrain test8k 4k8k 4k

Potential Datasets

TasksDatasets
Questions AnsweringPubMedQA, Children book corpus (CBT), ELI5, TruthfulQA, NarrativeQA
Scientific writingPeer Read, arXiv, TOEFL11
Story generationWritingPrompts
News Article writingXSum
Web TextWiki40b, WebText, Avax tweets dataset, Climate Change Tweets Ids
Opinion statementsr/ChangeMyView (CMV) Reddit subcommunity, Yelp , IMDB Dataset
Comprehension and ReasoningSciGen, ROCStories Corpora, HellaSwag, SQuAD

Detectors

<div align="left"> <img src="image.png" alt="Detector" width="800"> </div>

Watermark Technology

PaperLink
A watermark for large language models.Static Badge Static Badge
On the Reliability of Watermarks for Large Language ModelsStatic Badge Static Badge
A Private Watermark for Large Language ModelsStatic Badge Static Badge
Distillation-Resistant Watermarking for Model Protection in NLPStatic Badge Static Badge
Watermarking Pre-trained Language Models with BackdooringStatic Badge

Zero-shot Methods

PaperLink
DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability CurvatureStatic Badge Static Badge
Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability CurvatureStatic Badge Static Badge
Efficient Detection of LLM-generated Texts with a Bayesian Surrogate ModelStatic Badge
DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated TextStatic Badge Static Badge
GLTR: Statistical Detection and Visualization of Generated TextStatic Badge Static Badge
HowkGPT: Investigating the Detection of ChatGPT-generated University Student Homework through Context-Aware Perplexity AnalysisStatic Badge
Intrinsic Dimension Estimation for Robust Detection of AI-Generated TextsStatic Badge

Fine-tuning LMs Methods

PaperLink
How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and DetectionStatic Badge Static Badge
Multiscale Positive-Unlabeled Detection of AI-Generated TextsStatic Badge Static Badge
Real or fake? Learning to discriminate machine from human generated textStatic Badge
Automatic Detection of Generated Text is Easiest when Humans are FooledStatic Badge
Stylometric Detection of AI-Generated Text in Twitter TimelinesStatic Badge
TweepFake: about Detecting Deepfake TweetsStatic Badge Static Badge
Towards a Robust Detection of Language Model Generated Text: Is ChatGPT that Easy to Detect?Static Badge
Deepfake Text Detection in the WildStatic Badge Static Badge
ArguGPT: evaluating, understanding and identifying argumentative essays generated by GPT modelsStatic Badge Static Badge
Check Me If You Can: Detecting ChatGPT-Generated Academic Writing using CheckGPTStatic Badge
GPT-Sentinel: Distinguishing Human and ChatGPT Generated ContentStatic Badge
Neural Deepfake Detection with Factual Structure of TextStatic Badge
ConDA: Contrastive Domain Adaptation for AI-generated Text DetectionStatic Badge Static Badge

Adversarial Learning Methods

PaperLink
RADAR: Robust AI-Text Detection via Adversarial LearningStatic Badge
OUTFOX: LLM-generated Essay Detection through In-context Learning with Adversarially Generated ExamplesStatic Badge Static Badge
Red Teaming Language Model Detectors with Language ModelsStatic Badge Static Badge
Is ChatGPT Involved in Texts? Measure the Polish Ratio to Detect ChatGPT-Generated TextStatic Badge

LLMs as Detector

PaperLink
Fighting fire with fire: Can chatgpt detect ai-generated text?Static Badge Static Badge
OUTFOX: LLM-generated Essay Detection through In-context Learning with Adversarially Generated ExamplesStatic Badge Static Badge
GPT Paternity Test: GPT Generated Text Detection with GPT Genetic InheritanceStatic Badge

Related Works

Other Surveys

PaperLink
Automatic Detection of Machine Generated Text: A Critical SurveyStatic Badge
The Science of Detecting LLM-Generated TextsStatic Badge
Machine Generated Text: A Comprehensive Survey of Threat Models and Detection MethodsStatic Badge
Computer-Generated Text Detection Using Machine Learning: A Systematic ReviewStatic Badge
Attribution and Obfuscation of Neural Text Authorship: A Data Mining PerspectiveStatic Badge

🚩 Citation

If our research helps you, please kindly cite our paper.

@article{wu2023survey,
      title={A Survey on LLM-gernerated Text Detection: Necessity, Methods, and Future Directions}, 
      author={Junchao Wu and Shu Yang and Runzhe Zhan and Yulin Yuan and Derek F. Wong and Lidia S. Chao},
      journal      = {CoRR},
      volume       = {abs/2310.14724},
      year         = {2023},
      url          = {https://arxiv.org/abs/2310.14724},
      eprinttype   = {arXiv},
      eprint       = {2310.14724},

Contributing

Contributions are welcome! If you have any ideas, suggestions, or bug reports, please open an issue or submit a pull request. We appreciate your contributions to making LLM-generated Text Detection work even better.