Awesome

Awesome LLM-generated Text Detection

The powerful ability of large language models (LLMs) to understand, follow, and generate complex languages has enabled LLM-generated texts to flood many areas of our daily lives at an incredible rate, with potentially negative impacts and risks on society and academia. As LLMs continue to expand, how can we detect LLM-generated texts to help minimize the threat posed by the misuse of LLMs?

¹ Junchao Wu, ¹ Shu Yang, ¹ Runzhe Zhan, ¹ ² Yulin Yuan, ¹ Derek Fai Wong, ¹ Lidia Sam Chao

¹ University of Macau, ² Peking University

📢 News

• [2024.09.26] ✨ Our benchmark paper is accepted by NeurIPS 2024 D&B track. We released DetectRL, a benchmark for real-world LLM-generated text detection, provide real utility to researchers on the topic and practitioners looking for consistent evaluation methods. Please refer to arXiv: DetectRL: Benchmarking LLM-Generated Text Detection in Real-World Scenarios and Github Repo DetectRL for details.
• [2023.10.24] Our survey paper is now available on arXiv: A Survey on LLM-generated Text Detection: Necessity, Methods, and Future Directions.
• [2023.05.01] : We began to explore the topic of LLM-generated Text Detection.

🔍 Table of Contents

• Awesome LLM-generated Text Detection
  • 📢 News
  • � Table of Contents
  • 📃 Papers
    • Overview
    • Datasets
      • Benchmarks
      • Potential Datasets
    • Detector
      • Watermark Technology
      • Zero-shot Methods
      • Fine-tuning LMs Methods
      • Adversarial Learning Methods
      • LLMs as Detector
    • Related Works
      • Other Surveys
  • 🚩 Citation
  • Contributing

📃 Papers

Overview

A survey and reflection on the latest research breakthroughs in LLM-generated Text detection, including data, detectors, metrics, current issues and future directions. Please refer to our article/paper for more details.

Datasets

Benchmarks

Potential Datasets

Detectors

Watermark Technology

Paper	Link
A watermark for large language models.
On the Reliability of Watermarks for Large Language Models
A Private Watermark for Large Language Models
Distillation-Resistant Watermarking for Model Protection in NLP
Watermarking Pre-trained Language Models with Backdooring

Zero-shot Methods

Paper	Link
DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature
Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature
Efficient Detection of LLM-generated Texts with a Bayesian Surrogate Model
DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text
GLTR: Statistical Detection and Visualization of Generated Text
HowkGPT: Investigating the Detection of ChatGPT-generated University Student Homework through Context-Aware Perplexity Analysis
Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts

Fine-tuning LMs Methods

Paper	Link
How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection
Multiscale Positive-Unlabeled Detection of AI-Generated Texts
Real or fake? Learning to discriminate machine from human generated text
Automatic Detection of Generated Text is Easiest when Humans are Fooled
Stylometric Detection of AI-Generated Text in Twitter Timelines
TweepFake: about Detecting Deepfake Tweets
Towards a Robust Detection of Language Model Generated Text: Is ChatGPT that Easy to Detect?
Deepfake Text Detection in the Wild
ArguGPT: evaluating, understanding and identifying argumentative essays generated by GPT models
Check Me If You Can: Detecting ChatGPT-Generated Academic Writing using CheckGPT
GPT-Sentinel: Distinguishing Human and ChatGPT Generated Content
Neural Deepfake Detection with Factual Structure of Text
ConDA: Contrastive Domain Adaptation for AI-generated Text Detection

Adversarial Learning Methods

Paper	Link
RADAR: Robust AI-Text Detection via Adversarial Learning
OUTFOX: LLM-generated Essay Detection through In-context Learning with Adversarially Generated Examples
Red Teaming Language Model Detectors with Language Models
Is ChatGPT Involved in Texts? Measure the Polish Ratio to Detect ChatGPT-Generated Text

LLMs as Detector

Paper	Link
Fighting fire with fire: Can chatgpt detect ai-generated text?
OUTFOX: LLM-generated Essay Detection through In-context Learning with Adversarially Generated Examples
GPT Paternity Test: GPT Generated Text Detection with GPT Genetic Inheritance

Related Works

Other Surveys

Paper	Link
Automatic Detection of Machine Generated Text: A Critical Survey
The Science of Detecting LLM-Generated Texts
Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods
Computer-Generated Text Detection Using Machine Learning: A Systematic Review
Attribution and Obfuscation of Neural Text Authorship: A Data Mining Perspective

🚩 Citation

If our research helps you, please kindly cite our paper.

@article{wu2023survey,
      title={A Survey on LLM-gernerated Text Detection: Necessity, Methods, and Future Directions}, 
      author={Junchao Wu and Shu Yang and Runzhe Zhan and Yulin Yuan and Derek F. Wong and Lidia S. Chao},
      journal      = {CoRR},
      volume       = {abs/2310.14724},
      year         = {2023},
      url          = {https://arxiv.org/abs/2310.14724},
      eprinttype   = {arXiv},
      eprint       = {2310.14724},

Contributing

Contributions are welcome! If you have any ideas, suggestions, or bug reports, please open an issue or submit a pull request. We appreciate your contributions to making LLM-generated Text Detection work even better.