Awesome

Awesome-LLM4IE-Papers

🔥🔥🔥 The article has been accepted by Frontiers of Computer Science (FCS).

Awesome papers about generative Information extraction using LLMs

The organization of papers is discussed in our survey: Large Language Models for Generative Information Extraction: A Survey.

If you find any relevant academic papers that have not been included in our research, please submit a request for an update. We welcome contributions from everyone.

If any suggestions or mistakes, please feel free to let us know via email at derongxu@mail.ustc.edu.cn and chenweicw@mail.ustc.edu.cn. We appreciate your feedback and help in improving our work.

If you find our survey useful for your research, please cite the following paper:

@article{xu2024large,
  title={Large language models for generative information extraction: A survey},
  author={Xu, Derong and Chen, Wei and Peng, Wenjun and Zhang, Chao and Xu, Tong and Zhao, Xiangyu and Wu, Xian and Zheng, Yefeng and Wang, Yang and Chen, Enhong},
  journal={Frontiers of Computer Science},
  volume={18},
  number={6},
  pages={186357},
  year={2024},
  publisher={Springer}
}

📒 Table of Contents

Information Extraction tasks
Information Extraction Techniques
Specific Domain
Evaluation and Analysis
Project and Toolkit
⏰ Recently Updated Papers (After 2024/09/04, the updated papers is here~)
⭐️ Datasets (with Download Link~)

💡 News

Update Logs
- The details can be find in <code>./update_new_papers_list</code>.
- 2024/09/04 Add 22 papers
- 2024/06/06 Add 41 papers
- 2024/03/30 Add 27 papers
- 2024/03/29 Add 20 papers

Information Extraction tasks

A taxonomy by various tasks.

Named Entity Recognition

Models targeting only ner tasks.

Entity Typing

Paper	Venue	Date	Code
Calibrated Seq2seq Models for Efficient and Generalizable Ultra-fine Entity Typing	EMNLP Findings	2023-12	GitHub
Generative Entity Typing with Curriculum Learning	EMNLP	2022-12	GitHub

Entity Identification & Typing

Paper	Venue	Date	Code
Granular Entity Mapper: Advancing Fine-grained Multimodal Named Entity Recognition and Grounding	EMNLP Findings	2024
Double-Checker: Large Language Model as a Checker for Few-shot Named Entity Recognition	EMNLP Findings	2024	GitHub
VerifiNER: Verification-augmented NER via Knowledge-grounded Reasoning with Large Language Models	ACL	2024	GitHub
ProgGen: Generating Named Entity Recognition Datasets Step-by-step with Self-Reflexive Large Language Models	ACL Findings	2024	GitHub
Rethinking Negative Instances for Generative Named Entity Recognition	ACL Findings	2024	GitHub
LLMs as Bridges: Reformulating Grounded Multimodal Named Entity Recognition	ACL Findings	2024	GitHub
RT: a Retrieving and Chain-of-Thought framework for few-shot medical named entity recognition	Others	2024-05	GitHub
P-ICL: Point In-Context Learning for Named Entity Recognition with Large Language Models	Arxiv	2024-06	GitHub
Astro-NER -- Astronomy Named Entity Recognition: Is GPT a Good Domain Expert Annotator?	Arxiv	2024-05
Know-Adapter: Towards Knowledge-Aware Parameter-Efficient Transfer Learning for Few-shot Named Entity Recognition	COLING	2024
ToNER: Type-oriented Named Entity Recognition with Generative Language Model	COLING	2024
CHisIEC: An Information Extraction Corpus for Ancient Chinese History	COLING	2024	GitHub
Astronomical Knowledge Entity Extraction in Astrophysics Journal Articles via Large Language Models	Others	2024-04
LTNER: Large Language Model Tagging for Named Entity Recognition with Contextualized Entity Marking	Arxiv	2024-04	GitHub
Enhancing Software-Related Information Extraction via Single-Choice Question Answering with Large Language Models	Others	2024-04
Knowledge-Enriched Prompt for Low-Resource Named Entity Recognition	TALLIP	2024-04
VANER: Leveraging Large Language Model for Versatile and Adaptive Biomedical Named Entity Recognition	Arxiv	2024-04	GitHub
LLMs in Biomedicine: A study on clinical Named Entity Recognition	Arxiv	2024-04
Out of Sesame Street: A Study of Portuguese Legal Named Entity Recognition Through In-Context Learning	ResearchGate	2024-04
Mining experimental data from Materials Science literature with Large Language Models: an evaluation study	Arxiv	2024-04	GitHub
LinkNER: Linking Local Named Entity Recognition Models to Large Language Models using Uncertainty	WWW	2024
Self-Improving for Zero-Shot Named Entity Recognition with Large Language Models	NAACL Short	2024	GitHub
On-the-fly Definition Augmentation of LLMs for Biomedical NER	NAACL	2024	GitHub
MetaIE: Distilling a Meta Model from LLM for All Kinds of Information Extraction Tasks	Arxiv	2024-03	GitHub
Distilling Named Entity Recognition Models for Endangered Species from Large Language Models	Arxiv	2024-03
Augmenting NER Datasets with LLMs: Towards Automated and Refined Annotation	Arxiv	2024-03
ConsistNER: Towards Instructive NER Demonstrations for LLMs with the Consistency of Ontology and Context	AAAI	2024
Embedded Named Entity Recognition using Probing Classifiers	Arxiv	2024-03	GitHub
In-Context Learning for Few-Shot Nested Named Entity Recognition	Arxiv	2024-02
LLM-DA: Data Augmentation via Large Language Models for Few-Shot Named Entity Recognition	Arxiv	2024-02
Structured information extraction from scientific text with large language models	Nature Communications	2024-02	GitHub
NuNER: Entity Recognition Encoder Pre-training via LLM-Annotated Data	Arxiv	2024-02
A Simple but Effective Approach to Improve Structured Language Model Output for Information Extraction	Arxiv	2024-02
PaDeLLM-NER: Parallel Decoding in Large Language Models for Named Entity Recognition	Arxiv	2024-02
Small Language Model Is a Good Guide for Large Language Model in Chinese Entity Relation Extraction	Arxiv	2024-02
C-ICL: Contrastive In-context Learning for Information Extraction	Arxiv	2024-02
UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition	ICLR	2024	GitHub
Improving Large Language Models for Clinical Named Entity Recognition via Prompt Engineering	Arxiv	2024-01	GitHub
2INER: Instructive and In-Context Learning on Few-Shot Named Entity Recognition	EMNLP Findings	2023-12
In-context Learning for Few-shot Multimodal Named Entity Recognition	EMNLP Findings	2023-12
Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples!	EMNLP Findings	2023-12	GitHub
Learning to Rank Context for Named Entity Recognition Using a Synthetic Dataset	EMNLP	2023-12	GitHub
LLMaAA: Making Large Language Models as Active Annotators	EMNLP Findings	2023-12	GitHub
Prompting ChatGPT in MNER: Enhanced Multimodal Named Entity Recognition with Auxiliary Refined Knowledge	EMNLP Findings	2023-12	GitHub
GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer	Arxiv	2023-11	GitHub
GPT Struct Me: Probing GPT Models on Narrative Entity Extraction	WI-IAT	2023-10	GitHub
GPT-NER: Named Entity Recognition via Large Language Models	Arxiv	2023-10	GitHub
Prompt-NER: Zero-shot Named Entity Recognition in Astronomy Literature via Large Language Models	Arxiv	2023-10
Inspire the Large Language Model by External Knowledge on BioMedical Named Entity Recognition	Arxiv	2023-09
One Model for All Domains: Collaborative Domain-Prefx Tuning for Cross-Domain NER	IJCAI	2023-09	GitHub
Chain-of-Thought Prompt Distillation for Multimodal Named Entity Recognition and Multimodal Relation Extraction	Arxiv	2023-08
Learning In-context Learning for Named Entity Recognition	ACL	2023-07	GitHub
Debiasing Generative Named Entity Recognition by Calibrating Sequence Likelihood	ACL Short	2023-07
Entity-to-Text based Data Augmentation for various Named Entity Recognition Tasks	ACL Findings	2023-07
Large Language Models as Instructors: A Study on Multilingual Clinical Entity Extraction	BioNLP	2023-07	GitHub
NAG-NER: a Unified Non-Autoregressive Generation Framework for Various NER Tasks	ACL Industry	2023-07
Unified Named Entity Recognition as Multi-Label Sequence Generation	IJCNN	2023-06
PromptNER : Prompting For Named Entity Recognition	Arxiv	2023-06
Does Synthetic Data Generation of LLMs Help Clinical Text Mining?	Arxiv	2023-04
Unified Text Structuralization with Instruction-tuned Language Models	Arxiv	2023-03
Structured information extraction from complex scientific text with fine-tuned large language models	Arxiv	2022-12	Demo
LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Prompting	COLING	2022-10	GitHub
De-bias for generative extraction in unified NER task	ACL	2022-05
InstructionNER: A Multi-Task Instruction-Based Generative Framework for Few-shot NER	Arxiv	2022-03
Document-level Entity-based Extraction as Template Generation	EMNLP	2021-11	GitHub
A Unified Generative Framework for Various NER Subtasks	ACL	2021-08	GitHub
Template-Based Named Entity Recognition Using BART	ACL Findings	2021-08	GitHub

Relation Extraction

Models targeting only RE tasks.

Relation Classification

Paper	Venue	Date	Code
Enhancing Software-Related Information Extraction via Single-Choice Question Answering with Large Language Models	Others	2024-04
CRE-LLM: A Domain-Specific Chinese Relation Extraction Framework with Fine-tuned Large Language Model	Arxiv	2024-04	GitHub
Recall, Retrieve and Reason: Towards Better In-Context Relation Extraction	IJCAI	2024-04
Empirical Analysis of Dialogue Relation Extraction with Large Language Models	IJCAI	2024-04
Meta In-Context Learning Makes Large Language Models Better Zero and Few-Shot Relation Extractors	IJCAI	2024-04
Retrieval-Augmented Generation-based Relation Extraction	Arxiv	2024-04	GitHub
Relation Extraction Using Large Language Models: A Case Study on Acupuncture Point Locations	Arxiv	2024-04
STAR: Boosting Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language Models	AAAI	2024-03
Grasping the Essentials: Tailoring Large Language Models for Zero-Shot Relation Extraction	Arxiv	2024-02
Chain of Thought with Explicit Evidence Reasoning for Few-shot Relation Extraction	EMNLP Findings	2023-12
GPT-RE: In-context Learning for Relation Extraction using Large Language Models	EMNLP	2023-12	GitHub
Guideline Learning for In-context Information Extraction	EMNLP	2023-12
Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples!	EMNLP Findings	2023-12	GitHub
LLMaAA: Making Large Language Models as Active Annotators	EMNLP Findings	2023-12	GitHub
Improving Unsupervised Relation Extraction by Augmenting Diverse Sentence Pairs	EMNLP	2023-12	GitHub
Revisiting Large Language Models as Zero-shot Relation Extractors	EMNLP Findings	2023-12
Mastering the Task of Open Information Extraction with Large Language Models and Consistent Reasoning Environment	Arxiv	2023-10
Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractors	ACL Findings	2023-07	GitHub
How to Unleash the Power of Large Language Models for Few-shot Relation Extraction?	ACL Workshop	2023-07	GitHub
Sequence generation with label augmentation for relation extraction	AAAI	2023-06	GitHub
Does Synthetic Data Generation of LLMs Help Clinical Text Mining?	Arxiv	2023-04
DORE: Document Ordered Relation Extraction based on Generative Framework	EMNLP Findings	2022-12
REBEL: Relation Extraction By End-to-end Language generation	EMNLP Findings	2021-11	GitHub

Relation Triplet

Paper	Venue	Date	Code
ERA-CoT: Improving Chain-of-Thought through Entity Relationship Analysis	ACL	2024	GitHub
AutoRE: Document-Level Relation Extraction with Large Language Models	ACL Demos	2024	GitHub
Meta In-Context Learning Makes Large Language Models Better Zero and Few-Shot Relation Extractors	IJCAI	2024-04
Consistency Guided Knowledge Retrieval and Denoising in LLMs for Zero-shot Document-level Relation Triplet Extraction	WWW	2024
Improving Recall of Large Language Models: A Model Collaboration Approach for Relational Triple Extraction	COLING	2024	GitHub
Unlocking Instructive In-Context Learning with Tabular Prompting for Relational Triple Extraction	COLING	2024
A Simple but Effective Approach to Improve Structured Language Model Output for Information Extraction	Arxiv	2024-02
Structured information extraction from scientific text with large language models	Nature Communications	2024-02	GitHub
Document-Level In-Context Few-Shot Relation Extraction via Pre-Trained Language Models	Arxiv	2024-02	GitHub
Small Language Model Is a Good Guide for Large Language Model in Chinese Entity Relation Extraction	Arxiv	2024-02
Efficient Data Learning for Open Information Extraction with Pre-trained Language Models	EMNLP Findings	2023-12
Mastering the Task of Open Information Extraction with Large Language Models and Consistent Reasoning Environment	Arxiv	2023-10
Unified Text Structuralization with Instruction-tuned Language Models	Arxiv	2023-03
Document-level Entity-based Extraction as Template Generation	EMNLP	2021-11	GitHub

Relation Strict

Paper	Venue	Date	Code
MetaIE: Distilling a Meta Model from LLM for All Kinds of Information Extraction Tasks	Arxiv	2024-03	GitHub
Distilling Named Entity Recognition Models for Endangered Species from Large Language Models	Arxiv	2024-03
CHisIEC: An Information Extraction Corpus for Ancient Chinese History	COLING	2024-03	GitHub
An Autoregressive Text-to-Graph Framework for Joint Entity and Relation Extraction	AAAI	2024-03	GitHub
C-ICL: Contrastive In-context Learning for Information Extraction	Arxiv	2024-02
REBEL: Relation Extraction By End-to-end Language generation	EMNLP Findings	2021-11	GitHub

Event Extraction

Models targeting only EE tasks.

Event Detection

Paper	Venue	Date
Improving Event Definition Following For Zero-Shot Event Detection	Arxiv	2024-03
Mastering the Task of Open Information Extraction with Large Language Models and Consistent Reasoning Environment	Arxiv	2023-10
Unified Text Structuralization with Instruction-tuned Language Models	Arxiv	2023-03
Unleash GPT-2 Power for Event Detection	ACL	2021-08

Event Argument Extraction

Paper	Venue	Date	Code
LLMs Learn Task Heuristics from Demonstrations: A Heuristic-Driven Prompting Strategy for Document-Level Event Argument Extraction	ACL	2024	GitHub
Beyond Single-Event Extraction: Towards Efficient Document-Level Multi-Event Argument Extraction	ACL Findings	2024	GitHub
KeyEE: Enhancing Low-Resource Generative Event Extraction with Auxiliary Keyword Sub-Prompt	Others	2024-04	GitHub
MetaIE: Distilling a Meta Model from LLM for All Kinds of Information Extraction Tasks	Arxiv	2024-03	GitHub
Leveraging ChatGPT in Pharmacovigilance Event Extraction: An Empirical Study	EACL	2024-02	GitHub
ULTRA: Unleash LLMs' Potential for Event Argument Extraction through Hierarchical Modeling and Pair-wise Refinement	Arxiv	2024-01
Context-Aware Prompt for Generation-based Event Argument Extraction with Diffusion Models	CIKM	2023-10
Contextualized Soft Prompts for Extraction of Event Arguments	ACL Findings	2023-07
AMPERE: AMR-Aware Prefix for Generation-Based Event Argument Extraction Model	ACL	2023-07	GitHub
Code4Struct: Code Generation for Few-Shot Event Structure Prediction	ACL	2023-07	GitHub
Event Extraction as Question Generation and Answering	ACL short	2023-07	GitHub
Global Constraints with Prompting for Zero-Shot Event Argument Classification	EACL Findings	2023-05
Prompt for extraction? PAIE: prompting argument interaction for event argument extraction	ACL	2022-05	GitHub

Event Detection & Argument Extraction

Paper	Venue	Date	Code
TextEE: Benchmark, Reevaluation, Reflections, and Future Challenges in Event Extraction	ACL Findings	2024	GitHub
EventRL: Enhancing Event Extraction with Outcome Supervision for Large Language Models	Arxiv	2024-02
Guideline Learning for In-context Information Extraction	EMNLP	2023-12
DemoSG: Demonstration-enhanced Schema-guided Generation for Low-resource Event Extraction	EMNLP Findings	2023-12	GitHub
Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples!	EMNLP Findings	2023-12	GitHub
DICE: Data-Efficient Clinical Event Extraction with Generative Models	ACL	2023-07	GitHub
A Monte Carlo Language Model Pipeline for Zero-Shot Sociopolitical Event Extraction	NeurIPS Workshop	2023-10
STAR: Boosting Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language Models	AAAI	2024-03
DEGREE: A Data-Efficient Generative Event Extraction Model	NAACL	2022-07	GitHub
ClarET: Pre-training a correlation-aware context-to-event transformer for event-centric generation and classification	ACL	2022-05	GitHub
Dynamic prefix-tuning for generative template-based event extraction	ACL	2022-05
Text2event: Controllable sequence-to- structure generation for end-to-end event extraction	ACL	2021-08	GitHub
Document-level event argument extraction by conditional generation	NAACL	2021-06	GitHub

Universal Information Extraction

Unified models targeting multiple IE tasks.

NL-LLMs based

Paper	Venue	Date	Code
Diluie: constructing diverse demonstrations of in-context learning with large language model for unified information extraction	Others	2024-04	GitHub
ChatUIE: Exploring Chat-based Unified Information Extraction using Large Language Models	COLING	2024
YAYI-UIE: A Chat-Enhanced Instruction Tuning Framework for Universal Information Extraction	Arxiv	2024-04
Set Learning for Generative Information Extraction	EMNLP	2023-12
GIELLM: Japanese General Information Extraction Large Language Model Utilizing Mutual Reinforcement Effect	Arxiv	2023-11
InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction	Arxiv	2023-04	GitHub
Zero-Shot Information Extraction via Chatting with ChatGPT	Arxiv	2023-02	GitHub
GenIE: Generative Information Extraction	NAACL	2022-07	GitHub
DEEPSTRUCT: Pretraining of Language Models for Structure Prediction	ACL Findings	2022-05	GitHub
Unified Structure Generation for Universal Information Extraction	ACL	2022-05	GitHub
Structured prediction as translation between augmented natural languages	ICLR	2021-01	GitHub

Code-LLMs based

Paper	Venue	Date	Code
KnowCoder: Coding Structured Knowledge into LLMs for Universal Information Extraction	ACL	2024	GitHub
GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction	ICLR	2024	GitHub
Retrieval-Augmented Code Generation for Universal Information Extraction	Arxiv	2023-11
CODEIE: Large Code Generation Models are Better Few-Shot Information Extractors	ACL	2023-07	GitHub
CodeKGC: Code Language Model for Generative Knowledge Graph Construction	ACM TALLIP	2024-03	GitHub

Information Extraction Techniques

A taxonomy by techniques.

Supervised Fine-tuning

Paper	Venue	Date	Code
Rethinking Negative Instances for Generative Named Entity Recognition	ACL Findings	2024	GitHub
Beyond Single-Event Extraction: Towards Efficient Document-Level Multi-Event Argument Extraction	ACL Findings	2024	GitHub
AutoRE: Document-Level Relation Extraction with Large Language Models	ACL Demos	2024	GitHub
Recall, Retrieve and Reason: Towards Better In-Context Relation Extraction	IJCAI	2024-04
Empirical Analysis of Dialogue Relation Extraction with Large Language Models	IJCAI	2024-04
An Autoregressive Text-to-Graph Framework for Joint Entity and Relation Extraction	AAAI	2024	GitHub
Improving Recall of Large Language Models: A Model Collaboration Approach for Relational Triple Extraction	COLING	2024	GitHub
ToNER: Type-oriented Named Entity Recognition with Generative Language Model	COLING	2024
CHisIEC: An Information Extraction Corpus for Ancient Chinese History	COLING	2024	GitHub
KeyEE: Enhancing Low-Resource Generative Event Extraction with Auxiliary Keyword Sub-Prompt	Others	2024-04	GitHub
VANER: Leveraging Large Language Model for Versatile and Adaptive Biomedical Named Entity Recognition	Arxiv	2024-04	GitHub
LLMs in Biomedicine: A study on clinical Named Entity Recognition	Arxiv	2024-04
Mining experimental data from Materials Science literature with Large Language Models: an evaluation study	Arxiv	2024-04	GitHub
CRE-LLM: A Domain-Specific Chinese Relation Extraction Framework with Fine-tuned Large Language Model	Arxiv	2024-04	GitHub
Relation Extraction Using Large Language Models: A Case Study on Acupuncture Point Locations	Arxiv	2024-04
Improving Event Definition Following For Zero-Shot Event Detection	Arxiv	2024-03
Embedded Named Entity Recognition using Probing Classifiers	Arxiv	2024-03	GitHub
EventRL: Enhancing Event Extraction with Outcome Supervision for Large Language Models	Arxiv	2024-02
Structured information extraction from scientific text with large language models	Nature Communications	2024-02	GitHub
PaDeLLM-NER: Parallel Decoding in Large Language Models for Named Entity Recognition	Arxiv	2024-02
UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition	ICLR	2024	GitHub
GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction	ICLR	2024	GitHub
Set Learning for Generative Information Extraction	EMNLP	2023-12
Efficient Data Learning for Open Information Extraction with Pre-trained Language Models	EMNLP Findings	2023-12
DemoSG: Demonstration-enhanced Schema-guided Generation for Low-resource Event Extraction	EMNLP Findings	2023-12	GitHub
Calibrated Seq2seq Models for Efficient and Generalizable Ultra-fine Entity Typing	EMNLP Findings	2023-12
GIELLM: Japanese General Information Extraction Large Language Model Utilizing Mutual Reinforcement Effect	Arxiv	2023-11
GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer	Arxiv	2023-11	GitHub
Context-Aware Prompt for Generation-based Event Argument Extraction with Diffusion Models	CIKM	2023-10
Contextualized Soft Prompts for Extraction of Event Arguments	ACL Findings	2023-07
AMPERE: AMR-Aware Prefix for Generation-Based Event Argument Extraction Model	ACL	2023-07	GitHub
Debiasing Generative Named Entity Recognition by Calibrating Sequence Likelihood	ACL short	2023-07
DICE: Data-Efficient Clinical Event Extraction with Generative Models	ACL	2023-07	GitHub
Event Extraction as Question Generation and Answering	ACL short	2023-07	GitHub
NAG-NER: a Unified Non-Autoregressive Generation Framework for Various NER Tasks	ACL Industry	2023-07
Sequence generation with label augmentation for relation extraction	AAAI	2023-06	GitHub
Unified Named Entity Recognition as Multi-Label Sequence Generation	IJCNN	2023-06
InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction	Arxiv	2023-04	GitHub
Structured information extraction from complex scientific text with fine-tuned large language models	Arxiv	2022-12	Demo
Generative Entity Typing with Curriculum Learning	EMNLP	2022-12	GitHub
DORE: Document Ordered Relation Extraction based on Generative Framework	EMNLP Findings	2022-12
LasUIE: Unifying Information Extraction with Latent Adaptive Structure-aware Generative Language Model	NeurIPS	2022-10	GitHub
LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Prompting	COLING	2022-10	GitHub
GenIE: Generative Information Extraction	NAACL	2022-07	GitHub
DEGREE: A Data-Efficient Generative Event Extraction Model	NAACL	2022-07	GitHub
ClarET: Pre-training a correlation-aware context-to-event transformer for event-centric generation and classification	ACL	2022-05	GitHub
DEEPSTRUCT: Pretraining of Language Models for Structure Prediction	ACL Findings	2022-05	GitHub
Dynamic prefix-tuning for generative template-based event extraction	ACL	2022-05
Prompt for extraction? PAIE: prompting argument interaction for event argument extraction	ACL	2022-05	GitHub
Unified Structure Generation for Universal Information Extraction	ACL	2022-05	GitHub
De-bias for generative extraction in unified NER task	ACL	2022-05
Document-level Entity-based Extraction as Template Generation	EMNLP	2021-11	GitHub
REBEL: Relation Extraction By End-to-end Language generation	EMNLP Findings	2021-11	GitHub
A Unified Generative Framework for Various NER Subtasks	ACL	2021-08	GitHub
Template-Based Named Entity Recognition Using BART	ACL Findings	2021-08	GitHub
Text2event: Controllable sequence-to- structure generation for end-to-end event extraction	ACL	2021-08	GitHub
Document-level event argument extraction by conditional generation	NAACL	2021-06	GitHub
Structured prediction as translation between augmented natural languages	ICLR	2021-01	GitHub

Few-shot

Few-shot Fine-tuning

Paper	Venue	Date	Code
Diluie: constructing diverse demonstrations of in-context learning with large language model for unified information extraction	Others	2024-04	GitHub
KeyEE: Enhancing Low-Resource Generative Event Extraction with Auxiliary Keyword Sub-Prompt	Others	2024-04	GitHub
Meta In-Context Learning Makes Large Language Models Better Zero and Few-Shot Relation Extractors	IJCAI	2024-04
On-the-fly Definition Augmentation of LLMs for Biomedical NER	NAACL	2024-03	GitHub
DemoSG: Demonstration-enhanced Schema-guided Generation for Low-resource Event Extraction	EMNLP Findings	2023-12	GitHub
One Model for All Domains: Collaborative Domain-Prefx Tuning for Cross-Domain NER	IJCAI	2023-09	GitHub
LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Prompting	COLING	2022-10	GitHub
Unified Structure Generation for Universal Information Extraction	ACL	2022-05	GitHub
InstructionNER: A Multi-Task Instruction-Based Generative Framework for Few-shot NER	Arxiv	2022-03
Template-Based Named Entity Recognition Using BART	ACL Findings	2021-08	GitHub
Structured prediction as translation between augmented natural languages	ICLR	2021-01	GitHub

In-Context Learning

Paper	Venue	Date	Code
TextEE: Benchmark, Reevaluation, Reflections, and Future Challenges in Event Extraction	ACL Findings	2024	GitHub
RT: a Retrieving and Chain-of-Thought framework for few-shot medical named entity recognition	Others	2024-05	GitHub
P-ICL: Point In-Context Learning for Named Entity Recognition with Large Language Models	Arxiv	2024-06	GitHub
LTNER: Large Language Model Tagging for Named Entity Recognition with Contextualized Entity Marking	Arxiv	2024-04	GitHub
Enhancing Software-Related Information Extraction via Single-Choice Question Answering with Large Language Models	Others	2024-04
LLMs in Biomedicine: A study on clinical Named Entity Recognition	Arxiv	2024-04
Out of Sesame Street: A Study of Portuguese Legal Named Entity Recognition Through In-Context Learning	ResearchGate	2024-04
Mining experimental data from Materials Science literature with Large Language Models: an evaluation study	Arxiv	2024-04	GitHub
Empirical Analysis of Dialogue Relation Extraction with Large Language Models	IJCAI	2024-04
Self-Improving for Zero-Shot Named Entity Recognition with Large Language Models	NAACL Short	2024	GitHub
ConsistNER: Towards Instructive NER Demonstrations for LLMs with the Consistency of Ontology and Context	AAAI	2024
On-the-fly Definition Augmentation of LLMs for Biomedical NER	NAACL	2024	GitHub
CHisIEC: An Information Extraction Corpus for Ancient Chinese History	COLING	2024	GitHub
Unlocking Instructive In-Context Learning with Tabular Prompting for Relational Triple Extraction	COLING	2024
CodeKGC: Code Language Model for Generative Knowledge Graph Construction	ACM TALLIP	2024-03	GitHub
Document-Level In-Context Few-Shot Relation Extraction via Pre-Trained Language Models	Arxiv	2024-02	GitHub
In-Context Learning for Few-Shot Nested Named Entity Recognition	Arxiv	2024-02
Leveraging ChatGPT in Pharmacovigilance Event Extraction: An Empirical Study	EACL	2024-02	GitHub
Heuristic-Driven Link-of-Analogy Prompting: Enhancing Large Language Models for Document-Level Event Argument Extraction	Arxiv	2024-02
LinkNER: Linking Local Named Entity Recognition Models to Large Language Models using Uncertainty	WWW	2024
Small Language Model Is a Good Guide for Large Language Model in Chinese Entity Relation Extraction	Arxiv	2024-02
C-ICL: Contrastive In-context Learning for Information Extraction	Arxiv	2024-02
Improving Large Language Models for Clinical Named Entity Recognition via Prompt Engineering	Arxiv	2024-01	GitHub
Chain of Thought with Explicit Evidence Reasoning for Few-shot Relation Extraction	EMNLP Findings	2023-12
GPT-RE: In-context Learning for Relation Extraction using Large Language Models	EMNLP	2023-12	GitHub
Guideline Learning for In-context Information Extraction	EMNLP	2023-12
Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples!	EMNLP Findings	2023-12	GitHub
Retrieval-Augmented Code Generation for Universal Information Extraction	Arxiv	2023-11
Mastering the Task of Open Information Extraction with Large Language Models and Consistent Reasoning Environment	Arxiv	2023-10
GPT-NER: Named Entity Recognition via Large Language Models	Arxiv	2023-10	GitHub
GPT Struct Me: Probing GPT Models on Narrative Entity Extraction	WI-IAT	2023-10	GitHub
Learning In-context Learning for Named Entity Recognition	ACL	2023-07	GitHub
Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractors	ACL Findings	2023-07	GitHub
Code4Struct: Code Generation for Few-Shot Event Structure Prediction	ACL	2023-07	GitHub
CODEIE: Large Code Generation Models are Better Few-Shot Information Extractors	ACL	2023-07	GitHub
How to Unleash the Power of Large Language Models for Few-shot Relation Extraction?	ACL Workshop	2023-07	GitHub
PromptNER : Prompting For Named Entity Recognition	Arxiv	2023-06	GitHub
Unified Text Structuralization with Instruction-tuned Language Models	Arxiv	2023-03

Zero-shot

Zero-shot Prompting

Paper	Venue	Date	Code
ERA-CoT: Improving Chain-of-Thought through Entity Relationship Analysis	ACL	2024	GitHub
Astronomical Knowledge Entity Extraction in Astrophysics Journal Articles via Large Language Models	Others	2024-04
Mining experimental data from Materials Science literature with Large Language Models: an evaluation study	Arxiv	2024-04	GitHub
Empirical Analysis of Dialogue Relation Extraction with Large Language Models	IJCAI	2024-04
Retrieval-Augmented Generation-based Relation Extraction	Arxiv	2024-04	GitHub
Relation Extraction Using Large Language Models: A Case Study on Acupuncture Point Locations	Arxiv	2024-04
Meta In-Context Learning Makes Large Language Models Better Zero and Few-Shot Relation Extractors	IJCAI	2024-04
Self-Improving for Zero-Shot Named Entity Recognition with Large Language Models	NAACL Short	2024	GitHub
CodeKGC: Code Language Model for Generative Knowledge Graph Construction	ACM TALLIP	2024-03	GitHub
On-the-fly Definition Augmentation of LLMs for Biomedical NER	NAACL	2024-03	GitHub
Leveraging ChatGPT in Pharmacovigilance Event Extraction: An Empirical Study	EACL	2024-02	GitHub
A Simple but Effective Approach to Improve Structured Language Model Output for Information Extraction	Arxiv	2024-02
Small Language Model Is a Good Guide for Large Language Model in Chinese Entity Relation Extraction	Arxiv	2024-02
Improving Large Language Models for Clinical Named Entity Recognition via Prompt Engineering	Arxiv	2024-01	GitHub
Improving Unsupervised Relation Extraction by Augmenting Diverse Sentence Pairs	EMNLP	2023-12	GitHub
Prompt-NER: Zero-shot Named Entity Recognition in Astronomy Literature via Large Language Models	Arxiv	2023-10
Revisiting Large Language Models as Zero-shot Relation Extractors	EMNLP Findings	2023-10
Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractors	ACL Findings	2023-07	GitHub
Code4Struct: Code Generation for Few-Shot Event Structure Prediction	ACL	2023-07	GitHub
A Monte Carlo Language Model Pipeline for Zero-Shot Sociopolitical Event Extraction	NeurIPS Workshop	2023-10
Global Constraints with Prompting for Zero-Shot Event Argument Classification	EACL Findings	2023-05
Zero-Shot Information Extraction via Chatting with ChatGPT	Arxiv	2023-02	GitHub

Cross-Domain Learning

Paper	Venue	Date	Code
KnowCoder: Coding Structured Knowledge into LLMs for Universal Information Extraction	ACL	2024	GitHub
VerifiNER: Verification-augmented NER via Knowledge-grounded Reasoning with Large Language Models	ACL	2024	GitHub
Rethinking Negative Instances for Generative Named Entity Recognition	ACL Findings	2024	GitHub
IEPile: Unearthing Large-Scale Schema-Based Information Extraction Corpus	ACL Short	2024	GitHub
Diluie: constructing diverse demonstrations of in-context learning with large language model for unified information extraction	Others	2024-04	GitHub
Advancing Entity Recognition in Biomedicine via Instruction Tuning of Large Language Models	Bioinformatics	2024-03	GitHub
ChatUIE: Exploring Chat-based Unified Information Extraction using Large Language Models	COLING	2024
ULTRA: Unleash LLMs' Potential for Event Argument Extraction through Hierarchical Modeling and Pair-wise Refinement	Arxiv	2024-01
YAYI-UIE: A Chat-Enhanced Instruction Tuning Framework for Universal Information Extraction	Arxiv	2024-04
GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction	ICLR	2024	GitHub
UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition	ICLR	2024	GitHub
InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction	Arxiv	2023-04	GitHub
DEEPSTRUCT: Pretraining of Language Models for Structure Prediction	ACL Findings	2022-05	GitHub
Multilingual generative language models for zero-shot cross-lingual event argument extraction	ACL	2022-05	GitHub

Cross-Type Learning

Paper	Venue	Date	Code
Document-level event argument extraction by conditional generation	NAACL	2021-06	GitHub

Data Augmentation

Data Annotation

Paper	Venue	Date	Code
Astro-NER -- Astronomy Named Entity Recognition: Is GPT a Good Domain Expert Annotator?	Arxiv	2024-05
MetaIE: Distilling a Meta Model from LLM for All Kinds of Information Extraction Tasks	Arxiv	2024-03	GitHub
Augmenting NER Datasets with LLMs: Towards Automated and Refined Annotation	Arxiv	2024-03
NuNER: Entity Recognition Encoder Pre-training via LLM-Annotated Data	Arxiv	2024-02
Leveraging ChatGPT in Pharmacovigilance Event Extraction: An Empirical Study	EACL	2024-02	GitHub
LLM-DA: Data Augmentation via Large Language Models for Few-Shot Named Entity Recognition	Arxiv	2024-02
LLMaAA: Making Large Language Models as Active Annotators	EMNLP Findings	2023-12	GitHub
Improving Unsupervised Relation Extraction by Augmenting Diverse Sentence Pairs	EMNLP	2023-12	GitHub
Semi-automatic Data Enhancement for Document-Level Relation Extraction with Distant Supervision from Large Language Models	EMNLP	2023-12	GitHub
How to Unleash the Power of Large Language Models for Few-shot Relation Extraction?	ACL Workshop	2023-07	GitHub
Large Language Models as Instructors: A Study on Multilingual Clinical Entity Extraction	bioNLP Workshop	2023-07	GitHub
Does Synthetic Data Generation of LLMs Help Clinical Text Mining?	Arxiv	2023-04
Unleash GPT-2 Power for Event Detection	ACL	2021-08

Knowledge Retrieval

Paper	Venue	Date	Code
LLMs as Bridges: Reformulating Grounded Multimodal Named Entity Recognition	ACL Findings	2024	GitHub
Consistency Guided Knowledge Retrieval and Denoising in LLMs for Zero-shot Document-level Relation Triplet Extraction	WWW	2024
Learning to Rank Context for Named Entity Recognition Using a Synthetic Dataset	EMNLP	2023-12	GitHub
Prompting ChatGPT in MNER: Enhanced Multimodal Named Entity Recognition with Auxiliary Refined Knowledge	EMNLP Findings	2023-12	GitHub
Chain-of-Thought Prompt Distillation for Multimodal Named Entity Recognition and Multimodal Relation Extraction	Arxiv	2023-08

Inverse Generation

Paper	Venue	Date	Code
Distilling Named Entity Recognition Models for Endangered Species from Large Language Models	Arxiv	2024-03
Improving Event Definition Following For Zero-Shot Event Detection	Arxiv	2024-03
ProgGen: Generating Named Entity Recognition Datasets Step-by-step with Self-Reflexive Large Language Models	ACL Findings	2024	GitHub
Grasping the Essentials: Tailoring Large Language Models for Zero-Shot Relation Extraction	Arxiv	2024-02
Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extraction	EMNLP	2023-12	GitHub
Entity-to-Text based Data Augmentation for various Named Entity Recognition Tasks	ACL Findings	2023-07
Event Extraction as Question Generation and Answering	ACL Short	2023-07	GitHub
STAR: Boosting Low-Resource Event Extraction by Structure-to-Text Data Generation with Large Language Models	AAAI	2024-03

Synthetic Datasets for Instruction-tuning

Paper	Venue	Date	Code
Rethinking Negative Instances for Generative Named Entity Recognition	ACL Findings	2024	GitHub
UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition	ICLR	2024-01	GitHub
GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer	Arxiv	2023-11	GitHub
Chain-of-Thought Prompt Distillation for Multimodal Named Entity Recognition and Multimodal Relation Extraction	Arxiv	2023-08

Prompts Design

Question Answer

Paper	Venue	Date	Code
Knowledge-Enriched Prompt for Low-Resource Named Entity Recognition	TALLIP	2024-04
Enhancing Software-Related Information Extraction via Single-Choice Question Answering with Large Language Models	Others	2024-04
Revisiting Large Language Models as Zero-shot Relation Extractors	EMNLP Findings	2023-12
Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractors	ACL Findings	2023-07	GitHub
Zero-Shot Information Extraction via Chatting with ChatGPT	Arxiv	2023-02	GitHub

Chain of Thought

Paper	Venue	Date	Code
RT: a Retrieving and Chain-of-Thought framework for few-shot medical named entity recognition	Others	2024-05	GitHub
Inspire the Large Language Model by External Knowledge on BioMedical Named Entity Recognition	Arxiv	2023-09
Chain-of-Thought Prompt Distillation for Multimodal Named Entity Recognition and Multimodal Relation Extraction	Arxiv	2023-08
Revisiting Relation Extraction in the era of Large Language Models	ACL	2023-07	GitHub
Zero-shot Temporal Relation Extraction with ChatGPT	BioNLP	2023-07
PromptNER : Prompting For Named Entity Recognition	Arxiv	2023-06

Self-Improvement

Paper	Venue	Date	Code
ProgGen: Generating Named Entity Recognition Datasets Step-by-step with Self-Reflexive Large Language Models	ACL Findings	2024	GitHub
ULTRA: Unleash LLMs' Potential for Event Argument Extraction through Hierarchical Modeling and Pair-wise Refinement	Arxiv	2024-01
Self-Improving for Zero-Shot Named Entity Recognition with Large Language Models	NAACL Short	2024	GitHub

Constrained Decoding Generation

Paper	Venue	Date	Code
An Autoregressive Text-to-Graph Framework for Joint Entity and Relation Extraction	AAAI	2024-03	GitHub
Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning	EMNLP	2024-01	GitHub
DORE: Document Ordered Relation Extraction based on Generative Framework	EMNLP Findings	2022-12
Autoregressive Structured Prediction with Language Models	EMNLP Findings	2022-12	GitHub
Unified Structure Generation for Universal Information Extraction	ACL	2022-05	GitHub

Specific Domain

Paper	Domain	Venue	Date	Code
Granular Entity Mapper: Advancing Fine-grained Multimodal Named Entity Recognition and Grounding	Multimodal	EMNLP Findings	2024
LLMs as Bridges: Reformulating Grounded Multimodal Named Entity Recognition	Multimodal	ACL Findings	2024	GitHub
RT: a Retrieving and Chain-of-Thought framework for few-shot medical named entity recognition	Medical	Others	2024-05	GitHub
Astro-NER -- Astronomy Named Entity Recognition: Is GPT a Good Domain Expert Annotator?	Astronomy	Arxiv	2024-05
Astronomical Knowledge Entity Extraction in Astrophysics Journal Articles via Large Language Models	Astronomy	Others	2024-04
VANER: Leveraging Large Language Model for Versatile and Adaptive Biomedical Named Entity Recognition	Biomedical	Arxiv	2024-04	GitHub
LLMs in Biomedicine: A study on clinical Named Entity Recognition	Biomedical	Arxiv	2024-04
Enhancing Software-Related Information Extraction via Single-Choice Question Answering with Large Language Models	Software	Others	2024-04
Out of Sesame Street: A Study of Portuguese Legal Named Entity Recognition Through In-Context Learning	Legal	ResearchGate	2024-04
Mining experimental data from Materials Science literature with Large Language Models: an evaluation study	Scientific	Arxiv	2024-04	GitHub
Relation Extraction Using Large Language Models: A Case Study on Acupuncture Point Locations	Acupuncture Point	Arxiv	2024-04
Advancing Entity Recognition in Biomedicine via Instruction Tuning of Large Language Models	Biomedical	Bioinformatics	2024-03	GitHub
Distilling Named Entity Recognition Models for Endangered Species from Large Language Models	Endangered Species	Arxiv	2024-03
CHisIEC: An Information Extraction Corpus for Ancient Chinese History	Historical	COLING	2024-03	GitHub
On-the-fly Definition Augmentation of LLMs for Biomedical NER	Biomedical	NAACL	2024-03	GitHub
Improving LLM-Based Health Information Extraction with In-Context Learning	Health	Others	2024-03
Structured information extraction from scientific text with large language models	Scientific	Nat. Commun.	2024-02	GitHub
Leveraging ChatGPT in Pharmacovigilance Event Extraction: An Empirical Study	Pharmacovigilance	EACL	2024-02	GitHub
Structured information extraction from scientific text with large language models	Scientific	Nat. Commun.	2024-02	GitHub
Combining prompt‑based language models and weak supervision for labeling named entity recognition on legal documents	Legal	Others	2024-02
Improving Large Language Models for Clinical Named Entity Recognition via Prompt Engineering	Clinical	Arxiv	2024-01	GitHub
Impact of Sample Selection on In-Context Learning for Entity Extraction from Scientific Writing	Scientific	EMNLP Findings	2023-12	GitHub
Prompting ChatGPT in MNER: Enhanced Multimodal Named Entity Recognition with Auxiliary Refined Knowledge	Multimodal	ENMLP Findings	2023-12	GitHub
In-context Learning for Few-shot Multimodal Named Entity Recognition	Multimodal	ENMLP Findings	2023-12
PolyIE: A Dataset of Information Extraction from Polymer Material Scientific Literature	Polymer Material	Arxiv	2023-11	GitHub
Prompt-NER: Zero-shot Named Entity Recognition in Astronomy Literature via Large Language Models	Astronomical	Arxiv	2023-10
Inspire the Large Language Model by External Knowledge on BioMedical Named Entity Recognition	Biomedical	Arxiv	2023-09
Chain-of-Thought Prompt Distillation for Multimodal Named Entity Recognition and Multimodal Relation Extraction	Multimodal	Arxiv	2023-08
DICE: Data-Efficient Clinical Event Extraction with Generative Models	Clinical	ACL	2023-07	GitHub
How far is Language Model from 100% Few-shot Named Entity Recognition in Medical Domain	Medical	Arxiv	2023-07	GitHub
Large Language Models as Instructors: A Study on Multilingual Clinical Entity Extraction	Multilingual / Clinical	BioNLP	2023-07	GitHub
Does Synthetic Data Generation of LLMs Help Clinical Text Mining?	Clinical	Arxiv	2023-04
Yes but.. Can ChatGPT Identify Entities in Historical Documents	Historical	JCDL	2023-03
Zero-shot Clinical Entity Recognition using ChatGPT	Clinical	Arxiv	2023-03
Structured information extraction from complex scientific text with fine-tuned large language models	Scientific	Arxiv	2022-12	Demo
Multilingual generative language models for zero-shot cross-lingual event argument extraction	Multilingual	ACL	2022-05	GitHub

Evaluation and Analysis

Paper	Venue	Date	Code
TextEE: Benchmark, Reevaluation, Reflections, and Future Challenges in Event Extraction	ACL Findings	2024	GitHub
IEPile: Unearthing Large-Scale Schema-Based Information Extraction Corpus	ACL Short	2024	GitHub
CHisIEC: An Information Extraction Corpus for Ancient Chinese History	COLING	2024	GitHub
GenRES: Rethinking Evaluation for Generative Relation Extraction in the Era of Large Language Models	NAACL	2024	GitHub
Empirical Analysis of Dialogue Relation Extraction with Large Language Models	IJCAI	2024
Astro-NER -- Astronomy Named Entity Recognition: Is GPT a Good Domain Expert Annotator?	Arxiv	2024-05
Relation Extraction Using Large Language Models: A Case Study on Acupuncture Point Locations	Arxiv	2024-04
Mining experimental data from Materials Science literature with Large Language Models: an evaluation study	Arxiv	2024-04	GitHub
Distilling Named Entity Recognition Models for Endangered Species from Large Language Models	Arxiv	2024-03
LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities	Arxiv	2024-02	GitHub
Few shot clinical entity recognition in three languages: Masked language models outperform LLM prompting	Arxiv	2024-02
Information Extraction from Legal Wills: How Well Does GPT-4 Do?	EMNLP Findings	2023-12	GitHub
Information Extraction in Low-Resource Scenarios: Survey and Perspective	Arxiv	2023-12	GitHub
Empirical Study of Zero-Shot NER with ChatGPT	EMNLP	2023-12	GitHub
NERetrieve: Dataset for Next Generation Named Entity Recognition and Retrieval	EMNLP Findings	2023-12	GitHub
Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction	EMNLP	2023-12	GitHub
PolyIE: A Dataset of Information Extraction from Polymer Material Scientific Literature	Arxiv	2023-11	GitHub
XNLP: An Interactive Demonstration System for Universal Structured NLP	Arxiv	2023-08	Demo
A Zero-shot and Few-shot Study of Instruction-Finetuned Large Language Models Applied to Clinical and Biomedical Tasks	Arxiv	2023-07
How far is Language Model from 100% Few-shot Named Entity Recognition in Medical Domain	Arxiv	2023-07	GitHub
Revisiting Relation Extraction in the era of Large Language Models	ACL	2023-07	GitHub
Zero-shot Temporal Relation Extraction with ChatGPT	BioNLP	2023-07
InstructIE: A Chinese Instruction-based Information Extraction Dataset	Arxiv	2023-05	GitHub
Is Information Extraction Solved by ChatGPT? An Analysis of Performance, Evaluation Criteria, Robustness and Errors	Arxiv	2023-05	GitHub
Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness	Arxiv	2023-04	GitHub
Exploring the Feasibility of ChatGPT for Event Extraction	Arxiv	2023-03
Yes but.. Can ChatGPT Identify Entities in Historical Documents	JCDL	2023-03
Zero-shot Clinical Entity Recognition using ChatGPT	Arxiv	2023-03
Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Again	EMNLP Findings	2022-12	GitHub
Large Language Models are Few-Shot Clinical Information Extractors	EMNLP	2022-12	Huggingface

Project and Toolkit

Paper	Type	Venue	Date	Link
ONEKE	Project	-	-	Link
TechGPT-2.0: A Large Language Model Project to Solve the Task of Knowledge Graph Construction	Project	Arxiv	2024-01	Link
CollabKG: A Learnable Human-Machine-Cooperative Information Extraction Toolkit for (Event) Knowledge Graph Construction	Toolkit	Arxiv	2023-07	Link

Paper	Venue	Date	Code
Timeline-based Sentence Decomposition with In-Context Learning for Temporal Fact Extraction	ACL	2024-08	GitHub
Epidemic Information Extraction for Event-Based Surveillance using Large Language Models	ICICT	2024-08
SpeechEE: A Novel Benchmark for Speech Event Extraction	ACM MM	2024-08	GitHub
HybridRAG: Integrating Knowledge Graphs and Vector Retrieval Augmented Generation for Efficient Information Extraction	Arxiv	2024-08
Knowledge AI: Fine-tuning NLP Models for Facilitating Scientific Knowledge Extraction and Understanding	Arxiv	2024-08
Target Prompting for Information Extraction with Vision Language Model	Arxiv	2024-08
Evaluating Named Entity Recognition Using Few-Shot Prompting with Large Language Models	Arxiv	2024-08	GitHub
Utilizing Large Language Models for Named Entity Recognition in Traditional Chinese Medicine against COVID-19 Literature: Comparative Study	Arxiv	2024-08
CLLMFS: A Contrastive Learning enhanced Large Language Model Framework for Few-Shot Named Entity Recognition	ECAI	2024-08
LLMs are not Zero-Shot Reasoners for Biomedical Information Extraction	Arxiv	2024-08
Label Alignment and Reassignment with Generalist Large Language Model for Enhanced Cross-Domain Named Entity Recognition	Arxiv	2024-07
MMM: Multilingual Mutual Reinforcement Effect Mix Datasets & Test with Open-domain Information Extraction Large Language Models	Arxiv	2024-08	GitHub
FsPONER: Few-shot Prompt Optimization for Named Entity Recognition in Domain-specific Scenarios	ECAI	2024-07	GitHub
Adapting Multilingual LLMs to Low-Resource Languages with Knowledge Graphs via Adapters	KaLLM workshop	2024-07	GitHub
Show Less, Instruct More: Enriching Prompts with Definitions and Guidelines for Zero-Shot NER	Arxiv	2024-07
Large Language Models Struggle in Token-Level Clinical Named Entity Recognition	AMIA	2024-08
GLiNER multi-task: Generalist Lightweight Model for Various Information Extraction Tasks	Arxiv	2024-08
Retrieval Augmented Instruction Tuning for Open NER with Large Language Models	Arxiv	2024-06	GitHub
Beyond Boundaries: Learning a Universal Entity Taxonomy across Datasets and Languages for Open Named Entity Recognition	Arxiv	2024-06	GitHub
Fighting Against the Repetitive Training and Sample Dependency Problem in Few-shot Named Entity Recognition	IEEE Access	2024-06	GitHub
llmNER: (Zero\|Few)-Shot Named Entity Recognition, Exploiting the Power of Large Language Models	Arxiv	2024-06	GitHub
Assessing the Performance of Chinese Open Source Large Language Models in Information Extraction Tasks	Arxiv	2024-06

Datasets

* denotes the dataset is multimodal. # refers to the number of categories or sentences.

<table> <thead> <tr> <th align="center">Task</th> <th align="center">Dataset</th> <th align="center">Domain</th> <th align="center">#Class</th> <th align="center">#Train</th> <th align="center">#Val</th> <th align="center">#Test</th> <th align="center">Link</th> </tr> </thead> <tbody> <tr> <td align="center" rowspan="32" ><strong>NER</strong></td> <td align="center">ACE04</td> <td align="center">News</td> <td align="center">7</td> <td align="center">6202</td> <td align="center">745</td> <td align="center">812</td> <td align="center"><a href="https://catalog.ldc.upenn.edu/LDC2005T09">Link</a></td> </tr> <tr> <td align="center">ACE05</td> <td align="center">News</td> <td align="center">7</td> <td align="center">7299</td> <td align="center">971</td> <td align="center">1060</td> <td align="center"><a href="https://catalog.ldc.upenn.edu/LDC2006T06">Link</a></td> </tr> <tr> <td align="center">BC5CDR</td> <td align="center">Biomedical</td> <td align="center">2</td> <td align="center">4560</td> <td align="center">4581</td> <td align="center">4797</td> <td align="center"><a href="https://biocreative.bioinformatics.udel.edu/tasks/biocreative-v/track-3-cdr/">Link</a></td> </tr> <tr> <td align="center">Broad Twitter Corpus</td> <td align="center">Social Media</td> <td align="center">3</td> <td align="center">6338</td> <td align="center">1001</td> <td align="center">2000</td> <td align="center"><a href="https://github.com/GateNLP/broad_twitter_corpus?">Link</a></td> </tr> <tr> <td align="center">CADEC</td> <td align="center">Biomedical</td> <td align="center">1</td> <td align="center">5340</td> <td align="center">1097</td> <td align="center">1160</td> <td align="center"><a href="https://data.csiro.au/collection/csiro:10948?v=3&d=true">Link</a></td> </tr> <tr> <td align="center">CoNLL03</td> <td align="center">News</td> <td align="center">4</td> <td align="center">14041</td> <td align="center">3250</td> <td align="center">3453</td> <td align="center"><a href="https://www.clips.uantwerpen.be/conll2003/ner/">Link</a></td> </tr> <tr> <td align="center">CoNLLpp</td> <td align="center">News</td> <td align="center">4</td> <td align="center">14041</td> <td align="center">3250</td> <td align="center">3453</td> <td align="center"><a href="https://github.com/ZihanWangKi/CrossWeigh">Link</a></td> </tr> <tr> <td align="center">CrossNER-AI</td> <td align="center">Artificial Intelligence</td> <td align="center">14</td> <td align="center">100</td> <td align="center">350</td> <td align="center">431</td> <td align="center" rowspan="5" ><a href="https://github.com/zliucr/CrossNER">Link</a></td> </tr> <tr> <td align="center">CrossNER-Literature</td> <td align="center">Literary</td> <td align="center">12</td> <td align="center">100</td> <td align="center">400</td> <td align="center">416</td> </tr> <tr> <td align="center">CrossNER-Music</td> <td align="center">Musical</td> <td align="center">13</td> <td align="center">100</td> <td align="center">380</td> <td align="center">465</td> </tr> <tr> <td align="center">CrossNER-Politics</td> <td align="center">Political</td> <td align="center">9</td> <td align="center">199</td> <td align="center">540</td> <td align="center">650</td> </tr> <tr> <td align="center">CrossNER-Science</td> <td align="center">Scientific</td> <td align="center">17</td> <td align="center">200</td> <td align="center">450</td> <td align="center">543</td> </tr> <tr> <td align="center">FabNER</td> <td align="center">Scientific</td> <td align="center">12</td> <td align="center">9435</td> <td align="center">2182</td> <td align="center">2064</td> <td align="center"><a href="https://huggingface.co/datasets/DFKI-SLT/fabner">Link</a></td> </tr> <tr> <td align="center">Few-NERD</td> <td align="center">General</td> <td align="center">66</td> <td align="center">131767</td> <td align="center">18824</td> <td align="center">37468</td> <td align="center"><a href="https://github.com/thunlp/Few-NERD">Link</a></td> </tr> <tr> <td align="center">FindVehicle</td> <td align="center">Traffic</td> <td align="center">21</td> <td align="center">21565</td> <td align="center">20777</td> <td align="center">20777</td> <td align="center"><a href="https://github.com/GuanRunwei/VehicleFinder-CTIM">Link</a></td> </tr> <tr> <td align="center">GENIA</td> <td align="center">Biomedical</td> <td align="center">5</td> <td align="center">15023</td> <td align="center">1669</td> <td align="center">1854</td> <td align="center"><a href="https://github.com/ljynlp/W2NER?tab=readme-ov-file#3-dataset">Link</a></td> </tr> <tr> <td align="center">HarveyNER</td> <td align="center">Social Media</td> <td align="center">4</td> <td align="center">3967</td> <td align="center">1301</td> <td align="center">1303</td> <td align="center"><a href="https://github.com/brickee/HarveyNER">Link</a></td> </tr> <tr> <td align="center">MIT-Movie</td> <td align="center">Social Media</td> <td align="center">12</td> <td align="center">9774</td> <td align="center">2442</td> <td align="center">2442</td> <td align="center"><a href="https://tianchi.aliyun.com/dataset/145106">Link</a></td> </tr> <tr> <td align="center">MIT-Restaurant</td> <td align="center">Social Media</td> <td align="center">8</td> <td align="center">7659</td> <td align="center">1520</td> <td align="center">1520</td> <td align="center"><a href="https://tianchi.aliyun.com/dataset/145105">Link</a></td> </tr> <tr> <td align="center">MultiNERD</td> <td align="center">Wikipedia</td> <td align="center">16</td> <td align="center">134144</td> <td align="center">10000</td> <td align="center">10000</td> <td align="center"><a href="https://github.com/Babelscape/multinerd">Link</a></td> </tr> <tr> <td align="center">NCBI</td> <td align="center">Biomedical</td> <td align="center">4</td> <td align="center">5432</td> <td align="center">923</td> <td align="center">940</td> <td align="center"><a href="https://www.ncbi.nlm.nih.gov/CBBresearch/Dogan/DISEASE/">Link</a></td> </tr> <tr> <td align="center">OntoNotes 5.0</td> <td align="center">General</td> <td align="center">18</td> <td align="center">59924</td> <td align="center">8528</td> <td align="center">8262</td> <td align="center"><a href="https://catalog.ldc.upenn.edu/LDC2013T19">Link</a></td> </tr> <tr> <td align="center">ShARe13</td> <td align="center">Biomedical</td> <td align="center">1</td> <td align="center">8508</td> <td align="center">12050</td> <td align="center">9009</td> <td align="center"><a href="https://physionet.org/content/shareclefehealth2013/1.0/">Link</a></td> </tr> <tr> <td align="center">ShARe14</td> <td align="center">Biomedical</td> <td align="center">1</td> <td align="center">17404</td> <td align="center">1360</td> <td align="center">15850</td> <td align="center"><a href="https://physionet.org/content/shareclefehealth2014task2/1.0/">Link</a></td> </tr> <tr> <td align="center">SNAP<sup>*<sup></td> <td align="center">Social Media</td> <td align="center">4</td> <td align="center">4290</td> <td align="center">1432</td> <td align="center">1459</td> <td align="center"><a href="https://www.modelscope.cn/datasets/caijiong_sijun/MoRE-processed-data/files">Link</a></td> </tr> <tr> <td align="center">Temporal Twitter Corpus (TTC)</td> <td align="center">Social Meida</td> <td align="center">3</td> <td align="center">10000</td> <td align="center">500</td> <td align="center">1500</td> <td aligh="center"><a href="https://github.com/shrutirij/temporal-twitter-corpus">Link</a></td> </tr> <tr> <td align="center">Tweebank-NER</td> <td align="center">Social Media</td> <td align="center">4</td> <td align="center">1639</td> <td align="center">710</td> <td align="center">1201</td> <td aligh="center"><a href="https://github.com/mit-ccc/TweebankNLP">Link</a></td> </tr> <tr> <td align="center">Twitter2015<sup>*<sup></td> <td align="center">Social Media</td> <td align="center">4</td> <td align="center">4000</td> <td align="center">1000</td> <td align="center">3357</td> <td align="center"><a href="https://www.modelscope.cn/datasets/caijiong_sijun/MoRE-processed-data/files">Link</a></td> </tr> <tr> <td align="center">Twitter2017<sup>*<sup></td> <td align="center">Social Media</td> <td align="center">4</td> <td align="center">3373</td> <td align="center">723</td> <td align="center">723</td> <td align="center"><a href="https://www.modelscope.cn/datasets/caijiong_sijun/MoRE-processed-data/files">Link</a></td> </tr> <tr> <td align="center">TwitterNER7</td> <td align="center">Social Media</td> <td align="center">7</td> <td align="center">7111</td> <td align="center">886</td> <td align="center">576</td> <td aligh="center"><a href="https://huggingface.co/datasets/tner/tweetner7">Link</a></td> </tr> <tr> <td align="center">WikiDiverse<sup>*<sup></td> <td align="center">News</td> <td align="center">13</td> <td align="center">6312</td> <td align="center">755</td> <td align="center">757</td> <td aligh="center"><a href="https://github.com/wangxw5/wikidiverse?tab=readme-ov-file#get-the-data">Link</a></td> </tr> <tr> <td align="center">WNUT2017</td> <td align="center">Social Media</td> <td align="center">6</td> <td align="center">3394</td> <td align="center">1009</td> <td align="center">1287</td> <td aligh="center"><a href="https://tianchi.aliyun.com/dataset/144349">Link</a></td> </tr> <tr> <td align="center" rowspan="11"><strong>RE</strong></td> <td align="center">ACE05</td> <td align="center">News</td> <td align="center">7</td> <td align="center">10051</td> <td align="center">2420</td> <td align="center">2050</td> <td align="center"><a href="https://catalog.ldc.upenn.edu/LDC2006T06">Link</a></td> </tr> <tr> <td align="center">ADE</td> <td align="center">Biomedical</td> <td align="center">1</td> <td align="center">3417</td> <td align="center">427</td> <td align="center">428</td> <td align="center"><a href="http://lavis.cs.hs-rm.de/storage/spert/public/datasets/ade/">Link</a></td> </tr> <tr> <td align="center">CoNLL04</td> <td align="center">News</td> <td align="center">5</td> <td align="center">922</td> <td align="center">231</td> <td align="center">288</td> <td align="center"><a href="http://lavis.cs.hs-rm.de/storage/spert/public/datasets/conll04/">Link</a></td> </tr> <tr> <td align="center">DocRED</td> <td align="center">Wikipedia</td> <td align="center">96</td> <td align="center">3008</td> <td align="center">300</td> <td align="center">700</td> <td align="center"><a href="https://github.com/thunlp/DocRED">Link</a></td> </tr> <tr> <td align="center">MNRE<sup>*<sup></td> <td align="center">Social Media</td> <td align="center">23</td> <td align="center">12247</td> <td align="center">1624</td> <td align="center">1614</td> <td align="center"><a href="https://github.com/thecharm/MNRE">Link</a></td> </tr> <tr> <td align="center">NYT</td> <td align="center">News</td> <td align="center">24</td> <td align="center">56196</td> <td align="center">5000</td> <td align="center">5000</td> <td align="center"><a href="https://github.com/thunlp/OpenNRE/blob/master/benchmark/download_nyt10.sh">Link</a></td> </tr> <tr> <td align="center">Re-TACRED</td> <td align="center">News</td> <td align="center">40</td> <td align="center">58465</td> <td align="center">19584</td> <td align="center">13418</td> <td align="center"><a href="https://github.com/gstoica27/Re-TACRED">Link</a></td> </tr> <tr> <td align="center">SciERC</td> <td align="center">Scientific</td> <td align="center">7</td> <td align="center">1366</td> <td align="center">187</td> <td align="center">397</td> <td align="center"><a href="http://lavis.cs.hs-rm.de/storage/spert/public/datasets/scierc/">Link</a></td> </tr> <tr> <td align="center">SemEval2010</td> <td align="center">General</td> <td align="center">19</td> <td align="center">6507</td> <td align="center">1493</td> <td align="center">2717</td> <td align="center"><a href="https://github.com/thunlp/OpenNRE/blob/master/benchmark/download_semeval.sh">Link</a></td> </tr> <tr> <td align="center">TACRED</td> <td align="center">News</td> <td align="center">42</td> <td align="center">68124</td> <td align="center">22631</td> <td align="center">15509</td> <td align="center"><a href="https://nlp.stanford.edu/projects/tacred/">Link</a></td> </tr> <tr> <td align="center">TACREV</td> <td align="center">News</td> <td align="center">42</td> <td align="center">68124</td> <td align="center">22631</td> <td align="center">15509</td> <td align="center"><a href="https://github.com/DFKI-NLP/tacrev">Link</a></td> </tr> <tr> <td align="center" rowspan="7"><strong>EE</strong></td> <td align="center">ACE05</td> <td align="center">News</td> <td align="center">33/22</td> <td align="center">17172</td> <td align="center">923</td> <td align="center">832</td> <td align="center"><a href="https://catalog.ldc.upenn.edu/LDC2006T06">Link</a></td> </tr> <tr> <td align="center">CASIE</td> <td align="center">Cybersecurity</td> <td align="center">5/26</td> <td align="center">11189</td> <td align="center">1778</td> <td align="center">3208</td> <td align="center"><a href="https://github.com/Ebiquity/CASIE">Link</a></td> </tr> <tr> <td align="center">GENIA11</td> <td align="center">Biomedical</td> <td align="center">9/11</td> <td align="center">8730</td> <td align="center">1091</td> <td align="center">1092</td> <td align="center"><a href="https://bionlp-st.dbcls.jp/GE/2011/eval-test/">Link</a></td> </tr> <tr> <td align="center">GENIA13</td> <td align="center">Biomedical</td> <td align="center">13/7</td> <td align="center">4000</td> <td align="center">500</td> <td align="center">500</td> <td align="center"><a href="https://bionlp-st.dbcls.jp/GE/2013/eval-test/">Link</a></td> </tr> <tr> <td align="center">PHEE</td> <td align="center">Biomedical</td> <td align="center">2/16</td> <td align="center">2898</td> <td align="center">961</td> <td align="center">968</td> <td align="center"><a href="https://github.com/ZhaoyueSun/PHEE">Link</a></td> </tr> <tr> <td align="center">RAMS</td> <td align="center">News</td> <td align="center">139/65</td> <td align="center">7329</td> <td align="center">924</td> <td align="center">871</td> <td align="center"><a href="https://nlp.jhu.edu/rams/">Link</a></td> </tr> <tr> <td align="center">WikiEvents</td> <td align="center">Wikipedia</td> <td align="center">50/59</td> <td align="center">5262</td> <td align="center">378</td> <td align="center">492</td> <td align="center"><a href="https://github.com/raspberryice/gen-arg">Link</a></td> </tr> </tbody> </table>