Awesome

Low-resource Information Extraction 🚀

🍎 The repository is a paper set on low-resource information extraction (IE), mainly including NER, RE and EE, which is generally categorized into two paradigms:

• Traditional Low-Resource IE approaches
  • Exploiting Higher-Resource Data
  • Developing Stronger Data-Efficient Models
  • Optimizing Data and Models Together
• LLM-based Low-Resource IE approaches
  • Direct Inference Without Tuning
  • Model Specialization With Tuning

🤗 We strongly encourage the researchers who want to promote their fantastic work for the community to make pull request and update their papers in this repository!

📖 Survey Paper: Information Extraction in Low-Resource Scenarios: Survey and Perspective (ICKG 2024) [paper]

🗂️ Slides:

• Data-Efficient Knowledge Graph Construction, 高效知识图谱构建 (Tutorial on CCKS 2022) [slides]
• Efficient and Robust Knowledge Graph Construction (Tutorial on AACL-IJCNLP 2022) [paper, slides]
• Open-Environment Knowledge Graph Construction and Reasoning: Challenges, Approaches, and Opportunities (Tutorial on IJCAI 2023) [slides]

Content

Preliminaries

• 🛠️ Low-Resource IE Toolkits
  • Traditional Toolkits
  • LLM-Based Toolkits
• 📊 Low-Resource IE Datasets
  • Low-Resource NER
  • Low-Resource RE
  • Low-Resource EE
• 📖 Related Surveys/Analysis on Low-Resource IE
  • Information Extraction
  • Low-Resource NLP Learning

🍎Traditional Methods🍎

• 1. Exploiting Higher-Resource Data
  • 1.1 Weakly Supervised Augmentation
  • 1.2 Multimodal Augmentation
  • 1.3 Multi-Lingual Augmentation
  • 1.4 Auxiliary Knowledge Enhancement
• 2. Developing Stronger Data-Efficient Models
  • 2.1 Meta Learning
  • 2.2 Transfer Learning
  • 2.3 Fine-Tuning PLM
• 3. Optimizing Data and Models Together
  • 3.1 Multi-Task Learning
  • 3.2 Task Reformulation
  • 3.3 Prompt-Tuning PLM

🍏LLM-Based Methods🍏

• Direct Inference Without Tuning
  • Instruction Prompting
  • Code Prompting
  • In-Context Learning
• Model Specialization With Tuning
  • Prompt-Tuning LLM
  • Fine-Tuning LLM

How to Cite

Preliminaries

🛠️ Low-Resource IE Toolkits

Traditional Toolkits

• DeepKE: A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Population [paper, project]
• OpenUE: An Open Toolkit of Universal Extraction from Text [paper, project]
• Zshot: An Open-source Framework for Zero-Shot Named Entity Recognition and Relation Extraction [paper, project]
• OpenNRE [paper, project]
• OmniEvent [paper1, paper2, project]

LLM-Based Toolkits

• CollabKG: A Learnable Human-Machine-Cooperative Information Extraction Toolkit for (Event) Knowledge Graph Construction [paper]
• GPT4IE [project]
• ChatIE [paper, project]
• TechGPT: Technology-Oriented Generative Pretrained Transformer [project]
• TechGPT-2.0: A Large Language Model Project to Solve the Task of Knowledge Graph Construction [paper, project]
• AutoKG: LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities [paper, project]
• KnowLM [project]

📊 Low-Resource IE Datasets

Low-Resource NER

• {Few-NERD}: Few-NERD: A Few-shot Named Entity Recognition Dataset (EMNLP 2021) [paper, data]

Low-Resource RE

• {FewRel}: FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation (EMNLP 2018) [paper, data]
• {FewRel2.0}: FewRel 2.0: Towards More Challenging Few-Shot Relation Classification (EMNLP 2019) [paper, data]
• {Wiki-ZSL}: ZS-BERT: Towards Zero-Shot Relation Extraction with Attribute Representation Learning (NAACL 2021) [paper, data]
• {Entail-RE}: Low-resource Extraction with Knowledge-aware Pairwise Prototype Learning (Knowledge-Based Systems, 2022) [paper, data]
• {LREBench}: Towards Realistic Low-resource Relation Extraction: A Benchmark with Empirical Baseline Study (EMNLP 2022, Findings) [paper, data]

Low-Resource EE

• {FewEvent}: Meta-Learning with Dynamic-Memory-Based Prototypical Network for Few-Shot Event Detection (WSDM 2020) [paper, data]
• {Causal-EE}: Low-resource Extraction with Knowledge-aware Pairwise Prototype Learning (Knowledge-Based Systems, 2022) [paper, data]
• {OntoEvent}: OntoED: Low-resource Event Detection with Ontology Embedding (ACL 2021) [paper, data]
• {FewDocAE}: Few-Shot Document-Level Event Argument Extraction (ACL 2023) [paper, data]

📖 Related Surveys and Analysis on Low-Resource IE

Information Extraction

NER

• A Survey on Recent Advances in Named Entity Recognition from Deep Learning Models (COLING 2018) [paper]
• A Survey on Deep Learning for Named Entity Recognition (TKDE, 2020) [paper]
• Few-Shot Named Entity Recognition: An Empirical Baseline Study (EMNLP 2021) [paper]
• Few-shot Named Entity Recognition: definition, taxonomy and research directions (TIST, 2023) [paper]
• Comprehensive Overview of Named Entity Recognition: Models, Domain-Specific Applications and Challenges (arXiv, 2023) [paper]
• A Survey on Recent Advances in Named Entity Recognition (arXiv, 2024) [paper]

RE

• A Survey on Neural Relation Extraction (Science China Technological Sciences, 2020) [paper]
• Relation Extraction: A Brief Survey on Deep Neural Network Based Methods (ICSIM 2021) [paper]
• Revisiting Few-shot Relation Classification: Evaluation Data and Classification Schemes (TACL, 2021) [paper]
• Deep Neural Network-Based Relation Extraction: An Overview (Neural Computing and Applications, 2022) [paper]
• Revisiting Relation Extraction in the era of Large Language Models (ACL 2023) [paper]

EE

• A Survey of Event Extraction From Text (ACCESS, 2019) [paper]
• What is Event Knowledge Graph: A Survey (TKDE, 2022) [paper]
• A Survey on Deep Learning Event Extraction: Approaches and Applications (TNNLS, 2022) [paper]
• Event Extraction: A Survey (2022) [paper]
• Low Resource Event Extraction: A Survey (2022) [paper]
• Few-shot Event Detection: An Empirical Study and a Unified View (ACL 2023) [paper]
• Exploring the Feasibility of ChatGPT for Event Extraction (arXiv, 2023) [paper]
• A Reevaluation of Event Extraction: Past, Present, and Future Challenges (arXiv, 2023) [paper]
• ULTRA: Unleash LLMs' Potential for Event Argument Extraction through Hierarchical Modeling and Pair-wise Refinement (arXiv, 2023) [paper]

General IE

Traditional IE

• From Information to Knowledge: Harvesting Entities and Relationships from Web Sources (PODS 2010) [paper]
• Knowledge Base Population: Successful Approaches and Challenges (ACL 2011) [paper]
• Advances in Automated Knowledge Base Construction (NAACL-HLC 2012, AKBC-WEKEX workshop) [paper]
• Information Extraction (IEEE Intelligent Systems, 2015) [paper]
• Populating Knowledge Bases (Part of The Information Retrieval Series book series, 2018) [paper]
• A Survey on Open Information Extraction (COLING 2018) [paper]
• A Survey on Automatically Constructed Universal Knowledge Bases (Journal of Information Science, 2020) [paper]
• Machine Knowledge: Creation and Curation of Comprehensive Knowledge Bases (Foundations and Trends in Databases, 2021) [paper]
• A Survey on Knowledge Graphs: Representation, Acquisition and Applications (TNNLS, 2021) [paper]
• Neural Symbolic Reasoning with Knowledge Graphs: Knowledge Extraction, Relational Reasoning, and Inconsistency Checking (Fundamental Research, 2021) [paper]
• A Survey on Neural Open Information Extraction: Current Status and Future Directions (IJCAI 2022) [paper]
• A Survey of Information Extraction Based on Deep Learning (Applied Sciences, 2022) [paper]
• Generative Knowledge Graph Construction: A Review (EMNLP 2022) [paper]
• Multi-Modal Knowledge Graph Construction and Application: A Survey (TKDE, 2022) [paper]
• A Survey on Multimodal Knowledge Graphs: Construction, Completion and Applications (Mathematics, 2023) [paper]
• Construction of Knowledge Graphs: State and Challenges (Submitted to Semantic Web Journal, 2023) [paper]

LLM-based IE

• Empirical Study of Zero-Shot NER with ChatGPT (EMNLP 2023) [paper]
• Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples! (EMNLP 2023, Findings) [paper]
• Evaluating ChatGPT’s Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness (arXiv, 2023) [paper]
• Is Information Extraction Solved by ChatGPT? An Analysis of Performance, Evaluation Criteria, Robustness and Errors (arXiv, 2023) [paper]
• Improving Open Information Extraction with Large Language Models: A Study on Demonstration Uncertainty (arXiv, 2023) [paper]
• LOKE: Linked Open Knowledge Extraction for Automated Knowledge Graph Construction (arXiv, 2023) [paper]
• LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities (arXiv, 2023) [paper]
• Large Language Models for Generative Information Extraction: A Survey (arXiv, 2023) [paper]
• Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extraction (EMNLP 2023) [paper]
• LLMaAA: Making Large Language Models as Active Annotators (EMNLP 2023, Findings) [paper]
• Large Language Models and Knowledge Graphs: Opportunities and Challenges (TGDK, 2023) [paper]
• Unifying Large Language Models and Knowledge Graphs: A Roadmap (arXiv, 2023) [paper]
• Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applications (arXiv, 2023) [paper]
• Large Knowledge Model: Perspectives and Challenges (arXiv, 2023) [paper]
• Knowledge Bases and Language Models: Complementing Forces (RuleML+RR, 2023) [paper]
• StructGPT: A General Framework for Large Language Model to Reason over Structured Data (EMNLP 2023) [paper]
• Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction (EMNLP 2023) [paper]

Low-Resource NLP Learning

• A Survey of Zero-Shot Learning: Settings, Methods, and Applications (TIST, 2019) [paper]
• A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios (NAACL 2021) [paper]
• A Survey on Low-Resource Neural Machine Translation (IJCAI 2021) [paper]
• Generalizing from a Few Examples: A Survey on Few-shot Learning (ACM Computing Surveys, 2021) [paper]
• Knowledge-aware Zero-Shot Learning: Survey and Perspective (IJCAI 2021) [paper]
• Generalizing to Unseen Elements: A Survey on Knowledge Extrapolation for Knowledge Graphs (IJCAI 2023) [paper]
• Zero-shot and Few-shot Learning with Knowledge Graphs: A Comprehensive Survey (Proceedings of the IEEE, 2023) [paper]
• A Survey on Machine Learning from Few Samples (Pattern Recognition, 2023) [paper]
• Multi-Hop Knowledge Graph Reasoning in Few-Shot Scenarios (TKDE, 2023) [paper]
• An Empirical Survey of Data Augmentation for Limited Data Learning in NLP (TACL, 2023) [paper]
• Efficient Methods for Natural Language Processing: A Survey (TACL, 2023) [paper]

🍎 Traditional Methods 🍎

1 Exploiting Higher-Resource Data

Weakly Supervised Augmentation

• Distant Supervision for Relation Extraction without Labeled Data (ACL 2009) [paper]
• Modeling Missing Data in Distant Supervision for Information Extraction (TACL, 2013) [paper]
• Neural Relation Extraction with Selective Attention over Instances (ACL 2016) [paper]
• Automatically Labeled Data Generation for Large Scale Event Extraction (ACL 2017) [paper]
• CoType: Joint Extraction of Typed Entities and Relations with Knowledge Bases (WWW 2017) [paper]
• Adversarial Training for Weakly Supervised Event Detection (NAACL 2019) [paper]
• Local Additivity Based Data Augmentation for Semi-supervised NER (EMNLP 2020) [paper]
• BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervision (KDD 2020) [paper]
• Gradient Imitation Reinforcement Learning for Low Resource Relation Extraction (EMNLP 2021) [paper]
• Noisy-Labeled NER with Confidence Estimation (NAACL 2021) [paper]
• ANEA: Distant Supervision for Low-Resource Named Entity Recognition (ICLR 2021, Workshop of Practical Machine Learning For Developing Countries) [paper]
• Finding Influential Instances for Distantly Supervised Relation Extraction (COLING 2022) [paper]
• Better Sampling of Negatives for Distantly Supervised Named Entity Recognition (ACL 2023, Findings) [paper]
• Jointprop: Joint Semi-supervised Learning for Entity and Relation Extraction with Heterogeneous Graph-based Propagation (ACL 2023) [paper]

Multimodal Augmentation

• Visual Attention Model for Name Tagging in Multimodal Social Media (ACL 2018) [paper]
• Visual Relation Extraction via Multi-modal Translation Embedding Based Model (PAKDD 2018) [paper]
• Cross-media Structured Common Space for Multimedia Event Extraction (ACL 2020) [paper]
• Image Enhanced Event Detection in News Articles (AAAI 2020) [paper]
• Joint Multimedia Event Extraction from Video and Article (EMNLP 2021, Findings) [paper]
• Multimodal Relation Extraction with Efficient Graph Alignment (MM 2021) [paper]
• Hybrid Transformer with Multi-level Fusion for Multimodal Knowledge Graph Completion (SIGIR 2022) [paper]
• Good Visual Guidance Makes A Better Extractor: Hierarchical Visual Prefix for Multimodal Entity and Relation Extraction (NAACL 2022, Findings) [paper]

Multi-Lingual Augmentation

• Low-Resource Named Entity Recognition with Cross-lingual, Character-Level Neural Conditional Random Fields (IJCNLP 2017) [paper]
• Neural Relation Extraction with Multi-lingual Attention (ACL 2017) [paper]
• Improving Low Resource Named Entity Recognition using Cross-lingual Knowledge Transfer (IJCAI 2018) [paper]
• Event Detection via Gated Multilingual Attention Mechanism (AAAI 2018) [paper]
• Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning (COLING 2022) [paper]
• Cross-lingual Transfer Learning for Relation Extraction Using Universal Dependencies (Computer Speech & Language, 2022) [paper]
• Language Model Priming for Cross-Lingual Event Extraction (AAAI 2022) [paper]
• Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction (ACL 2022) [paper]
• PRAM: An End-to-end Prototype-based Representation Alignment Model for Zero-resource Cross-lingual Named Entity Recognition (ACL 2023, Findings) [paper]
• Retrieving Relevant Context to Align Representations for Cross-lingual Event Detection (ACL 2023, Findings) [paper]
• Hybrid Knowledge Transfer for Improved Cross-Lingual Event Detection via Hierarchical Sample Selection (ACL 2023) [paper]

Auxiliary Knowledge Enhancement

(1) Textual Knowledge (Type-related Knowledge & Synthesized Data)

• Zero-Shot Relation Extraction via Reading Comprehension (CoNLL 2017) [paper]
• Zero-Shot Open Entity Typing as Type-Compatible Grounding (EMNLP 2018) [paper]
• Description-Based Zero-shot Fine-Grained Entity Typing (NAACL 2019) [paper]
• Improving Event Detection via Open-domain Trigger Knowledge (ACL 2020) [paper]
• ZS-BERT: Towards Zero-Shot Relation Extraction with Attribute Representation Learning (NAACL 2021) [paper]
• MapRE: An Effective Semantic Mapping Approach for Low-resource Relation Extraction (EMNLP 2021) [paper]
• Distilling Discrimination and Generalization Knowledge for Event Detection via Delta-Representation Learning (ACL 2021) [paper]
• MELM: Data Augmentation with Masked Entity Language Modeling for Low-Resource NER (ACL 2022) [paper]
• Mask-then-Fill: A Flexible and Effective Data Augmentation Framework for Event Extraction (EMNLP 2022, Findings) [paper]
• Low-Resource NER by Data Augmentation With Prompting (IJCAI 2022) [paper]
• ACLM: A Selective-Denoising based Generative Data Augmentation Approach for Low-Resource Complex NER (ACL 2023) [paper]
• Entity-to-Text based Data Augmentation for Various Named Entity Recognition Tasks (ACL 2023, Findings) [paper]
• Improving Low-resource Named Entity Recognition with Graph Propagated Data Augmentation (ACL 2023, Short) [paper]
• GDA: Generative Data Augmentation Techniques for Relation Extraction Tasks (ACL 2023, Findings) [paper]
• Generating Labeled Data for Relation Extraction: A Meta Learning Approach with Joint GPT-2 Training (ACL 2023, Findings) [paper]
• RE-Matching: A Fine-Grained Semantic Matching Method for Zero-Shot Relation Extraction (ACL 2023) [paper]
• S2ynRE: Two-stage Self-training with Synthetic Data for Low-resource Relation Extraction (ACL 2023) [paper]
• Improving Unsupervised Relation Extraction by Augmenting Diverse Sentence Pairs (EMNLP 2023) [paper]
• DAFS: A Domain Aware Few Shot Generative Model for Event Detection (Machine Learning, 2023) [paper]
• Enhancing Few-shot NER with Prompt Ordering based Data Augmentation (arXiv, 2023) [paper]
• SegMix: A Simple Structure-Aware Data Augmentation Method (arXiv, 2023) [paper]
• Learning to Rank Context for Named Entity Recognition Using a Synthetic Dataset (EMNLP 2023) [paper]
• Semi-automatic Data Enhancement for Document-Level Relation Extraction with Distant Supervision from Large Language Models (EMNLP 2023) [paper]
• STAR: Boosting Low-Resource Event Extraction by Structure-to-Text Data Generation with Large Language Models (arXiv, 2023) [paper]
• LLM-DA: Data Augmentation via Large Language Models for Few-Shot Named Entity Recognition (arXiv, 2024) [paper]

(2) Structured Knowledge (KG & Ontology & Logical Rules)

• Leveraging FrameNet to Improve Automatic Event Detection (ACL 2016) [paper]
• DOZEN: Cross-Domain Zero Shot Named Entity Recognition with Knowledge Graph (SIGIR 2021) [paper]
• Connecting the Dots: Event Graph Schema Induction with Path Language Modeling (EMNLP 2020) [paper]
• Logic-guided Semantic Representation Learning for Zero-Shot Relation Classification (COLING 2020) [paper]
• NERO: A Neural Rule Grounding Framework for Label-Efficient Relation Extraction (WWW 2020) [paper]
• Knowledge-aware Named Entity Recognition with Alleviating Heterogeneity (AAAI 2021) [paper]
• OntoED: Low-resource Event Detection with Ontology Embedding (ACL 2021) [paper]
• Low-resource Extraction with Knowledge-aware Pairwise Prototype Learning (Knowledge-Based Systems, 2022) [paper]

2 Developing Stronger Data-Efficient Models

Meta Learning

For Low-Resource NER

• Few-shot Classification in Named Entity Recognition Task (SAC 2019) [paper]
• Enhanced Meta-Learning for Cross-Lingual Named Entity Recognition with Minimal Resources (AAAI 2020) [paper]
• MetaNER: Named Entity Recognition with Meta-Learning (WWW 2020) [paper]
• Meta-Learning for Few-Shot Named Entity Recognition (MetaNLP, 2021) [paper]
• Decomposed Meta-Learning for Few-Shot Named Entity Recognition (ACL 2022, Findings) [paper]
• Label Semantics for Few Shot Named Entity Recognition (ACL 2022, Findings) [paper]
• Few-Shot Named Entity Recognition via Meta-Learning (TKDE, 2022) [paper]
• Prompt-Based Metric Learning for Few-Shot NER (ACL 2023, Findings) [paper]
• Task-adaptive Label Dependency Transfer for Few-shot Named Entity Recognition (ACL 2023, Findings) [paper]
• HEProto: A Hierarchical Enhancing ProtoNet based on Multi-Task Learning for Few-shot Named Entity Recognition (CIKM 2023) [paper]
• Meta-Learning Triplet Network with Adaptive Margins for Few-Shot Named Entity Recognition (arXiv, 2023) [paper]
• Causal Interventions-based Few-Shot Named Entity Recognition (arXiv, 2023) [paper]
• MCML: A Novel Memory-based Contrastive Meta-Learning Method for Few Shot Slot Tagging (arXiv, 2023) [paper]

For Low-Resource RE

• Hybrid Attention-Based Prototypical Networks for Noisy Few-Shot Relation Classification (AAAI 2019) [paper]
• Few-shot Relation Extraction via Bayesian Meta-learning on Relation Graphs (ICML 2020) [paper]
• Bridging Text and Knowledge with Multi-Prototype Embedding for Few-Shot Relational Triple Extraction (COLING 2020) [paper]
• Meta-Information Guided Meta-Learning for Few-Shot Relation Classification (COLING 2020) [paper]
• Prototypical Representation Learning for Relation Extraction (ICLR 2021) [paper]
• Pre-training to Match for Unified Low-shot Relation Extraction (ACL 2022) [paper]
• Learn from Relation Information: Towards Prototype Representation Rectification for Few-Shot Relation Extraction (NAACL 2022, Findings) [paper]
• fmLRE: A Low-Resource Relation Extraction Model Based on Feature Mapping Similarity Calculation (AAAI 2023) [paper]
• Interaction Information Guided Prototype Representation Rectification for Few-Shot Relation Extraction (Electronics, 2023) [paper]
• Consistent Prototype Learning for Few-Shot Continual Relation Extraction (ACL 2023) [paper]
• RAPL: A Relation-Aware Prototype Learning Approach for Few-Shot Document-Level Relation Extraction (EMNLP 2023) [paper]
• Density-Aware Prototypical Network for Few-Shot Relation Classification (EMNLP 2023, Findings) [paper]
• Improving few-shot relation extraction through semantics-guided learning (Neural Networks, 2023) [paper]
• Generative Meta-Learning for Zero-Shot Relation Triplet Extraction (arXiv, 2023) [paper]

For Low-Resource EE

• Meta-Learning with Dynamic-Memory-Based Prototypical Network for Few-Shot Event Detection (WSDM 2020) [paper]
• Adaptive Knowledge-Enhanced Bayesian Meta-Learning for Few-shot Event Detection (ACL 2021, Findings) [paper]
• Few-Shot Event Detection with Prototypical Amortized Conditional Random Field (ACL 2021, Findings) [paper]
• Zero- and Few-Shot Event Detection via Prompt-Based Meta Learning (ACL 2023) [paper]
• MultiPLe: Multilingual Prompt Learning for Relieving Semantic Confusions in Few-shot Event Detection (CIKM 2023) [paper]

Transfer Learning

• Zero-Shot Transfer Learning for Event Extraction (ACL 2018) [paper]
• Transfer Learning for Named-Entity Recognition with Neural Networks (LREC 2018) [paper]
• Long-tail Relation Extraction via Knowledge Graph Embeddings and Graph Convolution Networks (NAACL 2019) [paper]
• Relation Adversarial Network for Low Resource Knowledge Graph Completion (WWW 2020) [paper]
• MZET: Memory Augmented Zero-Shot Fine-grained Named Entity Typing (COLING 2020) [paper]
• LearningToAdapt with Word Embeddings: Domain Adaptation of Named Entity Recognition Systems (Information Processing and Management, 2021) [paper]
• One Model for All Domains: Collaborative Domain-Prefx Tuning for Cross-Domain NER (IJCAI 2023) [paper]
• MANNER: A Variational Memory-Augmented Model for Cross Domain Few-Shot Named Entity Recognition (ACL 2023) [paper]
• Linguistic Representations for Fewer-shot Relation Extraction across Domains (ACL 2023) [paper]
• Few-Shot Relation Extraction With Dual Graph Neural Network Interaction (TNNLS, 2023) [paper]
• Leveraging Open Information Extraction for Improving Few-Shot Trigger Detection Domain Transfer (arXiv, 2023) [paper]

Fine-Tuning PLM

• Matching the Blanks: Distributional Similarity for Relation Learning (ACL 2019) [paper]
• Exploring Pre-trained Language Models for Event Extraction and Generation (ACL 2019) [paper]
• Coarse-to-Fine Pre-training for Named Entity Recognition (EMNLP 2020) [paper]
• CLEVE: Contrastive Pre-training for Event Extraction (ACL 2021) [paper]
• Unleash GPT-2 Power for Event Detection (ACL 2021) [paper]
• Efficient Zero-shot Event Extraction with Context-Definition Alignment (EMNLP 2022, Findings) [paper]
• Few-shot Named Entity Recognition with Self-describing Networks (ACL 2022) [paper]
• Query and Extract: Refining Event Extraction as Type-oriented Binary Decoding (ACL 2022, Findings) [paper]
• ClarET: Pre-training a Correlation-Aware Context-To-Event Transformer for Event-Centric Generation and Classification (ACL 2022) [paper]
• Unleashing Pre-trained Masked Language Model Knowledge for Label Signal Guided Event Detection (DASFAA 2023) [paper]
• A Multi-Task Semantic Decomposition Framework with Task-specific Pre-training for Few-Shot NER (CIKM 2023) [paper]
• Continual Contrastive Finetuning Improves Low-Resource Relation Extraction (ACL 2023) [paper]
• Unified Low-Resource Sequence Labeling by Sample-Aware Dynamic Sparse Finetuning (EMNLP 2023) [paper]
• GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer (arXiv, 2023) [paper]
• Synergistic Anchored Contrastive Pre-training for Few-Shot Relation Extraction (arXiv, 2023) [paper]

3 Optimizing Data and Models Together

Multi-Task Learning

(1) IE & IE-Related Tasks

NER, Named Entity Normalization (NEN)

• A Neural Multi-Task Learning Framework to Jointly Model Medical Named Entity Recognition and Normalization (AAAI 2019) [paper]
• MTAAL: Multi-Task Adversarial Active Learning for Medical Named Entity Recognition and Normalization (AAAI 2021) [paper]
• An End-to-End Progressive Multi-Task Learning Framework for Medical Named Entity Recognition and Normalization (ACL 2021) [paper]

Word Sense Disambiguation (WSD), Event Detection (ED)

• Similar but not the Same: Word Sense Disambiguation Improves Event Detection via Neural Representation Matching (EMNLP 2018) [paper]
• Graph Learning Regularization and Transfer Learning for Few-Shot Event Detection (SIGIR 2021) [paper]

(2) Joint IE & Other Structured Prediction Tasks

NER, RE

• GraphRel: Modeling Text as Relational Graphs for Joint Entity and Relation Extraction (ACL 2019) [paper]
• CopyMTL: Copy Mechanism for Joint Extraction of Entities and Relations with Multi-Task Learning (AAAI 2020) [paper]
• Joint Entity and Relation Extraction Model based on Rich Semantics (Neurocomputing, 2021) [paper]

NER, RE, EE

• Entity, Relation, and Event Extraction with Contextualized Span Representations (EMNLP 2019) [paper]

NER, RE, EE & Other Structured Prediction Tasks

• SPEECH: Structured Prediction with Energy-Based Event-Centric Hyperspheres (ACL 2023) [paper]
• Mirror: A Universal Framework for Various Information Extraction Tasks (EMNLP 2023) [paper]

Task Reformulation

• Zero-Shot Relation Extraction via Reading Comprehension (CoNLL 2017) [paper]
• Entity-Relation Extraction as Multi-Turn Question Answering (ACL 2019) [paper]
• A Unified MRC Framework for Named Entity Recognition (ACL 2020) [paper]
• Event Extraction as Machine Reading Comprehension (EMNLP 2020) [paper]
• Event Extraction by Answering (Almost) Natural Questions (EMNLP 2020) [paper]
• Text2Event: Controllable Sequence-to-Structure Generation for End-to-end Event Extraction (ACL 2021) [paper]
• Structured Prediction as Translation between Augmented Natural Languages (ICLR 2021) [paper]
• A Unified Generative Framework for Various NER Subtasks (ACL 2021) [paper]
• REBEL: Relation Extraction By End-to-end Language Generation (EMNLP 2021, Findings) [paper]
• GenIE: Generative Information Extraction (NAACL 2022) [paper]
• Learning to Ask for Data-Efficient Event Argument Extraction (AAAI 2022, Student Abstract) [paper]
• Complex Question Enhanced Transfer Learning for Zero-shot Joint Information Extraction (TASLP, 2023) [paper]
• Weakly-Supervised Questions for Zero-Shot Relation Extraction (EACL 2023) [paper]
• Event Extraction as Question Generation and Answering (ACL 2023, Short) [paper]
• Set Learning for Generative Information Extraction (EMNLP 2023) [paper]

Prompt-Tuning PLM

(1) Vanilla Prompt-Tuning

• Template-Based Named Entity Recognition Using BART (ACL 2021, Findings) [paper]
• Label Verbalization and Entailment for Effective Zero and Few-Shot Relation Extraction (EMNLP 2021) [paper]
• LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Prompting (COLING 2022) [paper]
• COPNER: Contrastive Learning with Prompt Guiding for Few-shot Named Entity Recognition (COLING 2022) [paper]
• Template-free Prompt Tuning for Few-shot NER (NAACL 2022) [paper]
• Textual Entailment for Event Argument Extraction: Zero- and Few-Shot with Multi-Source Learning (NAACL 2022, Findings) [paper]
• RelationPrompt: Leveraging Prompts to Generate Synthetic Data for Zero-Shot Relation Triplet Extraction (ACL 2022, Findings) [paper]
• Prompt for Extraction? PAIE: Prompting Argument Interaction for Event Argument Extraction (ACL 2022) [paper]
• Dynamic Prefix-Tuning for Generative Template-based Event Extraction (ACL 2022) [paper]
• Good Examples Make A Faster Learner Simple Demonstration-based Learning for Low-resource NER (ACL 2022) [paper]
• Prompt-Learning for Cross-Lingual Relation Extraction (IJCNN 2023) [paper]
• DSP: Discriminative Soft Prompts for Zero-Shot Entity and Relation Extraction (ACL 2023, Findings) [paper]
• Contextualized Soft Prompts for Extraction of Event Arguments (ACL 2023, Findings) [paper]
• The Art of Prompting: Event Detection based on Type Specific Prompts (ACL 2023, Short) [paper]
• Prompt for Extraction: Multiple Templates Choice Model for Event Extraction (KBS, 2024) [paper]
• UMIE: Unified Multimodal Information Extraction with Instruction Tuning (arXiv, 2024) [paper]

(2) Augmented Prompt-Tuning

• PTR: Prompt Tuning with Rules for Text Classification (AI Open, 2022) [paper]
• KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction (WWW 2022) [paper]
• Ontology-enhanced Prompt-tuning for Few-shot Learning (WWW 2022) [paper]
• Relation Extraction as Open-book Examination: Retrieval-enhanced Prompt Tuning (SIGIR 2022, Short) [paper]
• Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning (NeurIPS 2022) [paper]
• AugPrompt: Knowledgeable Augmented-Trigger Prompt for Few-Shot Event Classification (Information Processing & Management, 2022) [paper]
• Zero-Shot Event Detection Based on Ordered Contrastive Learning and Prompt-Based Prediction (NAACL 2022, Findings) [paper]
• DEGREE: A Data-Efficient Generation-Based Event Extraction Model (NAACL 2022) [paper]
• Retrieval-Augmented Generative Question Answering for Event Argument Extraction (EMNLP 2022) [paper]
• Unified Structure Generation for Universal Information Extraction (ACL 2022) [paper]
• LasUIE: Unifying Information Extraction with Latent Adaptive Structure-aware Generative Language Model (NeurIPS 2022) [paper]
• Universal Information Extraction as Unified Semantic Matching (AAAI 2023) [paper]
• Universal Information Extraction with Meta-Pretrained Self-Retrieval (ACL 2023) [paper]
• RexUIE: A Recursive Method with Explicit Schema Instructor for Universal Information Extraction (EMNLP 2023, Findings) [paper]
• Schema-aware Reference as Prompt Improves Data-Efficient Relational Triple and Event Extraction (SIGIR 2023) [paper]
• PromptNER: Prompt Locating and Typing for Named Entity Recognition (ACL 2023) [paper]
• Focusing, Bridging and Prompting for Few-shot Nested Named Entity Recognition (ACL 2023, Findings) [paper]
• Revisiting Relation Extraction in the era of Large Language Models (ACL 2023) [paper]
• AMPERE: AMR-Aware Prefix for Generation-Based Event Argument Extraction Model (ACL 2023) [paper]
• BertNet: Harvesting Knowledge Graphs with Arbitrary Relations from Pretrained Language Models (ACL 2023, Findings) [paper]
• Retrieve-and-Sample: Document-level Event Argument Extraction via Hybrid Retrieval Augmentation (ACL 2023) [paper]
• Easy-to-Hard Learning for Information Extraction (ACL 2023, Findings) [paper]
• DemoSG: Demonstration-enhanced Schema-guided Generation for Low-resource Event Extraction (EMNLP 2023, Findings) [paper]
• 2INER: Instructive and In-Context Learning on Few-Shot Named Entity Recognition (EMNLP 2023, Findings) [paper]
• Template-Free Prompting for Few-Shot Named Entity Recognition via Semantic-Enhanced Contrastive Learning (TNNLS, 2023) [paper]
• TaxonPrompt: Taxonomy-Aware Curriculum Prompt Learning for Few-Shot Event Classification (KBS, 2023) [paper]
• A Composable Generative Framework based on Prompt Learning for Various Information Extraction Tasks (IEEE Transactions on Big Data, 2023) [paper]
• Event Extraction With Dynamic Prefix Tuning and Relevance Retrieval (TKDE, 2023) [paper]
• MsPrompt: Multi-step Prompt Learning for Debiasing Few-shot Event Detection (Information Processing & Management, 2023) [paper]
• PromptNER: A Prompting Method for Few-shot Named Entity Recognition via k Nearest Neighbor Search (arXiv, 2023) [paper]
• TKDP: Threefold Knowledge-enriched Deep Prompt Tuning for Few-shot Named Entity Recognition (arXiv, 2023) [paper]
• OntoType: Ontology-Guided Zero-Shot Fine-Grained Entity Typing with Weak Supervision from Pre-Trained Language Models (arXiv, 2023) [paper]

🍏 LLM-Based Methods 🍏

Direct Inference Without Tuning

Instruction Prompting

• Exploring the Feasibility of ChatGPT for Event Extraction (arXiv, 2023) [paper]
• Zero-Shot Information Extraction via Chatting with ChatGPT (arXiv, 2023) [paper]
• Global Constraints with Prompting for Zero-Shot Event Argument Classification (EACL 2023, Findings) [paper]
• Revisiting Large Language Models as Zero-shot Relation Extractors (EMNLP 2023, Findings) [paper]
• Empirical Study of Zero-Shot NER with ChatGPT (EMNLP 2023) [paper]
• AutoKG: Efficient Automated Knowledge Graph Generation for Language Models (IEEE BigData 2023, GTA3 Workshop) [paper]
• PromptNER : Prompting For Named Entity Recognition (arXiv, 2023) [paper]
• Zero-shot Temporal Relation Extraction with ChatGPT (ACL 2023, BioNLP) [paper]
• Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness (arXiv, 2023) [paper]
• LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities (arXiv, 2023) [paper]
• Grasping the Essentials: Tailoring Large Language Models for Zero-Shot Relation Extraction (arXiv, 2024) [paper]
• A Simple but Effective Approach to Improve Structured Language Model Output for Information Extraction (arXiv, 2024) [paper]

Code Prompting

• Code4Struct: Code Generation for Few-Shot Event Structure Prediction (ACL 2023) [paper]
• CodeIE: Large Code Generation Models are Better Few-Shot Information Extractors (ACL 2023) [paper]
• ViStruct: Visual Structural Knowledge Extraction via Curriculum Guided Code-Vision Representation (EMNLP 2023) [paper]
• Retrieval-Augmented Code Generation for Universal Information Extraction (arXiv, 2023) [paper]
• CodeKGC: Code Language Model for Generative Knowledge Graph Construction (arXiv, 2023) [paper]
• GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction (arXiv, 2023) [paper]

In-Context Learning

• Learning In-context Learning for Named Entity Recognition (ACL 2023) [paper]
• How to Unleash the Power of Large Language Models for Few-shot Relation Extraction? (ACL 2023, SustaiNLP Workshop) [paper]
• GPT-RE: In-context Learning for Relation Extraction using Large Language Models (EMNLP 2023) [paper]
• In-context Learning for Few-shot Multimodal Named Entity Recognition (EMNLP 2023, Findings) [paper]
• Prompting ChatGPT in MNER: Enhanced Multimodal Named Entity Recognition with Auxiliary Refined Knowledge (EMNLP 2023, Findings) [paper]
• Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples! (EMNLP 2023, Findings) [paper]
• Guideline Learning for In-Context Information Extraction (EMNLP 2023) [paper]
• Chain of Thought with Explicit Evidence Reasoning for Few-shot Relation Extraction (EMNLP 2023, Findings) [paper]
• Pipeline Chain-of-Thought: A Prompt Method for Large Language Model Relation Extraction (IALP 2023) [paper]
• GPT-NER: Named Entity Recognition via Large Language Models (arXiv, 2023) [paper]
• In-Context Few-Shot Relation Extraction via Pre-Trained Language Models (arXiv, 2023) [paper]
• Self-Improving for Zero-Shot Named Entity Recognition with Large Language Models (arXiv, 2023) [paper]
• Is Information Extraction Solved by ChatGPT? An Analysis of Performance, Evaluation Criteria, Robustness and Errors (arXiv, 2023) [paper]
• GPT Struct Me: Probing GPT Models on Narrative Entity Extraction (arXiv, 2023) [paper]
• Improving Open Information Extraction with Large Language Models: A Study on Demonstration Uncertainty (arXiv, 2023) [paper]
• LOKE: Linked Open Knowledge Extraction for Automated Knowledge Graph Construction (arXiv, 2023) [paper]
• Zero- and Few-Shots Knowledge Graph Triplet Extraction with Large Language Models (arXiv, 2023) [paper]
• Chain-of-Thought Prompt Distillation for Multimodal Named Entity Recognition and Multimodal Relation Extraction (arXiv, 2023) [paper]
• Unlocking Instructive In-Context Learning with Tabular Prompting for Relational Triple Extraction (COLING 2024) [paper]
• LinkNER: Linking Local Named Entity Recognition Models to Large Language Models using Uncertainty (arXiv, 2024) [paper]
• GenRES: Rethinking Evaluation for Generative Relation Extraction in the Era of Large Language Models (arXiv, 2024) [paper]
• C-ICL: Contrastive In-context Learning for Information Extraction (arXiv, 2024) [paper]
• EventRL: Enhancing Event Extraction with Outcome Supervision for Large Language Models (arXiv, 2024) [paper]
• Small Language Model Is a Good Guide for Large Language Model in Chinese Entity Relation Extraction (arXiv, 2024) [paper]

Model Specialization With Tuning

Prompt-Tuning LLM

• DeepStruct: Pretraining of Language Models for Structure Prediction (ACL 2022, Findings) [paper]
• Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractors (ACL 2023, Findings) [paper]
• Instruct and Extract: Instruction Tuning for On-Demand Information Extraction (EMNLP 2023) [paper]
• UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition (ICLR 2024) [paper]
• InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction (arXiv, 2023) [paper]
• YAYI-UIE: A Chat-Enhanced Instruction Tuning Framework for Universal Information Extraction (arXiv, 2023) [paper]
• ChatUIE: Exploring Chat-based Unified Information Extraction using Large Language Models (COLING 2024) [paper]

Fine-Tuning LLM

• Fine-Tuning GPT Family (OpenAI, 2023) [Documentation]
• EventRL: Enhancing Event Extraction with Outcome Supervision for Large Language Models (arXiv, 2024) [paper]

How to Cite

📋 Thank you very much for your interest in our survey work. If you use or extend our survey, please cite the following paper:

@misc{2023_LowResIE,
    author    = {Shumin Deng and
                 Yubo Ma and
                 Ningyu Zhang and
                 Yixin Cao and
                 Bryan Hooi},
    title     = {Information Extraction in Low-Resource Scenarios: Survey and Perspective}, 
    journal   = {CoRR},
    volume    = {abs/2202.08063},
    year      = {2023},
    url       = {https://arxiv.org/abs/2202.08063}
}