Awesome

Resources on ChatGPT and Large Language Models

Collection of papers and related works for Large Language Models (ChatGPT, GPT-3, Codex etc.).

Contributors

This repository is contributed by the following contributors.

Organizers: Guilin Qi (漆桂林), Xiaofang Qi (戚晓芳)
Paper Collectors: Zafar Ali, Sheng Bi (毕胜), Yongrui Chen (陈永锐), Zizhuo Chen (陈孜卓), Xinbang Dai (戴鑫邦), Huan Gao (高桓), Nan Hu (胡楠), Shilong Hu (胡世龙), Jingqi Kang (康婧淇), Jiaqi Li (李嘉琦), Dehai Min (闵德海), Guilin Qi (漆桂林), Yiming Tan (谭亦鸣), Tongtong Wu (吴桐桐), Songlin Zhai (翟松林), Shenyu Zhang (张沈昱), Yuxin Zhang (张裕欣)
Maintainers: Runzhe Wang (王润哲), Shenyu Zhang (张沈昱)

The automation script of this repo is powered by Auto-Bibfile. If you'd like to commit to this repo, please modify bibtex.bib or related_works.json and re-generate README.md using python scripts/run.py.

Papers

Outline

Hyperlinks

Continual Learning

<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Can BERT Refrain from Forgetting on Sequential Tasks? A Probing Study, by Mingxu Tao, Yansong Feng and Dongyan Zhao
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Continual Pre-Training of Large Language Models: How to (re)warm your model?, by Kshitij Gupta, Benjamin Th'erien, Adam Ibrahim, Mats L. Richter, Quentin Anthony, Eugene Belilovsky, Irina Rish and Timoth'ee Lesort
<img src=https://img.shields.io/badge/ICLR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Continual Pre-training of Language Models, by Zixuan Ke, Yijia Shao, Haowei Lin, Tatsuya Konishi, Gyuhak Kim and Bing Liu
<img src=https://img.shields.io/badge/ACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Small Pre-trained Language Models Can be Fine-tuned as Large Models via Over-Parameterization, by Ze-Feng Gao, Kun Zhou, Peiyu Liu, Wayne Xin Zhao and Ji-Rong Wen
<img src=https://img.shields.io/badge/arXiv_preprint_arXiv:2309.06256-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Speciality vs Generality: An Empirical Study on Catastrophic Forgetting in Fine-tuning Foundation Models, by Lin, Yong, Tan, Lu, Lin, Hangyu, Zheng, Zeming, Pi, Renjie, Zhang, Jipeng, Diao, Shizhe, Wang, Haoxiang et al.
<img src=https://img.shields.io/badge/CVPR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Learning to Prompt for Continual Learning, by Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot et al.
<img src=https://img.shields.io/badge/T--PAMI-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Continual Learning Survey: Defying Forgetting in Classification Tasks, by Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Gregory G. Slabaugh and Tinne Tuytelaars
<img src=https://img.shields.io/badge/ACL_Findings-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> ELLE: Efficient Lifelong Pre-training for Emerging Data, by Yujia Qin, Jiajie Zhang, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun and Jie Zhou
<img src=https://img.shields.io/badge/NAACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora, by Xisen Jin, Dejiao Zhang, Henghui Zhu, Wei Xiao, Shang-Wen Li, Xiaokai Wei, Andrew O. Arnold and Xiang Ren
<img src=https://img.shields.io/badge/JAIR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Towards Continual Reinforcement Learning: A Review and Perspectives, by Khimya Khetarpal, Matthew Riemer, Irina Rish and Doina Precup
<img src=https://img.shields.io/badge/ACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Continual Pre-training of Language Models for Math Problem Understanding with Syntax-Aware Memory Network, by Zheng Gong, Kun Zhou, Xin Zhao, Jing Sha, Shijin Wang and Ji-Rong Wen
<img src=https://img.shields.io/badge/ICLR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5, by Chengwei Qin and Shafiq Joty We define a challenging yet practical problem as Lifelong Few-shot Language Learning and propose a unified framework for it based on prompt tuning of T5.
<img src=https://img.shields.io/badge/ICLR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Towards Continual Knowledge Learning of Language Models, by Joel Jang, Seonghyeon Ye, Sohee Yang, Joongbo Shin, Janghoon Han, Gyeonghun KIM, Stanley Jungkyu Choi and Minjoon Seo We propose a novel continual learning formulation named Continual Knowledge Learning which allows large language models to constantly obtain new and updated knowledge while mitigating forgetting of previous learned time-invariant knowledge.
<img src=https://img.shields.io/badge/ICLR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Pretrained Language Model in Continual Learning: A Comparative Study, by Tongtong Wu, Massimo Caccia, Zhuang Li, Yuan-Fang Li, Guilin Qi and Gholamreza Haffari To explore the layer-wise property of pretrained languge models in continual learning, we thoroughly compare the continual learning performance over the combination of 5 PLMs and 4 veins of CL methods on 3 benchmarks in 2 typical incremental settings.
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models, by Joel Jang, Seonghyeon Ye, Changho Lee, Sohee Yang, Joongbo Shin, Janghoon Han, Gyeonghun Kim and Minjoon Seo
<img src=https://img.shields.io/badge/ICML-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Models, by Adam Liska, Tom'as Kocisk'y, Elena Gribovskaya, Tayfun Terzi, Eren Sezener, Devang Agrawal, Cyprien de Masson d'Autume, Tim Scholtes et al.
<img src=https://img.shields.io/badge/NeurIPS-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning, by Zixuan Ke, Bing Liu, Nianzu Ma, Hu Xu and Lei Shu NeurIPS 2021, The key component of CTR is the CL-plugin inserted in BERT. A CL-plugin is a capsule network with a new transfer routing mechanism to encourage knowledge transfer among tasks and also to isolate task-specific knowledge to avoid forgetting.
<img src=https://img.shields.io/badge/EMNLP_Findings-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learning, by Jin, Xisen , Lin, Bill Yuchen , Rostami, Mohammad and Ren, Xiang We present a new learning setup, Continual Learning of Few-Shot Learners, to address challenges of both learning settings in a unified setup, with a hyper-network for task-specific adapter generation.
<img src=https://img.shields.io/badge/EACL-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Analyzing the Forgetting Problem in Pretrain-Finetuning of Open-domain Dialogue Response Models, by Tianxing He, Jun Liu, Kyunghyun Cho, Myle Ott, Bing Liu, James R. Glass and Fuchun Peng Our major finding is that after standard finetuning, the model forgets some of the important language generation skills acquired during large-scale pretraining. We propose an intuitive finetuning strategy named “mix-review”: : For each finetuning epoch, we mix the target dialogue data with a random subset of the pretraining data, mix_ratio is 4, decay is 0.9.
<img src=https://img.shields.io/badge/ACL_Findings-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters, by Ruize Wang, Duyu Tang, Nan Duan, Zhongyu Wei, Xuanjing Huang, Jianshu Ji, Guihong Cao, Daxin Jiang et al. We propose KADAPTER, a framework that retains the original parameters of the pre-trained model fixed and supports the development of versatile knowledge-infused model.
<img src=https://img.shields.io/badge/EMNLP-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Domain-Lifelong Learning for Dialogue State Tracking via Knowledge Preservation Networks, by Liu, Qingbin , Cao, Pengfei , Liu, Cao , Chen, Jiansong , Cai, Xunliang , Yang, Fan , He, Shizhu , Liu, Kang et al. This paper explores Domain-Lifelong Learning for Dialogue State Tracking, we propose Knowledge Preservation Network, which consists of multi-prototype enhanced retrospection and multi-strategy knowledge distillation, to solve the problems of expression diversity and combinatorial explosion in the DLL-DST task
<img src=https://img.shields.io/badge/EMNLP-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks, by Ke, Zixuan , Liu, Bing , Xu, Hu and Shu, Lei The key novelty is a contrastive continual learning method that enables both knowledge transfer across tasks and knowledge distillation from old tasks to the new task, which eliminates the need for task ids in testing.
<img src=https://img.shields.io/badge/EMNLP-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Lifelong Explainer for Lifelong Learners, by Situ, Xuelin , Maruf, Sameen , Zukerman, Ingrid , Paris, Cecile and Haffari, Gholamreza We propose a novel Lifelong Explanation approach that continuously trains a student explainer under the supervision of a teacher – an arbitrary explanation algorithm – on different tasks undertaken in LL. We also leverage the Experience Replay mechanism to prevent catastrophic forgetting in the student explainer.
<img src=https://img.shields.io/badge/EMNLP-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Unified Speaker Adaptation Approach for ASR, by Yingzhu Zhao, Chongjia Ni, Cheung-Chi Leung, Shafiq R. Joty, Eng Siong Chng and Bin Ma Prefix-based user identifier, Continual ASR / Architecture Search / Network Pruning.
<img src=https://img.shields.io/badge/SIGKDD-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Dynamic Language Models for Continuously Evolving Content, by Amba Hombaiah, Spurthi, Chen, Tao, Zhang, Mingyang, Bendersky, Michael and Najork, Marc
<img src=https://img.shields.io/badge/ACL-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Parameter-Efficient Transfer Learning with Diff Pruning, by Guo, Demi , Rush, Alexander and Kim, Yoon The approach learns a task-specific “diff” vector that extends the original pretrained parameters. As the number of tasks increases, diff pruning remains parameter-efficient, as it requires storing only a small diff vector for each task.
<img src=https://img.shields.io/badge/ACL-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Refining Sample Embeddings with Relation Prototypes to Enhance Continual Relation Extraction, by Cui, Li , Yang, Deqing , Yu, Jiaxin , Hu, Chengwei , Cheng, Jiayang , Yi, Jingjie and Xiao, Yanghua To fully utilize memorized samples, in this paper, we employ relation prototype to extract useful information of each relation.
<img src=https://img.shields.io/badge/ACL-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptation, by He, Ruidan , Liu, Linlin , Ye, Hai , Tan, Qingyu , Ding, Bosheng , Cheng, Liying , Low, Jiawei , Bing, Lidong et al. we first show that adapter-based tuning better mitigates forgetting issues than fine-tuning since it yields representations with less deviation from those generated by the initial PrLM. Effectiveness: it tendsto outperform fine-tuning on both low-resource and cross-lingual tasks; 2 it demonstrates higher stability under different learning rates compared to fine-tuning.
<img src=https://img.shields.io/badge/ACL-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Rational LAMOL: A Rationale-based Lifelong Learning Framework, by Kanwatchara, Kasidis , Horsuwan, Thanapapas , Lertvittayakumjorn, Piyawat , Kijsirikul, Boonserm and Vateekul, Peerapon Rational LAMOL enhances LAMOL, a recent LL model, by applying critical freezing guided by human rationales. When the human rationales are not available, we propose exploiting unsupervised generated rationales as substitutions.
<img src=https://img.shields.io/badge/NAACL--HLT-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution, by Garcia, Xavier , Constant, Noah , Parikh, Ankur and Firat, Orhan Introducing the catastrophic forgetting problem in incremental multi-language translation, and utilizing a vocabulary substitution manner to alleviate the above problem.
<img src=https://img.shields.io/badge/NAACL--HLT-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Continual Learning for Text Classification with Information Disentanglement Based Regularization, by Huang, Yufan , Zhang, Yanzhe , Chen, Jiaao , Wang, Xuezhi and Yang, Diyi Proposing a regularization-based method for continual text classification, introducing the next sentence prediction and task id prediction as auxiliary tasks.
<img src=https://img.shields.io/badge/NAACL--HLT-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Incremental Few-shot Text Classification with Multi-round New Classes: Formulation, Dataset and System, by Xia, Congying , Yin, Wenpeng , Feng, Yihao and Yu, Philip Proposing a new setting and respective benchmark for few-shot incremental text classification, modeling continual text classification with text entailment.
<img src=https://img.shields.io/badge/NAACL--HLT-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Hyperparameter-free Continuous Learning for Domain Classification in Natural Language Understanding, by Hua, Ting , Shen, Yilin , Zhao, Changsheng , Hsu, Yen-Chang and Jin, Hongxia Inspired by EWC and proposing a hyperparameter-free (Fisher information-based) sampling method for memory replay.
<img src=https://img.shields.io/badge/EACL-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Lifelong Knowledge-Enriched Social Event Representation Learning, by Vijayaraghavan, Prashanth and Roy, Deb Proposing a rehearsal-based method, i.e.,Domain-Representative Episodic Memory Replay (DR-EMR), for lifelong event representation with embedding alignment and external social commonsense knowledge.
<img src=https://img.shields.io/badge/CoRR-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Lifelong Intent Detection via Multi-Strategy Rebalancing, by Qingbin Liu, Xiaoyan Yu, Shizhu He, Kang Liu and Jun Zhao We propose the lifelong intent detection task to handle continually emerging user intents. And, we propose multistrategy rebalancing to address multiple adverse effects caused by the data imbalance problem.
<img src=https://img.shields.io/badge/EMNLP-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting, by Sanyuan Chen, Yutai Hou, Yiming Cui, Wanxiang Che, Ting Liu and Xiangzhan Yu We propose a recall and learn mechanism, which adopts the idea of multi-task learning and jointly learns pretraining tasks and downstream tasks. Specifically, we introduce a Pretraining Simulation mechanism to recall the knowledge from pretraining tasks without data, and an Objective Shifting mechanism to focus the learning on downstream tasks gradually.
<img src=https://img.shields.io/badge/EMNLP_Findings-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learning, by Zhaojiang Lin, Andrea Madotto and Pascale Fung Proposing an adapter-based method for continual learning in text generation. One of the insights is a frozen PLM can be well-applied in continual learning.
<img src=https://img.shields.io/badge/EMNLP-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> An Empirical Investigation Towards Efficient Multi-Domain Language Model Pre-training, by Arumae, Kristjan , Sun, Qing and Bhatia, Parminder We find that elastic weight consolidation provides best overall scores yielding only a 0.33% drop in performance across seven generic tasks while remaining competitive in bio-medical tasks.
<img src=https://img.shields.io/badge/EMNLP-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Visually Grounded Continual Learning of Compositional Phrases, by Jin, Xisen , Du, Junyi , Sadhu, Arka , Nevatia, Ram and Ren, Xiang A novel continual learning setting and a new benchmark for continual caption generation, evaluated with exiting rehearsal-based methods
<img src=https://img.shields.io/badge/EMNLP-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Incremental Event Detection via Knowledge Consolidation Networks, by Cao, Pengfei , Chen, Yubo , Zhao, Jun and Wang, Taifeng Proposing a hybrid continual learning method for event detection, combining experience replay and Knowledge Distillation, focusing on (1) semantic ambiguity in NLP and (2) data imbalance between memory and current task.
<img src=https://img.shields.io/badge/EMNLP-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Multi-Task Incremental Learning Framework with Category Name Embedding for Aspect-Category Sentiment Analysis, by Dai, Zehui , Peng, Cheng , Chen, Huajie and Ding, Yadong Utilizing BERT for sentence and category encoding, preserving category encoding to prevent catastrophic forgetting.
<img src=https://img.shields.io/badge/EMNLP-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Efficient Meta Lifelong-Learning with Limited Memory, by Wang, Zirui , Mehta, Sanket Vaibhav , Poczos, Barnabas and Carbonell, Jaime A meta learning-enhanced version of MbPA (NeurIPS19), sharing the continual setting as well. Figure 1 is interesting.
<img src=https://img.shields.io/badge/EMNLP-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Lifelong Language Knowledge Distillation, by Chuang, Yung-Sung , Su, Shang-Yu and Chen, Yun-Nung Proposing a Knowledge Distillation-enhanced Method LLL based on LAMOL (ICLR 2020) model for continual learning, evaluated on text generation and text classification.
<img src=https://img.shields.io/badge/COLING-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Distill and Replay for Continual Language Learning, by Sun, Jingyuan , Wang, Shaonan , Zhang, Jiajun and Zong, Chengqing Proposing a distill and replay method (DnR) which follows the setting of LAMOL. As a distillation-based method, DnR also shows the ability in incrementally compressing the model size while still outperforming most of the baselines.
<img src=https://img.shields.io/badge/AAAI-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> ERNIE 2.0: A Continual Pre-Training Framework for Language Understanding, by Sun, Yu, Wang, Shuohuan, Li, Yukun, Feng, Shikun, Tian, Hao, Wu, Hua and Wang, Haifeng In order to extract the lexical, syntactic and semantic information from training corpora, we propose a continual pre-training framework named ERNIE 2.0 which incrementally builds pre-training tasks and then learn pre-trained models on these constructed tasks via continual multi-task learning.
<img src=https://img.shields.io/badge/NeurIPS-2019-blue alt="img" style="zoom:100%; vertical-align: middle" /> Episodic Memory in Lifelong Language Learning, by Cyprien de Masson d'Autume, Sebastian Ruder, Lingpeng Kong and Dani Yogatama MbPA++. This paper proposes the use of memory (a fixed memory network) in life-long learning to prevent catastrophic forgetting by means of experience replay and local adaptation.

Prompt Engineering

<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex, by Terry Yue Zhuo, Zhuang Li, Yujin Huang, Yuan-Fang Li, Weiqing Wang, Gholamreza Haffari and Fatemeh Shiri
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT, by Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> ChatGPT Prompt Patterns for Improving Code Quality, Refactoring, Requirements Elicitation, and Software Design, by Jules White, Sam Hays, Quchen Fu, Jesse Spencer-Smith and Douglas C. Schmidt
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPT, by Jiawei Zhang
<img src=https://img.shields.io/badge/-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Joint Prompt Optimization of Stacked LLMs using Variational Inference, by Alessandro Sordoni, Xingdi Yuan, Marc-Alexandre Cote, Matheus Pereira, Adam Trischler, Ziang Xiao, Arian Hosseini, Friederike Niedtner et al.
<img src=https://img.shields.io/badge/CVPR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Learning to Prompt for Continual Learning, by Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot et al.
<img src=https://img.shields.io/badge/NAACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Do Prompt-Based Models Really Understand the Meaning of Their Prompts?, by Albert Webson and Ellie Pavlick
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large Language Models Are Human-Level Prompt Engineers, by Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan and Jimmy Ba
<img src=https://img.shields.io/badge/ACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> An Information-theoretic Approach to Prompt Engineering Without Ground Truth Labels, by Taylor Sorensen, Joshua Robinson, Christopher Michael Rytting, Alexander Glenn Shaw, Kyle Jeffrey Rogers, Alexia Pauline Delorey, Mahmoud Khalil, Nancy Fulda et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Demystifying Prompts in Language Models via Perplexity Estimation, by Hila Gonen, Srini Iyer, Terra Blevins, Noah A. Smith and Luke Zettlemoyer
<img src=https://img.shields.io/badge/ACL_Findings-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models, by Robert L. Logan IV, Ivana Balazevic, Eric Wallace, Fabio Petroni, Sameer Singh and Sebastian Riedel
<img src=https://img.shields.io/badge/ACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Adversarial Soft Prompt Tuning for Cross-Domain Sentiment Analysis, by Hui Wu and Xiaodong Shi
<img src=https://img.shields.io/badge/ACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Fine-Grained Controllable Text Generation Using Non-Residual Prompting, by Fredrik Carlsson, Joey "Ohman, Fangyu Liu, Severine Verlinden, Joakim Nivre and Magnus Sahlgren
<img src=https://img.shields.io/badge/ACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> MSP: Multi-Stage Prompting for Making Pre-trained Language Models Better Translators, by Zhixing Tan, Xiangwen Zhang, Shuo Wang and Yang Liu
<img src=https://img.shields.io/badge/ACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Noisy Channel Language Model Prompting for Few-Shot Text Classification, by Sewon Min, Mike Lewis, Hannaneh Hajishirzi and Luke Zettlemoyer
<img src=https://img.shields.io/badge/ACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer, by Tu Vu, Brian Lester, Noah Constant, Rami Al-Rfou' and Daniel Cer
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Models, by Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen et al.
<img src=https://img.shields.io/badge/AutoML-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Meta-Adapters: Parameter Efficient Few-shot Fine-tuning through Meta-Learning, by Trapit Bansal, Salaheddin Alzubi, Tong Wang, Jay-Yoon Lee and Andrew McCallum
<img src=https://img.shields.io/badge/NeurIPS-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Sparse Structure Search for Delta Tuning, by Shengding Hu, Zhen Zhang, Ning Ding, Yadao Wang, Yasheng Wang, Zhiyuan Liu and Maosong Sun
<img src=https://img.shields.io/badge/WWW-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Ontology-enhanced Prompt-tuning for Few-shot Learning, by Hongbin Ye, Ningyu Zhang, Shumin Deng, Xiang Chen, Hui Chen, Feiyu Xiong, Xi Chen and Huajun Chen
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Pre-trained Language Models can be Fully Zero-Shot Learners, by Xuandong Zhao, Siqi Ouyang, Zhiguo Yu, Ming Wu and Lei Li
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Least-to-Most Prompting Enables Complex Reasoning in Large Language Models, by Denny Zhou, Nathanael Sch"arli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Olivier Bousquet et al. (1) 两阶段的prompt，第一阶段问题分解（通过in-context learning实现，context中包含了其他问题的分解示例），对于每个问题，分解出回答该问题需要先回答什么子问题； (2) 在第二阶段中，从后往前依次解决子问题，同样通过in-context learing得到，每次LLM的回答会参与组成下一个问题的prompt。
<img src=https://img.shields.io/badge/NeurIPS-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> The unreliability of explanations in few-shot prompting for textual reasoning, by Ye, Xi and Durrett, Greg
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Ask Me Anything: A simple strategy for prompting language models, by Simran Arora, Avanika Narayan, Mayee F. Chen, Laurel J. Orr, Neel Guha, Kush Bhatia, Ines Chami, Frederic Sala et al.
<img src=https://img.shields.io/badge/ACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Can Prompt Probe Pretrained Language Models? Understanding the Invisible Risks from a Causal View, by Boxi Cao, Hongyu Lin, Xianpei Han, Fangchao Liu and Le Sun
<img src=https://img.shields.io/badge/ACL_Findings-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Reframing Instructional Prompts to GPTk's Language, by Daniel Khashabi, Chitta Baral, Yejin Choi and Hannaneh Hajishirzi
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Toward Human Readable Prompt Tuning: Kubrick's The Shining is a good movie, and a good prompt too?, by Weijia Shi, Xiaochuang Han, Hila Gonen, Ari Holtzman, Yulia Tsvetkov and Luke Zettlemoyer
<img src=https://img.shields.io/badge/EMNLP_Findings-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Towards Unified Prompt Tuning for Few-shot Text Classification, by Jianing Wang, Chengyu Wang, Fuli Luo, Chuanqi Tan, Minghui Qiu, Fei Yang, Qiuhui Shi, Songfang Huang et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Model ensemble instead of prompt fusion: a sample-specific knowledge transfer method for few-shot prompt tuning, by Xiangyu Peng, Chen Xing, Prafulla Kumar Choubey, Chien-Sheng Wu and Caiming Xiong
<img src=https://img.shields.io/badge/EMNLP-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> FewshotQA: A simple framework for few-shot learning of question answering tasks using pre-trained text-to-text models, by Rakesh Chada and Pradeep Natarajan
<img src=https://img.shields.io/badge/EMNLP-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> The Power of Scale for Parameter-Efficient Prompt Tuning, by Brian Lester, Rami Al-Rfou and Noah Constant
<img src=https://img.shields.io/badge/ACL-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Prefix-Tuning: Optimizing Continuous Prompts for Generation, by Xiang Lisa Li and Percy Liang
<img src=https://img.shields.io/badge/CHI-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm, by Laria Reynolds and Kyle McDonell

Natural Language Understanding

<img src=https://img.shields.io/badge/OpenAI-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> GPT-4 Technical Report, <img src=https://img.shields.io/badge/GPT--4-yellow alt="img" style="zoom:100%; vertical-align: middle" /> by OpenAI
<img src=https://img.shields.io/badge/OpenAI-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> GPT-4 System Card, <img src=https://img.shields.io/badge/GPT--4-yellow alt="img" style="zoom:100%; vertical-align: middle" /> by OpenAI
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Knowledge Prompting in Pre-trained Language Model for Natural Language Understanding, by Jianing Wang, Wenkang Huang, Minghui Qiu, Qiuhui Shi, Hongbin Wang, Xiang Li and Ming Gao
<img src=https://img.shields.io/badge/EMNLP_Findings-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> VarMAE: Pre-training of Variational Masked Autoencoder for Domain-adaptive Language Understanding, by Dou Hu, Xiaolong Hou, Xiyang Du, Mengyuan Zhou, Lianxin Jiang, Yang Mo and Xiaofeng Shi
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Generating Training Data with Language Models: Towards Zero-Shot Language Understanding, by Yu Meng, Jiaxin Huang, Yu Zhang and Jiawei Han
<img src=https://img.shields.io/badge/ACL-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation, by Fuli Luo, Wei Wang, Jiahao Liu, Yijia Liu, Bin Bi, Songfang Huang, Fei Huang and Luo Si
<img src=https://img.shields.io/badge/NeurIPS-2019-blue alt="img" style="zoom:100%; vertical-align: middle" /> Unified Language Model Pre-training for Natural Language Understanding and Generation, by Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou et al.
<img src=https://img.shields.io/badge/OpenAI-2018-blue alt="img" style="zoom:100%; vertical-align: middle" /> Improving language understanding by generative pre-training, <img src=https://img.shields.io/badge/GPT--1-yellow alt="img" style="zoom:100%; vertical-align: middle" /> by Radford, Alec, Narasimhan, Karthik, Salimans, Tim, Sutskever, Ilya and others

Multimodal

<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Scaling Vision Transformers to 22 Billion Parameters, by Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski, Jonathan Heek, Justin Gilmer, Andreas Steiner, Mathilde Caron et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> PaLM-E: An Embodied Multimodal Language Model, <img src=https://img.shields.io/badge/PaLM--E-yellow alt="img" style="zoom:100%; vertical-align: middle" /> by Driess, Danny, Xia, Fei, Sajjadi, Mehdi SM, Lynch, Corey, Chowdhery, Aakanksha, Ichter, Brian, Wahid, Ayzaan, Tompson, Jonathan et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Learning Customized Visual Models with Retrieval-Augmented Knowledge, by Haotian Liu, Kilho Son, Jianwei Yang, Ce Liu, Jianfeng Gao, Yong Jae Lee and Chunyuan Li
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models, by Wu, Chenfei, Yin, Shengming, Qi, Weizhen, Wang, Xiaodong, Tang, Zecheng and Duan, Nan
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Aligning Text-to-Image Models using Human Feedback, by Kimin Lee, Hao Liu, Moonkyung Ryu, Olivia Watkins, Yuqing Du, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Let's Think Frame by Frame: Evaluating Video Chain of Thought with Video Infilling and Prediction, by Vaishnavi Himakunthala, Andy Ouyang, Daniel Rose, Ryan He, Alex Mei, Yujie Lu, Chinmay Sonar, Michael Saxon et al.
<img src=https://img.shields.io/badge/Mach._Intell._Res.-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Multimodal Pretraining from Monolingual to Multilingual, by Liang Zhang, Ludan Ruan, Anwen Hu and Qin Jin
<img src=https://img.shields.io/badge/Mach._Intell._Res.-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Compositional Prompting Video-language Models to Understand Procedure in Instructional Videos, by Guyue Hu, Bin He and Hanwang Zhang
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models, by Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li and Mohamed Elhoseiny
<img src=https://img.shields.io/badge/NeurIPS-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Tasks, by Wenhai Wang, Zhe Chen, Xiaokang Chen, Jiannan Wu, Xizhou Zhu, Gang Zeng, Ping Luo, Tong Lu et al.
<img src=https://img.shields.io/badge/CVPR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> CLIP-Event: Connecting Text and Images with Event Structures, by Manling Li, Ruochen Xu, Shuohang Wang, Luowei Zhou, Xudong Lin, Chenguang Zhu, Michael Zeng, Heng Ji et al.
<img src=https://img.shields.io/badge/COLING-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Are Visual-Linguistic Models Commonsense Knowledge Bases?, by Hsiu-Yu Yang and Carina Silberer
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Retrieval-Augmented Multimodal Language Modeling, by Michihiro Yasunaga, Armen Aghajanyan, Weijia Shi, Rich James, Jure Leskovec, Percy Liang, Mike Lewis, Luke Zettlemoyer et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Contrastive Language-Image Pre-Training with Knowledge Graphs, by Xuran Pan, Tianzhu Ye, Dongchen Han, Shiji Song and Gao Huang
<img src=https://img.shields.io/badge/CoRR-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> CLSEBERT: Contrastive Learning for Syntax Enhanced Code Pre-Trained Model, by Xin Wang, Yasheng Wang, Pingyi Zhou, Fei Mi, Meng Xiao, Yadao Wang, Li Li, Xiao Liu et al.
<img src=https://img.shields.io/badge/CVPR-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Less Is More: ClipBERT for Video-and-Language Learning via Sparse Sampling, by Jie Lei, Linjie Li, Luowei Zhou, Zhe Gan, Tamara L. Berg, Mohit Bansal and Jingjing Liu
<img src=https://img.shields.io/badge/CoRR-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Transformer is All You Need: Multimodal Multitask Learning with a Unified Transformer, by Ronghang Hu and Amanpreet Singh
<img src=https://img.shields.io/badge/MM-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Pre-training Graph Transformer with Multimodal Side Information for Recommendation, by Yong Liu, Susen Yang, Chenyi Lei, Guoxin Wang, Haihong Tang, Juyong Zhang, Aixin Sun and Chunyan Miao
<img src=https://img.shields.io/badge/CoRR-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> UniViLM: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation, by Huaishao Luo, Lei Ji, Botian Shi, Haoyang Huang, Nan Duan, Tianrui Li, Xilin Chen and Ming Zhou
<img src=https://img.shields.io/badge/NeurIPS-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large-Scale Adversarial Training for Vision-and-Language Representation Learning, by Zhe Gan, Yen-Chun Chen, Linjie Li, Chen Zhu, Yu Cheng and Jingjing Liu
<img src=https://img.shields.io/badge/EMNLP-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Vokenization: Improving Language Understanding with Contextualized, Visual-Grounded Supervision, by Hao Tan and Mohit Bansal
<img src=https://img.shields.io/badge/ACL-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Integrating Multimodal Information in Large Pretrained Transformers, by Wasifur Rahman, Md. Kamrul Hasan, Sangwu Lee, AmirAli Bagher Zadeh, Chengfeng Mao, Louis-Philippe Morency and Mohammed E. Hoque
<img src=https://img.shields.io/badge/ICLR-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> VL-BERT: Pre-training of Generic Visual-Linguistic Representations, by Weijie Su, Xizhou Zhu, Yue Cao, Bin Li, Lewei Lu, Furu Wei and Jifeng Dai
<img src=https://img.shields.io/badge/CoRR-2019-blue alt="img" style="zoom:100%; vertical-align: middle" /> VisualBERT: A Simple and Performant Baseline for Vision and Language, by Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh and Kai-Wei Chang
<img src=https://img.shields.io/badge/NeurIPS-2019-blue alt="img" style="zoom:100%; vertical-align: middle" /> ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks, by Jiasen Lu, Dhruv Batra, Devi Parikh and Stefan Lee
<img src=https://img.shields.io/badge/ICCV-2019-blue alt="img" style="zoom:100%; vertical-align: middle" /> VideoBERT: A Joint Model for Video and Language Representation Learning, by Chen Sun, Austin Myers, Carl Vondrick, Kevin Murphy and Cordelia Schmid

Multilingual

<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> GeoMLAMA: Geo-Diverse Commonsense Probing on Multilingual Pre-Trained Language Models, by Da Yin, Hritik Bansal, Masoud Monajatipoor, Liunian Harold Li and Kai-Wei Chang

Reliability

<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective, by Jindong Wang, Xixu Hu, Wenxin Hou, Hao Chen, Runkai Zheng, Yidong Wang, Linyi Yang, Haojun Huang et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Prompting GPT-3 To Be Reliable, by Chenglei Si, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan L. Boyd-Graber and Lijuan Wang
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Plex: Towards Reliability using Pretrained Large Model Extensions, by Dustin Tran, Jeremiah Z. Liu, Michael W. Dusenberry, Du Phan, Mark Collier, Jie Ren, Kehang Han, Zi Wang et al.
<img src=https://img.shields.io/badge/NeurIPS-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Revisiting the Calibration of Modern Neural Networks, by Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran and Mario Lucic
<img src=https://img.shields.io/badge/NeurIPS-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Soft Calibration Objectives for Neural Networks, by Archit Karandikar, Nicholas Cain, Dustin Tran, Balaji Lakshminarayanan, Jonathon Shlens, Michael C. Mozer and Becca Roelofs

Robustness

<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Pretrained Transformers Do not Always Improve Robustness, by Swaroop Mishra, Bhavdeep Singh Sachdeva and Chitta Baral
<img src=https://img.shields.io/badge/ACL-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Pretrained Transformers Improve Out-of-Distribution Robustness, by Dan Hendrycks, Xiaoyuan Liu, Eric Wallace, Adam Dziedzic, Rishabh Krishnan and Dawn Song

Dialogue System

<img src=https://img.shields.io/badge/Mach._Intell._Res.-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> EVA2.0: Investigating Open-domain Chinese Dialogue Systems with Large-scale Pre-training, by Yuxian Gu, Jiaxin Wen, Hao Sun, Yi Song, Pei Ke, Chujie Zheng, Zheng Zhang, Jianzhu Yao et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> DiSTRICT: Dialogue State Tracking with Retriever Driven In-Context Tuning, by Praveen Venkateswaran, Evelyn Duesterwald and Vatche Isahagian
<img src=https://img.shields.io/badge/COLING-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Does GPT-3 Generate Empathetic Dialogues? A Novel In-Context Example Selection Method and Automatic Evaluation Metric for Empathetic Dialogue Generation, by Young-Jun Lee, Chae-Gyun Lim and Ho-Jin Choi
<img src=https://img.shields.io/badge/AAAI-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Fusing Task-Oriented and Open-Domain Dialogues in Conversational Agents, by Tom Young, Frank Xing, Vlad Pandelea, Jinjie Ni and Erik Cambria
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> GODEL: Large-Scale Pre-Training for Goal-Directed Dialog, by Baolin Peng, Michel Galley, Pengcheng He, Chris Brockett, Lars Liden, Elnaz Nouri, Zhou Yu, Bill Dolan et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Mind the Knowledge Gap: A Survey of Knowledge-enhanced Dialogue Systems, by Sagi Shaier, Lawrence Hunter and Katharina Kann
<img src=https://img.shields.io/badge/EMNLP-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Dialogue State Tracking with a Language Model using Schema-Driven Prompting, by Chia-Hsuan Lee, Hao Cheng and Mari Ostendorf
<img src=https://img.shields.io/badge/CoRR-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Few-Shot Bot: Prompt-Based Learning for Dialogue Systems, by Andrea Madotto, Zhaojiang Lin, Genta Indra Winata and Pascale Fung
<img src=https://img.shields.io/badge/NAACL-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems, by Derek Chen, Howard Chen, Yi Yang, Alexander Lin and Zhou Yu
<img src=https://img.shields.io/badge/NAACL-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Fine-grained Post-training for Improving Retrieval-based Dialogue Systems, by Janghoon Han, Taesuk Hong, Byoungjae Kim, Youngjoong Ko and Jungyun Seo
<img src=https://img.shields.io/badge/CoRR-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey, by Jinjie Ni, Tom Young, Vlad Pandelea, Fuzhao Xue, Vinay Adiga and Erik Cambria
<img src=https://img.shields.io/badge/WWW-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Slot Self-Attentive Dialogue State Tracking, by Fanghua Ye, Jarana Manotumruksa, Qiang Zhang, Shenghui Li and Emine Yilmaz
<img src=https://img.shields.io/badge/TACL-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Pretraining the Noisy Channel Model for Task-Oriented Dialogue, by Qi Liu, Lei Yu, Laura Rimell and Phil Blunsom
<img src=https://img.shields.io/badge/AAAI-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> UBAR: Towards Fully End-to-End Task-Oriented Dialog System with GPT-2, by Yunyi Yang, Yunhao Li and Xiaojun Quan
<img src=https://img.shields.io/badge/ACL-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> End-to-End Neural Pipeline for Goal-Oriented Dialogue Systems using GPT-2, by DongHoon Ham, Jeong-Gwan Lee, Youngsoo Jang and Kee-Eung Kim
<img src=https://img.shields.io/badge/NeurIPS-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Simple Language Model for Task-Oriented Dialogue, by Ehsan Hosseini-Asl, Bryan McCann, Chien-Sheng Wu, Semih Yavuz and Richard Socher

Recommender System

<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System, by Yunfan Gao, Tao Sheng, Youlin Xiang, Yun Xiong, Haofen Wang and Jiawei Zhang
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach, by Junjie Zhang, Ruobing Xie, Yupeng Hou, Wayne Xin Zhao, Leyu Lin and Ji-Rong Wen
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Recommender Systems in the Era of Large Language Models (LLMs), by Wenqi Fan, Zihuai Zhao, Jiatong Li, Yunqing Liu, Xiaowei Mei, Yiqi Wang, Jiliang Tang and Qing Li
<img src=https://img.shields.io/badge/TKDE-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Survey on Knowledge Graph-Based Recommender Systems, by Qingyu Guo, Fuzhen Zhuang, Chuan Qin, Hengshu Zhu, Xing Xie, Hui Xiong and Qing He
<img src=https://img.shields.io/badge/SIGIR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Are Graph Augmentations Necessary?: Simple Graph Contrastive Learning for Recommendation, by Junliang Yu, Hongzhi Yin, Xin Xia, Tong Chen, Lizhen Cui and Quoc Viet Hung Nguyen
<img src=https://img.shields.io/badge/TOIS-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Disentangled Representations Learning for Multi-Target Cross-Domain Recommendation, by Guo, Xiaobo, Li, Shaoshuai, Guo, Naicheng, Cao, Jiangxia, Liu, Xiaolei, Ma, Qiongxu, Gan, Runsheng and Zhao, Yunan
<img src=https://img.shields.io/badge/SIGIR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Rethinking Reinforcement Learning for Recommendation: A Prompt Perspective, by Xin Xin, Tiago Pimentel, Alexandros Karatzoglou, Pengjie Ren, Konstantina Christakopoulou and Zhaochun Ren
<img src=https://img.shields.io/badge/CoRR-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Advances and Challenges in Conversational Recommender Systems: A Survey, by Chongming Gao, Wenqiang Lei, Xiangnan He, Maarten de Rijke and Tat-Seng Chua
<img src=https://img.shields.io/badge/MM-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Pre-training Graph Transformer with Multimodal Side Information for Recommendation, by Yong Liu, Susen Yang, Chenyi Lei, Guoxin Wang, Haihong Tang, Juyong Zhang, Aixin Sun and Chunyan Miao
<img src=https://img.shields.io/badge/AAAI-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Towards Hands-Free Visual Dialog Interactive Recommendation, by Tong Yu, Yilin Shen and Hongxia Jin

Event Extraction

<img src=https://img.shields.io/badge/NAACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Word-Label Alignment for Event Detection: A New Perspective via Optimal Transport, by Amir Pouran Ben Veyseh and Thien Huu Nguyen
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Learning Cross-Task Dependencies for Joint Extraction of Entities, Events, Event Arguments, and Relations, by Minh Van Nguyen, Bonan Min, Franck Dernoncourt and Thien Nguyen
<img src=https://img.shields.io/badge/CVPR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> CLIP-Event: Connecting Text and Images with Event Structures, by Manling Li, Ruochen Xu, Shuohang Wang, Luowei Zhou, Xudong Lin, Chenguang Zhu, Michael Zeng, Heng Ji et al.
<img src=https://img.shields.io/badge/ECML-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Augmenting Open-Domain Event Detection with Synthetic Data from GPT-2, by Amir Pouran Ben Veyseh, Minh Van Nguyen, Bonan Min and Thien Huu Nguyen
<img src=https://img.shields.io/badge/EMNLP-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> SeqMix: Augmenting Active Sequence Labeling via Sequence Mixup, by Rongzhi Zhang, Yue Yu and Chao Zhang
<img src=https://img.shields.io/badge/ACL-2019-blue alt="img" style="zoom:100%; vertical-align: middle" /> Exploring Pre-trained Language Models for Event Extraction and Generation, by Sen Yang, Dawei Feng, Linbo Qiao, Zhigang Kan and Dongsheng Li

Event Relation Extraction

<img src=https://img.shields.io/badge/EMNLP-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Is ChatGPT a Good Causal Reasoner? A Comprehensive Evaluation, by Jinglong Gao, Xiao Ding, Bing Qin and Ting Liu
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Learning Cross-Task Dependencies for Joint Extraction of Entities, Events, Event Arguments, and Relations, by Minh Van Nguyen, Bonan Min, Franck Dernoncourt and Thien Nguyen
<img src=https://img.shields.io/badge/AAAI-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Selecting Optimal Context Sentences for Event-Event Relation Extraction, by Hieu Man, Nghia Trung Ngo, Linh Ngo Van and Thien Huu Nguyen
<img src=https://img.shields.io/badge/EMNLP_Findings-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Multilingual SubEvent Relation Extraction: A Novel Dataset and Structure Induction Method, by Viet Dac Lai, Hieu Man, Linh Ngo Van, Franck Dernoncourt and Thien Nguyen
<img src=https://img.shields.io/badge/COLING-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Event Causality Identification via Derivative Prompt Joint Learning, by Shirong Shen, Heng Zhou, Tongtong Wu and Guilin Qi
<img src=https://img.shields.io/badge/EMNLP-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Salience-Aware Event Chain Modeling for Narrative Understanding, by Xiyang Zhang, Muhao Chen and Jonathan May
<img src=https://img.shields.io/badge/EMNLP-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Joint Constrained Learning for Event-Event Relation Extraction, by Haoyu Wang, Muhao Chen, Hongming Zhang and Dan Roth

Data Argumentation

<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> ChatAug: Leveraging ChatGPT for Text Data Augmentation, by Haixing Dai, Zhengliang Liu, Wenxiong Liao, Xiaoke Huang, Zihao Wu, Lin Zhao, Wei Liu, Ninghao Liu et al.
<img src=https://img.shields.io/badge/ICLR-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Combining Ensembles and Data Augmentation Can Harm Your Calibration, by Yeming Wen, Ghassen Jerfel, Rafael Muller, Michael W. Dusenberry, Jasper Snoek, Balaji Lakshminarayanan and Dustin Tran
<img src=https://img.shields.io/badge/EMNLP_Findings-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation, by Kang Min Yoo, Dongju Park, Jaewook Kang, Sang-Woo Lee and Woo-Myoung Park
<img src=https://img.shields.io/badge/EMNLP-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> SeqMix: Augmenting Active Sequence Labeling via Sequence Mixup, by Rongzhi Zhang, Yue Yu and Chao Zhang

Data Annotation

<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Is GPT-3 a Good Data Annotator?, by Bosheng Ding, Chengwei Qin, Linlin Liu, Lidong Bing, Shafiq R. Joty and Boyang Li
<img src=https://img.shields.io/badge/EMNLP_Findings-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Want To Reduce Labeling Cost? GPT-3 Can Help, by Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu and Michael Zeng

Information Extraction

<img src=https://img.shields.io/badge/the_2024_Conference_of_the_North_American_Chapter_of_the_Association_for_Computational_Linguistics:Human_Language_Technologies(Volume_1:_Long_Papers)-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> TISE: A Tripartite In-context Selection Method for Event Argument Extraction, by Fu, Yanhe, Cao, Yanan, Wang, Qingyue and Liu, Yi
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> ULTRA: Unleash LLMs' Potential for Event Argument Extraction through Hierarchical Modeling and Pair-wise Refinement, by Xinliang Frederick Zhang, Carter Wood Blum, Temma Choji, Shalin Shah and Alakananda Vempala
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> ProgGen: Generating Named Entity Recognition Datasets Step-by-step with Self-Reflexive Large Language Models, by Yuzhao Heng, Chunyuan Deng, Yitong Li, Yue Yu, Yinghao Li, Rongzhi Zhang and Chao Zhang
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Simple but Effective Approach to Improve Structured Language Model Output for Information Extraction, by Yinghao Li, Rampi Ramprasad and Chao Zhang
<img src=https://img.shields.io/badge/AAAI-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Is a Large Language Model a Good Annotator for Event Extraction?, by Ruirui Chen, Chengwei Qin, Weifeng Jiang and Dongkyu Choi
<img src=https://img.shields.io/badge/ACL-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Unleashing the Power of Large Language Models in Zero-shot Relation Extraction via Self-Prompting, by Anonymous Submission
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Unified Label-Aware Contrastive Learning Framework for Few-Shot Named Entity Recognition, by Haojie Zhang and Yimeng Zhuang
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> LLM-DA: Data Augmentation via Large Language Models for Few-Shot Named Entity Recognition, by Junjie Ye, Nuo Xu, Yikun Wang, Jie Zhou, Qi Zhang, Tao Gui and Xuanjing Huang
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> On-the-fly Definition Augmentation of LLMs for Biomedical NER, by Monica Munnangi, Sergey Feldman, Byron C Wallace, Silvio Amir, Tom Hope and Aakanksha Naik
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Confidence Matters: Revisiting Intrinsic Self-Correction Capabilities of Large Language Models, by Loka Li, Guangyi Chen, Yusheng Su, Zhenhao Chen, Yixuan Zhang, Eric Xing and Kun Zhang
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Unlocking Instructive In-Context Learning with Tabular Prompting for Relational Triple Extraction, by Guozheng Li, Wenjun Ke, Peng Wang, Zijie Xu, Ke Ji, Jiajun Liu, Ziyu Shang and Qiqing Luo
<img src=https://img.shields.io/badge/AAAI-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Beyond Entities: A Large-Scale Multi-Modal Knowledge Graph with Triplet Fact Grounding, by Jingping Liu, Mingchuan Zhang, Weichen Li, Chao Wang, Shuang Li, Haiyun Jiang, Sihang Jiang, Yanghua Xiao et al.
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Small Language Model Is a Good Guide for Large Language Model in Chinese Entity Relation Extraction, by Xuemei Tang, Jun Wang and Qi Su
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Document-Level In-Context Few-Shot Relation Extraction via Pre-Trained Language Models, by Yilmazcan Ozyurt, Stefan Feuerriegel and Ce Zhang
<img src=https://img.shields.io/badge/COLING-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Improving Recall of Large Language Models: A Model Collaboration Approach for Relational Triple Extraction, by Zepeng Ding, Wenhao Huang, Jiaqing Liang, Yanghua Xiao and Deqing Yang
<img src=https://img.shields.io/badge/ICLR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction, by Oscar Sainz, Iker Garc'\ia-Ferrero, Rodrigo Agerri, Oier Lopez de Lacalle, German Rigau and Eneko Agirre
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> KnowCoder: Coding Structured Knowledge into LLMs for Universal Information Extraction, by Zixuan Li, Yutao Zeng, Yuxin Zuo, Weicheng Ren, Wenxuan Liu, Miao Su, Yucan Guo, Yantao Liu et al.
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Is There a One-Model-Fits-All Approach to Information Extraction? Revisiting Task Definition Biases, by Wenhao Huang, Qianyu He, Zhixu Li, Jiaqing Liang and Yanghua Xiao
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Retrieval Augmented Instruction Tuning for Open NER with Large Language Models, by Tingyu Xie, Jian Zhang, Yan Zhang, Yuanyuan Liang, Qi Li and Hongwei Wang
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Show Less, Instruct More: Enriching Prompts with Definitions and Guidelines for Zero-Shot NER, by Andrew Zamai, Andrea Zugarini, Leonardo Rigutini, Marco Ernandes and Marco Maggini
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Improving Event Definition Following For Zero-Shot Event Detection, by Zefan Cai, Po-Nien Kung, Ashima Suvarna, Mingyu Derek Ma, Hritik Bansal, Baobao Chang, P. Jeffrey Brantingham, Wei Wang et al.
<img src=https://img.shields.io/badge/Thirty--Eighth_{AAAI}Conference_on_Artificial_Intelligence,{AAAI} 2024,Thirty--Sixth_Conference_on_Innovative_Applications_of_Artificial Intelligence,{IAAI}_2024,Fourteenth_Symposium_on_Educational_Advances in_Artificial_Intelligence,{EAAI}_2014,_February_20--27,_2024,_Vancouver, Canada-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> STAR: Boosting Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language Models, by Mingyu Derek Ma, Xiaoxuan Wang, Po-Nien Kung, P. Jeffrey Brantingham, Nanyun Peng and Wei Wang
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Zero-Shot Information Extraction via Chatting with ChatGPT, by Xiang Wei, Xingyu Cui, Ning Cheng, Xiaobin Wang, Xin Zhang, Shen Huang, Pengjun Xie, Jinan Xu et al.
<img src=https://img.shields.io/badge/ACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Prompting Language Models for Linguistic Structure, by Terra Blevins, Hila Gonen and Luke Zettlemoyer
<img src=https://img.shields.io/badge/ACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Causality-aware Concept Extraction based on Knowledge-guided Prompting, by Siyu Yuan, Deqing Yang, Jinxi Liu, Shuyu Tian, Jiaqing Liang, Yanghua Xiao and Rui Xie
<img src=https://img.shields.io/badge/ACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Revisiting Relation Extraction in the era of Large Language Models, by Somin Wadhwa, Silvio Amir and Byron C. Wallace
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Is Information Extraction Solved by ChatGPT? An Analysis of Performance, Evaluation Criteria, Robustness and Errors, by Ridong Han, Tao Peng, Chaohao Yang, Benyou Wang, Lu Liu and Xiang Wan
<img src=https://img.shields.io/badge/ACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Learning In-context Learning for Named Entity Recognition, by Jiawei Chen, Yaojie Lu, Hongyu Lin, Jie Lou, Wei Jia, Dai Dai, Hua Wu, Boxi Cao et al.
<img src=https://img.shields.io/badge/ACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> WebIE: Faithful and Robust Information Extraction on the Web, by Chenxi Whitehouse, Clara Vania, Alham Fikri Aji, Christos Christodoulopoulos and Andrea Pierleoni
<img src=https://img.shields.io/badge/ACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractors, by Kai Zhang, Bernal Jimenez Gutierrez and Yu Su
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Revisiting Large Language Models as Zero-shot Relation Extractors, by Guozheng Li, Peng Wang and Wenjun Ke
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> PIVOINE: Instruction Tuning for Open-world Information Extraction, by Keming Lu, Xiaoman Pan, Kaiqiang Song, Hongming Zhang, Dong Yu and Jianshu Chen
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Product Attribute Value Extraction using Large Language Models, by Alexander Brinkmann, Roee Shraga and Christian Bizer
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> GPT-RE: In-context Learning for Relation Extraction using Large Language Models, by Zhen Wan, Fei Cheng, Zhuoyuan Mao, Qianying Liu, Haiyue Song, Jiwei Li and Sadao Kurohashi
<img src=https://img.shields.io/badge/ISWC-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Text2KGBench: A Benchmark for Ontology-Driven Knowledge Graph Generation from Text, by Nandana Mihindukulasooriya, Sanju Tiwari, Carlos F. Enguix and Kusum Lata
<img src=https://img.shields.io/badge/Findings_of_the_Association_for_Computational_Linguistics:_{EMNLP} 2023,_Singapore,_December_6--10,_2023-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> PIVOINE: Instruction Tuning for Open-world Entity Profiling, by Keming Lu, Xiaoman Pan, Kaiqiang Song, Hongming Zhang, Dong Yu and Jianshu Chen
<img src=https://img.shields.io/badge/-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Zero- and Few-Shots Knowledge Graph Triplet Extraction with Large Language Models, by Andrea Papaluca, Daniel Krefl, Sergio Mendez Rodriguez, Artem Lensky and Hanna Suominen
<img src=https://img.shields.io/badge/EMNLP-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Instruct and Extract: Instruction Tuning for On-Demand Information Extraction, by Yizhu Jiao, Ming Zhong, Sha Li, Ruining Zhao, Siru Ouyang, Heng Ji and Jiawei Han
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Zero- and Few-Shots Knowledge Graph Triplet Extraction with Large Language Models, by Andrea Papaluca, Daniel Krefl, Sergio Mendez Rodriguez, Artem Lenskiy and Hanna Suominen
<img src=https://img.shields.io/badge/EMNLP-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Empirical Study of Zero-Shot NER with ChatGPT, by Tingyu Xie, Qi Li, Jian Zhang, Yan Zhang, Zuozhu Liu and Hongwei Wang
<img src=https://img.shields.io/badge/ACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Extracting Multi-valued Relations from Language Models, by Sneha Singhania, Simon Razniewski and Gerhard Weikum
<img src=https://img.shields.io/badge/IJCNLP-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Zero-shot Triplet Extraction by Template Infilling, by Bosung Kim, Hayate Iso, Nikita Bhutani, Estevam Hruschka, Ndapa Nakashole and Tom M. Mitchell
<img src=https://img.shields.io/badge/CORR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> LLM Instruction-Example Adaptive Prompting (LEAP) Framework for Clinical Relation Extraction, by Anonymous Submission
<img src=https://img.shields.io/badge/EMNLP-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples!, by Yubo Ma, Yixin Cao, Yong Hong and Aixin Sun
<img src=https://img.shields.io/badge/EMNLP-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Chain of Thought with Explicit Evidence Reasoning for Few-shot Relation Extraction, by Xilai Ma, Jing Li and Min Zhang
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Self-Verification Improves Few-Shot Clinical Information Extraction, by Zelalem Gero, Chandan Singh, Hao Cheng, Tristan Naumann, Michel Galley, Jianfeng Gao and Hoifung Poon
<img src=https://img.shields.io/badge/EMNLP-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Guideline Learning for In-Context Information Extraction, by Chaoxu Pang, Yixuan Cao, Qiang Ding and Ping Luo
<img src=https://img.shields.io/badge/ACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> CodeIE: Large Code Generation Models are Better Few-Shot Information Extractors, by Peng Li, Tianxiang Sun, Qiong Tang, Hang Yan, Yuanbin Wu, Xuanjing Huang and Xipeng Qiu
<img src=https://img.shields.io/badge/the_61st_Annual_Meeting_of_the_Association_for_Computational Linguistics_(Volume_1:Long_Papers),{ACL}_2023,_Toronto,_Canada, July_9--14,_2023-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Code4Struct: Code Generation for Few-Shot Event Structure Prediction, by Xingyao Wang, Sha Li and Heng Ji
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large language models are few-shot clinical information extractors, by Monica Agrawal, Stefan Hegselmann, Hunter Lang, Yoon Kim and David A. Sontag
<img src=https://img.shields.io/badge/EMNLP_Findings-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Again, by Bernal Jimenez Gutierrez, Nikolas McNeal, Clayton Washington, You Chen, Lang Li, Huan Sun and Yu Su
<img src=https://img.shields.io/badge/CIKM-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> SPOT: Knowledge-Enhanced Language Representations for Information Extraction, by Jiacheng Li, Yannis Katsis, Tyler Baldwin, Ho-Cheol Kim, Andrew Bartko, Julian J. McAuley and Chun-Nan Hsu
<img src=https://img.shields.io/badge/ACL-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Leveraging Type Descriptions for Zero-shot Named Entity Recognition and Classification, by Rami Aly, Andreas Vlachos and Ryan McDonald

Domain Adaptive

<img src=https://img.shields.io/badge/COLING-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Domain Knowledge Enhanced Pre-Trained Language Model for Vertical Search: Case Study on Medicinal Products, by Kesong Liu, Jianhui Jiang and Feifei Lyu
<img src=https://img.shields.io/badge/EMNLP_Findings-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Snapshot-Guided Domain Adaptation for ELECTRA, by Daixuan Cheng, Shaohan Huang, Jianfeng Liu, Yuefeng Zhan, Hao Sun, Furu Wei, Denvy Deng and Qi Zhang
<img src=https://img.shields.io/badge/EMNLP_Findings-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> VarMAE: Pre-training of Variational Masked Autoencoder for Domain-adaptive Language Understanding, by Dou Hu, Xiaolong Hou, Xiyang Du, Mengyuan Zhou, Lianxin Jiang, Yang Mo and Xiaofeng Shi

Question Answering

<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> KB-Plugin: A Plug-and-play Framework for Large Language Models to Induce Programs over Low-resourced Knowledge Bases, by Jiajie Zhang, Shulin Cao, Linmei Hu, Ling Feng, Lei Hou and Juanzi Li
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> StructLM: Towards Building Generalist Models for Structured Knowledge Grounding, by Alex Zhuang, Ge Zhang, Tianyu Zheng, Xinrun Du, Junjie Wang, Weiming Ren, Stephen W. Huang, Jie Fu et al.
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Aligning Large Language Models to a Domain-specific Graph Database, by Yuanyuan Liang, Keren Tan, Tingyu Xie, Wenbiao Tao, Siyuan Wang, Yunshi Lan and Weining Qian
<img src=https://img.shields.io/badge/AAAI-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Code-Style In-Context Learning for Knowledge-Based Question Answering, by Zhijie Nie, Richong Zhang, Zhongyuan Wang and Xudong Liu
<img src=https://img.shields.io/badge/NAACL-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Prompting Few-shot Multi-hop Question Generation via Comprehending Type-aware Semantics, by Zefeng Lin, Weidong Chen, Yan Song and Yongdong Zhang
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Learn-Then-Reason Model Towards Generalization in Knowledge Base Question Answering, by Lingxi Zhang, Jing Zhang, Yanling Wang, Cuiping Li and Hong Chen
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Few-shot Transfer Learning for Knowledge Base Question Answering: Fusing Supervised Models with In-Context Learning, by Mayur Patidar, Riya Sawhney, Avinash Singh, Biswajit Chatterjee, Mausam and Indrajit Bhattacharya
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> LLM-Based Multi-Hop Question Answering with Knowledge Graph Integration in Evolving Environments, by Ruirui Chen, Weifeng Jiang, Chengwei Qin, Ishaan Singh Rawal, Cheston Tan, Dongkyu Choi, Bo Xiong and Bo Ai
<img src=https://img.shields.io/badge/ACL-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Few-shot Transfer Learning for Knowledge Base Question Answering: Fusing Supervised Models with In-Context Learning, by Mayur Patidar, Riya Sawhney, Avinash Kumar Singh, Biswajit Chatterjee, Mausam and Indrajit Bhattacharya
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Generate-on-Graph: Treat LLM as both Agent and KG in Incomplete Knowledge Graph Question Answering, by Yao Xu, Shizhu He, Jiabei Chen, Zihao Wang, Yangqiu Song, Hanghang Tong, Guang Liu, Kang Liu et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> ChatGPT versus Traditional Question Answering for Knowledge Graphs: Current Status and Future Directions Towards Knowledge Graph Chatbots, by Reham Omar, Omij Mangukiya, Panos Kalnis and Essam Mansour
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Evaluation of ChatGPT as a Question Answering System for Answering Complex Questions, by Yiming Tan, Dehai Min, Yu Li, Wenbo Li, Nan Hu, Yongrui Chen and Guilin Qi
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Bring Your Own KG: Self-Supervised Program Synthesis for Zero-Shot KGQA, by Dhruv Agarwal, Rajarshi Das, Sopan Khosla and Rashmi Gangadharaiah
<img src=https://img.shields.io/badge/-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Prophet: Prompting Large Language Models with Complementary Answer Heuristics for Knowledge-based Visual Question Answering, by Zhou Yu, Xuecheng Ouyang, Zhenwei Shao, Meng Wang and Jun Yu
<img src=https://img.shields.io/badge/-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> keqing: knowledge-based question answering is a nature chain-of-thought mentor of LLM, by Chaojie Wang, Yishi Xu, Zhong Peng, Chenxi Zhang, Bo Chen, Xinrun Wang, Lei Feng and Bo An
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Make a Choice! Knowledge Base Question Answering with In-Context Learning, by Chuanyuan Tan, Yuehe Chen, Wenbiao Shao and Wenliang Chen
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Few-shot In-context Learning for Knowledge Base Question Answering, by Tianle Li, Xueguang Ma, Alex Zhuang, Yu Gu, Yu Su and Wenhu Chen
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> In-Context Learning for Knowledge Base Question Answering for Unmanned Systems based on Large Language Models, by Yunlong Chen, Yaming Zhang, Jianfei Yu, Li Yang and Rui Xia
<img src=https://img.shields.io/badge/EMNLP-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Leveraging Structured Information for Explainable Multi-hop Question Answering and Reasoning, by Ruosen Li and Xinya Du
<img src=https://img.shields.io/badge/KDD-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Mask and Reason: Pre-Training Knowledge Graph Transformers for Complex Logical Queries, by Xiao Liu, Shiyu Zhao, Kai Su, Yukuo Cen, Jiezhong Qiu, Mengdi Zhang, Wei Wu, Yuxiao Dong et al.
<img src=https://img.shields.io/badge/ACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Sequence-to-Sequence Knowledge Graph Completion and Question Answering, by Apoorv Saxena, Adrian Kochsiek and Rainer Gemulla
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> RealTime QA: What's the Answer Right Now?, by Jungo Kasai, Keisuke Sakaguchi, Yoichi Takahashi, Ronan Le Bras, Akari Asai, Xinyan Yu, Dragomir R. Radev, Noah A. Smith et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Relation-aware Language-Graph Transformer for Question Answering, by Jinyoung Park, Hyeong Kyu Choi, Juyeon Ko, Hyeon-Jin Park, Ji-Hoon Kim, Jisu Jeong, Kyung-Min Kim and Hyunwoo J. Kim

Application

<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large Multi-Modal Models (LMMs) as Universal Foundation Models for AI-Native Wireless Systems, by Shengzhe Xu, Christo Kurisummoottil Thomas, Omar Hashash, Nikhil Muralidhar, Walid Saad and Naren Ramakrishnan
<img src=https://img.shields.io/badge/AAAI-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> EcomGPT: Instruction-Tuning Large Language Models with Chain-of-Task Tasks for E-commerce, by Yangning Li, Shirong Ma, Xiaobin Wang, Shen Huang, Chengyue Jiang, Haitao Zheng, Pengjun Xie, Fei Huang et al.
<img src=https://img.shields.io/badge/PLOS_Digital_Health-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models, by Kung, Tiffany H, Cheatham, Morgan, Medenilla, Arielle, Sillos, Czarina, De Leon, Lorie, Elepa~no, Camille, Madriaga, Maria, Aggabao, Rimel et al.
<img src=https://img.shields.io/badge/PLOS_Digital_Health-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> ChatGPT passing USMLE shines a spotlight on the flaws of medical education, by Mbakwe, Amarachi B, Lourentzou, Ismini, Celi, Leo Anthony, Mechanic, Oren J and Dagan, Alon
<img src=https://img.shields.io/badge/-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> BloombergGPT: A Large Language Model for Finance, by Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Is GPT-4 a Good Data Analyst?, by Liying Cheng, Xingxuan Li and Lidong Bing
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> LawBench: Benchmarking Legal Knowledge of Large Language Models, by Zhiwei Fei, Xiaoyu Shen, Dawei Zhu, Fengzhe Zhou, Zhuo Han, Songyang Zhang, Kai Chen, Zongwen Shen et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models, by Neel Guha, Julian Nyarko, Daniel E. Ho, Christopher R'e, Adam Chilton, Aditya Narayana, Alex Chohlas-Wood, Austin Peters et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Robot Learning in the Era of Foundation Models: A Survey, by Xuan Xiao, Jiahang Liu, Zhipeng Wang, Yanmin Zhou, Yong Qi, Qian Cheng, Bin He and Shuo Jiang
<img src=https://img.shields.io/badge/EvoMUSART-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Towards the Generation of Musical Explanations with GPT-3, by Stephen James Krol, Maria Teresa Llano and Jon McCormack

Meta Learning

<img src=https://img.shields.io/badge/ACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Meta-learning via Language Model In-context Tuning, <img src=https://img.shields.io/badge/BERT-yellow alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/DeBERTa-yellow alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/GPT--2-yellow alt="img" style="zoom:100%; vertical-align: middle" /> by Yanda Chen, Ruiqi Zhong, Sheng Zha, George Karypis and He He
<img src=https://img.shields.io/badge/NAACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> MetaICL: Learning to Learn In Context, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/GPT--2-yellow alt="img" style="zoom:100%; vertical-align: middle" /> by Sewon Min, Mike Lewis, Luke Zettlemoyer and Hannaneh Hajishirzi MetaICL proposes a supervised meta-training framework to enable LMs to more effectively learn a new task in context. In MetaICL, each meta-training example includes several training examples from one task that will be presented together as a single sequence to the LM, and the prediction of the final example is used to calculate the loss.

Generalizability

<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Conversation Regression Testing: A Design Technique for Prototyping Generalizable Prompt Strategies for Pre-trained Language Models, by J. D. Zamfirescu-Pereira, Bjoern Hartmann and Qian Yang
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Fine-tuning Pre-trained Language Models with Noise Stability Regularization, by Hang Hua, Xingjian Li, Dejing Dou, Cheng-Zhong Xu and Jiebo Luo
<img src=https://img.shields.io/badge/ACL_Findings-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Do Language Models Perform Generalizable Commonsense Inference?, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" /> by Peifeng Wang, Filip Ilievski, Muhao Chen and Xiang Ren

Language Model as Knowledge Base

<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Understanding Finetuning for Factual Knowledge Extraction from Language Models, by Mehran Kazemi, Sid Mittal and Deepak Ramachandran
<img src=https://img.shields.io/badge/ICML-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large Language Models Struggle to Learn Long-Tail Knowledge, by Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace and Colin Raffel
<img src=https://img.shields.io/badge/EACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Crawling The Internal Knowledge-Base of Language Models, by Roi Cohen, Mor Geva, Jonathan Berant and Amir Globerson
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Measuring and Modifying Factual Knowledge in Large Language Models, by Pouya Pezeshkpour
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> ChatGPT is not Enough: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modeling, by Linyao Yang, Hongyang Chen, Zhao Li, Xiao Ding and Xindong Wu
<img src=https://img.shields.io/badge/ACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Language Model Analysis for Ontology Subsumption Inference, by Yuan He, Jiaoyan Chen, Ernesto Jim'enez-Ruiz, Hang Dong and Ian Horrocks
<img src=https://img.shields.io/badge/ACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> BertNet: Harvesting Knowledge Graphs with Arbitrary Relations from Pretrained Language Models, by Shibo Hao, Bowen Tan, Kaiwen Tang, Bin Ni, Xiyan Shao, Hengzhe Zhang, Eric P. Xing and Zhiting Hu
<img src=https://img.shields.io/badge/ACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Text Augmented Open Knowledge Graph Completion via Pre-Trained Language Models, by Pengcheng Jiang, Shivam Agarwal, Bowen Jin, Xuan Wang, Jimeng Sun and Jiawei Han
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Rethinking Language Models as Symbolic Knowledge Graphs, by Vishwas Mruthyunjaya, Pouya Pezeshkpour, Estevam Hruschka and Nikita Bhutani
<img src=https://img.shields.io/badge/EMNLP-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Models, by Paul Youssef, Osman Alperen Koras, Meijie Li, J"org Schl"otterer and Christin Seifert
<img src=https://img.shields.io/badge/EMNLP_Findings-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Can Language Models Serve as Temporal Knowledge Bases?, by Ruilin Zhao, Feng Zhao, Guandong Xu, Sixiao Zhang and Hai Jin
<img src=https://img.shields.io/badge/ACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Finding Structural Knowledge in Multimodal-BERT, by Victor Milewski, Miryam de Lhoneux and Marie-Francine Moens
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Review on Language Models as Knowledge Bases, by Badr AlKhamissi, Millicent Li, Asli Celikyilmaz, Mona T. Diab and Marjan Ghazvininejad
<img src=https://img.shields.io/badge/Trans._Assoc._Comput._Linguistics-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Time-Aware Language Models as Temporal Knowledge Bases, by Bhuwan Dhingra, Jeremy R. Cole, Julian Martin Eisenschlos, Daniel Gillick, Jacob Eisenstein and William W. Cohen
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Prompting as Probing: Using Language Models for Knowledge Base Construction, by Dimitrios Alivanistos, Selene Baez Santamar'\ia, Michael Cochez, Jan-Christoph Kalo, Emile van Krieken and Thiviyan Thanapalasingam
<img src=https://img.shields.io/badge/EACL-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queries, by Benjamin Heinzerling and Kentaro Inui
<img src=https://img.shields.io/badge/ACL-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases, by Boxi Cao, Hongyu Lin, Xianpei Han, Le Sun, Lingyong Yan, Meng Liao, Tong Xue and Jin Xu
<img src=https://img.shields.io/badge/EMNLP-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Can Language Models be Biomedical Knowledge Bases?, by Mujeen Sung, Jinhyuk Lee, Sean S. Yi, Minji Jeon, Sungdong Kim and Jaewoo Kang
<img src=https://img.shields.io/badge/EMNLP-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts, by Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace and Sameer Singh
<img src=https://img.shields.io/badge/EMNLP-2019-blue alt="img" style="zoom:100%; vertical-align: middle" /> Language Models as Knowledge Bases?, by Fabio Petroni, Tim Rockt"aschel, Sebastian Riedel, Patrick S. H. Lewis, Anton Bakhtin, Yuxiang Wu and Alexander H. Miller

Retrieval-Augmented Language Model

<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Corrective Retrieval Augmented Generation, by Shi-Qi Yan, Jia-Chen Gu, Yun Zhu and Zhen-Hua Ling
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> In-Context Retrieval-Augmented Language Models, by Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown and Yoav Shoham
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Learning Customized Visual Models with Retrieval-Augmented Knowledge, by Haotian Liu, Kilho Son, Jianwei Yang, Ce Liu, Jianfeng Gao, Yong Jae Lee and Chunyuan Li
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> REPLUG: Retrieval-Augmented Black-Box Language Models, by Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer and Wen-tau Yih
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioning, by Zhuolin Yang, Wei Ping, Zihan Liu, Vijay Korthikanti, Weili Nie, De-An Huang, Linxi Fan, Zhiding Yu et al.
<img src=https://img.shields.io/badge/ACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> The Web Can Be Your Oyster for Improving Language Models, by Junyi Li, Tianyi Tang, Wayne Xin Zhao, Jingyuan Wang, Jian-Yun Nie and Ji-Rong Wen
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Knowledge Solver: Teaching LLMs to Search for Domain Knowledge from Knowledge Graphs, by Chao Feng, Xinyu Zhang and Zichu Fei
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Survey on Retrieval-Augmented Text Generation, by Huayang Li, Yixuan Su, Deng Cai, Yan Wang and Lemao Liu
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Retrieval-Augmented Multimodal Language Modeling, by Michihiro Yasunaga, Armen Aghajanyan, Weijia Shi, Rich James, Jure Leskovec, Percy Liang, Mike Lewis, Luke Zettlemoyer et al.
<img src=https://img.shields.io/badge/arXiv_preprint_arXiv-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Atlas: Few-shot learning with retrieval augmented language models, by Izacard, Gautier, Lewis, Patrick, Lomeli, Maria, Hosseini, Lucas, Petroni, Fabio, Schick, Timo, Dwivedi-Yu, Jane, Joulin, Armand et al.
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Training Language Models with Memory Augmentation, by Zexuan Zhong, Tao Lei and Danqi Chen
<img src=https://img.shields.io/badge/ICML-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Improving Language Models by Retrieving from Trillions of Tokens, by Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Jean-Baptiste Lespiau et al.
<img src=https://img.shields.io/badge/CoRR-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> REALM: Retrieval-Augmented Language Model Pre-Training, by Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat and Ming-Wei Chang
<img src=https://img.shields.io/badge/NeurIPS-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, by Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K"uttler, Mike Lewis et al.

Quality

<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Generating Sequences by Learning to Self-Correct, by Sean Welleck, Ximing Lu, Peter West, Faeze Brahman, Tianxiao Shen, Daniel Khashabi and Yejin Choi
<img src=https://img.shields.io/badge/Trans._Assoc._Comput._Linguistics-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Measuring and Improving Consistency in Pretrained Language Models, by Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard H. Hovy, Hinrich Sch"utze and Yoav Goldberg

Interpretability/Explainability

<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Rethinking the Role of Scale for In-Context Learning: An Interpretability-based Case Study at 66 Billion Scale, by Hritik Bansal, Karthik Gopalakrishnan, Saket Dingliwal, Sravan Bodapati, Katrin Kirchhoff and Dan Roth
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Are Hard Examples also Harder to Explain? A Study with Human and Model-Generated Explanations, by Swarnadeep Saha, Peter Hase, Nazneen Rajani and Mohit Bansal
<img src=https://img.shields.io/badge/ACL-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Prompting Contrastive Explanations for Commonsense Reasoning Tasks, by Bhargavi Paranjape, Julian Michael, Marjan Ghazvininejad, Hannaneh Hajishirzi and Luke Zettlemoyer

Data Generation

<img src=https://img.shields.io/badge/ICLR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learning, by Gao, Jiahui, Pi, Renjie, Yong, LIN, Xu, Hang, Ye, Jiacheng, Wu, Zhiyong, ZHANG, WEIZHONG, Liang, Xiaodan et al.
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> ZeroGen: Efficient Zero-shot Learning via Dataset Generation, by Jiacheng Ye, Jiahui Gao, Qintong Li, Hang Xu, Jiangtao Feng, Zhiyong Wu, Tao Yu and Lingpeng Kong
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Generating Training Data with Language Models: Towards Zero-Shot Language Understanding, by Yu Meng, Jiaxin Huang, Yu Zhang and Jiawei Han

Safety

<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Safety Assessment of Chinese Large Language Models, by Hao Sun, Zhexin Zhang, Jiawen Deng, Jiale Cheng and Minlie Huang

Graph Learning

<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large Language Model Meets Graph Neural Network in Knowledge Distillation, by Shengxiang Hu, Guobing Zou, Song Yang, Yanglan Gan, Bofeng Zhang and Yixin Chen
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs, by Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Hongzhi Wen, Xiaochi Wei, Shuaiqiang Wang, Dawei Yin et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Natural Language is All a Graph Needs, by Ruosong Ye, Caiqi Zhang, Runhui Wang, Shuyuan Xu and Yongfeng Zhang
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large Graph Models: A Perspective, by Ziwei Zhang, Haoyang Li, Zeyang Zhang, Yijian Qin, Xin Wang and Wenwu Zhu
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Unleashing the Power of Graph Learning through LLM-based Autonomous Agents, by Lanning Wei, Zhiqiang He, Huan Zhao and Quanming Yao
<img src=https://img.shields.io/badge/EMNLP-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Graph Meets LLM: A Novel Approach to Collaborative Filtering for Robust Conversational Understanding, by Zheng Chen, Ziyan Jiang, Fan Yang, Eunah Cho, Xing Fan, Xiaojiang Huang, Yanbin Lu and Aram Galstyan

Knowledge Storage and Locating

<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Journey to the Center of the Knowledge Neurons: Discoveries of Language-Independent Knowledge Neurons and Degenerate Knowledge Neurons, by Yuheng Chen, Pengfei Cao, Yubo Chen, Kang Liu and Jun Zhao
<img src=https://img.shields.io/badge/ICML-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Can Neural Network Memorization Be Localized?, by Pratyush Maini, Michael Curtis Mozer, Hanie Sedghi, Zachary Chase Lipton, J. Zico Kolter and Chiyuan Zhang
<img src=https://img.shields.io/badge/EMNLP-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Transformer Feed-Forward Layers Are Key-Value Memories, by Mor Geva, Roei Schuster, Jonathan Berant and Omer Levy

Knowledge Fusion

<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Unlocking the Power of Large Language Models for Entity Alignment, by Xuhui Jiang, Yinghan Shen, Zhichao Shi, Chengjin Xu, Wei Li, Zixuan Li, Jian Guo, Huawei Shen et al.
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Two Heads Are Better Than One: Integrating Knowledge from Knowledge Graphs and Large Language Models for Entity Alignment, by Linyao Yang, Hongyang Chen, Xiao Wang, Jing Yang, Fei-Yue Wang and Han Liu
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> LLM Augmented LLMs: Expanding Capabilities through Composition, by Rachit Bansal, Bidisha Samanta, Siddharth Dalmia, Nitish Gupta, Shikhar Vashishth, Sriram Ganapathy, Abhishek Bapna, Prateek Jain et al.
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Arcee's MergeKit: A Toolkit for Merging Large Language Models, by Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vlad Karpukhin, Brian Benedict, Mark McQuade and Jacob Solawetz
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> LangBridge: Multilingual Reasoning Without Multilingual Supervision, by Dongkeun Yoon, Joel Jang, Sungdong Kim, Seungone Kim, Sheikh Shafayat and Minjoon Seo
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaboration, by Shangbin Feng, Weijia Shi, Yike Wang, Wenxuan Ding, Vidhisha Balachandran and Yulia Tsvetkov
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Two Heads Are Better Than One: Integrating Knowledge from Knowledge Graphs and Large Language Models for Entity Alignment, by Linyao Yang, Hongyang Chen, Xiao Wang, Jing Yang, Fei-Yue Wang and Han Liu
<img src=https://img.shields.io/badge/The_Twelfth_International_Conference_on_Learning_Representations, {ICLR}_2024,_Vienna,_Austria,_May_7--11,_2024-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Revisit and Outstrip Entity Alignment: A Perspective of Generative Models, by Lingbing Guo, Zhuo Chen, Jiaoyan Chen, Yin Fang, Wen Zhang and Huajun Chen
<img src=https://img.shields.io/badge/ICLR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Dataless Knowledge Fusion by Merging Weights of Language Models, by Xisen Jin, Xiang Ren, Daniel Preotiuc-Pietro and Pengxiang Cheng
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> AdaMerging: Adaptive Model Merging for Multi-Task Learning, by Enneng Yang, Zhenyi Wang, Li Shen, Shiwei Liu, Guibing Guo, Xingwei Wang and Dacheng Tao
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Resolving Interference When Merging Models, by Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel and Mohit Bansal
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy, by Pingzhi Li, Zhenyu Zhang, Prateek Yadav, Yi-Lin Sung, Yu Cheng, Mohit Bansal and Tianlong Chen
<img src=https://img.shields.io/badge/-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Model Merging by Uncertainty-Based Gradient Matching, by Nico Daheim, Thomas Möllenhoff, Edoardo Maria Ponti, Iryna Gurevych and Mohammad Emtiyaz Khan
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Fusing Models with Complementary Expertise, by Hongyi Wang, Felipe Maia Polo, Yuekai Sun, Souvik Kundu, Eric P. Xing and Mikhail Yurochkin
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> CITING: Large Language Models Create Curriculum for Instruction Tuning, by Tao Feng, Zifeng Wang and Jimeng Sun
<img src=https://img.shields.io/badge/the_12th_Knowledge_Capture_Conference_2023,_{K--CAP} 2023,_Pensacola,_FL,_USA,_December_5--7,_2023-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> OLaLa: Ontology Matching with Large Language Models, by Sven Hertling and Heiko Paulheim

Agent

<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> AUTOACT: Automatic Agent Learning from Scratch via Self-Planning, by Shuofei Qiao, Ningyu Zhang, Runnan Fang, Yujie Luo, Wangchunshu Zhou, Yuchen Eleanor Jiang, Chengfei Lv and Huajun Chen
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Reason for Future, Act for Now: A Principled Framework for Autonomous LLM Agents with Provable Sample Efficiency, by Zhihan Liu, Hao Hu, Shenao Zhang, Hongyi Guo, Shuqi Ke, Boyi Liu and Zhaoran Wang

LLM and GNN

<img src=https://img.shields.io/badge/KDD-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> All in One: Multi-Task Prompting for Graph Neural Networks, by Xiangguo Sun, Hong Cheng, Jia Li, Bo Liu and Jihong Guan
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Graph Neural Prompting with Large Language Models, by Yijun Tian, Huan Song, Zichen Wang, Haozhu Wang, Ziqing Hu, Fang Wang, Nitesh V. Chawla and Panpan Xu
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Graph Prompt Learning: A Comprehensive Survey and Beyond, by Xiangguo Sun, Jiawen Zhang, Xixi Wu, Hong Cheng, Yun Xiong and Jia Li
<img src=https://img.shields.io/badge/-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large Language Models on Graphs: A Comprehensive Survey, by Bowen Jin, Gang Liu, Chi Han, Meng Jiang, Heng Ji and Jiawei Han
<img src=https://img.shields.io/badge/KDD-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> GPPT: Graph Pre-training and Prompt Tuning to Generalize Graph Neural Networks, by Mingchen Sun, Kaixiong Zhou, Xin He, Ying Wang and Xin Wang

Vision LLM

<img src=https://img.shields.io/badge/{IEEE/CVF}_Conference_on_Computer_Vision_and_Pattern_Recognition, {CVPR}_2023,_Vancouver,_BC,_Canada,_June_17--24,_2023-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Visual Atoms: Pre-Training Vision Transformers with Sinusoidal Waves, by Sora Takashima, Ryo Hayamizu, Nakamasa Inoue, Hirokatsu Kataoka and Rio Yokota

LLM and KG

<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> InfuserKI: Enhancing Large Language Models with Knowledge Graphs via Infuser-Guided Knowledge Integration, by Fali Wang, Runxue Bao, Suhang Wang, Wenchao Yu, Yanchi Liu, Wei Cheng and Haifeng Chen
<img src=https://img.shields.io/badge/IEEE_Transactions_on_Knowledge_and_Data_Engineering-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Give Us the Facts: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modeling, by Yang, Linyao, Chen, Hongyang, Li, Zhao, Ding, Xiao and Wu, Xindong
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> GraphRAG: Unlocking LLM discovery on narrative private data, by Jonathan Larson, Steven Truitt
<img src=https://img.shields.io/badge/NeurIPS-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> GraphAdapter: Tuning Vision-Language Models With Dual Knowledge Graph, by Xin Li, Dongze Lian, Zhihe Lu, Jiawang Bai, Zhibo Chen and Xinchao Wang
<img src=https://img.shields.io/badge/AAAI-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Learning Hierarchical Prompt with Structured Linguistic Knowledge for Vision-Language Models, by Yubin Wang, Xinyang Jiang, De Cheng, Dongsheng Li and Cairong Zhao
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> UrbanKGent: A Unified Large Language Model Agent Framework for Urban Knowledge Graph Construction, by Yansong Ning and Hao Liu
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Can Language Models Act as Knowledge Bases at Scale?, by Qiyuan He, Yizhong Wang and Wenya Wang
<img src=https://img.shields.io/badge/NAACL-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Survey, by Garima Agrawal, Tharindu Kumarage, Zeyad Alghami and Huan Liu
<img src=https://img.shields.io/badge/AAAI-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Mitigating Large Language Model Hallucinations via Autonomous Knowledge Graph-Based Retrofitting, by Xinyan Guan, Yanjiang Liu, Hongyu Lin, Yaojie Lu, Ben He, Xianpei Han and Le Sun
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> An Enhanced Prompt-Based LLM Reasoning Scheme via Knowledge Graph-Integrated Collaboration, by Yihao Li, Ru Zhang, Jianyi Liu and Gongshen Liu
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> LLMs Instruct LLMs: An Extraction and Editing Method, by Xin Zhang, Tianjie Ju, Huijia Liang, Ying Fu and Qin Zhang
<img src=https://img.shields.io/badge/the_47th_International_{ACM}_{SIGIR}Conference_on Research_and_Development_in_Information_Retrieval,{SIGIR}_2024,_Washington DC,_USA,_July_14--18,_2024-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answering, by Zhentao Xu, Mark Jerome Cruz, Matthew Guevara, Tie Wang, Manasi Deshpande, Xiaofeng Wang and Zheng Li
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> HyKGE: A Hypothesis Knowledge Graph Enhanced Framework for Accurate and Reliable Medical LLMs Responses, by Xinke Jiang, Ruizhe Zhang, Yongxin Xu, Rihong Qiu, Yue Fang, Zhiyuan Wang, Jinyi Tang, Hongxin Ding et al.
<img src=https://img.shields.io/badge/The_Twelfth_International_Conference_on_Learning_Representations, {ICLR}_2024,_Vienna,_Austria,_May_7--11,_2024-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph, by Jiashuo Sun, Chengjin Xu, Lumingyuan Tang, Saizhuo Wang, Chen Lin, Yeyun Gong, Lionel M. Ni, Heung-Yeung Shum et al.
<img src=https://img.shields.io/badge/ACM_Trans._Asian_Low_Resour._Lang._Inf._Process.-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> CodeKGC: Code Language Model for Generative Knowledge Graph Construction, by Zhen Bi, Jing Chen, Yinuo Jiang, Feiyu Xiong, Wei Guo, Huajun Chen and Ningyu Zhang
<img src=https://img.shields.io/badge/-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Fusing Knowledge Graphs and Large Language Models, by Rudy Agovic
<img src=https://img.shields.io/badge/-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> RAG with a Neo4j Knowledge Graph: How it Works and How to Set It Up, by Neo4j
<img src=https://img.shields.io/badge/-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Making Large Language Models Perform Better in Knowledge Graph Completion, by Yichi Zhang, Zhuo Chen, Wen Zhang and Huajun Chen
<img src=https://img.shields.io/badge/TGDK-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large Language Models and Knowledge Graphs: Opportunities and Challenges, by Jeff Z. Pan, Simon Razniewski, Jan-Christoph Kalo, Sneha Singhania, Jiaoyan Chen, Stefan Dietze, Hajira Jabeen, Janna Omeliyanenko et al.
<img src=https://img.shields.io/badge/EMNLP-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Evaluating the Knowledge Base Completion Potential of GPT, by Blerta Veseli, Simon Razniewski, Jan-Christoph Kalo and Gerhard Weikum

Others

<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Toward General Design Principles for Generative AI Applications, by Justin D. Weisz, Michael J. Muller, Jessica He and Stephanie Houde
<img src=https://img.shields.io/badge/NeurIPS-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Unsupervised Representation Learning from Pre-trained Diffusion Probabilistic Models, by Zijian Zhang, Zhou Zhao and Zhijie Lin
<img src=https://img.shields.io/badge/AAAI-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Parsing as Pretraining, by David Vilares, Michalina Strzyz, Anders S\ogaard and Carlos G'omez-Rodr'\iguez
<img src=https://img.shields.io/badge/AAAI-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Unsupervised Deep Learning via Affinity Diffusion, by Jiabo Huang, Qi Dong, Shaogang Gong and Xiatian Zhu
<img src=https://img.shields.io/badge/-2019-blue alt="img" style="zoom:100%; vertical-align: middle" /> HellaSwag: Can a Machine Really Finish Your Sentence?, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" /> by Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi and Yejin Choi
<img src=https://img.shields.io/badge/CVPR-2009-blue alt="img" style="zoom:100%; vertical-align: middle" /> Learning to detect unseen object classes by between-class attribute transfer, by Christoph H. Lampert, Hannes Nickisch and Stefan Harmeling

Related Works

Git Repos

Awesome-ChatGPT, ChatGPT资料汇总学习，持续更新......
Awesome ChatGPT Prompts, In this repository, you will find a variety of prompts that can be used with ChatGPT.
ChatRWKV, ChatRWKV is like ChatGPT but powered by my RWKV (100% RNN) language model, which is the only RNN (as of now) that can match transformers in quality and scaling, while being faster and saves VRAM. Training sponsored by Stability EleutherAI.
ChatGPT-Hub, ChatGPT资源汇总
PaLM-rlhf-pytorch, Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture.
BAAI-WuDao/Data, “悟道”项目构建了高质量的数据集，用于支撑大模型的训练和测评工作，本仓库提供所有开源数据集的链接。
Colossal-AI, Colossal-AI provides a collection of parallel components for you. We aim to support you to write your distributed deep learning models just like how you write your model on your laptop. We provide user-friendly tools to kickstart distributed training and inference in a few lines.

Articles

Exploring Prompt Injection Attacks, by Jose Selvi Prompt Injection is a new vulnerability that is affecting some AI/ML models and, in particular, certain types of language models using prompt-based learning.
ChatGPT发展历程、原理、技术架构详解和产业未来, by 陈巍 本文将介绍ChatGPT的特点、功能、技术架构、局限、产业应用、投资机会和未来。作者本人曾担任华为系自然语言处理（ NLP ）企业的首席科学家。

Blogs

How does GPT Obtain its Ability?, by Yao Fu Tracing emergent abilities of language models to their sources.
Open source solution replicates ChatGPT training process, Colossal-AI, as one of the hottest open-source solutions for large AI models, presents an open-source low-cost ChatGPT equivalent implementation process.

Demos

CPM-Bee, CPM-Bee是一个开源的双语预训练语言模型，参数量为10B，拥有十余种原生能力和强大的通用语言能力，并支持结构化输入和输出。

Reports

Lectures

Chain of Thought Prompting for Large Language Model Reasoning,

Related Works

Git Repos

Awesome-ChatGPT, ChatGPT资料汇总学习，持续更新......
Awesome ChatGPT Prompts, In this repository, you will find a variety of prompts that can be used with ChatGPT.
ChatRWKV, ChatRWKV is like ChatGPT but powered by my RWKV (100% RNN) language model, which is the only RNN (as of now) that can match transformers in quality and scaling, while being faster and saves VRAM. Training sponsored by Stability EleutherAI.
ChatGPT-Hub, ChatGPT资源汇总
PaLM-rlhf-pytorch, Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture.
BAAI-WuDao/Data, “悟道”项目构建了高质量的数据集，用于支撑大模型的训练和测评工作，本仓库提供所有开源数据集的链接。
Colossal-AI, Colossal-AI provides a collection of parallel components for you. We aim to support you to write your distributed deep learning models just like how you write your model on your laptop. We provide user-friendly tools to kickstart distributed training and inference in a few lines.

Articles

Exploring Prompt Injection Attacks, by Jose Selvi Prompt Injection is a new vulnerability that is affecting some AI/ML models and, in particular, certain types of language models using prompt-based learning.
ChatGPT发展历程、原理、技术架构详解和产业未来, by 陈巍 本文将介绍ChatGPT的特点、功能、技术架构、局限、产业应用、投资机会和未来。作者本人曾担任华为系自然语言处理（ NLP ）企业的首席科学家。

Blogs

How does GPT Obtain its Ability?, by Yao Fu Tracing emergent abilities of language models to their sources.
Open source solution replicates ChatGPT training process, Colossal-AI, as one of the hottest open-source solutions for large AI models, presents an open-source low-cost ChatGPT equivalent implementation process.

Demos

CPM-Bee, CPM-Bee是一个开源的双语预训练语言模型，参数量为10B，拥有十余种原生能力和强大的通用语言能力，并支持结构化输入和输出。

Reports

Lectures

Chain of Thought Prompting for Large Language Model Reasoning,

<img src="https://avatars.githubusercontent.com/u/77821103?s=400&u=17b0ffcd148c697c9f604d8ed4241ffa8fb62257&v=4" alt="img" style="zoom:25%; vertical-align: middle" /> Researcher Recruitment 科研人员招聘

Knowledge Science and Engineering Lab is recruiting researchers! You are welcome to apply for the following positions:

Research Assistant: Bachelor degree or above, proficient in Python/Java, familiar with machine learning espicially deep learning models.
Postdoctoral Fellow: Doctoral research in Artificial Intelligence, published at least 3 high-quality papers.
Lecturer, Associate Professor and Professor

If you are interested in our research and meet the above requirements, feel free to contact Prof. Guilin Qi.

知识科学与工程实验室正在招聘科研人员！欢迎申请以下岗位：

科研助理：本科学历以上，精通Python/Java，熟悉机器学习，特别是深度学习模型。
博士后：博士研究人工智能相关方向，发表至少3篇高水平论文。
讲师、副教授、教授等教职

如果您对我们的研究工作感兴趣并满足以上要求，欢迎您与漆桂林教授联系。

Awesome

Resources on ChatGPT and Large Language Models

Contributors

Papers

Outline

Hyperlinks

Evaluation

Survey

In-Context Learning

Instruction Tuning

RLHF

Pre-Training Techniques

Mixtures of Experts

Knowledge Enhanced

Knowledge Distillation

Knowledge Generation

Knowledge Editing

Reasoning

Chain of Thought

Multi-Step Reasoning

Arithmetic Reasoning

Symbolic Reasoning

Chain of Verification

Knowledge Graph Embedding

Federated Learning

Distributed AI

Selective Annotation

Program and Code Generation

Code Representation

Code Fixing

Code Review

Program Generation

Software Engineering

AIGC

Controllable Text Generation

Continual Learning

Prompt Engineering

Natural Language Understanding

Multimodal

Multilingual

Reliability

Robustness

Dialogue System

Recommender System

Event Extraction

Event Relation Extraction

Data Argumentation

Data Annotation

Information Extraction

Domain Adaptive

Question Answering

Application

Meta Learning

Generalizability

Language Model as Knowledge Base

Retrieval-Augmented Language Model

Quality

Interpretability/Explainability

Data Generation

Safety

Graph Learning

Knowledge Storage and Locating

Knowledge Fusion

Agent

LLM and GNN

Vision LLM

LLM and KG

Others

Related Works

Git Repos

Articles

Blogs

Demos

Reports

Lectures

Related Works

Git Repos

Articles

Blogs

Demos