Awesome

GPT-NER: Named Entity Recognition via Large LanguageModels

Introduction

This repo contains code for the paper GPT-NER: Named Entity Recognition via Large LanguageModels.

@article{wang2023gpt,
  title={GPT-NER: Named Entity Recognition via Large Language Models},
  author={Wang, Shuhe and Sun, Xiaofei and Li, Xiaoya and Ouyang, Rongbin and Wu, Fei and Zhang, Tianwei and Li, Jiwei and Wang, Guoyin},
  journal={arXiv preprint arXiv:2304.10428},
  year={2023}
}

Usage

Requirements

• python>=3.7.3
• openai==0.27.2
• simcse==0.4

This repor mainly use two addtional packages: SimCSE and OpenAI. So, if you want to know more about the arguments used in codes, please refer to the corresponding documents.

Proposed Dataset

For the full NER dataset, we follow MRC-NER for preprocessing, and you can directly download these here.

For sampled 100-dataset, we have put them on the Google Drive.

Few-shot Demonstrations Retrieval

For sentence-level embeddings, run openai_access/extract_mrc_knn.py.

Note that you should change the directory for the input/output file and the used SimCSE model. In this repo, the model sup-simcse-roberta-large is used for SimCSE, and you can find it here.

OpenAI Access

We follow the official steps to access the GPT-* Models, and the document can be found here. Before you run our scripts, you need to add OPENAI_API_KEY, which you can find it in your account profile, to the environment variable by the command export OPENAI_API_KEY="YOUR_KEY".

To get preditions, please run openai_access/scripts/access_ai.sh, and the used arguments are listed in file openai_access/get_results_mrc_knn.py.

For self-verification, please run openai_access/scripts/verify.sh, and the used arguments are listed in file openai_access/verify_results.py.

Note that accessing to the GPT-3 is very expensive, we thus strongly advise you to start from our sampled 100-dataset.

Evaluate

We use span-level precession, recall and F1-score for evaluation, and to do this, please run the script openai_access/scripts/compute_f1.sh.

Results

Table 1: Results of sampled 100 pieces of data for two Flat NER datasets: CoNLL2003 and OntoNotes5.0.

Table 2: Results of full data for two Flat NER datasets: CoNLL2003 and OntoNotes5.0.

Table 3: Results of full data for three Nested NER datasets: ACE2004, ACE2005 and GENIA.

Contact

If you have any issues or questions about this repo, feel free to contact wangshuhe@stu.pku.edu.cn.