Home

Awesome

Can ChatGPT Assess Human Personalities? <br> A General Evaluation Framework

By Haocong Rao, Cyril Leung, and Chunyan Miao. In EMNLP 2023 (Findings).

Introduction

This is a simplified implementation of automated testing codes for “Can ChatGPT Assess Human Personalities? A General Evaluation Framework”. The codes are used to produce one independent testing result of personality assessment. Users may customize the codes to execute multiple testings to reproduce the complete result in the paper.

Environment

pip install openai, ChatGPT_lite, asyncio, requests, brotli, numpy, csv, json

Usage

python Query_GPT.py --model ChatGPT --api_key [input your api key]

# Default options: --model GPT4 
# --model [GPT4, ChatGPT, InstructGPT]

An example result returned from Crawler_16personalities.py is shown as:

Subject: Men
Trait: Extraverted (E) 59 | Introverted (I) 41
Trait: Intuitive (N) 62 | Observant (S) 38
Trait: Thinking (T) 48 | Feeling (F) 52
Trait: Judging (J) 63 | Prospecting (P) 37
Trait: Assertive (A) 40 | Turbulent (T) 60
Character: enfj-protagonist
Dic. Judge: ENFJ-T Protagonist

Note: Query_ChatGPT.py (Line 160) requires the session token from the ChatGPT interface, and it is free and connected using ChatGPT_lite library. Query_InstructGPT.py (Line 8) requires the API key generated from your OpenAI account.

Configuration

response = openai.Completion.create(
model="text-davinci-003",  # choose your testing LLM
prompt=[statement + instruction],
temperature=1.0,
max_tokens=2000,
top_p=1,
frequency_penalty=0,
presence_penalty=0)

Update

image

LLMs-PA UI Usage

1. Menu overview (API Key Required)

image

2. Query different subjects (65 Subjects)

image

3. Make a single-question query

image

4. Query all questions

image

5. Obtain personality results

image

For more details on these functions, please refer to the framework in our paper.

Citation

If you found this repository useful, please consider citing:

@article{rao2023can,
  title={Can {ChatGPT} Assess Human Personalities? A General Evaluation Framework},
  author={Rao, Haocong and Leung, Cyril and Miao, Chunyan},
  journal={arXiv preprint arXiv:2303.01248},
  year={2023}
}

License

ChatGPT-MBTI is released under the MIT License.