Awesome

PsyDI

English | 简体中文(Simplified Chinese) | 🔥PsyDI Paper

Introduction to PsyDI

PsyDI is a multi-modal and interactive chatbot for psychological assessments. Its objective is to explore users' potential cognitive styles through interactive analysis of their multimodal inputs, finally determining their Myers-Briggs Type Indicator (MBTI). Additionally, PsyDI offers feedback in the form of customized figures and detailed analysis for each user. We are continuously improving PsyDI, with upcoming features such as an MBTI gallery. Your feedback is valuable to us!

Access PsyDI: English version | Chinese version

:boom: News!

• We've recently refreshed our collection of classical character galleries:

Outline

• Introduction to PsyDI
• News
• Outline
• Quick Start
• PsyDI Mini Pipeline
• Introduction
  • Evaluation Framework
  • Process Description
  • Iterative Refinement
• Evaluation
• Roadmap
• Running Frontend Locally
• Acnowledgements
• Feedback and Contribution
• Citation
• License

:star_struck: Quick Start

Getting started with PsyDI is easy! Follow these simple steps to begin your journey of self-discovery:

:rocket: Accessing PsyDI Online: Visit our website to access PsyDI directly online. No downloads or installations required!

:memo: Beginning the Quiz:

• Upon entering the quiz, you'll be prompted to choose a label that best represents you. This helps PsyDI tailor the assessment to your personality.
• Next, you'll be asked to share one of your recent favorite songs and your most recent thoughts. This information provides valuable insights into your current mindset and preferences.

:bulb: Exploring Your Personality:

• In the Explore chapter, PsyDI will ask you a series of questions to gain a basic understanding of who you are based on your provided tags. This initial interaction sets the stage for deeper exploration.

:speech_balloon: Interactive Chatting:

• PsyDI will chat with you to delve deeper into topics mentioned earlier. This interactive process typically consists of 12-15 questions, allowing PsyDI to get to know you better.

:sparkles: Detailed Analysis:

• Finally, PsyDI will provide you with a comprehensive analysis of your Myers-Briggs Type Indicator (MBTI) and characteristics. This analysis offers valuable insights into your personality traits and tendencies.
• Additionally, you'll receive a generated image that matches your temperament, adding a visual element to your understanding of yourself.

Now that you know the basics, dive in and start your journey with PsyDI today!

:key: PsyDI Mini Pipeline

We offer a mini version of the PsyDI pipeline as a demo, making it accessible for everyone to use. This mini pipeline allows users to input several initial posts and generates an initial MBTI score table. It then selects posts with the highest likelihood of having deep meaning to initiate a multi-turn dialogue with the user. After the dialogue, the pipeline updates the user's MBTI score table. Such mini pipeline is the core module of PsyDI as mentioned in the Introduction.

In this mini pipeline, we apply DeepSeek as a large language model example to provide multi-turn dialogue. Users can also replace it with any other language model with multi-turn dialogue capabilities.

Installation

pip3 install -r requirements.txt

Usage

Mini Pipeline with Our Released Score Model

REWARD_MODEL_PATH=<rm-path> API_KEY=<your-api-key> API_URL=https://api.deepseek.com MODEL_NAME=deepseek-chat python3 psydi_mini_agent.py

Mini Pipeline with Fake Debug Score Model

DEBUG=true API_KEY=<your-api-key> API_URL=https://api.deepseek.com MODEL_NAME=deepseek-chat python3 psydi_mini_agent.py

:books: Introduction

Evaluation Framework

At the core of PsyDI's evaluation process is a table of MBTI scores, following established psychological testing methods. Each row of this table represents the probability of the current user being assigned to a specific MBTI type, ranging from 0 to 100.

Process Description

PsyDI first converts the user's multimodal information into text form. The score model then evaluates these textual expressions, providing probabilities for each sentence being associated with any MBTI type. PsyDI iterates through the following three steps until determining the user's MBTI:

1. Post Selection: PsyDI selects the post with the highest scores under both of the two MBTI types with the Top-2 probabilities. This step aims to pinpoint the most ambiguous information and prompt further questions to clarify.

2. Dialog Interaction: PsyDI engages the user in multiple rounds of dialog, utilizing three interaction formats: multiple choice, forced choice, and free question and answer. All the question and answer pairs are integrated into a new post.

3. Table Update: PsyDI updates the MBTI score table with the new post, incorporating the insights gained from the dialog interaction.

Iterative Refinement

PsyDI iterates through these steps until it reaches a confident determination of the user's MBTI, continually refining its understanding through each interaction.

Evaluation

Score Model Evaluation

We have constructed a pair-wise MBTI dataset and compared the performance of multiple closed-source and open-source models with PsyDI on this dataset. PsyDI has demonstrated superior results over existing language models across various MBTI datasets, as shown in the following:

Pipeline Evaluation

We assessed the accuracy of the PsyDI Pipeline compared to existing MBTI tests in evaluating MBTI bots, with the results displayed on the left side of the figure below. Additionally, we evaluated the degree of dependence of the PsyDI Pipeline on initial dynamics and found that PsyDI can determine the user's true MBTI in subsequent conversations even under random initial dynamics, with the results shown on the right side of the figure:

Roadmap

• Online deployment of PsyDI
• Release all the frontend code and deployment scripts
• MBTI evaluation gallery
• Other language support (English)
• Release the backend code examples and prompts
• Release the generated datasets
• Release the trained reward model
• Technical report arxiv link
• More multi-modal and interactive questions and examples

Running Frontend Locally

You will first need to use the environment variables defined in .env.example to run PsyDI agent.

Note: You should not commit your .env file or it will expose secrets that will allow others to control access to your various OpenAI and authentication provider accounts.

Then you need to install node.js (v18.17.0 is recommended) and npm on your machine.

After setting up all the prerequisites, you can run the following commands to start the frontend:

npm install -g pnpm
pnpm install
pnpm dev --port 3001

Your app template should now be running on localhost:3001.

Acnowledgements

• Thanks JAAAAACKKKKKY for her contributions to the UI/UX design and artistic materials for this project."
• vercel/ai-chatbot

Feedback and Contribution

• File an issue on Github

• Discuss on PsyDI's WeChat group (i.e. add us on WeChat: ding314assist)

  <img src=https://github.com/opendilab/PsyDI/blob/main/assets/wechat.jpeg width=35% />

• Contact our email (opendilab@pjlab.org.cn)

We appreciate all the feedbacks and contributions to improve PsyDI, both algorithms and system designs.

Citation

@article{li2024psydi,
    title={PsyDI: Towards a Personalized and Progressively In-depth Chatbot for Psychological Measurements},
    author={Li, Xueyan and Chen, Xinyan and Niu, Yazhe and Hu, Shuai and Liu, Yu},
    journal={arXiv preprint arXiv:2408.03337},
    year={2024}
}

License

PsyDI is released under the Apache 2.0 license.