Home

Awesome

LLM Zoo: democratizing ChatGPT

<div align=center> <img src="assets/zoo.png" width = "640" alt="zoo" align=center /> </div>

⚑LLM Zoo is a project that provides data, models, and evaluation benchmark for large language models.⚑ [Tech Report]

✨ Latest News

πŸ€” Motivation

"AI supremacy" is understood as a company's absolute leadership and monopoly position in an AI field, which may even include exclusive capabilities beyond general artificial intelligence. This is unacceptable for AI community and may even lead to individual influence on the direction of the human future, thus bringing various hazards to human society.

🎬 Get started

Install

Run the following command to install the required packages:

pip install -r requirements.txt

CLI Inference

python -m llmzoo.deploy.cli --model-path /path/to/weights/

For example, for Phoenix, run

python -m llmzoo.deploy.cli --model-path FreedomIntelligence/phoenix-inst-chat-7b

and it will download the model from Hugging Face automatically. For Chimera, please follow this instruction to prepare the weights.

Check here for deploying a web application.

πŸ“š Data

Overview

We used the following two types of data for training Phoenix and Chimera:

<details><summary><b>Instruction data</b></summary>
+ Self-Instructed / Translated (Instruction, Input) in Language A
- ---(Step 1) Translation --->
+ (Instruction, Input) in Language B (B is randomly sampled w.r.t. the probability distribution of realistic languages)
- ---(Step 2) Generate--->
+ Output in Language B
+ (Role, Instruction, Input) seeds
- ---(Step 1) Self Instruct--->
+ (Role, Instruction, Input) samples
- ---(Step 2) generate output Instruct--->
+ (Role, Instruction, Input) ---> Output
</details> <details><summary><b>Conversation data</b></summary>
+ ChatGPT conversations shared on the Internet
- ---(Step 1) Crawl--->
+ Multi-round conversation data
</details>

Check InstructionZoo for the collection of instruction datasets.

Check GPT-API-Accelerate Tool for faster data generation using ChatGPT.

Download

🐼 Models

Overview of existing models

ModelBackbone#ParamsOpen-source modelOpen-source dataClaimed languagePost-training (instruction)Post-training (conversation)Release date
ChatGPT--❌❌multi11/30/22
Wenxin--❌❌zh03/16/23
ChatGLMGLM6Bβœ…βŒen, zh03/16/23
AlpacaLLaMA7Bβœ…βœ…en52K, en❌03/13/23
DollyGPT-J6Bβœ…βœ…en52K, en❌03/24/23
BELLEBLOOMZ7Bβœ…βœ…zh1.5M, zh❌03/26/23
GuanacoLLaMA7Bβœ…βœ…en, zh, ja, de534K, multi❌03/26/23
Chinese-LLaMA-AlpacaLLaMA7/13Bβœ…βœ…en, zh2M/3M, en/zh❌03/28/23
LuoTuoLLaMA7Bβœ…βœ…zh52K, zh❌03/31/23
VicunaLLaMA7/13Bβœ…βœ…en❌70K, multi03/13/23
KoalaLLaMA13Bβœ…βœ…en355K, en117K, en04/03/23
BAIZELLaMA7/13/30Bβœ…βœ…en52K, en111.5K, en04/04/23
Phoenix (Ours)BLOOMZ7Bβœ…βœ…multi40+40+04/08/23
Latin Phoenix: Chimera (Ours)LLaMA7/13Bβœ…βœ…multi (Latin)LatinLatin04/08/23
<details><summary><b>The key difference between existing models and ours.</b></summary>

The key difference in our models is that we utilize two sets of data, namely instructions and conversations, which were previously only used by Alpaca and Vicuna respectively. We believe that incorporating both types of data is essential for a recipe to achieve a proficient language model. The rationale is that the instruction data helps to tame language models to adhere to human instructions and fulfill their information requirements, while the conversation data facilitates the development of conversational skills in the model. Together, these two types of data complement each other to create a more well-rounded language model.

</details>

Phoenix (LLM across Languages)

<details><summary><b>The philosophy to name</b></summary>

The first model is named Phoenix. In Chinese culture, the Phoenix is commonly regarded as a symbol of the king of birds; as the saying goes "η™ΎιΈŸζœε‡€", indicating its ability to coordinate with all birds, even if they speak different languages. We refer to Phoenix as the one capable of understanding and speaking hundreds of (bird) languages. More importantly, Phoenix is the totem of "the Chinese University of Hong Kong, Shenzhen" (CUHKSZ); it goes without saying this is also for the Chinese University of Hong Kong (CUHK).

</details>
ModelBackboneDataLink
Phoenix-chat-7bBLOOMZ-7b1-mtConversationparameters
Phoenix-inst-chat-7bBLOOMZ-7b1-mtInstruction + Conversationparameters
Phoenix-inst-chat-7b-int4BLOOMZ-7b1-mtInstruction + Conversationparameters

Chimera (LLM mainly for Latin and Cyrillic languages)

<details><summary><b>The philosophy to name</b></summary>

The philosophy to name: The biggest barrier to LLM is that we do not have enough candidate names for LLMs, as LLAMA, Guanaco, Vicuna, and Alpaca have already been used, and there are no more members in the camel family. Therefore, we find a similar hybrid creature in Greek mythology, Chimera, composed of different Lycia and Asia Minor animal parts. Coincidentally, it is a hero/role in DOTA (and also Warcraft III). It could therefore be used to memorize a period of playing games overnight during high school and undergraduate time.

</details>
ModelBackboneDataLink
Chimera-chat-7bLLaMA-7bConversationparameters (delta)
Chimera-chat-13bLLaMA-13bConversationparameters (delta)
Chimera-inst-chat-7bLLaMA-7bInstruction + Conversationparameters (delta)
Chimera-inst-chat-13bLLaMA-13bInstruction + Conversationparameters (delta)

Due to LLaMA's license restrictions, we follow FastChat to release our delta weights. To use Chimera, download the original LLaMA weights and run the script:

python tools/apply_delta.py \
 --base /path/to/llama-13b \
 --target /output/path/to/chimera-inst-chat-13b \
 --delta FreedomIntelligence/chimera-inst-chat-13b-delta

CAMEL (Chinese And Medically Enhanced Langauge models)

<details><summary><b>The philosophy to name</b></summary>

The philosophy to name: Its Chinese name is HuatuoGPT or εŽδ½—GPT to commemorate the great Chinese physician named Hua Tuo (εŽδ½—), who lived around 200 AC. Training is already finished; we will release it in two weeks; some efforts are needed to deploy it in public cloud servers in case of massive requests.

</details>

Check our models in HuatuoGPT or try our demo . Similar biomedical models could be seen in biomedical LLMs.

<details><summary><b>More models in the future</b></summary>

Legal GPT (coming soon)

Vision-Language Models (coming soon)

Retrieval-augmented Models (coming soon)

</details>

🧐 Evaluation and Benchmark

We provide a bilingual, multidimensional comparison across different open-source models with ours.

Chinese

ModelRatio
Phoenix-inst-chat-7b vs. ChatGPT85.2%
Phoenix-inst-chat-7b vs. ChatGLM-6b94.6%
Phoenix-inst-chat-7b vs. Baidu-Wenxin96.8%
Phoenix-inst-chat-7b vs. MOSS-moon-003-sft109.7%
Phoenix-inst-chat-7b vs. BELLE-7b-2m122.7%
Phoenix-inst-chat-7b vs. Chinese-Alpaca-7b135.3%
Phoenix-inst-chat-7b vs. Chinese-Alpaca-13b125.2%

Observation: It shows that Phoenix-chat-7b achieves 85.2% performance of ChatGPT in Chinese. It slightly underperforms Baidu-Wenxin (96.8%) and ChatGLM-6b (94.6 %), both are not fully open-source; ChatGLM-6b only provides model weights without training data and details. Although Phoenix is a multilingual LLM, it achieves SOTA performance among all open-source Chinese LLMs.

wintielose
Phoenix vs. ChatGPT123553
Phoenix vs. ChatGLM-6b361153
Phoenix vs. Baidu-Wenxin292546
Phoenix vs. BELLE-7b-2m553114
Phoenix vs. Chinese-Alpaca-13b563113

Observation: It shows that the human evaluation results show the same trend as the automatic evaluation results.

English

ModelRatio
Chimera-chat-7b vs. ChatGPT85.2%
Chimera-chat-13b vs. ChatGPT92.6%
Chimera-inst-chat-13b vs. ChatGPT96.6%

πŸ‘Ύ Quantization

We offer int8 and int4 quantizations, which will largely reduce the GPU memory consumption, e.g., from ~28GB to ~7GB for phoenix.

Int8

You can directly obatin int8 version of phoenix by passing --load-8bit when using cli inference. E.g.,

python -m llmzoo.deploy.cli --model-path FreedomIntelligence/phoenix-inst-chat-7b --load-8bit

Int4

For int4 version, we take advantage of GPTQ. You can directly obatin int4 version of Phoenix by passing int4 version model and --load-4bit when using cli inference. This would require package AutoGPTQ be installed. E.g.,

python -m llmzoo.deploy.cli --model-path FreedomIntelligence/phoenix-inst-chat-7b-int4 --load-4bit

We use AutoGPTQ to support Phoenix via,

BUILD_CUDA_EXT=0 pip install auto-gptq[triton]

For Chimera, we can not share the int4 version parameters due to restrictions. And you can follow the example in our patched AutoGPTQ to conduct quantization by yourselves.

Thank yhyu13, please check the merged weight and GPTQ quantized weight for chimera in chimera-inst-chat-13b-hf and chimera-inst-chat-13b-gptq-4bit.

Inference in pure C/C++: You can refer to this link to run Chimera or Phoenix on your PC.

🏭 Deployment

Launch a controller

python -m llmzoo.deploy.webapp.controller

Launch a model worker

python -m llmzoo.deploy.webapp.model_worker --model-path /path/to/weights/

Launch a gradio web server

python -m llmzoo.deploy.webapp.gradio_web_server

Now, you can open your browser and chat with a model.

πŸ˜€ Training by yourself

Prepare the data

You can either download the phoenix-sft-data-v1 data or prepare your own data. Put your data on the path data/data.json.

Training

For Phoenix, run

bash scripts/train_phoenix_7b.sh

For Chimera, prepare the LLaMA weights following this instruction and run

bash scripts/train_chimera_7b.sh
bash scripts/train_chimera_13b.sh

πŸ€– Limitations

Our goal in releasing our models is to assist our community in better replicating ChatGPT/GPT4. We are not targeting competition with other competitors, as benchmarking models is a challenging task. Our models face similar models to those of ChatGPT/GPT4, which include:

πŸ™Œ Contributors

LLM Zoo is mainly contributed by:

As an open-source project, we are open to contributions. Feel free to contribute if you have any ideas or find any issue.

Acknowledgement

We are aware that our works are inspired by the following works, including but not limited to

Without these, nothing could happen in this repository.

Citation

@article{phoenix-2023,
  title={Phoenix: Democratizing ChatGPT across Languages},
  author={Zhihong Chen and Feng Jiang and Junying Chen and Tiannan Wang and Fei Yu and Guiming Chen and Hongbo Zhang and Juhao Liang and Chen Zhang and Zhiyi Zhang and Jianquan Li and Xiang Wan and Benyou Wang and Haizhou Li},
  journal={arXiv preprint arXiv:2304.10453},
  year={2023}
}
@misc{llm-zoo-2023,
  title={LLM Zoo: democratizing ChatGPT},
  author={Zhihong Chen and Junying Chen and Hongbo Zhang and Feng Jiang and Guiming Chen and Fei Yu and Tiannan Wang and Juhao Liang and Chen Zhang and Zhiyi Zhang and Jianquan Li and Xiang Wan and Haizhou Li and Benyou Wang},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/FreedomIntelligence/LLMZoo}},
}

We are from the School of Data Science, the Chinese University of Hong Kong, Shenzhen (CUHKSZ) and the Shenzhen Rsearch Institute of Big Data (SRIBD).

Star History

Star History Chart