Home

Awesome

WeMix-LLM

WeMix-LLM includes a series of LLMs and multimodal LLMs following the same paradigm. WeMix-LLM is built on LLaMA2-Accessory.

<img src="docs/opencompass_all.png" width="90%" />

Changelog

Setup

Please follow the Environment Setup of LLaMA2-Accessory.

Models

WeMix-LLaMA2: An Instruction-Following LLM

ModelWeMix-LLaMA2-70BLLaMA2-70BVicuna-33BWeMix-LLaMA2-7BLLaMA-2-7B-ChatVicuna-7BLLaMA-2-7B
OVERALL58.657.45049.644.843.441.6
EXAM62.357.349.245.540.140.535.5
LANGUAGE52.651.644.945.14439.644.1
KNOWLEDGE6967.761.359.454.351.753.3
UNDERSTANDING62.960.858.555.550.950.542.4
REASONING54.15544.747.441.439.940.1

Please refer to benchmark.md for more details.

WeMix-LLaMA2-13B-MM: A Multimodal LLM

wemix_weight=path/to/WeMix-LLaMA2-13B-MM

torchrun --nproc-per-node=2 demos/single_turn_mm.py \
--llama_config ${wemix_weight}/params.json --tokenizer_path ${wemix_weight}/tokenizer.model \
--pretrained_path ${wemix_weight}
ModelNoCapsFlickr30K
Flamingo-9B-61.5
Flamingo-80B-67.2
Unified-IO-XL100.0-
Kosmos-1-67.1
Kosmos-2-66.7
BLIP-2 (Vicuna-13B)103.971.6
InstructBLIP (Vicuna-13B)121.982.8
Shikra (Vicuna-13B)-73.9
Qwen-VL (Qwen-7B)121.485.8
Qwen-VL-Chat120.281.0
WeMix-LLaMA2-13B-MM114.786.0

The multimodal benchmark is still in progress. Stay tuned!🎉

<!-- ### Contributors -->

Acknowledgement

LLaMA2-Accessory, LLaMA-Adapter, LLaMA.

License

Llama 2 is licensed under the LLAMA 2 Community License, Copyright (c) Meta Platforms, Inc. All Rights Reserved.