Awesome

👋🤗🤗👋 Join our WeChat.

Easy and Efficient Fine-tuning LLMs --- 简单高效的大语言模型训练/部署

中文 | English

Introduction

LLamaTuner is an efficient, flexible and full-featured toolkit for fine-tuning LLM (Llama3, Phi3, Qwen, Mistral, ...)

Efficient

• Support LLM, VLM pre-training / fine-tuning on almost all GPUs. LLamaTuner is capable of fine-tuning 7B LLM on a single 8GB GPU, as well as multi-node fine-tuning of models exceeding 70B.
• Automatically dispatch high-performance operators such as FlashAttention and Triton kernels to increase training throughput.
• Compatible with DeepSpeed 🚀, easily utilizing a variety of ZeRO optimization techniques.

Flexible

• Support various LLMs (Llama 3, Mixtral, Llama 2, ChatGLM, Qwen, Baichuan, ...).
• Support VLM (LLaVA).
• Well-designed data pipeline, accommodating datasets in any format, including but not limited to open-source and custom formats.
• Support various training algorithms (QLoRA, LoRA, full-parameter fune-tune), allowing users to choose the most suitable solution for their requirements.

Full-featured

• Support continuous pre-training, instruction fine-tuning, and agent fine-tuning.
• Support chatting with large models with pre-defined templates.

Table of Contents

• Easy and Efficient Fine-tuning LLMs --- 简单高效的大语言模型训练/部署
  • Introduction
  • Table of Contents
  • Supported Models
  • Supported Training Approaches
  • Supported Datasets
    • Data Preprocessing
  • Model Zoo
  • Requirement
    • Hardware Requirement
  • Getting Started
    • Clone the code
  • Getting Started
    • QLora int4 Finetune
  • Known Issues and Limitations
  • License
  • Acknowledgements
  • Some lmm fine-tuning repos
  • Citation

Supported Models

Supported Training Approaches

Supported Datasets

As of now, we support the following datasets, most of which are all available in the Hugging Face datasets library.

• Stanford Alpaca
• Stanford Alpaca (Chinese)
• Hello-SimpleAI/HC3
• BELLE 2M (zh)
• BELLE 1M (zh)
• BELLE 0.5M (zh)
• BELLE Dialogue 0.4M (zh)
• BELLE School Math 0.25M (zh)
• BELLE Multiturn Chat 0.8M (zh)
• databricks-dolly-15k
• mosaicml/dolly_hhrlhf
• GPT-4 Generated Data
• Alpaca CoT
• UltraChat
• OpenAssistant/oasst1
• ShareGPT_Vicuna_unfiltered
• BIAI/OL-CC
• timdettmers/openassistant-guanaco
• Evol-Instruct
• OpenOrca
• Platypus
• OpenHermes

• DPO mixed (en&zh)
• Orca DPO Pairs (en)
• HH-RLHF (en)
• Open Assistant(en&zh)
• Nectar (en)
• Orca DPO (de)
• KTO mixed (en)

Please refer to data/README.md to learn how to use these datasets. If you want to explore more datasets, please refer to the awesome-instruction-datasets. Some datasets require confirmation before using them, so we recommend logging in with your Hugging Face account using these commands.

pip install --upgrade huggingface_hub
huggingface-cli login

Data Preprocessing

We provide a number of data preprocessing tools in the data folder. These tools are intended to be a starting point for further research and development.

• data_utils.py : Data preprocessing and formatting
• sft_dataset.py : Supervised fine-tuning dataset class and collator
• conv_dataset.py : Conversation dataset class and collator

Model Zoo

We provide a number of models in the Hugging Face model hub. These models are trained with QLoRA and can be used for inference and finetuning. We provide the following models:

Requirement

Hardware Requirement

* estimated

Getting Started

Clone the code

Clone this repository and navigate to the Efficient-Tuning-LLMs folder

git clone https://github.com/jianzhnie/LLamaTuner.git
cd LLamaTuner

Getting Started

QLora int4 Finetune

The train_qlora.py code is a starting point for finetuning and inference on various datasets. Basic command for finetuning a baseline model on the Alpaca dataset:

python train_qlora.py --model_name_or_path <path_or_name>

For models larger than 13B, we recommend adjusting the learning rate:

python train_qlora.py –learning_rate 0.0001 --model_name_or_path <path_or_name>

To find more scripts for finetuning and inference, please refer to the scripts folder.

Known Issues and Limitations

Here a list of known issues and bugs. If your issue is not reported here, please open a new issue and describe the problem.

1. 4-bit inference is slow. Currently, our 4-bit inference implementation is not yet integrated with the 4-bit matrix multiplication
2. Resuming a LoRA training run with the Trainer currently runs on an error
3. Currently, using bnb_4bit_compute_type='fp16' can lead to instabilities. For 7B LLaMA, only 80% of finetuning runs complete without error. We have solutions, but they are not integrated yet into bitsandbytes.
4. Make sure that tokenizer.bos_token_id = 1 to avoid generation issues.

License

LLamaTuner is released under the Apache 2.0 license.

Acknowledgements

We thank the Huggingface team, in particular Younes Belkada, for their support integrating QLoRA with PEFT and transformers libraries.

We appreciate the work by many open-source contributors, especially:

• LLaMa
• Vicuna
• xTuring
• Alpaca-LoRA
• Stanford Alpaca
• LLaMA-Factory
• Hugging Face
• Peft
• axolotl
• deepspeed
• Unsloth
• qlora
• bitsandbytes

Some lmm fine-tuning repos

• https://github.com/QwenLM
• https://github.com/InternLM
• https://github.com/ymcui/Chinese-LLaMA-Alpaca-3
• https://github.com/ymcui/Chinese-Mixtral
• https://github.com/SmartFlowAI/EmoLLM
• https://github.com/yangjianxin1/Firefly
• https://github.com/LiuHC0428/LAW-GPT

Citation

Please cite the repo if you use the data or code in this repo.

@misc{Chinese-Guanaco,
  author = {jianzhnie},
  title = {LLamaTuner: Easy and Efficient Fine-tuning LLMs},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/jianzhnie/LLamaTuner}},
}