Awesome
LLMFarm_core.swift
LLMFarm_core swift library to work with large language models (LLM). It allows you to load different LLMs with certain parameters.<br> Based on ggml and llama.cpp by Georgi Gerganov.
Also used sources from:
- rwkv.cpp by saharNooby.
- Mia by byroneverson.
Features
- MacOS (13+)
- iOS (16+)
- Various inferences
- Various sampling methods
- Metal (dont work on intel Mac)
- Model setting templates
- LoRA adapters support (read more)
- LoRA train support
- LoRA export as model support
- Restore context state (now only chat history)
Inferences
- LLaMA <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px">
- GPTNeoX
- Replit
- GPT2 + Cerebras <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px">
- Starcoder(Santacoder) <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px">
- RWKV (20B tokenizer)
- Falcon <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px">
- MPT <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px">
- Bloom <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px">
- StableLM-3b-4e1t <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px">
- Qwen <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px">
- Yi models <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px">
- Deepseek models <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px">
- Mixtral MoE <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px">
- PLaMo-13B <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px">
Note: For Falcon, Alpaca, GPT4All, Chinese LLaMA / Alpaca and Chinese LLaMA-2 / Alpaca-2, Vigogne (French), Vicuna, Koala, OpenBuddy (Multilingual), Pygmalion/Metharme, WizardLM, Baichuan 1 & 2 + derivations, Aquila 1 & 2, Mistral AI v0.1, Refact, Persimmon 8B, MPT, Bloom select llama inferece in model settings.
Sampling methods
- Temperature (temp, tok-k, top-p)
- Tail Free Sampling (TFS)
- Locally Typical Sampling
- Mirostat
- Greedy
- Grammar (dont work for GPTNeoX, GPT-2, RWKV)
- Classifier-Free Guidance
Installation
git clone https://github.com/guinmoon/llmfarm_core.swift
Swift Package Manager
Add llmfarm_core to your project using Xcode (File > Add Packages...) or by adding it to your project's Package.swift file:
dependencies: [
.package(url: "https://github.com/guinmoon/llmfarm_core.swift")
]
Build and Debug
To Debug llmfarm_core package, do not forget to comment .unsafeFlags(["-Ofast"]) in Package.swift.
Don't forget that the debug version is slower than the release version.
To build with QKK_64 support uncomment .unsafeFlags(["-DGGML_QKK_64"]) in Package.swift.
Usage
See examples in the Demo Project
Projects based on this library
App to run LLaMA and other large language models locally on iOS and MacOS.