Awesome
LLMFarm
<p align="center"> <a href="https://apps.apple.com/ru/app/llm-farm/id6461209867?l=en-GB&platform=iphone"><img width="178px" alt="Icon" src="dist/LLMFarm0.1.2_256.png"></a> <a href="https://testflight.apple.com/join/6SpPLIVM"><img width="178px" alt="Icon" src="dist/testflight.png"></a> </p> <p align="center"> <a href="https://apps.apple.com/ru/app/llm-farm/id6461209867?l=en-GB&platform=iphone"><strong>Install Stable</strong></a> <a href="https://testflight.apple.com/join/6SpPLIVM"><strong>Install Latest</strong></a> </p> <br> <p align="center"> <a href="https://t.me/llmfarm_chat"><img alt="Icon" height="32" src="dist/telegram_logo_128.png"></a> <a href="https://www.youtube.com/@LLMFarm-lib"><img alt="Icon" height="32" src="dist/youtube_logo_128.png"></a> <a href="https://boosty.to/llmfarm"><img alt="Icon" height="32" src="dist/boosty_icon.png"></a> <a href="https://llmfarm.site"><img alt="Icon" height="32" src="dist/www_logo_128.png"></a> <a href="https://github.com/guinmoon/LLMFarm/wiki"><img alt="Wiki" height="32" src="dist/wiki_logo_128.png"></a> </p> <p align="center"> <img alt="Icon" height="440px" src="dist/screen1.png"> <img alt="Icon" width="540px" src="dist/screen2.png"> </p>LLMFarm is an iOS and MacOS app to work with large language models (LLM). It allows you to load different LLMs with certain parameters.With LLMFarm, you can test the performance of different LLMs on iOS and macOS and find the most suitable model for your project.<br> Based on ggml and llama.cpp by Georgi Gerganov.
Also used sources from:
- rwkv.cpp by saharNooby
- Mia by byroneverson
- LlamaChat by alexrozanski
Features
- MacOS (13+)
- iOS (16+)
- Various inferences
- Various sampling methods
- Metal (dont work on intel Mac)
- Model setting templates
- LoRA adapters support
- LoRA finetune support
- LoRA export as model support
- Restore context state
- Apple Shortcuts
Inferences
- <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px"> LLaMA 1,2,3
- <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px"> Gemma
- <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px"> Phi models
- <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px"> GPT2 + Cerebras
- <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px"> Starcoder(Santacoder)
- <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px"> Falcon
- <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px"> MPT
- <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px"> Bloom
- <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px"> StableLM-3b-4e1t
- <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px"> Qwen
- <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px"> Yi models
- <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px"> Deepseek models
- <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px"> Mixtral MoE
- <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px"> PLaMo-13B
- <img src="dist/metal-96x96_2x.png" width="16px" heigth="16px"> Mamba
- RWKV (20B tokenizer)
- GPTNeoX
- Replit
Multimodal
- LLaVA 1.5 models, LLaVA 1.6 models
- BakLLaVA
- Obsidian
- ShareGPT4V
- MobileVLM 1.7B/3B models
- Yi-VL
- Moondream
Note: For Falcon, Alpaca, GPT4All, Chinese LLaMA / Alpaca and Chinese LLaMA-2 / Alpaca-2, Vigogne (French), Vicuna, Koala, OpenBuddy (Multilingual), Pygmalion/Metharme, WizardLM, Baichuan 1 & 2 + derivations, Aquila 1 & 2, Mistral AI v0.1, Refact, Persimmon 8B, MPT, Bloom select llama inferece
in model settings.
Sampling methods
- Temperature (temp, tok-k, top-p)
- Tail Free Sampling (TFS)
- Locally Typical Sampling
- Mirostat
- Greedy
- Grammar (dont work for GPTNeoX, GPT-2, RWKV)
- Classifier-Free Guidance
Getting Started
You can find answers to some questions in the FAQ section.
Inference options
When creating a chat, a JSON file is generated in which you can specify additional inference options. The chat files are located in the "chats" directory. You can see all inference options here.
Models
You can download some of the supported models here.
Development
llmfarm_core
has been moved to a separate repository. To build llmfarm, you need to clone this repository recursively:
git clone --recurse-submodules https://github.com/guinmoon/LLMFarm