Home

Awesome

Llama Coder

Llama Coder is a better and self-hosted Github Copilot replacement for VS Code. Llama Coder uses Ollama and codellama to provide autocomplete that runs on your hardware. Works best with Mac M1/M2/M3 or with RTX 4090.

VS Code Plugin

Features

Recommended hardware

Minimum required RAM: 16GB is a minimum, more is better since even smallest model takes 5GB of RAM. The best way: dedicated machine with RTX 4090. Install Ollama on this machine and configure endpoint in extension settings to offload to this machine. Second best way: run on MacBook M1/M2/M3 with enough RAM (more == better, but 10gb extra would be enough). For windows notebooks: it runs good with decent GPU, but dedicated machine with a good GPU is recommended. Perfect if you have a dedicated gaming PC.

Local Installation

Install Ollama on local machine and then launch the extension in VSCode, everything should work as it is.

Remote Installation

Install Ollama on dedicated machine and configure endpoint to it in extension settings. Ollama usually uses port 11434 and binds to 127.0.0.1, to change it you should set OLLAMA_HOST to 0.0.0.0.

Models

Currently Llama Coder supports only Codellama. Model is quantized in different ways, but our tests shows that q4 is an optimal way to run network. When selecting model the bigger the model is, it performs better. Always pick the model with the biggest size and the biggest possible quantization for your machine. Default one is stable-code:3b-code-q4_0 and should work everywhere and outperforms most other models.

NameRAM/VRAMNotes
stable-code:3b-code-q4_03GB
codellama:7b-code-q4_K_M5GB
codellama:7b-code-q6_K6GBm
codellama:7b-code-fp1614GBg
codellama:13b-code-q4_K_M10GB
codellama:13b-code-q6_K14GBm
codellama:34b-code-q4_K_M24GB
codellama:34b-code-q6_K32GBm

Troubleshooting

Most of the problems could be seen in output of a plugin in VS Code extension output.

Changelog

[0.0.14]

[0.0.13]

[0.0.12]

[0.0.11]

[0.0.10]

[0.0.9]

[0.0.8]

[0.0.7]

[0.0.6]

[0.0.4]