Home

Awesome

<p align='center'> <picture> <source width='300px' alt='White Refact Logo' media="(prefers-color-scheme: dark)" srcset="white-refact-logo.svg"> <img width='300px' alt="Black Refact Logo" src="refact-logo.svg"> </picture> </p>

This repo consists Refact WebUI for fine-tuning and self-hosting of code models, that you can later use inside Refact plugins for code completion and chat.

Discord Twitter Follow License Visual Studio JetBrains

self-hosting-refact

Running Refact Self-Hosted in a Docker Container

The easiest way to run the self-hosted server is a pre-build Docker image.

Install Docker with NVidia GPU support. On Windows you need to install WSL 2 first, one guide to do this.

Run docker container with following command:

docker run -d --rm --gpus all --shm-size=256m -p 8008:8008 -v refact-perm-storage:/perm_storage smallcloud/refact_self_hosting:latest

perm-storage is a volume that is mounted inside the container. All the configuration files, downloaded weights and logs are stored here.

To upgrade the docker, delete it using docker kill XXX (the volume perm-storage will retain your data), run docker pull smallcloud/refact_self_hosting and run it again.

Now you can visit http://127.0.0.1:8008 to see the server Web GUI.

<details><summary>Docker commands super short refresher</summary> Add your yourself to docker group to run docker without sudo (works for Linux): 
sudo usermod -aG docker {your user}

List all containers:

docker ps -a

Start and stop existing containers (stop doesn't remove them):

docker start XXX
docker stop XXX

Shows messages from a container:

docker logs -f XXX

Remove a container and all its data (except data inside a volume):

docker rm XXX

Check out or delete a docker volume:

docker volume inspect VVV
docker volume rm VVV
</details>

See CONTRIBUTING.md for installation without a docker container.

Setting Up Plugins

Download Refact for VS Code or JetBrains.

Go to plugin settings and set up a custom inference URL http://127.0.0.1:8008

<details><summary>JetBrains</summary> Settings > Tools > Refact.ai > Advanced > Inference URL </details> <details><summary>VSCode</summary> Extensions > Refact.ai Assistant > Settings > Infurl </details>

Supported models

ModelCompletionChatFine-tuningDeprecated
Refact/1.6B++
starcoder2/3b/base++
starcoder2/7b/base++
starcoder2/15b/base++
deepseek-coder/1.3b/base++
deepseek-coder/5.7b/mqa-base++
magicoder/6.7b+
mistral/7b/instruct-v0.1+
mixtral/8x7b/instruct-v0.1+
deepseek-coder/6.7b/instruct+
deepseek-coder/33b/instruct+
stable/3b/code+
llama3/8b/instruct+

Usage

Refact is free to use for individuals and small teams under BSD-3-Clause license. If you wish to use Refact for Enterprise, please contact us.

Custom installation

You can also install refact repo without docker:

pip install .

If you have a GPU with CUDA capability >= 8.0, you can also install it with flash-attention v2 support:

FLASH_ATTENTION_FORCE_BUILD=TRUE MAX_JOBS=4 INSTALL_OPTIONAL=TRUE pip install .

FAQ

Q: Can I run a model on CPU?

A: it doesn't run on CPU yet, but it's certainly possible to implement this.

Community & Support