Awesome

koi 🎣

koi is an open source plug-in for Krita that allows you to use AI to accelerate your art workflow!

Disclaimer ✋

In the interest of getting the open source community on board--I have released this plug-in early. In its current state you may run into issues (particularly during the setup process). If you do, I encourage you to open an issue here on GitHub and describe your problems so that it can be fixed it for you and others!

Overview 😄

The goal of this repository is to serve as a starting point for building increasingly useful tools for artists of all levels of experience to use.

Link to original twitter thread

This plug-in serves as a working example of how new A.I. models like Stable Diffusion can lower the barrier of entry to art so that anyone can enjoy making their dreams a reality!

Because this is an open source project I encourage you to try it out, break things, and come back with suggestions!

Getting Started 🏁

If you are new to git, or get stuck during the installation process, Lewington made a nice step-by-step video.

The easiest way to get started is to follow the plug-in installation process for krita. Then use the google colab backend server (button at the top of this readme)! This should give you a good introduction to the setup process and get you up and running fast!

Installation 🔨

Krita has a few plug-in installation methods, however, I will refer you to the one I use.

Step 1: Find your operating system's pykrita folder reference
Step 2: Clone the repository, and copy the koi folder and koi.desktop to pykrita. (restart krita now if it is open)
Step 3: Open Krita and navigate to the python plug-in menu reference
Step 4: Enable the koi plugin and restart Krita to load the plug-in.

The next thing you will need to do is setup the backend server that do all the computation!

Step 0: Ensure you have a GPU-accelerated installation of pytorch. (You can skip this step if you are using Colab or already have it)
- Follow the installation instructions on pytorch's official getting started.
Step 1: Get the latest version of HuggingFace's diffusers from source by going to the GitHub repo
- From here you can clone the repo git clone https://github.com/huggingface/diffusers.git & cd diffusers to move into the directory.
- Install the package with pip install -e .
Step 2: Install this package! I recommend moving out of the diffusers folder if you haven't already (eg. cd ..)
- git clone https://github.com/nousr/koi.git, then cd koi and pip install -e .

Note :raising_hand:

Before continuing, make sure you accept the terms of service for the diffusers model link to do so here.

Next, inside your terminal run the huggingface-cli login command and paste a token generated from here. If you don't want to repeat this step in the future you can then run git config --global credential.helper store. (note: only do this on a computer you trust)

Step 3: Run the server by typing python server.py
- If you did everything correctly you should see an address spit out after some time (eg. 127.0.0.1:8888)
Step 4: Open Krita, if you haven't already, and paste your address into the endpoint field of the plugin
- You will also need to append the actual API endpoint you are using. By default this is /api/img2img
- If you are using all of the default settings your endpoint field will look something like this http://127.0.0.1:8888/api/img2img

Inference 🖌️

This part is easy!

Step 1: Create a new canvas that is 512 x 512 (px) in size and make a single-layer sketch (note: these are temporary restrictions).
Step 2: Fill out the prompt field in the koi panel (default location is somewhere on the right of your screen).
Step 3: Make any additional changes you would like to the inference parameters (strength, steps, etc.)
Step 4: Copy and paste your server's endpoint to the associated field
Step 5: Click dream!

FAQ ❔

What does koi stand for?
- Krita Open(source) Img2Img: While support for StableDiffusion is first, the goal is to have this plug-in be compatible with any model!
Why the client/server setup?
- The goal is to make this as widely available as possible. The server can be run anywhere with a GPU (i.e. colab) and allow those with low-powered hardware to still use the plug-in!
I'm getting an error with "set-size"
- This usually happens when you either forget "/api/img2img" at the end of your endpoint when copying it into the plugin OR you have some issue with your backend server (check the output on your server for more information).