Awesome

Rivet Ollama Plugin

The Rivet Ollama Plugin is a plugin for Rivet to allow you to use Ollama to run and chat with LLMs locally and easily. It adds the following nodes:

• Ollama Chat
• Ollama Embedding
• Get Ollama Model
• List Ollama Models
• Pull Model to Ollama

Table of Contents

• Running Ollama
• Using the plugin
  • In Rivet
  • In the SDK
• Configuration
  • In Rivet
  • In the SDK
• Nodes
  • Ollama Chat
    • Inputs
    • Outputs
    • Editor Settings
  • Ollama Embedding
    • Inputs
    • Outputs
    • Editor Settings
  • Ollama Generate
    • Inputs
    • Outputs
    • Editor Settings
  • List Ollama Models
    • Inputs
    • Outputs
    • Editor Settings
  • Get Ollama Model
    • Inputs
    • Outputs
    • Editor Settings
  • Pull Model to Ollama
    • Inputs
    • Outputs
    • Editor Settings
• Local Development

Running Ollama

To run Ollama so that Rivet's default browser executor can communicate with it, you will want to start it with the following command:

OLLAMA_ORIGINS=* ollama serve

If you are using the node executor, you can omit the OLLAMA_ORIGINS environment variable.

Using the plugin

In Rivet

To use this plugin in Rivet:

1. Open the plugins overlay at the top of the screen.
2. Search for "rivet-plugin-ollama"
3. Click the "Add" button to install the plugin into your current project.

In the SDK

1. Import the plugin and Rivet into your project:

   import * as Rivet from "@ironclad/rivet-node";
   import RivetPluginOllama from "rivet-plugin-ollama";
   

2. Initialize the plugin and register the nodes with the globalRivetNodeRegistry:

   Rivet.globalRivetNodeRegistry.registerPlugin(RivetPluginOllama(Rivet));
   

   (You may also use your own node registry if you wish, instead of the global one.)

3. The nodes will now work when ran with runGraphInFile or createProcessor.

Configuration

In Rivet

By default, the plugin will attempt to connect to Ollama at http://localhost:11434. If you would like you change this, you can open the Settings window, navigate to the Plugins area, and you will see a Host setting for Ollama. You can change this to the URL of your Ollama instance. For some users it works using http://127.0.0.1:11434 instead.

In the SDK

When using the SDK, you can pass a host option to the plugin to configure the host:

Using createProcessor or runGraphInFile, pass in via pluginSettings in RunGraphOptions:

await createProcessor(project, {
  ...etc,
  pluginSettings: {
    ollama: {
      host: "http://localhost:11434",
    },
  },
});

Nodes

Ollama Chat

The main node of the plugin. Functions similarly to the Chat Node built in to Rivet. Uses /api/chat route

Inputs

Outputs

Editor Settings

Ollama Embedding

Embedding models are models that are trained specifically to generate vector embeddings: long arrays of numbers that represent semantic meaning for a given sequence of text. The resulting vector embedding arrays can then be stored in a database, which will compare them as a way to search for data that is similar in meaning.

Inputs

See Editor Settings for all possible inputs.

Outputs

Editor Settings

Ollama Generate

Previously the main node of the plugin. Allows you to send prompts to Ollama and receive responses from the LLMs installed with deep customization options even including custom prompt formats. Uses /api/generate route

Inputs

Additional inputs available with toggles in the editor.

Outputs

Editor Settings

List Ollama Models

Lists the models installed in Ollama.

Inputs

This node has no inputs.

Outputs

Editor Settings

This node has no editor settings.

Get Ollama Model

Gets the model with the given name from Ollama.

Inputs

See Editor Settings for all possible inputs.

Outputs

Editor Settings

Pull Model to Ollama

Downloads a model from the Ollama library to the Ollama server.

Inputs

See Editor Settings for all possible inputs.

Outputs

Editor Settings

Local Development

1. Run yarn dev to start the compiler and bundler in watch mode. This will automatically recombine and rebundle your changes into the dist folder. This will also copy the bundled files into the plugin install directory.
2. After each change, you must restart Rivet to see the changes.