Home

Awesome

Guidance API: An Extension for oobabooga/text-generation-webui

Guidance API is a powerful extension for oobabooga/text-generation-webui that integrates the feature-rich and easy-to-use interface of OOGA with the robust capabilities of Guidance. By facilitating network calls for Guidance, this API brings out the full potential of modern language models in a streamlined and efficient manner.

Features

Note the "select" tag in guidance is currently WIP


Getting Started

Example of flags for config in webui.py

CMD_FLAGS = " --chat --model-menu  --model decapoda-research_llama-7b-hf --extensions guidance_api"

Then in guidance:

import guidance
import json,requests
import re,sys

guidance.llm = guidance.llms.TGWUI("http://127.0.0.1:9555")

character_maker = guidance("""The following is a character profile for an RPG game in JSON format.
```json
{
    "id": "{{id}}",
    "description": "{{description}}",
    "name": "{{gen 'name'}}",
    "class": "{{gen 'class'}}",

}```""")

# generate a character
res=character_maker(
    id="e1f491f7-7ab8-4dac-8c20-c92b5e7d883d",
    description="A quick and nimble fighter.",
)

print(res)

Feel free to submit feedback, this repository is under active development