Awesome
llm.ts
Call 30+ LLMs with a single API.
- Send multiple prompts to multiple LLMs and get the results back in a single response.
- Zero dependencies (under 10kB minified)
- Bring your own API keys
- Works anywhere (Node, Deno, browser)
Install
npm
npm install llm.ts
yarn
yarn add llm.ts
Example
import { LLM, MODEL } from 'llm.ts';
(async function () {
await new LLM({
apiKeys: {
openAI: process.env.OPENAI_API_KEY ?? '',
cohere: process.env.COHERE_API_KEY ?? '',
huggingface: process.env.HF_API_TOKEN ?? '',
}
}).completion({
prompt: [
'Repeat the following sentence: "I am a robot."',
'Repeat the following sentence: "I am a human."',
],
model: [
// use the model name
'text-ada-001',
// or specify a specific provider
'cohere/command-nightly',
// or use enums to avoid typos
MODEL.HF_GPT2,
],
}).then(resp => {
console.log(resp);
})
})()
{
"created": 1683079463217,
"choices": [
{
"text": "\n\nI am a robot.",
"index": 0,
"model": "text-ada-001",
"promptIndex": 0,
"created": 1683079462
},
{
"text": "\n\nI am a human.",
"index": 1,
"model": "text-ada-001",
"promptIndex": 1,
"created": 1683079462
},
{
"text": "\nI am a robot.",
"index": 2,
"model": "command-nightly",
"promptIndex": 0,
"created": 1683079463217
},
{
"text": "\nI am a human.",
"index": 3,
"model": "command-nightly",
"promptIndex": 1,
"created": 1683079463216
},
{
"text": " \"Is that your question? I was expecting the answer.\" \"Then why do you think you are being asked!\" 1. \"What are you?\" \"What are you?\" \"Why are you",
"index": 4,
"model": "gpt2",
"promptIndex": 0,
"created": 1683079463088
},
{
"text": " — this quote is most often cited in reference to the Qur'an. (e.g. Ibn `Allaahu `udayyyih, Al-Rai`an, Al",
"index": 5,
"model": "gpt2",
"promptIndex": 1,
"created": 1683079463091
}
]
}
Models supported (want to add one, open a PR!):
- text-ada-001
- text-babbage-001
- text-curie-001
- text-davinci-002
- text-davinci-003
- cohere-command
- cohere-command-nightly
- cohere-command-light
- cohere-command-light-nightly
- gpt2
- bloom-1b
- bloom-3b
- bloom-7b1
- llama-7b
- llama-13b
- llama-30b
- llama-65b
- gptj-6b
- gptj-2.7b
- gpt-neo-125m
- gpt-neo-1.3b
- gpt-neo-20b
- cerebras-gpt-111m
- cerebras-gpt-1.3b
- cerebras-gpt-2.7b
- santacoder
- codegen-350m
- codegen-2b
- stablelm-tuned-3b
- stablelm-tuned-7b
- pythia-70m
- pythia-160m
- pythia-12b
- distilgpt2
Hosting providers supported (want to add one, open a PR!):
- OpenAI
- Cohere
- HuggingFace
License
MIT