Awesome

batched-chatgpt

Extremely easy-to-use ChatGPT batched API caller.
It only supports single turn conversation!

Installation

pip install batched-chatgpt

Quickstart

It requires OPENAI_API_KEY in your environment variable.

export OPENAI_API_KEY=<your_api_key>

Or in python code,

import os
os.environ['OPENAI_API_KEY'] = "<your_api_key>"

Simple version

from batched_chatgpt import call_chatgpt

resp = call_chatgpt(
    human_message=PROMPTS,  # list of str
)

Features

Autosaves the responses if pkl_path is specified. (auto-detect new filename and use pickle to save)
- Autosaves per single API call
- saves like ['blabla', 'blablabla', None, None, None, ...] and None is a placeholder not responded by ChatGPT.
- That means, no None will be returned if everything goes well.
Auto retry with customizable timeout.
Customizable chunk size, and auto multiprocessing.
Reserving the order of input list.

Need more customization?

from batched_chatgpt import call_chatgpt

resp = call_chatgpt(
    human_message=PROMPTS,  # list of str
    system_message=['You are a helpful assistant.'] * len(prompts),
    model_name=CHATGPT_VERSION_NAME,  # default is 'gpt-3.5-turbo'
    temperature=TEMPERATURE,  # default 0.0
    chunk_size=CONCURRENCY_NUM,
    timeout_each=TIMEOUT_EACH,
    sleep_between_chunk=SLEEP_BETWEEN_CHUNK,
    pkl_path=file_dir,  # ex) "result.pkl'
    verbose=True
)

human_message: list of human message
system_message: list of system prompt. It can be a str or a list of str that has same length to human_message
model_name: ChatGPT API name (ex: "gpt-4-1106-preview")
temperature: Controls randomness of the generated text
chunk_size: The number of examples which simultaneously send in one batch
timeout_each: API call timeout (for each batch)
sleep_between_chunk: sleep time between batches
pkl_path: Specifies the path where output will be saved. By default, outputs are not saved.
verbose: If true, debugging message will be printed.

Requirements

langchain
langchain-openai
openai