Home

Awesome

RestGPT

This is the code for the paper RestGPT: Connecting Large Language Models with Real-World RESTful APIs.

This work aims to construct a large language model based autonomous agent, RestGPT, to control real-world applications, such as movie database and music player. To achieve this, we connect LLMs with RESTful APIs and tackle the practical challenges of planning, API calling, and response parsing. To fully evaluate the performance of RestGPT, we propose RestBench, a high-quality benchmark which consists of two real-world scenarios and human-annotated instructions with gold solution paths.

intro

What's New

RestGPT

RestGPT adopts an iterative coarse-to-fine online planning framework and uses an executor to call RESTful APIs. Here is an overview of RestGPT.

model

Modules:

Here is an example of using TMDB movie database to search for the number of movies directed by Sofia Coppola.

example

Data

We also introduce RestBench to evaluate the performance of RestGPT. RestBench is a high-quality test set consisting of TMDB movie database and Spotify music player scenarios. We collect realistic user instructions with human-annotated gold solution paths. Here are examples of RestBench:

TMDB example:

Spotify example:

Below is the statistics of the data. We report the number of instructions with different lengths of solution path:

Scenario#APIsLen-1Len-2Len-3Len-4Avg. Len.Total
TMDB545662722.3100
Spotify408182292.657

Setup

pip install langchain colorama tiktoken spotipy openai

create logs folder

Get OpenAI key from OpenAI, TMDB key from https://developer.themoviedb.org/docs/getting-started, and Spotify key from https://developer.spotify.com/documentation/web-api

Fill in your own key in config.yaml

(Optional) Initialize the Spotify Environment

WARNING: this will remove all your data from spotify!

python init_spotify.py

Run

The code can be run using the following command:

python run.py

Then Input the scenario (TMDB/Spotify) and instruction.

We also provide two scripts to run RestGPT on RestBench:

# TMDB
python run_tmdb.py

# Spotify, please open Spotify on your device
python run_spotify.py

run_tmdb.py will sequentially execute all instructions of RestBench-TMDB. Regarding RestBench-Spotify, you should manually modify the query_idx before executing the instructions.

Citation

If you find this repo useful, please cite us.

@misc{song2023restgpt,
      title={RestGPT: Connecting Large Language Models with Real-World RESTful APIs}, 
      author={Yifan Song and Weimin Xiong and Dawei Zhu and Wenhao Wu and Han Qian and Mingbo Song and Hailiang Huang and Cheng Li and Ke Wang and Rong Yao and Ye Tian and Sujian Li},
      year={2023},
      eprint={2306.06624},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}