Awesome

RLLTE Copilot: LLM-Empowered Assistant for RL

Introduction

Copilot is the first attempt to integrate an LLM into an RL framework, which aims to help developers reduce the learning cost and facilitate application construction. We follow the design of LocalGPT that interacts privately with documents using the power of GPT. The source documents are first ingested by an instructor embedding tool to create a local vector database. After that, a local LLM is used to understand questions and create answers based on the database. In practice, we utilize Vicuna-7B as the base model and build the database using various corpora, including API documentation, tutorials, and RL references. The powerful understanding ability of the LLM model enables the copilot to accurately answer questions about the use of the framework and any other questions of RL. Moreover, no additional training is required, and users are free to replace the base model according to their computing power. In future work, we will further enrich the corpus and add the code completion function to build a more intelligent copilot for RL.

Usage

Online

We plan to deploy the Copilot on the Hugging Face platform. Currently, we're dealing with the problem of computing power. The online server is coming soon.

Offline

Firstly, clone the repository by:

git clone https://github.com/RLE-Foundation/rllte-copilot.git

Then install the necessary dependencies:

pip install -r requirements.txt

Finally, open a terminal and run the app.py:

python app.py

Launch the browser and you'll see the following page:

RLLTE Copilot: LLM-Empowered Assistant for RL

FAQ

• How to change the base LLM model?

Currently, our project utilizes the 'Llama-2-7b-Chat-GGUF' model due to its superior performance. However, we plan to expand the model selection options in the future. Thus, users will be able to select from a range of models to suit their specific needs.