Home

Awesome

AI Chat: Your (Witty) AI Companions

<img alt="hero.png" style="width:100%; max-width: 512px" src="docs/assets/hero.png"/>

This application is your bridge between popular chat platforms like Discord and Slack and the powerful minds of LLMs like OpenAI and Google Gemini. It empowers you to build AI bots with unique personalities that can seamlessly engage in conversations with users.

Features

Getting Started

Prerequisites

Run Locally

  1. Create Your Encore App:
encore app create my-ai-chat --example=https://github.com/encoredev/ai-chat

(Feel free to replace my-ai-chat with a name that tickles your fancy)

  1. Set Your OpenAI API Key (Optional):
cd my-ai-chat
encore secret set OpenAIKey --type dev,local,pr

Paste your OpenAI API key when prompted.

  1. Run Your App:
encore run

Encore will build and start your application, providing you with a local URL (e.g., http://127.0.0.1:4000). Open this URL in your browser to see your creation - the local chat interface!

<img alt="Local chat interface demonstration" style="width:100%; max-width: 512px" src="docs/assets/bots.gif"/>

Deploy to the Cloud

Ready to share your bots with the world? Encore makes deploying to a free testing environment a breeze:

git push

Encore will guide you through the deployment process and provide a link to your live application.

<img alt="Encore deployment overview" style="width:100%; max-width: 512px" src="docs/assets/deploy.png"/>

Once the deployment is complete, click Overview and copy the URL to see your bots in action on the web!

<img alt="Encore deployment overview" style="width:100%; max-width: 512px" src="docs/assets/overview.png"/>

Under the Hood

AI Chat is a microservices-based application, with each service handling a specific aspect of the chatbot ecosystem. The services uses a combination of Encore APIs, pub/sub messaging, and WebSocket communication to orchestrate the flow of messages between chat platforms and LLM providers. You can explore the services and their interactions in the Local Dashboard.

<img alt="System design diagram" style="width:100%; max-width: 768px" src="docs/assets/system-design.png"/>

Key Components

Main Flow

  1. A user sends a message in a connected chat channel
  2. The corresponding chat integration (Discord, Slack, or local) receives the message
  3. The integration publishes a message to the chat service
  4. The chat service identifies any bots in the channel and fetches their profiles and the channel's message history
  5. The chat service sends the message to the LLM service
  6. The LLM service crafts a prompt including the bot's persona and the ongoing conversation
  7. The prompt is sent to the chosen LLM provider (OpenAI or Gemini)
  8. The LLM provider streams responses through pubsub back to the LLM service
  9. The LLM service parses the responses and relays them back to the chat service
  10. The chat service delivers the bot's witty (or not-so-witty) responses to the appropriate chat integration

Integrating Your LLMs

LLMs are the heart and soul of your bots, providing the intelligence and personality that make them shine. This application is built to make it easy to integrate with popular LLM providers, and it comes pre-configured to work with OpenAI and Google Gemini. The only thing you need to do is set your credentials as Encore secrets, and you're ready to start generating bots with your chosen LLM provider.

Adding OpenAI Credentials

To enable OpenAI as an LLM provider, you'll need to set your OpenAI API key as an Encore secret. Here's how you can do it:

  1. Get Your OpenAI API Key:
  1. Add Your API Key as an Encore Secret:
encore secret set OpenAIKey --type dev,local,pr
  1. Generate Bots: All done! You can now generate bots with OpenAI as the LLM, just call the bot.Create endpoint with openai as the provider.

Adding Gemini Credentials

To enable Gemini as an LLM provider, you'll need to set your Google Cloud credentials as an Encore secret. Here's how you can do it: To enable Gemini as an LLM provider, you'll need to set your Google Cloud credentials as an Encore secret. Here's how you can do it:

  1. Create a GCP Service Account:
  1. Create a JSON Key:
  1. Add JSON Key as an Encore Secret:
encore secret set --type dev,local,pr GeminiJSONCredentials < <downloaded json>.json
  1. Generate Bots: All done! You can now generate bots with Gemini as the LLM, just call the bot.Create endpoint with gemini as the provider.

Integrating your Chat Platforms

The application is designed to make it easy to integrate with any chat platform, but it comes pre-configured to work with Discord and Slack. It also includes a local chat service with an easy-to-use web interface which is hosted on the API server. The local chat service is enabled by default, but Slack and Discord require additional setup to connect your bots to these platforms.

Using the Local Chat

Configuring Slack

To be able to use Slack as a chat platform, you'll need to create a Slack app and add it to your workspace. Here's how you can do it:

  1. Create the Encore App:
  1. Create a Slack App:
  1. Activate Bot Events:
  1. Install the App to Your Workspace:
  1. Add the Slack Bot Token:
encore secret set SlackToken --type local
  1. Create Your Chat Bots

Configuring Discord

To be able to use Discord as a chat platform, you'll need to create a Discord bot and add it to your server. Here's how you can do it:

  1. Create the Encore App:
  1. Create a Discord Bot:
  1. Configure Install Settings:
  1. Grant Privileged Gateway Intents:
  1. Copy the Bot Token:
encore secret set DiscordToken --type local
  1. Install the Bot:
  1. Invite the Bot to a Channel (Optional):
  1. Create Your Chat Bots

Create Your Chat Bots

The Slack and Discord integrations does not come with a custom-made UI for adding bots to channels. Until you've built your own UI (or maybe addded support for slash commands?), you can use the Encore Dashboards to add bots to channels:

  1. Open the Service Catalog
  1. Create a Bot Profile:
  1. Find a Chat Channel ID:
  1. Add the Bot to a Channel:
  1. Verify Your Bot:
<img alt="slack-message.gif" style="width:100%; max-width: 386px" src="docs/assets/slack-message.gif"/> <img alt="discord-message.gif" style="width:100%; max-width: 386px" src="docs/assets/discord-message.gif"/>

You've successfully integrated your chat platforms and LLMs, ready to unleash your AI-powered conversationalists! Now go forth and create bots that will make people laugh, think, or maybe even question the nature of reality (but no pressure!).

Development

Configuring Ngrok

If you're developing locally and want to connect to external services like Slack, you'll need to use a service like ngrok to create a secure tunnel to your local development server. This app is pre-configured to work with ngrok, so you can easily test your bots on Slack and Discord without deploying them to the cloud.

  1. Create a Ngrok Account
  1. Add Tunnel Authtoken
encore secret set NGrokToken --type local
  1. Create a Custom Ngrok Domain
encore secret set NGrokDomain --type local
  1. Update Your Apps (If Needed)

Updating the Local Chat Web Interface

The local chat service uses a web interface to interact with the chat service. If you want to make changes to the interface, you'll need to rebuild it before you deploy your changes.

  1. Install npm*
brew install npm
  1. Install Dependencies
npm install
  1. Build the Interface
npm run build
  1. Start the Interface
encore run
  1. Start the Interface in Dev Mode
npm run dev