Awesome
AI Chat: Your (Witty) AI Companions
<img alt="hero.png" style="width:100%; max-width: 512px" src="docs/assets/hero.png"/>This application is your bridge between popular chat platforms like Discord and Slack and the powerful minds of LLMs like OpenAI and Google Gemini. It empowers you to build AI bots with unique personalities that can seamlessly engage in conversations with users.
Features
- Multi-platform Versatility: Deploy your bots on Discord, Slack, or a local web-based chat for testing. Your digital puppets, your stage!
- Personality is Key: Craft engaging bots with distinct personas and conversational styles. Witty, wise, or just plain weird - the choice is yours!
- LLM Buffet: Pick and choose your LLM flavor - OpenAI and Google Gemini are ready to be your bots' brains.
- Streamlined Development with Encore: Built on Encore (https://encore.dev/), simplifying development and deployment so you can focus on crafting brilliant bot personalities.
Getting Started
Prerequisites
- Encore: You'll need Encore to get started. Follow the installation instructions at https://encore.dev/docs/install.
- OpenAI API Key: To use OpenAI's models, grab an API key from https://platform.openai.com/api-keys.
Run Locally
- Create Your Encore App:
encore app create my-ai-chat --example=https://github.com/encoredev/ai-chat
(Feel free to replace my-ai-chat
with a name that tickles your fancy)
- Set Your OpenAI API Key (Optional):
cd my-ai-chat
encore secret set OpenAIKey --type dev,local,pr
Paste your OpenAI API key when prompted.
- Run Your App:
encore run
Encore will build and start your application, providing you with a local URL (e.g., http://127.0.0.1:4000
). Open this URL in your browser to see your creation - the local chat interface!
Deploy to the Cloud
Ready to share your bots with the world? Encore makes deploying to a free testing environment a breeze:
git push
Encore will guide you through the deployment process and provide a link to your live application.
<img alt="Encore deployment overview" style="width:100%; max-width: 512px" src="docs/assets/deploy.png"/>Once the deployment is complete, click Overview and copy the URL to see your bots in action on the web!
<img alt="Encore deployment overview" style="width:100%; max-width: 512px" src="docs/assets/overview.png"/>Under the Hood
AI Chat is a microservices-based application, with each service handling a specific aspect of the chatbot ecosystem. The services uses a combination of Encore APIs, pub/sub messaging, and WebSocket communication to orchestrate the flow of messages between chat platforms and LLM providers. You can explore the services and their interactions in the Local Dashboard.
<img alt="System design diagram" style="width:100%; max-width: 768px" src="docs/assets/system-design.png"/>Key Components
- Chat Service: The orchestrator service, routing messages between chat platforms and LLM providers.
- Discord Service: Handles the integration with the Discord API.
- Slack Service: Manages the art of conversation with the Slack API.
- Local Service: Provides a cozy web-based chat interface for testing and development.
- Bot Service: Responsible for creating, storing, and managing bot profiles.
- LLM Service: Formats prompts for LLMs, processes responses, and gracefully handles multiple LLM providers.
- OpenAI Service: Interfaces with OpenAI's API for chat completions and image generation.
- Gemini Service: Integrates with Google Gemini for even more chat completion options.
Main Flow
- A user sends a message in a connected chat channel
- The corresponding chat integration (Discord, Slack, or local) receives the message
- The integration publishes a message to the chat service
- The chat service identifies any bots in the channel and fetches their profiles and the channel's message history
- The chat service sends the message to the LLM service
- The LLM service crafts a prompt including the bot's persona and the ongoing conversation
- The prompt is sent to the chosen LLM provider (OpenAI or Gemini)
- The LLM provider streams responses through pubsub back to the LLM service
- The LLM service parses the responses and relays them back to the chat service
- The chat service delivers the bot's witty (or not-so-witty) responses to the appropriate chat integration
Integrating Your LLMs
LLMs are the heart and soul of your bots, providing the intelligence and personality that make them shine. This application is built to make it easy to integrate with popular LLM providers, and it comes pre-configured to work with OpenAI and Google Gemini. The only thing you need to do is set your credentials as Encore secrets, and you're ready to start generating bots with your chosen LLM provider.
Adding OpenAI Credentials
To enable OpenAI as an LLM provider, you'll need to set your OpenAI API key as an Encore secret. Here's how you can do it:
- Get Your OpenAI API Key:
- Visit https://platform.openai.com/api-keys and copy your API key.
- Add Your API Key as an Encore Secret:
encore secret set OpenAIKey --type dev,local,pr
- Generate Bots:
All done! You can now generate bots with OpenAI as the LLM, just call the
bot.Create
endpoint withopenai
as the provider.
Adding Gemini Credentials
To enable Gemini as an LLM provider, you'll need to set your Google Cloud credentials as an Encore secret. Here's how you can do it: To enable Gemini as an LLM provider, you'll need to set your Google Cloud credentials as an Encore secret. Here's how you can do it:
- Create a GCP Service Account:
- Head over to the Google Cloud Console.
- Click
Create Service Account
and give your new account a name and description. - Grant your service account the
Vertex AI User
role. - Click
Done
.
- Create a JSON Key:
- Click on your newly created service account and then on
Keys
->Add Key
->Create New
. - Choose the
JSON
format and clickCreate
. - Download the JSON file.
- Add JSON Key as an Encore Secret:
encore secret set --type dev,local,pr GeminiJSONCredentials < <downloaded json>.json
- Generate Bots:
All done! You can now generate bots with Gemini as the LLM, just call the
bot.Create
endpoint withgemini
as the provider.
Integrating your Chat Platforms
The application is designed to make it easy to integrate with any chat platform, but it comes pre-configured to work with Discord and Slack. It also includes a local chat service with an easy-to-use web interface which is hosted on the API server. The local chat service is enabled by default, but Slack and Discord require additional setup to connect your bots to these platforms.
Using the Local Chat
- The local chat UI is by default served from the root of the API server (locally at
http://localhost:4000/
). - You can disable the local chat service by setting the
Enabled
property tofalse
in thechat/provider/local/config.cue
file.
Configuring Slack
To be able to use Slack as a chat platform, you'll need to create a Slack app and add it to your workspace. Here's how you can do it:
- Create the Encore App:
- Complete the steps in the Getting Started section to create your Encore app.
- Create a Slack App:
- Visit https://api.slack.com/apps and click
Create New App
. - Choose
From an app manifest
and clickNext
. - Pick the workspace for your bot and click
Next
. - Copy the bot manifest and paste it into the text box.
- Replace the
<bot-domain>
placeholder with either<env-name>-<app-name>-<slug>.encr.app
(e.g.staging-my-chatbot-tur3.encr.app
) or the ngrok domain you configured for local development - Click
Next
and thenCreate
.
- Activate Bot Events:
- On the bot settings page, click
Event Subscriptions
. - Start the Encore app.
- If the
Request URL
is yellow, click onRetry
.
- Install the App to Your Workspace:
- On the settings page, click
OAuth & Permissions
and thenInstall to Workspace
. - Select a channel for your bot and click
Allow
.
- Add the Slack Bot Token:
- Copy the
Bot User OAuth Token
from theOAuth & Permissions
page. - Add it as an Encore secret:
encore secret set SlackToken --type local
- Create Your Chat Bots
- Proceed to the Create Your Chat Bots section to add bots to your channels.
Configuring Discord
To be able to use Discord as a chat platform, you'll need to create a Discord bot and add it to your server. Here's how you can do it:
- Create the Encore App:
- Complete the steps in the Getting Started section to create your Encore app.
- Create a Discord Bot:
- Go to Developer Portal Applications and click
New Application
. - Give your Discord app a name and click
Create
.
- Configure Install Settings:
- Click
Installation
. - Select
Discord Provided Link
inInstall Link
. - Under
Default Install Settings
, add thebot
scope and these permissions: - Connect
- Manage Web Hooks
- Read Message History
- Read Messages/View Channels
- Send Messages
- Grant Privileged Gateway Intents:
- Click
Bot
and thenPrivileged Gateway Intents
. - Enable these intents:
- Server Members Intent
- Message Content Intent
- Copy the Bot Token:
- On the
Bot
page, clickReset Token
. - Copy the token and add it as an Encore secret:
encore secret set DiscordToken --type local
- Install the Bot:
- Copy the Install Link and paste it into your browser.
- Grant your bot access to a server.
- Invite the Bot to a Channel (Optional):
- If you want your bot to join private conversations, invite it to specific channels.
- Create Your Chat Bots
- Proceed to the Create Your Chat Bots section to add bots to your channels.
Create Your Chat Bots
The Slack and Discord integrations does not come with a custom-made UI for adding bots to channels. Until you've built your own UI (or maybe addded support for slash commands?), you can use the Encore Dashboards to add bots to channels:
- Open the Service Catalog
- Visit the Local Dashboard or the Cloud Dashboard.
- Click on your service
- Click on
Service Catalog
- Create a Bot Profile:
- Select the
bot.Create
endpoint. - Give your bot a name, an engaging prompt, and enter
openai
as the LLM. - Click
Call API
. - Copy the bot ID in the response.
- Find a Chat Channel ID:
- Select the
chat.ListChannels
. - Click
Call API
. - Copy the
id
of the channel you want your bot to join.
- Add the Bot to a Channel:
- Select the
chat.AddBotToChannel
endpoint. - Enter the bot ID and the channel ID.
- Click
Call API
.
- Verify Your Bot:
- Check your Slack/Discord channel; your bot should now be present and ready to chat!
You've successfully integrated your chat platforms and LLMs, ready to unleash your AI-powered conversationalists! Now go forth and create bots that will make people laugh, think, or maybe even question the nature of reality (but no pressure!).
Development
Configuring Ngrok
If you're developing locally and want to connect to external services like Slack, you'll need to use a service like ngrok to create a secure tunnel to your local development server. This app is pre-configured to work with ngrok, so you can easily test your bots on Slack and Discord without deploying them to the cloud.
- Create a Ngrok Account
- Visit https://ngrok.com/ and create an account.
- Add Tunnel Authtoken
- Log in to your ngrok account and click on
Authtokens
. - Click
Add Tunnel Authtoken
. - Copy the token and add it as an Encore secret:
encore secret set NGrokToken --type local
- Create a Custom Ngrok Domain
- Navigate to
Domains
->New Domain
in your ngrok dashboard. - Create a memorable domain name.
- Copy the domain name and add it as another Encore secret:
encore secret set NGrokDomain --type local
- Update Your Apps (If Needed)
- For some integrations (like Slack), you'll need to update the Slack app to use your ngrok domain.
Updating the Local Chat Web Interface
The local chat service uses a web interface to interact with the chat service. If you want to make changes to the interface, you'll need to rebuild it before you deploy your changes.
- Install npm*
- If you don't have npm installed, you can download it from https://www.npmjs.com/get-npm, or use your package manager, e.g.
brew install npm
- Install Dependencies
- Navigate to the
chat/provider/local/static
directory and run:
npm install
- Build the Interface
- Run the following command to build the web interface:
npm run build
- Start the Interface
- Start the local chat service and open the interface in your browser:
encore run
- Start the Interface in Dev Mode
- Alternatively, you can start the interface in development mode to get hot reloading and other goodies:
npm run dev
- You still need to start the local chat service with
encore run
.