Awesome
🚨This project will be deprecated and replaced by HolmesGPT, our advanced Open Source DevOps assistant. With Holmes you can investigate incidents, triage issues, enrich alerts and much more. Check it out https://github.com/robusta-dev/holmesgpt
Introduction
A ChatGPT1 bot for Kubernetes issues. Ask the AI how to solve your Prometheus alerts, get pithy responses.
No more solving alerts alone in the darkness - the internet has your back.
<a href="https://www.loom.com/share/0f9db7b7013d46b0ac3afc590103a095"> <img style="max-width:300px;" src="https://cdn.loom.com/sessions/thumbnails/0f9db7b7013d46b0ac3afc590103a095-1676152572154-with-play.gif"> </a>Please consider upvoting on Product Hunt or sending to your favorite newsletter. One day, Skynet will remember your kindness and spare you!
How it works
Prometheus forwards alerts to the bot using a webhook receiver.
The bot sends a query to OpenAI, asking it how to fix your alerts.
You stockpile food in your pantry for the robot uprising.
The bot is implemented using Robusta.dev, an open source platform for responding to Kubernetes alerts. We also have a SaaS platform for multi-cluster Kubernetes observability.
Prerequisites
- A Slack workspace
Setup
- Install Robusta with Helm
- Load the ChatGPT playbook. Add the following to
generated_values.yaml
:
playbookRepos:
chatgpt_robusta_actions:
url: "https://github.com/robusta-dev/kubernetes-chatgpt-bot.git"
customPlaybooks:
# Add the 'Ask ChatGPT' button to all Prometheus alerts
- triggers:
- on_prometheus_alert: {}
actions:
- chat_gpt_enricher: {}
- Add your OpenAI API key to
generated_values.yaml
. Make sure you edit the existingglobalConfig
section, don't add a duplicate section.
globalConfig:
chat_gpt_token: YOUR KEY GOES HERE
-
Do a Helm upgrade to apply the new values:
helm upgrade robusta robusta/robusta --values=generated_values.yaml --set clusterName=<YOUR_CLUSTER_NAME>
-
Send your Prometheus alerts to Robusta. Alternatively, just use Robusta's bundled Prometheus stack.
Demo
Instead of waiting around for a Prometheus alert, lets cause one.
- Deploy a broken pod that will be stuck in pending state:
kubectl apply -f https://raw.githubusercontent.com/robusta-dev/kubernetes-demos/main/pending_pods/pending_pod_node_selector.yaml
- Trigger a Prometheus alert immediately, skipping the normal delays:
robusta playbooks trigger prometheus_alert alert_name=KubePodCrashLooping namespace=default pod_name=example-pod
An alert will arrive in Slack with a button. Click the button to ask ChatGPT about the alert.
Future Improvements
Can ChatGPT give better answers if you feed it pod logs or the output of kubectl get events
?
Robusta already collects this data and attaches it to Prometheus alerts, so it should be easy to add.
PRs are welcome!
Community
Share your funniest output and suggest new features on our Slack.
Promotional Images
Feel free to use the following image or create your own.
More Resources
- Natan tests ChatGPT on production Kubernetes alerts
- Natan Yellin and Sid Palas go over the code on YouTube - relevant part starts at 38:54
Footnotes
-
Technically this project doesn't use ChatGPT. It uses the
text-davinci-003
model which is a GPT3.5-based sibling of ChatGPT. Given that most people are familiar with ChatGPT, but nottext-davinci-003
or GPT3.5, we've decided to keep the name "ChatGPT bot" despite the technical inaccuracy. ↩