Awesome
Grafana OnCall
<img width="400px" src="docs/img/logo.png">Developer-friendly incident response with brilliant Slack integration.
<!-- markdownlint-disable MD013 MD033 --> <table> <tbody> <tr> <td width="75%"><img src="docs/img/screenshot.png"></td> <td><div align="center"><a href="https://grafana.com/docs/oncall/latest/mobile-app/">Android & iOS</a>:<br><img src="docs/img/screenshot_mobile.png"></div></td> </tr> </tbody> </table> <!-- markdownlint-enable MD013 MD033 -->- Collect and analyze alerts from multiple monitoring systems
- On-call rotations based on schedules
- Automatic escalations
- Phone calls, SMS, Slack, Telegram notifications
Getting Started
[!IMPORTANT]
These instructions are for using Grafana 11 or newer. You must enable the feature toggle forexternalServiceAccounts
. This is already done for the docker files and helm charts. If you are running Grafana separately see the Grafana documentation on how to enable this.
We prepared multiple environments:
- production
- developer
- hobby (described in the following steps)
-
Download
docker-compose.yml
:curl -fsSL https://raw.githubusercontent.com/grafana/oncall/dev/docker-compose.yml -o docker-compose.yml
-
Set variables:
echo "DOMAIN=http://localhost:8080 # Remove 'with_grafana' below if you want to use existing grafana # Add 'with_prometheus' below to optionally enable a local prometheus for oncall metrics # e.g. COMPOSE_PROFILES=with_grafana,with_prometheus COMPOSE_PROFILES=with_grafana # to setup an auth token for prometheus exporter metrics: # PROMETHEUS_EXPORTER_SECRET=my_random_prometheus_secret # also, make sure to enable the /metrics endpoint: # FEATURE_PROMETHEUS_EXPORTER_ENABLED=True SECRET_KEY=my_random_secret_must_be_more_than_32_characters_long" > .env
-
(Optional) If you want to enable/setup the prometheus metrics exporter (besides the changes above), create a
prometheus.yml
file (replacingmy_random_prometheus_secret
accordingly), next to yourdocker-compose.yml
:echo "global: scrape_interval: 15s evaluation_interval: 15s scrape_configs: - job_name: prometheus metrics_path: /metrics/ authorization: credentials: my_random_prometheus_secret static_configs: - targets: [\"host.docker.internal:8080\"]" > prometheus.yml
NOTE: you will need to setup a Prometheus datasource using
http://prometheus:9090
as the URL in the Grafana UI. -
Launch services:
docker-compose pull && docker-compose up -d
-
Provision the plugin (If you run Grafana outside the included docker files install the plugin before these steps):
If you are using the included docker compose file use
admin
/admin
credentials andlocalhost:3000
to perform this task. If you have configured Grafana differently adjust your credentials and hostnames accordingly.# Note: onCallApiUrl 'engine' and grafanaUrl 'grafana' use the name from the docker compose file. If you are # running your grafana or oncall engine instance with another hostname adjust accordingly. curl -X POST 'http://admin:admin@localhost:3000/api/plugins/grafana-oncall-app/settings' -H "Content-Type: application/json" -d '{"enabled":true, "jsonData":{"stackId":5, "orgId":100, "onCallApiUrl":"http://engine:8080", "grafanaUrl":"http://grafana:3000"}}' curl -X POST 'http://admin:admin@localhost:3000/api/plugins/grafana-oncall-app/resources/plugin/install'
-
Start using OnCall, log in to Grafana with credentials as defined above:
admin
/admin
-
Enjoy! Check our OSS docs if you want to set up Slack, Telegram, Twilio or SMS/calls through Grafana Cloud.
Troubleshooting
Here are some API calls that can be made to help if you are having difficulty connecting Grafana and OnCall. (Modify parameters to match your credentials and environment)
# Use this to get more information about the connection between Grafana and OnCall
curl -X GET 'http://admin:admin@localhost:3000/api/plugins/grafana-oncall-app/resources/plugin/status'
# If you added a user or changed permissions and don't see it show up in OnCall you can manually trigger sync.
# Note: This is called automatically when the app is loaded (page load/refresh) but there is a 5 min timeout so
# that it does not generate unnecessary activity.
curl -X POST 'http://admin:admin@localhost:3000/api/plugins/grafana-oncall-app/resources/plugin/sync'
Update version
To update your Grafana OnCall hobby environment:
# Update Docker image
docker-compose pull engine
# Re-deploy
docker-compose up -d
After updating the engine, you'll also need to click the "Update" button on the plugin version page. See Grafana docs for more info on updating Grafana plugins.
Join community
<img width="200px" src="docs/img/slack.png"> <img width="200px" src="docs/img/GH_discussions.png">
Have a question, comment or feedback? Don't be afraid to open an issue!
Stargazers over time
Further Reading
- Automated migration from other on-call tools - Migrator
- Documentation - Grafana OnCall
- Overview Webinar - YouTube
- How To Add Integration - How to Add Integration
- Blog Post - Announcing Grafana OnCall, the easiest way to do on-call management
- Presentation - Deep dive into the Grafana, Prometheus, and Alertmanager stack for alerting and on-call management