Awesome
Brigade Metrics: Monitoring for Brigade 2
<img width="100" align="left" src="logo.png">Brigade Metrics adds monitoring capabilities to a Brigade 2 installation. It utilizes Brigade APIs to export time series metrics to Prometheus and makes visualizations of those metrics available through a Grafana dashboard.
<br clear="left"/>Installation
Prerequisites:
-
A Kubernetes cluster:
- For which you have the
admin
cluster role - That is already running Brigade 2
- Capable of provisioning a public IP address for a service of type
LoadBalancer
. (This means you won't have much luck running the gateway locally in the likes of kind or minikube unless you're able and willing to mess with port forwarding settings on your router, which we won't be covering here.)
- For which you have the
-
kubectl
,helm
(commands below require Helm 3.7.0+), andbrig
(the Brigade 2 CLI)
1. Create a Service Account
Note: To proceed beyond this point, you'll need to be logged into Brigade 2
as the "root" user (not recommended) or (preferably) as a user with the ADMIN
role. Further discussion of this is beyond the scope of this documentation.
Please refer to Brigade's own documentation.
Using Brigade 2's brig
CLI, create a service account:
$ brig service-account create \
--id brigade-metrics \
--description brigade-metrics
Make note of the token returned. This value will be used in another step. It is your only opportunity to access this value, as Brigade does not save it.
Authorize this service account with read-only access to Brigade:
$ brig role grant READER \
--service-account brigade-metrics
2. Installing Brigade Metrics
For now, we're using the GitHub Container Registry (which is an OCI registry) to host our Helm chart. Helm 3.7 has experimental support for OCI registries. In the event that the Helm 3.7 dependency proves troublesome for users, or in the event that this experimental feature goes away, or isn't working like we'd hope, we will revisit this choice before going GA.
First, be sure you are using Helm 3.7.0 or greater and enable experimental OCI support:
$ export HELM_EXPERIMENTAL_OCI=1
Use the following command to extract the full set of configuration options from
the chart. Here we're storing a copy at ~/brigade-metrics-values.yaml
:
$ helm inspect values oci://ghcr.io/brigadecore/brigade-metrics \
--version v0.4.1 > ~/brigade-metrics-values.yaml
Edit the configuration (~/brigade-metrics-values.yaml
in this example). At
minimum, you will need to make the following changes:
-
Set the value of
exporter.brigade.apiAddress
to the address of your Brigade 2 API server. This should utilize the internal DNS hostname by which that API server is reachable within your Kubernetes cluster. This value is defaulted tohttps://brigade-apiserver.brigade.svc.cluster.local
, but may need to be updated if you installed Brigade 2 in a different namespace. -
Set the value of
exporter.brigade.apiToken
to the service account token that was generated earlier. -
grafana.host
: Set this to the host name where you'd like the dashboard (Grafana) to be accessible. -
Specify a username and password for the metrics dashboard by setting values for
grafana.auth.username
andgrafana.auth.password
. -
grafana.service.type
: If you plan to enable ingress (advanced), you can leave this as its default --ClusterIP
. If you do not plan to enable ingress, you probably will want to change this value toLoadBalancer
.
Install Brigade Metrics, referencing your edited configuration:
$ helm install brigade-metrics \
oci://ghcr.io/brigadecore/brigade-metrics \
--version v0.4.1 \
--create-namespace \
--namespace brigade-metrics \
--values ~/brigade-metrics-values.yaml \
--wait \
--timeout 300s
3. (RECOMMENDED) Create a DNS Entry
If you overrode defaults and set grafana.service.type
to LoadBalancer
, use
this command to find the gateway's public IP address:
$ kubectl get svc brigade-metrics-grafana \
--namespace brigade-metrics \
--output jsonpath='{.status.loadBalancer.ingress[0].ip}'
If you overrode defaults and enabled support for an ingress controller, you probably know what you're doing well enough to track down the correct IP without our help. 😉
With this public IP in hand, edit your name servers and add an A
record
pointing your domain to the public IP.
4. Accessing the Dashboard
If you overrode defaults and set grafana.service.type
to LoadBalancer
, then
the dashboard should be accessible over HTTPS at the public IP address or DNS
hostname.
If you kept the default setting of ClusterIP
for grafana.service.type
, then
use port forwarding to expose the dashboard on your local network interface:
$ kubectl port-forward \
service/brigade-metrics-grafana \
--namespace brigade-metrics \
8443:443
In this case, the dashboard should be accessible at https://localhost:8443
.
Expect to receive a cert warning.
Log in using the username and password you selected in the previous section.
Contributing
The Brigade project accepts contributions via GitHub pull requests. The Contributing document outlines the process to help get your contribution accepted.
Support & Feedback
We have a slack channel! Kubernetes/#brigade Feel free to join for any support questions or feedback, we are happy to help. To report an issue or to request a feature open an issue here
Code of Conduct
Participation in the Brigade project is governed by the CNCF Code of Conduct.