Awesome

Consul Exporter

Export Consul service health to Prometheus.

To run it:

make
./consul_exporter [flags]

Exported Metrics

Metric	Meaning	Labels
consul_up	Was the last query of Consul successful
consul_raft_peers	How many peers (servers) are in the Raft cluster
consul_serf_lan_members	How many members are in the cluster
consul_serf_lan_member_info	Information of member in the cluster.	member, role, version
consul_serf_lan_member_status	Status of member in the cluster. 1=Alive, 2=Leaving, 3=Left, 4=Failed.	member
consul_serf_wan_member_info	Information of member in the wan cluster.	member, dc, role, version
consul_serf_wan_member_status	Status of member in the wan cluster. 1=Alive, 2=Leaving, 3=Left, 4=Failed.	member, dc
consul_catalog_services	How many services are in the cluster
consul_service_tag	Tags of a service.	service_id, node, tag
consul_catalog_service_node_healthy	Is this service healthy on this node	service_id, node, service_name
consul_health_node_status	Status of health checks associated with a node	check, node, status
consul_health_service_status	Status of health checks associated with a service	check, node, service_id, service_name, status
consul_catalog_kv	The values for selected keys in Consul's key/value catalog. Keys with non-numeric values are omitted	key
consul_service_checks	Link the Consul service ID with check name if available	service_id,service_name, check_id, check_name, node

Flags

./consul_exporter --help

consul.agent-only: Only export metrics about services registered on local agent.
consul.allow_stale: Allows any Consul server (non-leader) to service a read.
consul.ca-file: File path to a PEM-encoded certificate authority used to validate the authenticity of a server certificate.
consul.cert-file: File path to a PEM-encoded certificate used with the private key to verify the exporter's authenticity.
consul.health-summary: Collects information about each registered service and exports consul_catalog_service_node_healthy. This requires n+1 Consul API queries to gather all information about each service. Health check information are available via consul_health_service_status as well, but only for services which have a health check configured. Defaults to true. Disable using --no-consul.heatlh-summary.
consul.key-file: File path to a PEM-encoded private key used with the certificate to verify the exporter's authenticity.
consul.insecure: Disable TLS host verification.
consul.require_consistent: Forces the read to be fully consistent.
consul.server: Address (host and port) of the Consul instance we should connect to. This could be a local agent (localhost:8500, for instance), or the address of a Consul server.
consul.server-name: When provided, this overrides the hostname for the TLS certificate. It can be used to ensure that the certificate name matches the hostname we declare.
consul.timeout: Timeout on HTTP requests to consul.
consul.request-limit: Limit the maximum number of concurrent requests to consul, 0 means no limit.
log.format: Set the log target and format. Example: logger:syslog?appname=bob&local=7 or logger:stdout?json=true
log.level: Logging level. info by default.
web.listen-address: Address to listen on for web interface and telemetry.
web.telemetry-path: Path under which to expose metrics.
version: Show application version.

Key/Value Checks

This exporter supports grabbing key/value pairs from Consul's KV store and exposing them to Prometheus. This can be useful, for instance, if you use Consul KV to store your intended cluster size, and want to graph that value against the actual value found via monitoring.

kv.filter: Only store keys that match this regex pattern.
kv.prefix: Prefix under which to look for KV pairs.

A prefix must be supplied to activate this feature. Pass / if you want to search the entire keyspace.

Environment variables

The consul_exporter supports all environment variables provided by the official consul/api package, including CONSUL_HTTP_TOKEN to set the ACL token.

Useful Queries

Are my services healthy?

min(consul_catalog_service_node_healthy) by (service_name)

Values of 1 mean that all nodes for the service are passing. Values of 0 mean at least one node for the service is not passing.

What service nodes are failing?

sum by (node, service_name)(consul_catalog_service_node_healthy == 0)

What service checks are critical?

consul_health_service_status{status="critical"} == 1

You can query for the following health check states: "maintenance", "critical", "warning" or "passing"

Using Docker

You can deploy this exporter using the prom/consul-exporter Docker image.

For example:

docker pull prom/consul-exporter

docker run -d -p 9107:9107 prom/consul-exporter --consul.server=172.17.0.1:8500

Keep in mind that your container needs to be able to communicate with the Consul server or agent. Use an IP accessible from the container or set the --dns and --dns-search options of the docker run command:

docker run -d -p 9107:9107 --dns=172.17.0.1 --dns-search=service.consul \
        prom/consul-exporter --consul.server=consul:8500

TLS and basic authentication

The consul_exporter supports TLS and basic authentication. To use TLS and/or basic authentication, you need to pass a configuration file using the --web.config.file parameter. The format of the file is described in the exporter-toolkit repository.