Home

Awesome

SMI Metrics API

This repository is an implementation of the Traffic Metrics Spec which follows the format of the official Metrics API being built on top of kubernetes to get metrics about pods and nodes. Here, the metrics are about the inbound and outbound requests i.e their Golden Metrics like p99_latency, p55_latency, success_count, etc for a particular workload.

Installation

For Linkerd:

helm template chart --set adapter=linkerd | kubectl apply -f -

For Istio

helm template chart --set adapter=istio | kubectl apply -f -

Roadmap

The API supports linkerd and Istio right now. The support for Consul is being worked up on right now and the API and responses will have the same structure unless there are no changes to the spec.

Working

The SMI metrics api is a Kubernetes APIService as seen in the installation manifest, which is a way of extending the Kubernetes API.

We will perform installation of the SMI Metrics API w.r.t linkerd. Make sure linkerd is installed and is running as per the instructions here, This API can be installed by running the following command

helm template chart -f dev.yaml -f linkerd.yaml --name dev | kubectl apply -f -

The installation of the APIService can be verified by running

kubectl get apiservice | grep smi
v1alpha1.metrics.smi-spec.io           default/dev-smi-metrics   True        25h
v1alpha1.metrics.smi-spec.io           default/dev-smi-metrics   True        25h

The APIService first informs the kubernetes API about itself and the resource types that it exposes, which can be viewed by running

kubectl api-resources | grep smi
NAME                              SHORTNAMES   APIGROUP                       NAMESPACED   KIND
daemonsets                                     metrics.smi-spec.io            true         TrafficMetrics
deployments                                    metrics.smi-spec.io            true         TrafficMetrics
namespaces                                     metrics.smi-spec.io            false        TrafficMetrics
pods                                           metrics.smi-spec.io            true         TrafficMetrics
statefulsets                                   metrics.smi-spec.io            true         TrafficMetrics

This means that, the APIService can be queried regarding the above mentioned resource types.

Now that the SMI APIService is installed, metric queries can be done through the kubernetes API.

Because metrics only return the last 30s by default, if you'd like to see the metrics, make sure there is something happening on your cluster. The easiest way to do this is to run linkerd dashboard and see the traffic generated by viewing that page.

kubectl get --raw /apis/metrics.smi-spec.io/v1alpha1/namespaces/linkerd/deployments/linkerd-web | jq

Output:

{
  "kind": "TrafficMetrics",
  "apiVersion": "metrics.smi-spec.io/v1alpha1",
  "metadata": {
    "name": "linkerd-web",
    "namespace": "linkerd",
    "selfLink": "/apis/metrics.smi-spec.io/v1alpha1/namespaces/linkerd/deployments/linkerd-web",
    "creationTimestamp": "2019-06-17T14:26:22Z"
  },
  "timestamp": "2019-06-17T14:26:22Z",
  "window": "30s",
  "resource": {
    "kind": "Deployment",
    "namespace": "linkerd",
    "name": "linkerd-web"
  },
  "edge": {
    "direction": "from",
    "resource": null
  },
  "metrics": [
    {
      "name": "p99_response_latency",
      "unit": "ms",
      "value": "296875m"
    },
    {
      "name": "p90_response_latency",
      "unit": "ms",
      "value": "268750m"
    },
    {
      "name": "p50_response_latency",
      "unit": "ms",
      "value": "162500m"
    },
    {
      "name": "success_count",
      "value": "73492m"
    },
    {
      "name": "failure_count",
      "value": "0"
    }
  ]
}

As we can see, the golden metrics of a particular resource can be retrieved by querying the API with a path format /apis/metrics.smi-spec.io/v1alpha1/namespaces/{Namespace}/{Kind}/{ResourceName}

Queries for golden metrics on edges i.e paths associated with a particular resource, for example linkerd-controller can also be done by adding /edges to the path.

kubectl get --raw /apis/metrics.smi-spec.io/v1alpha1/namespaces/linkerd/deployments/linkerd-controller/edges | jq

Output:

{
  "kind": "TrafficMetricsList",
  "apiVersion": "metrics.smi-spec.io/v1alpha1",
  "metadata": {
    "selfLink": "/apis/metrics.smi-spec.io/v1alpha1/namespaces/linkerd/deployments/linkerd-controller/edges"
  },
  "resource": {
    "kind": "Deployment",
    "namespace": "linkerd",
    "name": "linkerd-controller"
  },
  "items": [
    {
      "kind": "TrafficMetrics",
      "apiVersion": "metrics.smi-spec.io/v1alpha1",
      "metadata": {
        "name": "linkerd-controller",
        "namespace": "linkerd",
        "selfLink": "/apis/metrics.smi-spec.io/v1alpha1/namespaces/linkerd/deployments/linkerd-controller/edges",
        "creationTimestamp": "2019-06-17T14:51:57Z"
      },
      "timestamp": "2019-06-17T14:51:57Z",
      "window": "30s",
      "resource": {
        "kind": "Deployment",
        "namespace": "linkerd",
        "name": "linkerd-controller"
      },
      "edge": {
        "direction": "from",
        "resource": {
          "kind": "Deployment",
          "namespace": "linkerd",
          "name": "linkerd-web"
        }
      },
      "metrics": [
        {
          "name": "p99_response_latency",
          "unit": "ms",
          "value": "294"
        },
        {
          "name": "p90_response_latency",
          "unit": "ms",
          "value": "240"
        },
        {
          "name": "p50_response_latency",
          "unit": "ms",
          "value": "150"
        },
        {
          "name": "success_count",
          "value": "28580m"
        },
        {
          "name": "failure_count",
          "value": "0"
        }
      ]
    },
    {
      "kind": "TrafficMetrics",
      "apiVersion": "metrics.smi-spec.io/v1alpha1",
      "metadata": {
        "name": "linkerd-controller",
        "namespace": "linkerd",
        "selfLink": "/apis/metrics.smi-spec.io/v1alpha1/namespaces/linkerd/deployments/linkerd-controller/edges",
        "creationTimestamp": "2019-06-17T14:51:58Z"
      },
      "timestamp": "2019-06-17T14:51:57Z",
      "window": "30s",
      "resource": {
        "kind": "Deployment",
        "namespace": "linkerd",
        "name": "linkerd-controller"
      },
      "edge": {
        "direction": "to",
        "resource": {
          "kind": "Deployment",
          "namespace": "linkerd",
          "name": "linkerd-prometheus"
        }
      },
      "metrics": [
        {
          "name": "p99_response_latency",
          "unit": "ms",
          "value": "368"
        },
        {
          "name": "p90_response_latency",
          "unit": "ms",
          "value": "247199m"
        },
        {
          "name": "p50_response_latency",
          "unit": "ms",
          "value": "120731m"
        },
        {
          "name": "success_count",
          "value": "1008100m"
        },
        {
          "name": "failure_count",
          "value": "0"
        }
      ]
    }
  ]
}

Development

Admin

TODO

API Questions

Internal details

Contributing

Please refer to CONTRIBUTING.md for more information on contributing.