Home

Awesome

kube-rebac-authorizer

Welcome to a prototype of how to integrate Kubernetes Authorization with Relation-Based Access Control (ReBAC)!

The core idea is that this project provides an Authorizer implementation that the Kubernetes API server can webhook to, replacing the need for both the RBAC authorizer and Node authorizer, deployed in practically all Kubernetes clusters, while offering a wider, general-purpose, way to write authorization rules for your Kubernetes cluster or kcp-like generic control planes.

The work in this repository is described and core idea is proposed in a Cloud Native Rejekts talk from November 5, 2023.

I highly recommend watching this talk before reading the rest of the README: https://www.youtube.com/live/tWWBzsZLrIw?t=396

I also highly recommend checking out the talk presentation: https://speakerdeck.com/luxas/beyond-rbac-implementing-relation-based-access-control-for-kubernetes-with-openfga

:star: Features

:handshake: IMPORTANT: While this project demonstrates a re-implementation of the RBAC and Node authorizers in Kubernetes, the aim is not to make those obsolete in any way. The aim is to demonstrate how the concepts relate, showcase that ReBAC can offer what we already have, and more, as well as experiment with making Kubernetes' (in-core and out-of-core) authorization primitives even better in the future.

:warning: WARNING: This is a prototype which, although close, is not (at the time of writing) 100% feature-compatible with neither the RBAC nor the Node authorizer, and this project is not in any way production-ready. Use it at your own risk.

Looking for feedback!

The goal here is to discuss in the Cloud Native / Kubernetes community how we can apply the great recent developments of open source ReBAC projects.

:rocket: Get in touch! I'm looking for feedback on the approach. I'm planning to bring this to the Kubernetes Special Interest Group for Authorization soon.

You can open an issue, star :star: this repo so I see your interest, or reach out to me on Twitter (I'm @kubernetesonarm) or the CNCF or Kubernetes Slacks (I'm @luxas). Looking forward to speak with you!

What is Relation Based Access Control?

ReBAC is an evolution of Role-Based Access Control and Attribute-Based Access Control, and was popularized by the Google Zanzibar paper. The core idea is that the authorization state (who X can do Y on resource Z) is modelled as a graph with nodes (for example, users and documents) and relations (directed edges) between them (user named lucas is related to document secret through the relation read).

graph LR
  user:lucas -- read --> document:secret

A ReBAC service, such as the CNCF project OpenFGA used here, provides an API to read and write what the authorization state looks like (what nodes and edges there are), define a schema/model for how the graph can be constructed, and for querying the graph (e.g., does this user node have access to this document node? or what documents does this user have access to through this relation?).

ReBAC can be an effective tool in combating the Roles Explosion phenomenon often seen in RBAC setups, which practically limits the usefulness of RBAC deployments, although in theory it would be possible to specify more fine-grained RBAC definitions than is practical. In order to understand the true usefulness of ReBAC, however, let's dive into how the authorization state is specified!

Authorization Model

Each node and relation has specific type, e.g., a node representing a user named lucas could be named user:lucas where user is the node type, and lucas is the node identifier. Furthermore, a node of a certain type can be related to nodes of other types through specific types of relations (e.g., read, write, admin, and so forth), as described by the authorization model. You can think of the authorization model as schema validation (e.g., OpenAPI / JSON Schemas) for the graph, defining what can relate to what and how.

TODO: Insert picture from the slides on the type model.

Definition Schema (Tuples)

The graph describing the authorization state is made up by a list of Tuples. A Tuple describes a direct edge in the graph of the form {source_node, relation, destination_node}. The source node is often referred to as the subject node or user node of the tuple, and the destination node the object node.

Note: The fact that relations have a direction means that even though user:lucas is related to document:secret through the access relation, there is no way for anyone related to document:secret to traverse towards the user:lucas node (unless another relation from document to user is added, of course). This has direct implications for parent-child relationships: if you want to model that someone with access to a parent (e.g., something general) should be able to access child nodes as well (something specific), then you create a relation from the parent to the child, but another relation from child to parent is needed if you want to model that people with access to the child (e.g., something specific) should also get access to the parent (that is more general).

There are two types of ways to specify the subject node(s) of a Tuple:

Evaluation Schema (Rewrites)

Although UserSets are powerful, as seen above, sometimes they are not quite enough. With the most recent example, we could say that a set of users can relate to one document through a certain relation. But if we want to give the owners of the clients folder (folder:clients#owner) access to another document, say document:marketingplan, we would need another Tuple as follows: {folder:clients#owner, editor, document:marketingplan}. This needs to be done, as otherwise we don't know which documents belong to which folders. However, this gets tedious quickly.

The solution to this is to define rules, rewrites, of the defined tuple data, that are computed at evaluation (query) time, e.g., when issuing a Check request asking "is this subject node related to this object node through this relation", or in more simple terms "can this actor perform this action on this object?"

We'd like to just define relations between users and folders, and folders and documents, and then dynamically resolve the (possibly recursive) relations between user -> folder -> document. We can do this by specifying simple {user:alice, owner, folder:clients} and {folder:clients, contains, document:customercase}-like tuples, and defining rules for how the ReBAC engine shall resolve (or rewrite) these edges. Notably, at this point, there is no relation between user and document nodes at all (only between user and folder, and folder and document).

The Zanzibar paper proposes several rewrites, of which one is the Tuple to UserSet rewrite. With the most recent example, we could define a rule on the document type, which says that "any subject node that is an owner of a folder node that I am connected with through the contains relation, will be related to me as editor". In this case, the rewrite creates a path between user and document nodes. The name comes from that, in this case, the {folder:clients, contains, document:customercase} Tuple with subject type folder is traversed "backwards" (or forwards, depending on how you think about it), in order to yield a UserSet of the subject types of the owner relation (here, user).

There are other rewrites available as well, but surprisingly few at the end of the day, given that Google is using it for authorization in all of their services TODO link. The other ones are:

Kubernetes Authorization Model

Kubernetes API server

The Kubernetes API server is an extensible, uniform, and declarative REST API server. Let's look at the implications of this:

Kubernetes Authentication

One of the first stages that happen for an API server request is authentication, where some user-provided credential (often a client certificate or JWT token) is resolved into what Kubernetes calls UserInfo, with the following fields:

Kubernetes Authorization

Authorization is executed after the authentication stage, but before the HTTP request body (if any) is deserialized. In other words, at this stage, authorization decisions cannot be made based on the user-provided data, only based on metadata. Additionally, the current API server data is also not available for making decisions.

Essentially, the parameters defined in the API server section, together with the user info, make up the attributes that are at the Authorizer's disposal for producing a response.

Kubernetes also provides an API, SubjectAccessReview, which allows an actor check if another actor is authorized to perform a given action.

Kubernetes supports multiple Authorizers, where each Authorizer is asked serially from the first to the last. Each Authorizer can produce an Allow, Deny or NoOpinion response. If Allowed, the API server short-circuits and proceeds to the next stage of the API request process. If NoOpinion is returned, the API server asks the next Authorizer in the chain, or denies the request, if there is no further authorizer to ask. If Denied, it short-circuits as well, and denies the request. In summary, order matters for the authorizer list.

The de-facto default authorizer chain consists of the builtin Node, and RBAC authorizers. Let's cover them next.

Kubernetes' RBAC Authorizer

Kubernetes implements declarative Role-Based Access Control through defining the ClusterRole, Role, ClusterRoleBinding and RoleBinding types, where the Cluster prefix indicates that the role or binding applies cluster-wide, and absence means the role or binding applies in a given namespace.

RBAC rules are purely additive, in other words the RBAC authorizer will return other Allow, or NoOpinion, never Deny.

A (Cluster)RoleBinding defines a mapping between a set of users or groups (matching username and groups of UserInfo) that are bound to one (Cluster)Role. A (Cluster)Role defines a set of policies for allowed requests to match. Each policy defines:

An allowed request must contain a matching apiGroup, resource and verb.

Optionally, the rule can contain a set of resourceNames to match as well, for when the request's name must be in the list too.

The RBAC authorizer is built into the API server in Go code, but the authorizer receives RBAC objects through a regular client through REST requests (i.e. it does not "bypass" the normal request path), so it is topologically the same as running the RBAC authorizer outside the API server binary, barring that there isn't any need in-binary to issue an HTTP request querying for the authorization decision, which is a necessity out-of-tree. The RBAC authorizer makes decisions based on the RBAC API objects it has cached in-memory (and which are updated through a standard API server watch TODO link)

Notably, the RBAC storage layer (not the authorizer) prevents privilege escalation when creating/updating Roles and RoleBindings by making sure the user trying to change a policy already has access to all the policies that would be applied to the new role or binding.

RBAC supports aggregated ClusterRoles (such that one ClusterRole can inherit policies from others), but this is done through a controller copying source PolicyRules to the aggregated ClusterRole's Rules. With ReBAC, we can just create an edge between one ClusterRole to another without further ado.

Additionally, RBAC is extensible, and allows users and programs to define custom verbs in addition to the ones used by the API server. This can be used in conjunction with the SubjectAccessReview API, where operators, admission controllers or extensions can delegate the authorization decision, e.g., whether a user is able to set a specific field, to the authorizers already used by the API server, for which the user already has a defined authorization state.

Kubernetes' Node Authorizer

API objects in Kubernetes follow the "control through choreography" design pattern, where the idea is to design the types such that they can be actuated or reconciled (user's desired state is turned into the actual state of the system through controller actions) independently, thus favoring multiple semi-independent objects over one big, conflated object. The coupling between API objects is loose, meaning, unlike databases, missing references between objects can momentarily exist (the idea is that, while the link and referenced object might due to necessity have to be created out of order, they will eventually appear).

Kubernetes being a highly dynamic system poses a challenge to RBAC: Sometimes one wants to provide the policy in a relation/graph-based manner, as one might not know the API object names in beforehand. For example, in the Pod definition there is a nodeName field, which the Kubernetes scheduler sets once it has found a suitable node to schedule the workload on. The node agent, kubelet, needs, by necessity, to be able to read the Pod specification and linked objects the Pod needs for execution (Secrets, PersistentVolumes, etc.) in order to be able to know which containers to run and how. However, it is not ideal to let every node agent read every Pod, Secret, PersistentVolume, etc., as then it's enough to have one Node hacked to have all Secrets leaked. While RBAC allows specifying resourceNames, which in theory would make it possible to assign an individual Role per Node, with needing a list of Pods, Secrets, etc. to access, such a design is not ideal. As we don't know the names of the Pods and Secrets up-front, there would need to be a controller updating RBAC roles dynamically, but the churn/write rate of such a ClusterRole would be relatively high, which would result in a lot of conflicts. Furthermore, instead of relying on a graph-based design, specifying, e.g., that one Pod references two Secrets, the whole tree of linked resources would need to be flattened in the ClusterRole.

Kubernetes has solved this in a specific way through the Node authorizer, built in Go code directly into the API server. The Node authorizer watches Node, Pod, Secret, etc. resources and builds a graph in-memory, from which it then computes authorization decisions.

Kubernetes + ReBAC = possible?!?

The idea with this project is to generalize both the RBAC and Node authorizers into one, by using a Zanzibar-inspired ReBAC implementation. For this PoC, I chose to integrate with OpenFGA, a CNCF project implementing Zanzibar-style fine-grained authorization.

Further, I want to prototype with the possibilities that using a native, general-purpose graph-based authorization engine brings to the table.

But first, how can we replicate the existing behavior?

Mapping RBAC API Objects to Tuples

We need to define Zanzibar node types and relations between the nodes from the Kubernetes types. In addition, we need to create a mapping function from an API object of a certain kind to a list of Tuples.

Let's see an example on how to map RBAC API objects to Tuples, and what kind of authorization model we'd need:

The authorizer would now be along the lines of:

This works fairly well, and is a good start for re-implementing RBAC with ReBAC. However, Kubernetes RBAC allows defining wildcards, such as verbs=* or resources=*. This implementation does currently not support that. Thus, enter contextual check tuples.

Providing Context for Check Requests

Sometimes, it is useful to provide contextual tuples when checking if there is a path in the graph between two nodes, especially when all possible nodes and edges have not been created in the name of resource efficiency (e.g., why create a controller that watches every single object in the cluster and creates one node in the graph for each, if all authorization policies are based on collections and not instances?)

In our above example, in order to support the wildcard, we can send a contextual tuple (that is never saved to the database) with the check request that "gets us the last mile" in the graph, from a wildcard node that might exist in the graph, e.g., resource:apps.* that would allow the user to access any resource in the apps API group, to the specific node part of the check request, e.g., resource:apps.deployments. As we want to create this "forwarding rule" for any verb, we create a new resource to resource relation (e.g., with name wildcardmatch), and specify a "Tuple to UserSet" rewrite (see above for the definition) such that if the user had, e.g., get access on resource:apps.*, then they also have get access on resource:apps.deployments, as long as the contextual tuple resource:apps.* is related to resource:apps.deployments through wildcardmatch.

Furthermore, we can have an anyverb relation on the resource type, which corresponds to verb=*, for which we specify a "computed UserSet" for all the other verbs, such that a user is considered to be related to resource:<id> as get if they are directly related, related through the wildcard match, or are related through the anyverb relation.

A similar thing can be done if we have a get request for a specific API object (e.g., Deployment foo). If the user can get all Deployments in the cluster, then surely they can get that one specific Deployment. As the target node is now more specific (resourceinstance:apps/deployments/foo through get), there needs to be a contextual "forwarding" from resource:apps.deployments to resourceinstance:apps/deployments/foo using a contextual tuple and a tuple to UserSet rewrite similar to wildcardmatch earlier.

Finally, in order to reduce the amount of check requests from the Authorizer to the ReBAC engine, we can bind users contextually to groups for that specific check request, and thus make do with only one check request. This can be done without Kubernetes knowing in general which users belong to which groups, we only know for a specific authorization request that just now the user with this name belongs to this group, as that info comes from the authenticator. Thus, a contextual tuple is enough to implement this.

Mapping Node Authorizer API Objects to Relations

With the above, we implemented most (not all) of the RBAC functionality in pseudocode. Let's proceed to sketch on how the Node authorizer can be reimplemented. First and foremost, just as the Node authorizer, we need to list and watch all Nodes, Pods, and other related resources, and create one node in the graph for each API object, and create edges between related objects.

Let's look at a couple of API objects and what information they contain:

With this setup, the node user:system:node:<node-name> is related as get to all core.node, core.pod, and core.secret resources it needs to have access to, just like the Node authorizer. The ReBAC authorizer would either make a contextual forwarding node from e.g., resourceinstance:core.pods/foo to core.pods:<namespace>/foo, or perform two check requests if it knows that this Kubernetes type is "fine-grained".

Generically Building an Authorization Model

Now, what similarities are there from the above mappings from a Kubernetes API object and authorization style to the "ReBAC way"?

Each Kubernetes API kind/resource should define the following:

Given this information, an authorization model can automatically be generated in a way that, e.g., OpenFGA understands.

Reconciling Tuples in a Generic Way

Furthermore, these mapping rules are more or less declarative, given we can use something like Common Expression Language TODO link for writing the mapping functions that extract the node ID strings from the API object. With that, we can create a declarative CustomResourceDefinition API that lets the user define their wanted graph-based authorization logic without:

  1. writing Go code
  2. creating a webhook authorizer that multiplexes requests to other authorizers (as Kubernetes currently supports only one), and creating yet one webhook authorizer with its own deployment challenges
  3. writing their own graph engine like Kubernetes did
  4. trying to figure out a scalable data model for authorization data (Zanzibar already provided one that empirically works)

One important aspect of reconciliation of the tuples from the Kubernetes API (being authoritative in the case of RBAC and Node authorization, but in general the source could be other places too) to a ReBAC engine like OpenFGA, is how to detect deletions in (or "shrinking of") the desired state.

Consider that you had a ClusterRoleBinding foo with two subjects, user:lucas and group:admins, pointing to ClusterRole bar. This will generate three tuples, which are applied on the first reconciliation:

Now, imagine that, as the ClusterRoleBinding's subjects are mutable, someone goes and deletes the user:lucas subject. On the next reconciliation, the following Tuples are generated:

This means that, even though the ReBAC engine provides an idempotent "apply tuples" API, that is not enough. We need to inspect the tuples bound to the ClusterRoleBinding for the relations the ClusterRoleBinding defines or "owns", and compute what Tuples currently existing in the graph should be deleted.

Why is it important to specify what relations a Kubernetes API kind "owns" in the graph, and why did we mention some relations are "incoming" and some "outgoing"? Imagine that we reconcile the ClusterRole bar that lets subjects get deployments. It then generates the following tuple that it applies on the first reconciliation:

The second time we reconcile, say, the ClusterRole has not changed. In order to be able to process deletes, as highlighted above, if we now read all tuples touching the clusterrole:bar node in the graph, we get:

Oh, we got the clusterrolebinding -> clusterrole tuple as well! However, the ClusterRole has no knowledge about to which bindings it is bound, so it cannot generate tuples for those. Thus, unless we filter out the clusterrolebinding -> clusterrole tuple due to that that relation not being "owned" by the ClusterRole API object, we will either not be able to delete stale tuples, or we delete all unknown tuples from a given type's perspective.

TODO: talk about Kubernetes finalizers and whole object deletions.

Deployment Topologies

This project consists of two parts:

At the time of writing, the mapping policy is statically defined, but the idea is to create a CRD API for defining the authorization model and tuple mappings.

TODO: Add picture.

OpenFGA can be deployed with persistent storage (a SQL database, e.g., MySQL or Postgres) or used with an in-memory graph.

If

the in-memory backend can be a good alternative. In fact, this is exactly what Kubernetes does with RBAC and Node authorizer; authoritative data is stored in a consistent store (etcd) and then replicated to in-memory caches in the authorizers for fast response times and availability, trading consistency as per the CAP theorem. In case of a partition, the authorizer would happily respond based on potentially stale policy rules. But in this case, as it is coupled with the API server, the API server would be part of the partition as well, so the request fails regardless.

Running the demo

Setup

  1. Start an instance of OpenFGA locally, serving the gRPC port on localhost:8081.
  2. Start an etcd and API server container locally using Docker Compose, that have the following configuration:
    • Authentication is statically defined using the static-tokens authenticator, and there are three users:
      • one superuser user which can do anything as it bypasses authorization completely, being in the system:masters group.
      • one normal-user user which does not have any privileges from start, which lets us demo the RBAC authorizer reimplementation.
      • one system:node:foo-node user that lets us demo the Node authorizer reimplementation.
    • No RBAC or Node authorizers, only Webhook mode, which calls our ReBAC authorizer. The API server does not cache any authz requests.
    • API server serves HTTPS requests on localhost:6443.
  3. Start the rebac-authorizer binary using the deploy/default-config.yaml config file, and kubeconfig deploy/superuser-kubeconfig.yaml.

RBAC demo

export KUBECONFIG=demo/demo-kubeconfig.yaml

kubectl config use-context normal-user

# all denied
kubectl get pods
kubectl get pods foo
kubectl get pods -A
kubectl get pods -A -w

# use superuser mode
kubectl config use-context admin-user

# create only the role and no binding to our user
kubectl create clusterrole view-pods --verb get,list,watch --resource pods

kubectl config use-context normal-user

# still denied
kubectl get pods
kubectl get pods foo

kubectl config use-context admin-user

# bind our user to the role => now should become allowed
kubectl create clusterrolebinding normal-view-pods --clusterrole view-pods --user normal-user

kubectl config use-context normal-user

# all allowed
kubectl get pods
kubectl get pods foo
kubectl get pods -A
kubectl get pods -A -w

kubectl config use-context admin-user

# remove the "list" and "watch" verbs from the clusterrole, leave the "get"
EDITOR=nano kubectl edit clusterrole view-pods

kubectl config use-context normal-user

# not allowed anymore
kubectl get pods
# allowed
kubectl get pods foo
# allowed, in any namespace
kubectl -n sample-namespace get pods foo
# not allowed anymore
kubectl get pods -A -w

Node Authorization demo

export KUBECONFIG=demo/demo-kubeconfig.yaml
kubectl config use-context node-user

# all denied
kubectl get nodes
kubectl get node foo-node
kubectl get pods
kubectl get pods hello
kubectl get secrets
kubectl get secrets missioncritical
kubectl get secrets very-secret

kubectl config use-context admin-user

# apply the following objects:
# Node foo-node (node-kubeconfig.yaml has corresponding user system:node:foo-node)
# Pod hello which runs on foo-node and references Secrets missioncritical and very-secret
# Secret missioncritical
# Secret very-secret
# NOTE: No RBAC rules are changed here!
kubectl apply -f deploy/node-pod-secret.yaml

# Switch back to node context
kubectl config use-context node-user

# denied
kubectl get nodes
# allowed
kubectl get node foo-node
# denied
kubectl get pods
# allowed
kubectl get pods hello
# denied
kubectl get secrets
# allowed
kubectl get secrets missioncritical
# allowed
kubectl get secrets very-secret

Future improvements

Features yet to be implemented:

Request for Feedback!

The point of this project is to gather feedback and see how authorization for declarative control planes can be improved. Please comment what you think about this in a GitHub Issue, Discussion or on the Kubernetes or Cloud Native Slack! (I'm @luxas there as well.) Or shoot me a tweet at @kubernetesonarm.

Credits

Thanks to Jonathan Whitaker and Andrés Aguiar from the OpenFGA team that helped with questions about OpenFGA, and thanks to Jonathan for the Rejekts presentation together.

Project generated with Kubebuilder.

The authorization webhook implementation is heavily inspired by the authentication webhook in controller-runtime project. If desired by the community, I can contribute that part upstream.

As the RBAC reconciliation code is not easily vendorable from Kubernetes, I need to carry an extracted version of the RBAC storage in a forked repo. This file is outside the scope of the license and copyright of this project.

License

Apache 2.0