Home

Awesome

CSI Proxy

There's a KEP about moving CSI Drivers to HostProcess containers which involves making CSI Proxy a Go library, development for this work is happening in the library-development branch

CSI Proxy (which might be more aptly named "csi-node-proxy") is a binary that exposes a set of gRPC APIs (over named pipes) around local storage operations for nodes in Windows. A container, such as CSI node plugins, can mount the named pipes depending on operations it wants to exercise on the host and invoke the APIs. This allows a storage plugin to run as if were a CSI plugin on linux which have the ability to perform "privileged" actions on the windows host OS.

CSI Proxy High Level Architecture Diagram

In the above diagram, there are 3 communication channels being utilized for CSI proxy:

Each named pipe will support a specific version of an API (e.g. v1alpha1, v1beta1, v1beta2, v1) that targets a specific area of storage (e.g. disk, volume, file, SMB, iSCSI).

Overview

CSI drivers are recommended to be deployed as containers. Node plugin containers need to run with privileges to perform storage related operations. However, Windows does not support privileged containers currently. With CSIProxy, the node plugins can now be deployed as unprivileged pods that use the proxy to perform privileged storage operations on the node. Kubernetes administrators will need to install and maintain csi-proxy.exe on all Windows nodes in a manner similar to kubelet.exe.

Compatibility

Recommended K8s Version: 1.18

Feature status

CSI Proxy is in a stable status (GA Blogpost), the latest versions of the API Groups are:

API GroupLatest VersionAPI Docs
Diskv1link
Filesystemv1link
SMBv1link
Volumev1link
iSCSIv1alpha2link to proto
Systemv1alpha1link to proto

Build

csi-proxy.exe binary has not published yet. You need to clone the repo and build it. The easiest way to build this binary is using the makefile in this repo.

git clone https://github.com/kubernetes-csi/csi-proxy
sudo make build

On successful execution of make build, the output binary csi-proxy.exe will be available under /bin directory.

Installation

csi-proxy.exe can be installed and run as binary or run as a Windows service on each Windows node. See the following as an example to run CSI Proxy as a web service.

$flags = "-windows-service -log_file=C:\etc\kubernetes\logs\csi-proxy.log -logtostderr=false"
sc.exe create csiproxy start= "auto" binPath= "C:\etc\kubernetes\node\bin\csi-proxy.exe $flags"
sc.exe failure csiproxy reset= 0 actions= restart/10000
sc.exe start csiproxy

If you are using kube-up to start a Windows cluster, node startup script will automatically run csi-proxy as a service. For GKE 1.18+, csi-proxy will be installed automatically.

Usage

Command line options

Setup for CSI Driver Deployment

Deploy and start csiproxy.exe on all Windows hosts in the cluster. Next, the named pipes can be mounted in a CSI node plugin DaemonSet YAML in the following manner:

kind: DaemonSet
apiVersion: apps/v1
metadata:
  name: csi-storage-node-win
spec:
  selector:
    matchLabels:
      app: csi-driver-win
  template:
    metadata:
      labels:
        app: csi-driver-win
    spec:
      serviceAccountName: csi-node-sa
      tolerations:
      - key: "node.kubernetes.io/os"
        operator: "Equal"
        value: "win1809"
        effect: "NoSchedule"
      nodeSelector:
        kubernetes.io/os: windows
      containers:
        - name: csi-driver-registrar
          image: k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0
          args:
            - "--v=5"
            - "--csi-address=unix://C:\\csi\\csi.sock"
            - "--kubelet-registration-path=C:\\var\\lib\\kubelet\\plugins\\pd.csi.storage.gke.io\\csi.sock"
          env:
            - name: KUBE_NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
          volumeMounts:
            - name: plugin-dir
              mountPath: C:\csi
            - name: registration-dir
              mountPath: C:\registration
        - name: csi-driver
          # placeholder, use your CSI driver
          image: org/csi-driver:win-v1
          args:
            - "--v=5"
            - "--endpoint=unix:/csi/csi.sock"
          volumeMounts:
            - name: kubelet-dir
              mountPath: C:\var\lib\kubelet
            - name: plugin-dir
              mountPath: C:\csi
            - name: csi-proxy-disk-pipe
              mountPath: \\.\pipe\csi-proxy-disk-v1
            - name: csi-proxy-volume-pipe
              mountPath: \\.\pipe\csi-proxy-volume-v1
            - name: csi-proxy-filesystem-pipe
              mountPath: \\.\pipe\csi-proxy-filesystem-v1
            - name: csi-proxy-smb-pipe
              mountPath: \\.\pipe\csi-proxy-smb-v1
      volumes:
        - name: csi-proxy-disk-pipe
          hostPath:
            path: \\.\pipe\csi-proxy-disk-v1
            type: ""
        - name: csi-proxy-volume-pipe
          hostPath:
            path: \\.\pipe\csi-proxy-volume-v1
            type: ""
        - name: csi-proxy-filesystem-pipe
          hostPath:
            path: \\.\pipe\csi-proxy-filesystem-v1
            type: ""
        - name: csi-proxy-smb-pipe
          hostPath:
            path: \\.\pipe\csi-proxy-smb-v1
            type: ""
        - name: registration-dir
          hostPath:
            path: C:\var\lib\kubelet\plugins_registry\
            type: Directory
        - name: kubelet-dir
          hostPath:
            path: C:\var\lib\kubelet\
            type: Directory
        - name: plugin-dir
          hostPath:
            path: C:\var\lib\kubelet\plugins\csi.org.io\
            type: DirectoryOrCreate

Community, discussion, contribution, and support

Check out development.md for instructions to set up a development environment to run CSI Proxy.

Learn how to engage with the Kubernetes community on the community page.

You can reach the maintainers of this project at:

Supported CSI Drivers

Code of conduct

Participation in the Kubernetes community is governed by the Kubernetes Code of Conduct.