Skip to content
Menu

Virtualization5 min read

Container Storage Interface (CSI)

Decoupling storage providers from Kubernetes — the plugin architecture

Why CSI Exists

Before CSI (In-tree Drivers)
  • Storage drivers compiled into K8s binaries
  • Every new storage backend = K8s code change + release cycle
  • Bugs in storage drivers could crash the kubelet
  • Vendors had to wait for K8s releases to ship fixes
  • Growing codebase — 20+ storage plugins in k8s.io/kubernetes
After CSI (Out-of-tree Drivers)
  • Standardized gRPC interface between K8s and storage providers
  • Storage vendors ship their own CSI drivers independently
  • Decoupled release cycles — vendor ships when ready
  • Same spec used by K8s, Mesos, Docker (K8s is the primary consumer)
  • Crashes isolated to driver pods, not kubelet

Note

Key insight: CSI is not a K8s-specific standard. It is a container orchestrator-agnostic specification. K8s just happens to be its biggest adopter. The spec defines a set of gRPC services that any storage provider must implement.

CSI Architecture

A CSI driver deploys two components in the cluster:

Controller Plugin (Deployment)
  • Usually 1 replica (with leader election for HA)
  • Handles cluster-level operations:
    • Create / delete volumes
    • Attach / detach volumes from nodes
    • Create / delete snapshots
    • Expand volumes
  • Runs anywhere in the cluster — doesn't need to be on the node with the storage
Node Plugin (DaemonSet)
  • Runs on every node that needs to mount volumes
  • Handles node-level operations:
    • Stage / unstage volumes on the node
    • Mount / unmount volumes into pods
    • Report node capabilities
  • Communicates via Unix socket on the node
  • Registered with kubelet via node-driver-registrar

Topology Diagram

Kubernetes Cluster
Controller (Deployment)

CSI Controller Container

CreateVolume, DeleteVolume, ControllerPublishVolume, ControllerUnpublishVolume, CreateSnapshot

Sidecar Containers

external-provisioner, external-attacher, external-snapshotter, external-resizer

Node 1 (DaemonSet Pod)

CSI Node Container

NodeStageVolume, NodePublishVolume, NodeUnstageVolume, NodeUnpublishVolume

node-driver-registrar

Registers plugin with kubelet

Node 2 (DaemonSet Pod)

CSI Node Container

NodeStageVolume, NodePublishVolume, NodeUnstageVolume, NodeUnpublishVolume

node-driver-registrar

Registers plugin with kubelet

Sidecar Containers

CSI drivers don't talk to the K8s API directly. Instead, sidecar containers (maintained by the K8s storage SIG) bridge the gap between K8s API objects and CSI gRPC calls:

Sidecar Role Watches
external-provisioner Creates/deletes PVs for PVCs PVC objects
external-attacher Attaches/detaches volumes to nodes VolumeAttachment objects
external-snapshotter Creates/restores volume snapshots VolumeSnapshot objects
external-resizer Expands volumes (online resize) PVC resize requests
node-driver-registrar Registers node plugin with kubelet (runs on node plugin pod)
livenessprobe Health check for the CSI driver (sidecar health endpoint)

Tip

Design pattern: The sidecars are the "K8s-aware" part. The CSI driver itself only needs to implement gRPC methods — it never imports K8s client libraries. This clean separation is what makes CSI portable across orchestrators.

Full Volume Lifecycle Through CSI

  1. Provision (Dynamic)
    User creates PVC → external-provisioner watches PVC → calls CreateVolume on CSI controller → storage backend creates volume → external-provisioner creates PV → PVC binds to PV.
  2. Attach
    Pod is scheduled to a node → kubelet creates VolumeAttachment object → external-attacher watches → calls ControllerPublishVolume on CSI controller → storage backend makes volume accessible to node (e.g., iSCSI login, EBS attach API call).
  3. Stage (Node)
    kubelet calls NodeStageVolume on CSI node plugin → format filesystem if needed (mkfs.ext4) → mount to a global staging path on the node (e.g., /var/lib/kubelet/plugins/kubernetes.io/csi/...).
  4. Mount (Node)
    kubelet calls NodePublishVolume on CSI node plugin → bind mount from the staging path to the pod's volume directory (e.g., /var/lib/kubelet/pods/<uid>/volumes/...).
  5. Unmount
    Pod terminates → kubelet calls NodeUnpublishVolume → removes the bind mount from the pod's directory.
  6. Unstage
    No more pods using the volume on this node → kubelet calls NodeUnstageVolume → unmount from the global staging path.
  7. Detach
    kubelet deletes VolumeAttachmentexternal-attacher calls ControllerUnpublishVolume → storage backend detaches volume from node (e.g., EBS detach API call).
  8. Delete
    PVC deleted + Delete reclaim policy → external-provisioner calls DeleteVolume → storage backend deletes volume → PV deleted.

Visual Flow: Mount Path

  1. CreateVolume

    Storage backend creates volume

  2. ControllerPublish

    Volume attached to node

  3. NodeStage

    Format + mount to staging path

  4. NodePublish

    Bind mount into pod

  1. NodeUnpublish

    Unmount from pod

  2. NodeUnstage

    Unmount from staging

  3. ControllerUnpublish

    Detach from node

  4. DeleteVolume

    Backend deletes volume

Volume Snapshots

Snapshot Abstraction (mirrors PV/PVC pattern)

  1. VolumeSnapshot

    User's request (like PVC)

  2. VolumeSnapshotContent

    Actual snapshot resource (like PV)

VolumeSnapshotClass defines the snapshot provider — analogous to StorageClass for provisioning.

Creating a Snapshot

yaml
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: my-snapshot
spec:
  volumeSnapshotClassName: csi-snapclass
  source:
    persistentVolumeClaimName: my-data   # PVC to snapshot

Restoring from a Snapshot

yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: restored-data
spec:
  storageClassName: fast-ssd
  dataSource:                           # restore from snapshot
    kind: VolumeSnapshot
    name: my-snapshot
    apiGroup: snapshot.storage.k8s.io
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi

Note

Cross-reference: For Proxmox storage and ZFS snapshots, see Virt 10 — Storage & HA.

Ephemeral CSI Volumes

Short-lived volumes created and deleted with the pod (like emptyDir, but via CSI). The pod spec includes the CSI volume definition inline — no PVC needed.

Use Cases

  • Secret injection: secrets-store-csi-driver mounts secrets from HashiCorp Vault, AWS Secrets Manager, or Azure Key Vault directly into pods
  • Node-local temp storage: CSI drivers that provide node-local SSDs as ephemeral storage with better performance than emptyDir
  • Identity injection: Workload identity tokens mounted as files
yaml
apiVersion: v1
kind: Pod
metadata:
  name: my-app
spec:
  containers:
    - name: app
      image: my-app:latest
      volumeMounts:
        - name: secrets
          mountPath: /mnt/secrets
          readOnly: true
  volumes:
    - name: secrets
      csi:
        driver: secrets-store.csi.k8s.io
        readOnly: true
        volumeAttributes:
          secretProviderClass: vault-secrets

Tip

Tip: Ephemeral CSI volumes don't create PV/PVC objects. They are defined inline in the pod spec, created when the pod starts, and destroyed when the pod is deleted. The lifecycle is managed entirely by the kubelet and the CSI node plugin.

CSI Driver Examples

Driver Storage Typical Use
ebs.csi.aws.com AWS EBS Block storage (RWO)
efs.csi.aws.com AWS EFS File storage (RWX)
pd.csi.storage.gke.io GCE Persistent Disk Block storage (RWO)
disk.csi.azure.com Azure Disk Block storage (RWO)
cephfs.csi.ceph.com CephFS Distributed file storage (RWX)
rbd.csi.ceph.com Ceph RBD Block storage (RWO)
secrets-store.csi.k8s.io Vault / cloud secrets Secret injection (ephemeral)

Note

Finding CSI drivers: The official list is at kubernetes-csi.github.io/docs/drivers.html. Most cloud providers maintain their own CSI drivers with Helm charts for easy installation.

CSI vs In-tree: Summary

Aspect In-tree (legacy) CSI (modern)
Code location Inside k8s.io/kubernetes Separate repo, vendor-maintained
Release cycle Tied to K8s releases Independent
Crash impact Can crash kubelet Isolated to driver pod
Interface Go interfaces compiled in gRPC over Unix socket
Portability K8s only Any CSI-compliant orchestrator
Status Deprecated, being migrated Standard, GA since K8s 1.13

Warning

Migration: All in-tree storage plugins are being migrated to CSI. The CSIMigration feature gate (GA since 1.25 for most drivers) transparently redirects in-tree volume API calls to the equivalent CSI driver. If you are using kubernetes.io/aws-ebs as a provisioner, it is already being handled by ebs.csi.aws.com behind the scenes.

Solidnines — solidnines.com