How Kubernetes CSI driver works
Why
How many times we are asked to install a CSI Driver (using kubectl/Helm) on a vendor Kubernetes cluster in order to use the same provider storage capabilities?
Each provider has its own way to do it, someone has installation from scratch others have a easier way using sort of add-on/extension.
And because everything which exists could break, it is strongly suggested to know better what's behind this CSI driver in order to troubleshoot it better in the future. That's the way of this post 😎
Intro to CSI
CSI (Container Storage Interface) is a standard for exposing storage systems to container orchestrators like Kubernetes. The CSI driver in Kubernetes acts as an intermediary between the Kubernetes cluster and the underlying storage systems, allowing for dynamic provisioning, attaching, and mounting of storage volumes to containers.
Thanks to this standard we can easly provision/use storages as a service like Amazon EFS/EBS, Azure File Share, Azure Disk and so on.
CSI Architecture in Kubernetes
container-storage-interface
CO stands for Container Orchestrator.
Generally talking (forget for a while Kubernetes), CSI needs two components:
- Node Plugin: A gRPC endpoint serving CSI RPCs that MUST be run on the Node whereupon an SP-provisioned volume will be published.
- Controller Plugin: A gRPC endpoint serving CSI RPCs that MAY be run anywhere.
Returning to Kubernetes, the following could be a diagram of CSI implemented inside Kubernetes.
+--------------------------------------+
| |
| Kubernetes Control Plane |
| |
| +------------+ +--------------+ |
| | API Server | | Controller | |
| | | | Manager | |
| +------------+ +--------------+ |
| | | |
+--------------------------------------+
| |
CSI Specification |
| |
+--------------------------------------+
| |
| CSI Driver |
| |
| +------------+ +--------------+ |
| | Kubelet | | External | |
| | | | Provisioner | |
| +------------+ +--------------+ |
| | | |
+--------------------------------------+
| |
CSI gRPC API | | Storage-specific
| | protocols (e.g., iSCSI, NFS)
+--------------------------------------+
| |
| External Storage System |
| |
+--------------------------------------+
The architecture of a CSI driver in Kubernetes involves multiple components working together:
- CSI Driver
The CSI driver is the main component responsible for interacting with the storage systems. It implements the CSI specification and exposes the necessary APIs for Kubernetes to manage storage operations. The driver communicates with the Kubernetes control plane and the storage system to perform actions such as provisioning, attaching, and mounting volumes. - Kubernetes Control Plane: The Kubernetes control plane consists of components like the API server, controller manager, and scheduler. These components interact with the CSI driver using the CSI specification in order to manage the storage lifecycle.
- Kubelet: The kubelet runs on each node in the Kubernetes cluster and is responsible for managing containers. It interacts with the CSI driver to orchestrate storage operations on the node level. When a pod is scheduled on a node and requires storage volumes, the kubelet communicates with the CSI driver to attach and mount the volumes to the appropriate containers.
- External Provisioner: In some cases, external provisioners are used alongside CSI drivers to handle volume provisioning. An external provisioner watches for specific events in Kubernetes, such as PersistentVolumeClaim (PVC) creation, and interacts with the CSI driver to dynamically create volumes on-demand from the storage system.
- Storage System: The underlying storage system is where the actual data is stored. It can be a local disk, a network-attached storage (NAS), or a cloud-based storage service. The CSI driver communicates with the storage system to perform actions like creating volumes, attaching them to nodes, and mounting them to containers.
The CSI driver architecture provides a standardized way for Kubernetes to manage storage operations across different storage systems. It allows for pluggable and extensible storage support, enabling administrators to choose and integrate various storage solutions with their Kubernetes clusters.
So, thank to CSI driver we are able to attach an external storage (like AWS EFS) to our pods using the following standard way.
+-----------------------------------+
| Kubernetes Cluster |
| |
+-------------------+---------------+
|
+---------------------+-----------------+
| |
| PersistentVolume |
| |
+---------------------+-----------------+
|
+---------------------+-----------------+
| |
| PersistentVolumeClaim |
| |
+---------------------+-----------------+
|
+---------------------+-----------------+
| |
| Pod |
| |
+---------------------+-----------------+
Links
- CSI specification
container-storage-interface- Kubernetes CSI specific
![[k8s] Automatically pull images from GitLab container registry without change the tag](/content/images/size/w750/2024/01/urunner-gitlab.png)
