Dynamic Database Storage with the Amazon EBS CSI Driver for Amazon EKS
How to implement dynamic provisioning of Amazon EBS volumes for your self-managed databases in Kubernetes with the EBS CSI Driver Add-On.
Leah Tucker
Amazon Employee
Published Aug 30, 2023
Last Modified Jun 10, 2024
In today's cloud computing landscape, enterprises and SaaS platforms face challenges in managing complex databases and real-time data processing. These require storage solutions that are both robust and scalable. AWS offers Container Storage Interface (CSI) drivers for its various storage services, including Amazon Elastic Block Store (EBS), which enable you to choose the right type of storage for your specific workload requirements. The EBS CSI driver provides a Kubernetes-native way to provision and manage block storage on AWS. This allows you to dynamically provision storage, recover quickly from failures, and ensure data consistency, all within the context of Amazon EKS. Whether you're dealing with self-managed databases like PostgreSQL, MySQL, or MongoDB, the EBS CSI driver simplifies the deployment and scaling of these storage solutions.
Building on the Amazon EKS cluster from part 1 of our series, this tutorial dives into setting up the EBS CSI Driver Add-On. Included in the cluster configuration for the previous tutorial is the installation and association of the EBS CSI Driver Add-On and the OpenID Connect (OIDC) endpoint. For part one of this series, see Building an Amazon EKS Cluster Preconfigured to Run High Traffic Microservices. Alternatively, to set up an existing cluster with the components required for this tutorial, use the instructions in Create an IAM OpenID Connect (OIDC) endpoint and Create an IAM Role for Service Account (IRSA) in EKS official documentation.
| About | |
|---|---|
| ✅ AWS experience | 200 - Intermediate |
| ⏱ Time to complete | 30 minutes |
| 🧩 Prerequisites | - AWS Account |
| 📢 Feedback | Any feedback, issues, or just a 👍 / 👎 ? |
| ⏰ Last Updated | 2023-08-30 |
In this tutorial, you'll use the EBS CSI Driver Add-On installed on your Amazon EKS cluster, enabling dynamic provisioning of EBS volumes for database storage, and deploy a sample workload for a MySQL database.
Before you begin this tutorial, you need to:
- Install the latest version of kubectl. To check your version, run:
kubectl version --short. - Install the latest version of eksctl. To check your version, run:
eksctl info.
This tutorial is the second part of a series on managing high traffic microservices platforms using Amazon EKS, and it's dedicated to dynamic provisioning and storage solutions with the EBS CSI Driver. This tutorial shows not only how to provision storage within the cluster but also introduces the concept of dynamic provisioning. It covers the following components:
- Authentication: Utilize the pre-configured IAM Role for Service Account (IRSA) specifically designed for the EBS CSI Driver. This role, used within a designated AWS Availability Zone, works with the OpenID Connect (OIDC) endpoint to ensure secure communication between Kubernetes pods and AWS services.
- EBS CSI Driver Add-On Set-Up: Amazon EKS add-ons, such as the EBS CSI Driver Add-On, are curated software add-ons that include the latest security patches and bug fixes. The AWS Management Console notifies you of new versions for an Amazon EKS add-on. If you choose to install the EBS CSI driver manually, you are responsible for keeping it up to date. Note that EBS volumes are bound to a specific AWS Availability Zone and can only be accessed by nodes within the same zone, making it suitable for workloads running nodes within the same availability zone.
- Sample Application Deployment: Build and provision storage for self-managed databases using a sample MySQL database. Utilize custom annotations and parameters for the EBS CSI Driver, specifically the 'ReadWriteOnce' access mode, to instruct the EBS CSI Driver to handle storage provisioning for databases. For more examples, see Examples in the EBS CSI Driver GitHub repository.
Note that if you're still within your initial 12-month AWS Free Tier period, it's important to note that usage of EBS volumes beyond the free tier will result in additional AWS charges.
Before interacting with your Amazon EKS cluster using Helm or other command-line tools, it's essential to define specific environment variables that encapsulate your cluster's details. These variables will be used in subsequent commands, ensuring that they target the correct cluster and resources.
- First, confirm that you are operating within the correct cluster context. This ensures that any subsequent commands are sent to the intended Kubernetes cluster. You can verify the current context by executing the following command:
- Define the
CLUSTER_NAMEenvironment variable for your EKS cluster. Replace the sample value for clusterregion.
- Define the
CLUSTER_REGIONenvironment variable for your EKS cluster. Replace the sample value for clusterregion.
- Define the
CLUSTER_VPCenvironment variable for your EKS cluster.
- Define the
ACCOUNT_IDenvironment variable for the account associated with your EKS cluster.
The
ebs-csi-controller-sa service account is crucial for managing EBS volumes in Kubernetes. Make sure it's correctly set up in the kube-system namespace on your cluster.The expected output should look like this:
If you do not already have an IAM Role for Service Account (IRSA) set up for the EBS CSI Driver, or you receive an error for the above command, the following command will create the IAM Role for Service Account (IRSA) along with the service account names
ebs-csi-controller-sa. Note that you must have an OpenID Connect (OIDC) endpoint associated with your cluster before you run this command.Verify that the Amazon EBS CSI Driver Add-On was successfully installed using the following command:
The expected output should look like this:
Optionally, if you do not already have the EBS CSI Driver Add-On installed, or you receive an error, install the add-on by running the following command.
The expected output should look like this:
In this section, you will create a Kubernetes secret to securely store the MySQL credentials. By using a secret, you can keep sensitive information such as passwords and user names out of your application code and configuration files. This enhances the security of your application by allowing you to manage access to this information separately from the rest of your application. The secret will be referenced in the StatefulSet to provide the necessary environment variables to the MySQL container.
- Create an
.envfile and paste the following contents.
- In the same directory as the
.envfile you just created, run the following command to create a Kubernetes secret with sensitive credentials.
This file contains the necessary Kubernetes manifests to set up the Amazon EBS CSI Driver on an Amazon EKS cluster. It defines a StorageClass named
ebs-sc that sets up storage provisioning on AWS EBS, with a binding mode that waits for the first consumer; and a Pod named mysql-ebs that runs a Linux OS container. It also specifies the EBS CSI Driver as the provisioner (i.e., ebs.csi.aws.com). Together, these components provide a complete example of defining, claiming, and utilizing persistent storage within a Kubernetes cluster.- Create a Kubernetes manifest called mysql-ebs.yaml and paste the following contents into it.
- Deploy the Kubernetes resources in
mysql-ebs.yaml:
When you deploy the workload, the EBS CSI driver uses the
StorageClass to provision an EBS volume based on the specified configuration. The volumeClaimTemplates in the StatefulSet define the desired properties of the volume, such as accessModes, storageClassName, and resource requests. The EBS CSI driver then binds the volume to the StatefulSet's Pod, allowing the MySQL container to utilize the persistent storage.In this section, you'll validate that the volume has been dynamically provisioned and mounted using the following command:
You can see the recently attached EBS volume on the Linux OS. If everything is functioning correctly, a 30G volume should be visible at the
/var/lib/mysql path. This verifies that the Pod has successfully created and attached the volume to the Linux OS. The expected output should look like this:After finishing with this tutorial, for better resource management, you may want to delete the specific resources you created. You can delete the service account and other resources with the following commands:
Upon completion of this tutorial, you have successfully navigated the complex process of setting up data storage for stateful workloads using the EBS CSI Driver on your Amazon EKS cluster. This comprehensive journey included deploying a sample application and configuring essential components that are vital to the robustness of your system. With the completion of this tutorial, you've not only set up the EBS CSI Driver for dynamic provisioning of Amazon EBS volumes but also laid the groundwork for a scalable and resilient storage solution. To further enhance your new setup, explore enabling additional features such as volume scheduling, snap-shotting, and resizing capabilities. To learn more, see the EBS CSI Driver documentation on GitHub.
Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.