Amazon EKS Auto Mode ENABLED - Build your super-powered cluster
Deploy a Fully Functional Amazon EKS Cluster with Auto Mode Using Terraform – See How It Simplifies Operations
Published Feb 2, 2025
Last Modified Feb 3, 2025
TL;DR: The Github Repo here setheliot/eks_auto_mode enables you to setup a fully functional Amazon EKS cluster (and app) with Auto Mode enabled, using Terraform.
If you're running on AWS and love Kubernetes, Amazon EKS is a natural choice. It removes the burden of managing the Kubernetes control plane and makes it easy to run Kubernetes workloads on Amazon EC2 or go serverless with AWS Fargate.
Until recently, EKS still required some burden—you had to install and maintain controllers for load balancing and auto scaling, manage IAM roles and policies, and configure IAM configurations for CSI drivers (EBS, EFS, FSx) for persistent storage.
But now, with EKS Auto Mode, announced at AWS re:Invent 2024, much of this complexity is gone! I created a GitHub repo: setheliot/eks_auto_mode for hands-on learning—feel free to dive in now and create an EKS cluster with Auto Mode enabled. Or stick around here, and this blog will take you deeper into what Auto Mode does (and doesn’t) do and how it can benefit you.
EKS Auto Mode automates:
- Compute: It creates new nodes when pods can't fit onto existing ones, and identifies low utilization nodes for deletion.
- Networking: It configures AWS Load Balancers for Kubernetes Service and Ingress resources, to expose cluster apps to the internet.
- Storage: It creates EBS Volumes to back Kubernetes storage resources.
Glad you asked! 😁
The setheliot/eks_auto_mode Github repo uses Terraform to deploy an Amazon EKS cluster with Auto Mode enabled, including a demo app running on the cluster. Using that, I will walk you through how to deploy with Auto Mode enabled, and what Auto Mode actually does for you.
Let's start with enabling it. The following code is what sets Auto Mode to enabled.
with Auto Mode enabled
Code here.
And the following is what the code above replaces.
without Auto Mode
Code here
Note: for my EKS Cluster code without Auto Mode, I am using the GitHub repos: setheliot/xyz_infra_poc and setheliot/xyz_app_poc.
To enable Auto Mode on EKS, you set
enabled=true on cluster_compute_config.But here is something that is a little tricky. Even though this is under
cluster_compute_config, this is actually enabling Auto Mode for not just compute, but for everything:- compute
- networking (load balancing)
- storage
These three are controlled by three separate flags in the SDK for
create_cluster, but you get them all when setting enabled=true using the Terraform eks module.The above code serves two purposes. One (as just stated) is to enable Auto Mode for everything. The second is to set up how Auto Mode handles compute in our Kubernetes cluster. Let's look at what it is doing.
In the code without Auto Mode, you can see we need to setup a Node Group. This makes use of the Cluster Autoscaler which sets up an AWS Auto Scaling Group for EKS nodes. You provide instance types, AMIs, and parameters about the number of nodes.
By comparison, when using Auto Mode enabled you only specify a node pool. EKS Auto Mode uses Karpenter, an open-source Kubernetes autoscaler. By selecting the
general-purpose node pool you are letting EKS Auto Mode select for you the instance size and number of nodes. If, however, you need more control over such things, you can create custom node pools.
Persistent storage in a Kubernetes cluster remains available to any container it's configured for and persists independently of the node lifecycle. In the following code we use an Amazon Elastic Block Store (EBS) volume for storage.
with Auto Mode enabled
Code here
without Auto Mode
Note the extra configuration to create and attach the IAM policy.
Code here
Ok. there are a few things going on here.
With Auto Mode enabled, you don’t need to add IAM permissions for the CSI (Container Storage Interface) driver (
AmazonEBSCSIDriverPolicy). Auto Mode has its own CSI driver and permissions are already set up.Note: EKS purists may notice I took a shortcut when setting up CSI permissions in the "without Auto Mode" case. I should be using IRSA (IAM roles for service accounts), but am not. In the new repo "with Auto Mode enabled" I do indeed use IRSA when needed later (to give my application necessary permissions).
Setting up
StorageClass looks pretty similar. However note the different storage_provisioner values. ebs.csi.eks.amazonaws.comwith Auto Mode enabledebs.csi.amazonaws.comusing the older CSI driver
This can be a little tricky, so make sure you include that
eks. It also shows how with Auto Mode enabled, it is using a new CSI driver, specific to Auto Mode.And from here, the
PersistentVolumeClaim setup is the same. Also, later, when configuring your pods, you would reference the PersistentVolumeClaim the same way to create the PersistentVolumes. (Here if you are curious).Load balancing enables you to route requests to the pods in your cluster. Let's take a look at how Auto Mode enabled helps with this.
We will break this into two parts
with Auto Mode enabled
Nothing 🚫; Zilch 0️⃣; Nada 🙅♂️
without Auto Mode
Code here
Code here
And here we can see the big win for Auto Mode enabled. It is great that AWS Load Balancer Controller (LBC) exists, but installing it and its IAM roles was no fun. Amazon EKS with Auto Mode enabled eliminates this chore. It provides its own controller for creating load balancers, including all permissions already configured to use it.
Here is the code to create the Application Load Balancer (ALB).
with Auto Mode enabled
Code here
without Auto Mode
Code here
At first, it looks like Auto Mode adds extra work here. Let's look more closely.
IngressClass
Using Auto Mode requires we create an
IngressClass. If you look at the "without Auto Mode" code there is no such resource, but you can see it referenced:ingress_class_name = local.ingress_class_name
The
IngressClass referenced was automatically created by Helm when installing the AWS Load Balancer Controller (LBC).IngressClassParams
IngressClassParams is optional when using LBC, but mandatory for Auto Mode.Here’s where Terraform presents a challenge:
IngressClassParamsis not a standard Kubernetes resource; it is a custom resource managed inside the cluster.- Terraform does not have a built-in resource to create it, so we must use a workaround—executing the AWS CLI via Terraform to apply it.
Controller specification
The configuration that determines whether to use LBC (without Auto Mode) or use the Auto Mode controller is in the
IngressClass. With Auto Mode enabled you can see in the code above we use:controller = "eks.amazonaws.com/alb"
This explicitly tells EKS Auto Mode to manage the ALB.
But without Auto Mode (using LBC), this information is in the
IngressClass that was created by Helm. It looks like this:controller = ingress.k8s.aws/alb
Earlier I mentioned IRSA (IAM Roles for Service Accounts), which is the best way to grant your pods access to the AWS resources they need. And while having IRSA is more convenient than assigning broad permissions to node IAM roles or manually managing static credentials in
Secrets, it still requires setting up an OIDC provider and configuring roles properly. Here is the code that uses IRSA to give the application pods authorization to access an Amazon DynamoDB table. Auto Mode enabled does not help at all with this. I would like to see it help to set up trust relationships with OIDC, which are going to be common across IAM roles used with IRSA. Hopefully, the EKS team is working on this!
With Auto Mode enabled, Amazon EKS simplifies compute, storage, and networking setup—making Kubernetes on AWS easier than ever. Ready to try it? Check out the setheliot/eks_auto_mode repo and deploy your first Auto Mode cluster today!"
Seth Eliot is Principal Resilience Architect at Arpio, where he helps AWS customers implement Disaster Recovery and Cyber-Resilience strategies to protect their workloads. Before that, he was Principal Engineer at AWS and Microsoft.