Building an Amazon EKS Cluster Preconfigured to Run High Traffic Microservices

The modern digital landscape thrives on high traffic platforms, such as bustling online marketplaces, media streaming services, and real-time data analytics applications. These platforms often face unpredictable surges in user activity, like a sudden influx of ticket sales due to a concert announcement or a streaming service being swamped during a much-anticipated series premiere. Such scenarios require an architecture that can dynamically scale and swiftly recover from failures. Microservices are key to meeting these demands, but the real challenge lies in their efficient orchestration. Amazon EKS stands out as a solution, offering streamlined deployment and scaling of services, allowing each one to independently handle spikes without affecting others. With the robust support of AWS's infrastructure, Amazon EKS ensures both performance and reliability.

This tutorial demonstrates rapid Amazon EKS cluster deployment using eksctl "quickstart" templates, comparing the traditional managed node groups approach with the simplified EKS Auto Mode configuration. This use case-specific template creates and sets up a cluster preconfigured and ready to run your dynamic frontends, such as interactive web dashboards, and data-intensive backends, such as analytics engines or recommendation systems.

About
✅ AWS experience	200 - Intermediate
⏱ Time to complete	30 minutes
🧩 Prerequisites	- AWS Account
📢 Feedback	Any feedback, issues, or just a 👍 / 👎 ?
⏰ Last Updated	2023-08-29

Prerequisites

Before you begin this tutorial, you need to:

Install the latest version of kubectl. To check your version, run: kubectl version --short.
Install the latest version of eksctl. To check your version, run: eksctl info.

Overview

This tutorial is the first part of a series on managing high traffic microservices platforms using Amazon EKS, and it's dedicated to pre-configuring a cluster with the components it needs to run microservice applications with data-intensive workloads. Using the eksctl cluster template that follows, you'll build a robust, scalable, and secure Amazon EKS cluster with managed node groups. This template not only enables application workloads but also fortifies the cluster with an additional layer of security, fully aligned with best practices for production environments. It configures the following components:

Autoscaling: Managed node groups use an m5.large instance type, providing a balance of resources. With a minimum size of "2" and a maximum size of "5", node groups can dynamically scale. The volume size is set to "100", ensuring ample capacity, and required subnet tags allow the Kubernetes Cluster Autoscaler (CA) to dynamically scale your cluster.
Authentication: Necessary EKS Pod Identities mappings to enable communication between Kubernetes pods and AWS services. This includes the AWS Load Balancer Controller (LBC) used to expose applications, Amazon EFS CSI Driver for persistent data storage, Kubernetes External DNS to automatically manage DNS records, and Cert Manager to streamline management of SSL/TLS certificates. Additionally, an OpenID Connect (OIDC) endpoint enables seamless and secure communication.
Add-ons: Latest versions of the following add-ons, including "vpc-cni" to enable the Amazon VPC Container Network Interface, "coredns" to facilitate DNS resolution, "kube-proxy" to maintain network rules on each Amazon EC2 node, and the EBS CSI Driver Add-On.
Public/Private Networking: Managed node groups utilize private networking and a NAT gateway to bolster security by limiting direct internet access. The AWS Load Balancer Controller (LBC) manages and securely distributes all incoming web traffic to private subnets.
Monitoring: An Amazon CloudWatch IAM policy is attached to the EKS Pod Identity, aiding optional components like CloudWatch Container Insights to collect and summarize metrics and logs.

Note that if you're still within your initial 12-month AWS Free Tier period, certain Amazon EC2 instances for managed node groups and additional AWS services may not be included in the Free Tier, and charges may apply based on your usage.

Step 1: Configure the Cluster

In this section, you will configure the Amazon EKS cluster to meet the specific demands of high-traffic microservice applications. By creating this cluster-config.yaml file, you'll define the settings for IAM roles, scalable resources, private networking, and monitoring. These configurations are essential for ensuring that the cluster is robust, scalable, and secure, with optimized performance for dynamic scalability and data persistence.

To Create the Cluster Config

Create a cluster-config.yaml file and paste the following contents into it. Replace the region with your preferred region.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: managednodes-quickstart
  region: us-east-2
  version: "1.32"
  tags:
    # Add more cloud tags if needed for billing
    environment: managednodes-quickstarts

# The IAM section is for how EKS Pod Identity grants pods access to AWS services for your cluster.
iam:
  withOIDC: true
  podIdentityAssociations: 
  - namespace: kube-system
    serviceAccountName: aws-load-balancer-controller
    roleName: pod-identity-role-alb
    wellKnownPolicies:
      awsLoadBalancerController: true

  - namespace: kube-system
    serviceAccountName: efs-csi-controller-sa
    roleName: pod-identity-role-efs
    wellKnownPolicies:
      efsCSIController: true      

  - namespace: kube-system
    serviceAccountName: external-dns
    roleName: pod-identity-role-external-dns
    wellKnownPolicies:
      externalDNS: true          

  - namespace: cert-manager
    serviceAccountName: cert-manager
    roleName: pod-identity-role-cert-manager
    wellKnownPolicies:
      certManager: true         

  - namespace: kube-system
    serviceAccountName: cluster-autoscaler
    roleName: pod-identity-role-cluster-autoscaler
    wellKnownPolicies:
      autoScaler: true  

  - namespace: amazon-cloudwatch
    serviceAccountName: aws-node
    roleName: pod-identity-role-cloudwatch-agent
    permissionPolicyARNs: ["arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy"]

managedNodeGroups:
  - name: managed-ng
    instanceType: m5.large
    minSize: 2
    desiredCapacity: 3
    maxSize: 5
    # launch nodegroup in private subnets
    privateNetworking: true
    volumeSize: 100
    volumeType: gp3
    # Encrypt Worker Nodes Amazon EBS Volumes by default
    volumeEncrypted: true
    labels: 
      node-class: "production-workload" 
      role: "worker"   
    tags:
      nodegroup-role: worker
      env: prod     
      # EC2 tags required for cluster-autoscaler auto-discovery - these tags are automatically applied to a managed nodegroup autoscaling group
      k8s.io/cluster-autoscaler/enabled: "true"
      k8s.io/cluster-autoscaler/managednodes-quickstart: "owned"      

addonsConfig:
# automatically resolve (and apply) the recommended pod identity configuration
  autoApplyPodIdentityAssociations: true

addons:
  - name: eks-pod-identity-agent # required for `iam.podIdentityAssociations`
    tags:
      team: eks
  - name: vpc-cni
    version: latest  
  - name: aws-ebs-csi-driver
    version: latest  
  - name: coredns
    version: latest # auto discovers the latest available
  - name: kube-proxy
    version: latest

cloudWatch:
 clusterLogging:
   enableTypes: ["*"]
    # Sets the number of days to retain the logs for (see [CloudWatch docs](https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutRetentionPolicy.html#API_PutRetentionPolicy_RequestSyntax)).
    # By default, log data is stored in CloudWatch Logs indefinitely.
   logRetentionInDays: 60

Step 2: Create the Cluster

Now, we're ready to create our Amazon EKS cluster. This process takes several minutes to complete. If you'd like to monitor the status, see the AWS CloudFormation console.

Create the EKS cluster using the cluster-config.yaml.

1
eksctl create cluster -f cluster-config.yaml

Upon completion, you should see the following response output:

1
2024-08-21 11:20:40 [✔]  EKS cluster "managednodes-quickstart" in "us-east-2" region is ready

When the previous command completes, verify that all of your nodes have reached the Ready state with the following command:

1
kubectl get nodes

The expected output should look like this:

1
2
3
4
NAME                                            STATUS   ROLES    AGE     VERSION
ip-192-168-123-223.us-east-2.compute.internal   Ready    <none>   5m13s   v1.32.0-eks-036c24b
ip-192-168-153-14.us-east-2.compute.internal    Ready    <none>   5m9s    v1.32.0-eks-036c24b
ip-192-168-179-206.us-east-2.compute.internal   Ready    <none>   5m20s   v1.32.0-eks-036c24b

Step 3: Verify Cluster Node and Pod Health

Verify that the Amazon EBS CSI Driver was successfully installed with the following command:

1
kubectl get deployment ebs-csi-controller -n kube-system

The expected output should look like this:

1
2
NAME                 READY   UP-TO-DATE   AVAILABLE   AGE
ebs-csi-controller   2/2     2            2           15m

Verify all the Pod Identity associations and how it maps a role to a service account in a namespace in the cluster with the following command:

1
eksctl get podidentityassociation --cluster managednodes-quickstart --region us-east-2

Verify all the addons in the cluster with the following command:

1
eksctl get addons --cluster managednodes-quickstart --region us-east-2

The expected output should look like this:

1
2
3
4
5
6
7
NAME                    VERSION                 STATUS  ISSUES  IAMROLE UPDATE AVAILABLE        CONFIGURATION VALUES    POD IDENTITY ASSOCIATION ROLES
aws-ebs-csi-driver      v1.38.1-eksbuild.2      ACTIVE  0                                                               arn:aws:iam::111111111111:role/eksctl-managednodes-quickstartx-addon-aws-ebs-Role1-K7ij8B5IzOiT
coredns                 v1.11.4-eksbuild.2      ACTIVE  0
eks-pod-identity-agent  v1.3.4-eksbuild.1       ACTIVE  0               v1.3.5-eksbuild.1
kube-proxy              v1.32.0-eksbuild.2      ACTIVE  0
vpc-cni                 v1.19.2-eksbuild.1      ACTIVE  0                                                               arn:aws:iam::111111111111:role/eksctl-managednodes-quickstartx-addon-vpc-cni-Role1-8vJZRTZGBNAh
metrics-server          v0.7.2-eksbuild.1       ACTIVE  0

Congratulations! You've successfully set up the foundational infrastructure of your Amazon EKS cluster using the EKS standard mode, laying the essential groundwork for deploying high-traffic microservices workloads. This setup provides you with the groundwork needed to begin deploying applications, but keep in mind that additional configurations are required for a fully optimized, production-ready environment.

(Optional) Deploy a Sample Application

Now, we’re all set to launch a sample application and enable its accessibility on the internet through an Application Load Balancer (ALB). For this type of workload, you will need to install and configure additional cluster capabilities such as AWS Load Balancer Controller (LBC). For step-by-step guidance, check out the tutorial at Exposing and Grouping Applications using the AWS Load Balancer Controller (LBC) on an EKS IPv4 Cluster. This tutorial will guide you through the required Ingress annotations for the AWS LBC, an essential mechanism for controlling external access to services within an EKS cluster. You’ll also explore Ingress Groupings, a sophisticated feature that amalgamates multiple Ingress resources into one ALB, enhancing both efficiency and ALB management.

Leveraging Amazon EKS Auto Mode

If you want to further simplify cluster management responsibilities and provision a production-ready cluster configuration out of the box, you can leverage Amazon EKS Auto Mode. EKS Auto Mode significantly reduces the complexity of managing production workloads while maintaining enterprise-grade security and reliability standards. Let's explore how Auto Mode simplifies the process while maintaining the performance requirements for high-traffic microservices.

Feature	EKS Auto Mode	Standard Mode
Node Group Management	Automatic scaling and updates	Manual configuration required
Security Compliance	Pre-configured security policies and controls	Manual security configuration
Load balancing	Pre-integrated with with Application Load Balancer	Manual setup required
Production Readiness	Immediate - preconfigured best practices	Requires manual implementation of best practices
Cost Optimization	Built-in cost optimization	Separate cost management setup needed

To Create the Cluster Config for Auto Mode

Create a cluster-config.yaml file and paste the following contents into it. Replace the region with your preferred region.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: automode-quickstart
  region: us-east-2
  version: "1.32"
  tags:
    # Add more cloud tags if needed for billing
    environment: automode-quickstarts

autoModeConfig:
  # defaults to false
  enabled: true
    

Create the EKS cluster using the cluster-config.yaml.

1
eksctl create cluster -f cluster-config.yaml

Congratulations! You've successfully set a production ready Amazon EKS cluster for your production workloads using the EKS Auto Mode. Now, we’re all set to launch a sample application and enable its accessibility on the internet through an Application Load Balancer (ALB). Using Auto Mode, you do not need to install and configure AWS Load Balancer Controller (LBC) cluster capabilities.

Clean Up

To avoid incurring future charges, you should delete the resources created during this tutorial. You can delete the EKS cluster with the following command:

1
eksctl delete cluster -f ./cluster-config.yaml

Upon completion, you should see the following response output:

1
2025-02-03 16:12:56 [✔]  all cluster resources were deleted

Conclusion

In this tutorial, you've not only successfully set up an Amazon EKS cluster optimized for deploying microservice applications but also laid the foundation for a seamless integration between AWS and Kubernetes. The Auto Mode transforms EKS cluster deployment from a complex, time-consuming process into a streamlined operation that delivers production-ready infrastructure in minutes rather than hours or days. For the standard mode, by configuring the necessary EKS Pod Identities, node groups, and other essential components, you've established the infrastructure that ensures smooth communication with AWS services and drivers needed to run high-traffic microservices. To fully leverage your new standard mode setup, remember that the installation of the ExternalDNS, Cluster Autoscaler, Container Insights, and AWS Load Balancer Controller are still required. With these final installations, you'll have a robust, fully operational environment ready for your microservice application deployment, all while taking advantage of the unique strengths of both Amazon EKS and Kubernetes.

Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.

Select your cookie preferences

Site Terms, Privacy, and more.

Building an Amazon EKS Cluster Preconfigured to Run High Traffic Microservices

Deploy a preconfigured Amazon EKS cluster optimized for high-demand microservice applications using an eksctl "quickstart" template.

Prerequisites

Overview

Step 1: Configure the Cluster

To Create the Cluster Config

Step 2: Create the Cluster

Step 3: Verify Cluster Node and Pod Health

(Optional) Deploy a Sample Application

Leveraging Amazon EKS Auto Mode

To Create the Cluster Config for Auto Mode

Clean Up

Conclusion

Comments