Container Cost Optimization on AWS

• Savings Plans and Reserved Instances
  • 1 year or 3 year commitments
  • Highly recommended for steady state workloads that is always there to reduce costs significantly
  • Not only for EC2 instances, but also also highly recommended for databases such as Amazon RDS
• EC2 Spot
  • You can save up to 90% with EC2 Spot instances compared to EC2 On-Demand instances
  • If your workload is stateless, Spot is a really good option to save costs. Try to make your workload stateless.
  • Spot with Karpenter is amazing!
  • Karpenter supports both On-Demand and Spot purchase options in the same NodePool.
• Karpenter as cluster autoscaling tool for your worker nodes
  • A lot of my customers made a lot of cost optimization benefits with Karpenter and Spot
  • Karpenter makes it possible to have a heterogeneous cluster
  • You can use different instance types, different instance sizes, different architecture types, different purchase options with just a single NodePool. You can of course use mutliple NodePools — in fact, it is a very valid strategy to use NodePools for Spot, GPU workloads, and maybe one for general purpose to cover the majority of workloads
  • I often hear from customers that Karpenter is faster than Cluster Autoscaler. Also, Cluster Autoscaler assumes that the node group is homogeneous which makes it necessary that you set up many managed node groups to support various instance types and sizes and purchase options which is hard to keep overview and maintain
  • Use Horizontal Pod Autoscaler (HPA) or Vertical Pod Autoscaler (VPA) and/ or Kubernetes Event-Driven Autosclaer (KEDA) for pod autoscaling on application level, and use Karpenter for cluster autoscaling for your worker nodes
  • Karpenter comes with node right-sizing feature to consolidate workload and to delete unused nodes or to replace nodes with smaller nodes
• EKS Auto Mode to reduce operational overhead and Total Cost of Ownership
  • With EKS Auto Mode, you don’t need to maintain cluster core capabilities such as VPC CNI, Load balancer controller, CSI Driver for EBS volumes, etc. They come with EKS Auto Mode. Especially small startups with small teams often have no time and this is very valuable for them
• Graviton
  • Graviton is ARM chips, designed by AWS, it is cheaper and faster than comparable EC2 x86 instances
  • it needs some engineering effort to build multi-architecture images, but it is a great way to reduce costs if your containers support ARM architecture
• Don’t use self-managed node groups for your EKS cluster, but rather managed node (MNG) groups, but even better is Karpenter NodePools to have a heterogeneous Kubernetes cluster. MNGs reduce your operational overhead. Customers who don’t use EKS Auto Mode yet, usually have an MNG where they install Karpenter and other core components, and the remaining EC2 instances are managed by Karpenter.
• Reduce inter-AZ traffic
  • AWS charges for Inter-AZ traffic for EC2 instances. Use Kubernetes Topology Aware Routing to route traffic to the closest pod available to try to keep pod-to-pod communication within the same AZ
• Monitoring
  • Use monitoring tools. Some examples are AWS Cost and Usage based dashboards, such as CUDOS, or SCAD Containers Cost Allocation. Find them here.
• Ask EKS MCP Server for cost optimization recommendation for your EKS clusters. github.com/awslabs/mcp
  • e.g.in Amazon Q Developer CLI, or Cursor

Summary