Select your cookie preferences

We use essential cookies and similar tools that are necessary to provide our site and services. We use performance cookies to collect anonymous statistics, so we can understand how customers use our site and make improvements. Essential cookies cannot be deactivated, but you can choose “Customize” or “Decline” to decline performance cookies.

If you agree, AWS and approved third parties will also use cookies to provide useful site features, remember your preferences, and display relevant content, including relevant advertising. To accept or decline all non-essential cookies, choose “Accept” or “Decline.” To make more detailed choices, choose “Customize.”

AWS Logo
Menu
Redefining Kubernetes Scaling with Smart EC2 Spot Allocation

Redefining Kubernetes Scaling with Smart EC2 Spot Allocation

This blog covers Karpenter's groundbreaking approach to Kubernetes scaling using smart EC2 Spot allocation. It delves into implementing price-capacity-optimized strategies, diversifying instance types, and setting up mixed Spot/On-Demand configurations. Learn how to overcome traditional autoscaling challenges, achieve rapid and efficient scaling, and significantly reduce costs while maintaining high availability in your Kubernetes clusters.

karpentercontainerscost-optimizationkubernetesec2-spot
Mahima saran
Amazon Employee
Published Nov 20, 2024
Comments

Challenges with Traditional cluster Auto-scalers

Traditionally, Kubernetes users faced a dilemma in managing cluster capacity. They relied on tools like Kubernetes Cluster Autoscaler (CAS) or Amazon EC2 Auto Scaling Groups to avoid wasted node resources. However, these solutions came with significant limitations:
  1. Lack of Intelligent Decision-Making: Cluster auto-scaler, for instance, doesn't make nuanced scaling decisions. It simply provisions a new node of a predefined size when a pending pod's resource requirements exceed the available capacity on existing nodes.
  2. Resource Inefficiency: This approach often leads to sub-optimal resource allocation:
    • Oversized Nodes: When nodes are too large, it results in underutilized infrastructure and unnecessary costs.
    • Undersized Nodes: Conversely, if nodes are too small, it increases overhead and drives up expenses due to the need for more nodes.
  3. Inflexibility: The predetermined node sizes limit the ability to adapt to diverse and changing workload needs efficiently.
This rigid scaling mechanism often resulted in a trade-off between cost efficiency and performance, leaving many organizations struggling to find the right balance in their Kubernetes environments.
The need for a more intelligent, flexible, and cost-effective scaling solution became increasingly apparent as Kubernetes deployments grew in complexity and scale.

Enter Karpenter: A Solution to Kubernetes Scaling Woes

Karpenter is an open-source tool developed by AWS for managing Kubernetes clusters and seamless integration with Kubernetes, support for Amazon EC2 Spot Instances to reduce costs, dynamic resource allocation, and scalability. It is designed to address these challenges:
1. Rapid Scaling: Karpenter can provision new nodes in seconds, not minutes.
2. Efficient Resource Allocation: It chooses the most appropriate instance types based on workload requirements.
3. Simplified Management: No need for predefined node groups; Karpenter manages everything dynamically.
4. Enhanced Flexibility: Easily leverage diverse instance types and purchasing options, including EC2 Spot instances.
Let's explore how to optimize workloads using Karpenter with EC2 Spot instances, including the price-capacity-optimized allocation strategy.

1. Basic Karpenter Setup with EC2 Spot

Here's a simple Karpenter provisioner that uses EC2 Spot instances: This provisioner tells Karpenter to use Spot instances for scaling.
1
2
3
4
5
6
7
8
9
10
11
apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
name: spot-provisioner
spec:
requirements:
- key: karpenter.sh/capacity-type
operator: In
values: ["spot"]
providerRef:
name: default

2. Implementing Price-Capacity-Optimized Allocation Strategy

EC2 Spot instances offer a price-capacity-optimized allocation strategy, which Karpenter can leverage. This strategy selects Spot instances with the lowest probability of interruption while considering price. Here's how to implement it
1
2
3
4
5
6
7
8
9
10
11
12
13
apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
name: spot-price-capacity-optimized
spec:
requirements:
- key: karpenter.sh/capacity-type
operator: In
values: ["spot"]
providerRef:
name: default
provider:
spotAllocationStrategy: price-capacity-optimized
The `spotAllocationStrategy: price-capacity-optimized` line instructs Karpenter to use the price-capacity-optimized strategy when selecting Spot instances. This approach:
  • Balances cost savings with instance availability
  • Reduces the likelihood of Spot interruptions
  • Optimizes for both price and capacity stability
3. Diversifying Instance Types
To further enhance the effectiveness of the price-capacity-optimized strategy, diversify your instance types:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
name: diverse-spot-provisioner
spec:
requirements:
- key: karpenter.sh/capacity-type
operator: In
values: ["spot"]
- key: node.kubernetes.io/instance-type
operator: In
values: ["c5.large", "c5a.large", "m5.large", "m5a.large", "r5.large", "r5a.large"]
provider:
spotAllocationStrategy: price-capacity-optimized
This configuration allows Karpenter to choose from various instance types, increasing the chances of finding optimal Spot capacity.
4. Implementing Fallback and Mixed Strategies
For critical workloads, you might want to use a mix of Spot and On-Demand instances:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
name: mixed-provisioner
spec:
requirements:
- key: karpenter.sh/capacity-type
operator: In
values: ["spot", "on-demand"]
provider:
spotAllocationStrategy: price-capacity-optimized
onDemandAllocationStrategy: lowest-price
weights:
- key: karpenter.sh/capacity-type
operator: In
values: ["spot"]
weight: 90
- key: karpenter.sh/capacity-type
operator: In
values: ["on-demand"]
weight: 10
This setup:
  • Prefers Spot instances (90% weight) but allows for On-Demand (10% weight)
  • Uses price-capacity-optimized strategy for Spot
  • Uses lowest-price strategy for On-Demand when needed
5. Monitoring and Optimization
To ensure you're getting the most out of your price-capacity-optimized strategy, monitor key metrics:
1
2
3
4
5
6
7
aws cloudwatch get-metric-statistics \
--namespace AWS/EC2Spot \
--metric-name SpotInstanceInterruption \
--statistics Sum \
--period 3600 \
--start-time 2023-01-01T00:00:00 \
--end-time 2023-01-31T23:59:59
This command retrieves Spot instance interruption data, helping you assess the effectiveness of your price-capacity-optimized strategy.
Summary
While Cluster Autoscaler was once the go-to solution for Kubernetes scaling, it has struggled to keep pace with the demands of modern, dynamic workloads. Its limitations often result in resource inefficiencies, including over-provisioned clusters, underutilized nodes, and unnecessary costs.
Karpenter features enable teams to swiftly adjust resources, scale more efficiently, and continuously optimize their Kubernetes environments. But Karpenter's impact goes beyond mere autoscaling. It has evolved into a comprehensive workload management solution, earning widespread adoption among DevOps teams. Its appeal stems from the fine-grained control it offers over scheduling decisions and resource allocation, allowing for more strategic and efficient cluster operations.
 

Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.

Comments

Comments

Log in to comment