5 ways to cost optimize your Infrastructure as Code
Learn five code snippets that you can add to your existing AWS CloudFormation templates to prevent cost waste.
Builders! Have you been asked to increase efficiency in your AWS accounts?
Today, we’ll share five code snippets you can add to your Infrastructure as Code(IaC) to prevent cost waste. For for each code snippet, we will tell you why you need the code, what the change will do, and the code you can copy for AWS CloudFormation template.
- Amazon CloudWatch Log Group Retention
- Amazon Simple Storage Service (Amazon S3) Lifecycle rules for unused objects
- AWS Graviton for AWS Managed Services
- gp3 for Amazon Elastic Block Store (Amazon EBS) volumes
- Amazon Elastic File System (Amazon EFS) Infrequent Access
When creating resources, such as an AWS Lambda functions, if you do not create an Amazon CloudWatch Log group, AWS will create it for you. When logs are created, the default retention policy is Never expire which means you will store, and more importantly, pay for those logs forever! But if you create the CloudWatch log group upon resource provisioning, then you can define the retention period yourself.
This code can be added to your template when you create an AWS lambda function.
1
2
3
4
5
LambdaLogGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Sub "/aws/lambda/${LambdaFunctionResource}"
RetentionInDays: 14
Check out AWS::Logs::LogGroup in the Documentation
In the example code above, we have set it to two weeks, but remember this should be configured for your applications run frequency. For example, if you run the AWS Lambda function every hour, then maybe you only need a week’s worth of data. (Please check your company policy on data retention). You can track the impact in Amazon Cost Explorer. Looking at Usage type that is like TimedStorage-ByteHrs when filtered to Amazon CloudWatch Service.
When storing objects in Amazon S3, there are two overlooked types of objects that could be costing you money, and you aren’t even using them!
Delete Markers - A delete marker in Amazon S3 is a placeholder (or marker) for a versioned object that was requested to be deleted when a bucket has versioning-enabled. The object will not be deleted in this situation, but the delete marker makes Amazon S3 behave as if it is deleted. You can end up storing and paying for hundreds or thousands of previous versions that you thought were deleted.
Multi Part Uploads (MPUs) - Amazon S3’s multipart upload feature allows you to upload a single object to an S3 bucket as a set of parts. If the complete multipart upload request isn’t sent successfully, Amazon S3 will not assemble the parts and will not create any object. The parts remain in your Amazon S3 account until the multipart upload completes or is aborted, and you pay for the parts that are stored in Amazon S3.
This code snippets covers both of the overlooked objects and should be added to your code for Amazon S3 Buckets. Ensure to replace the mybucket with your unique bucket name.
1
2
3
4
5
6
7
8
9
10
11
S3Bucket:
Type: 'AWS::S3::Bucket'
Properties:
BucketName:"mybucket"
LifecycleConfiguration:
Rules:
- Id: delete-incomplete-mpu-7days
Prefix: ''
AbortIncompleteMultipartUpload:
DaysAfterInitiation: 7
ExpiredObjectDeleteMarker: True
Check out ExpiredObjectDeleteMarker in the Documentation
Adding the code above to every bucket you deploy will ensure you don’t waste money on storage you are not using. Use Amazon S3 Lens enables you to identify these objects so you can add the code snippet and start saving.
AWS Graviton processors are designed by AWS to deliver the best price performance for your cloud workloads. The processors are available with these managed services and is a great way to get started with AWS Graviton, where you won’t need to recompile your code. This change offers a range of price/performance improvements. Below is for AWS Lambda, add two lines to your code and it saves 10%.
This snippet shows the Architecture property you need to add to your AWS Lambda function to use an AWS Graviton processor.
1
2
3
4
5
6
7
8
LambdaFunctionResource:
Type: AWS::Lambda::Function
Properties:
FunctionName: MyLambdaFunction
Description: LambdaFunction of python3.10
Runtime: python3.10
Architectures:
- "arm64"
Check out AWS::Lambda::Function in the Documentation.
Amazon Elastic Block Storage gp3 volumes arrived in 2020, and yet we still see customers using gp2 when they could be making a 20% cost saving by changing. Volumes under 1TB can be moved immediately over to gp3 without any downtime or performance impact. Volumes over 1TB should have their IOPs requirements reviewed. You can find any volumes that would suit gp3 by using this query on you AWS Cost & Usage Report.
The below snippet shows the change in volume type to move your volume to gp3.
1
2
3
4
5
6
BlockDeviceMappings:
- DeviceName: "/dev/sdm"
Ebs:
VolumeType: "gp3"
DeleteOnTermination: "true"
VolumeSize: "10"
Even if you make this change after a volume has been deployed, you will have no down time.
Check out AWS::EC2::Volume in the Documentation.
Intelligent-Tiering uses Lifecycle Management to monitor the access patterns of your workload and automatically transition files that are not accessed. Files will be moved from performance-optimized storage classes, to their corresponding cost-optimized Infrequent Access (IA) storage class. Take advantage of IA storage pricing that is up to 91% lower than EFS Standard.
This snippet shows the lifecycle policies to add to your EFS resource.
1
2
3
4
5
6
FileSystemResource:
Type: 'AWS::EFS::FileSystem'
Properties:
LifecyclePolicies:
- TransitionToIA: AFTER_30_DAYS
- TransitionToPrimaryStorageClass: AFTER_1_ACCESS
Check out AWS::EFS::FileSystem in the Documentation.
This code should be deployed for file systems that contain files that are not accessed every day to reduce your storage costs. Review any latency considerations in the EFS FAQs.
In this blog post, we looked at five code snippets that you can add to your Infrastructure as Code(IaC) to prevent cost waste.
If you are interested in how you can set up a CI/CD pipeline to deploy changes to your CloudFormation stacks, have a look at this tutorial.
Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.