logo
Menu
CloudCycle

CloudCycle

Set lifecycle for your cloud resources to avoid surprising costs

Published May 26, 2024

The Problem

I have recently forgot to Turn off / Terminate two ec2 instances running m5x.large and t3.medium that runs about 1 month and I just received a notification of my AWS monthly bill and shocked me!
At the first place I thought my account was hacked, But turns out that I just left two AWS EC2 Instances running. 🥲😔

The Solution (CloudCycle)

So, that's why I came up with a basic solution and share it. So now, whether I forgot to turn it off / terminate a running AWS Resources, I don't have to worry about it anymore, if I have properly set the desired lifecycle of my cloud resources.
I have created a lambda function which get's executed every 15 minutes to check if the supported resources are due to termination or not.
Now, if you or your team forgot to terminate cloud resources, the CloudCycle will do the job for you.

Tools & Language used

  1. Golang - Since lambda function bills will be based on the duration of execution. At first, I developed this by using python and I have realized that I need to consider it's performance and execution time.
  2. Terraform - Used only for deployment and examples.

Architecture

Architecture
Architecture
This setup will utilized the schedule expression of event bridge, In which event bridge get's executed every 15 minutes. In doing so, lambda function get validate if the supported resources is valid for termination.
But why termination? Instead of turning it off?
Of course there's a free tool available called Cloud-Custodian. But still, sometimes turning it off isn't enough and it can lead to lots of unused resources and can still occur minimal cost.

Sample Code for EC2 Instance Service

Below are the sample codes for EC2 Instance service.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
package services

import (
"context"
"github.com/aws/aws-sdk-go-v2/service/ec2"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/service/ec2/types"
"github.com/redopsbay/cloudcycle/internal"
"github.com/redopsbay/cloudcycle/internal/schedule"
"fmt"
)

type EC2 struct {
InstanceId string
CloudCycle string
MarkForTermination bool
}

type EC2Instances struct {
Instances []EC2
}

func GetEC2Instances(ctx context.Context, client *ec2.Client) ([]types.Reservation, error) {
filters := []types.Filter{{
Name: aws.String("tag-key"),
Values: []string{internal.TagKey},
},
}

DescribeInputs := ec2.DescribeInstancesInput{
Filters: filters,
}

instances, err := client.DescribeInstances(ctx, &DescribeInputs)

if err != nil {
panic(err)
return []types.Reservation{}, err
}

return instances.Reservations, nil
}

func MarkInstancesForTermination(reservations []types.Reservation) (EC2Instances, error) {
var instances EC2Instances

for _, reservation := range reservations {
for _, instance := range reservation.Instances {
for _, tag := range instance.Tags {
if *tag.Key == internal.TagKey {
Lifecycle, err := schedule.GetLifeCycle(instance.LaunchTime, *tag.Value)
if err != nil {
return EC2Instances{}, err
}

if schedule.ValidForTermination(Lifecycle) {
ec2instance := EC2{
InstanceId: *instance.InstanceId,
CloudCycle: *tag.Value,
MarkForTermination: true,
}
instances.Instances = append(instances.Instances, ec2instance)

} else {
ec2instance := EC2{
InstanceId: *instance.InstanceId,
CloudCycle: *tag.Value,
MarkForTermination: false,
}
instances.Instances = append(instances.Instances, ec2instance)
}
}
}
}
}
return instances, nil
}

func StartEC2InstanceTermination(ctx context.Context, client *ec2.Client) error {
var instanceIds []string

reservations, err := GetEC2Instances(ctx, client)
if err != nil {
fmt.Println("Unable to get instances.")
return err
}

instances, err := MarkInstancesForTermination(reservations)
if err != nil {
fmt.Println("Unable to mark instances for termination.")
return err
}

for _, instance := range instances.Instances {
if instance.MarkForTermination {
instanceIds = append(instanceIds, instance.InstanceId)
fmt.Printf("\nInstanceID: %s, ForTermination: %t, CloudCycle: %s\n",
instance.InstanceId,
instance.MarkForTermination,
instance.CloudCycle)
}
}

TerminatedOutput, err := client.TerminateInstances(ctx, &ec2.TerminateInstancesInput{
InstanceIds: instanceIds,
})

for _, state := range TerminatedOutput.TerminatingInstances {
if *state.CurrentState.Code == 0 {
fmt.Printf("InstanceID: %s, State: Pending for Termination", *state.InstanceId)
} else if *state.CurrentState.Code == 32 {
fmt.Printf("InstanceID: %s, State: Shutting down", *state.InstanceId)
} else if *state.CurrentState.Code == 48 {
fmt.Printf("InstanceID: %s, State: Shutting down", *state.InstanceId)
} else if *state.CurrentState.Code == 16 {
fmt.Printf("InstanceID: %s, State: Still running", *state.InstanceId)
} else if *state.CurrentState.Code == 64 {
fmt.Printf("InstanceID: %s, State: Stopping", *state.InstanceId)
} else if *state.CurrentState.Code == 80 {
fmt.Printf("InstanceID: %s, State: Stopped", *state.InstanceId)
} else {
fmt.Printf("InstanceID: %s, State: Unknown", *state.InstanceId)
}
}

return nil

}

Objective

My objective with CloudCycle is to automatically cleanup supported resources based on the specified duration or lifecycle through resource tagging, And to support the commonly used resources that causes AWS Bills to grow even though the cloud resources is not needed anymore.
No more story telling!
For complete documentation and project link, you can proceed directly to my GitHub repo below.
This repo is open for contributors!!! Some documents and cloud resources are currently WORK-IN-PROGRESS

Usage

Just specify that tag and set your desired lifecycle for supported resources with CloudCycle Key. Below are the supported duration.
SuffixesDetailSample Value
mMinutes60m
hHours2h
dDays7d

How does it works?

CloudCycle will get all the supported resources with a tagged key CloudCycle and it will simply compare the current time vs launch time of the supported resources with the specified key/value pair resource tag if the supported resources are valid for termination.
Below are the sample terraform code.
Specify ec2 lifecycle by 24 hours from it's launch date.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
data "aws_ami" "ubuntu" {
most_recent = true

filter {
name = "name"
values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"]
}

filter {
name = "virtualization-type"
values = ["hvm"]
}

owners = ["099720109477"] # Canonical
}

resource "aws_instance" "web" {
ami = data.aws_ami.ubuntu.id
instance_type = "t3.micro"

tags = {
CloudCycle = "1d" // This ec2 instance will be terminated within 24 hours from it's launch date.
}
}

Sample Terraform Deployment Usage

For deployment, you can refer to the github repo deployment page https://github.com/redopsbay/cloudcycle/blob/master/deploy/README.md

Benefit's

Sometimes it's better to let go than stay strong. Leaving up unused resources will incur costs.
And lastly, No nightmare's, No poverty! 😂🤣

Reference

Comments