GitHub Actions workflow for deploying a scheduled task using ECS and EventBridge Scheduler

GitHub Actions workflow for deploying a scheduled task using AWS ECS and EventBridge Scheduler

Published Jan 23, 2024
Cron jobs are repetitive tasks scheduled to run periodically at fixed times, dates, or intervals. It typically automates system maintenance or administration. Some workloads that are still running on non-containerized platforms (VMs, bare metal, etc.) are suitable to be moved to Serveless with multiple alternatives, depending on the context of each task.
Considering AWS services, for most of the options EventBridge Scheduler will be used to manage tasks as it is capable of invoking lots of AWS services. One of them is invoking a containerized application, or ECS task.
Amazon EventBridge Scheduler is a serverless scheduler that allows you to create, run, and manage tasks from one central, managed service. Highly scalable, EventBridge Scheduler allows you to schedule millions of tasks that can invoke more than 270 AWS services and over 6,000 API operations. Without the need to provision and manage infrastructure, or integrate with multiple services, EventBridge Scheduler provides you with the ability to deliver schedules at scale and reduce maintenance costs.
(EventBridge Scheduler is recommended to be used instead of CloudWatch Scheduler with EventBridge rules)
I worked on an application running on EC2 to ECS that is still using its cron jobs. Cron jobs were migrated to EventBridge Scheduler. Our CI/CD uses GitHub Actions and Terraform. AWS provides actions that can create and deploy an ECS task definition (the container blueprint) to an ECS service, but there is no action to deploy an ECS task to the EventBridge Scheduler, as the cron task is not executed under a service.
To deploy the new code we have to write some code to the GitHub Action and I think it might benefit others in a similar context. We use Terraform as Infrastructure as a Code, so keep this in mind if you need to adapt to your IaaS solution.
There will be the full Yaml file here but I will comment parts of it separately afterwards.
This pipeline will create the scheduled task definition, check out the IaaS repository, change the Scheduler task definition ARN and create a PR in the IaaS repository. We are using that also as a way to have approval to deploy to Production, but it can be automated if needed.
Let's comment on some parts:
It is good to separate the image creation from the deployment. This input is required assuming the image was created and published to the registry. This promotes reusability and flexibility.
Environment is a required input and this pipeline can be executed against any environment you defined in your repository.
Although the action name is "deploy task definition" it will create the task, but not deploy, as this is only possible when you provide a service input (by the time this article is being written). But we are not deploying to a service though, so the action will only create the task definition, but the EventBridge Scheduler remains calling the same task definition it was invoking before the creation of this task definition.
Using AWS CLI was an alternative we considered, but changing the target of the EventBridge scheduler becomes a little bit confusing and brings some cognitive complexity in case we need to change something. We decided then to fetch the IaaS repository and control the task definition version to be the target of the scheduler in Terraform code, so we could also be sure any dependency that the target change could have would be managed by Terraform, instead of another CLI change in this pipeline. We checkout the IaaS repo and save the path as ./working-path to keep the workspace clean. The name is your choice.
This is where we use sed to search and replace the ARN in the terraform code. We scape the ARN before applying sed to not mess with the search regexp.
The terraform code expected to be changed will be something like this:
Links:

Comments