Explore Amazon GuardDuty ECS Runtime Monitoring

In the cloud, we need to do lots of scans for thousands of vulnerabilities and threats in lots of different services. Some of them are related to static code scanning, dynamic code scanning, supply chain attack protection, API testing, etc. In addition to these, runtime protection and detecting threats in real-time is a challenging but required process for us. It’s important to know what is happening in my environment now, whether these are required processes or whether someone is attacking me or not.

In re:Invent 2023, the AWS team announced Amazon GuardDuty ECS Runtime Monitoring to detect potential runtime security issues in Amazon ECS clusters for AWS Fargate and Amazon EC2. As you can guess, AWS Fargate operations for GuardDuty Runtime Monitoring are mostly handled by AWS (agent configurations are handled by AWS, enabling is our responsibility). For the EC2, we need to set up our monitoring environment.

In this blog, we’ll use and test Amazon GuardDuty ECS Runtime Monitoring for EC2. Let’s start together! 💫

It’s so easy to enable Runtime Monitoring on Amazon GuardDuty. But I just want to be sure it’s detecting my findings or not in runtime. To test this, I need an isolated AWS account and GuardDuty tester environment. For testing, I use this and build the environment with CloudFormation stacks. After building we have an ECS cluster with two EC2s: Linux and Windows Target, a red team server for attacking, and a bastion host to connect. This is the architecture in the GitHub link:

For our EC2 targets in ECS, we can install the agent manually, use it as AMI for future servers or we can use AWS Systems Manager Documents. I decided to install my Linux target in ECS (It seems like there is no Windows support for now.) We will use the manual installation for this blog, but if you want to use AWS Systems Manager you can see this:

Before the installation, you need to do two things:

It’s important to know your servers. You need to read carefully the prerequisites for Amazon EC2 instance support from here. AWS recommends that you use the latest Amazon ECS-optimized AMIs (dated September 29, 2023 or later), or use Amazon ECS agent version v1.77.0.

You need to verify the GuardDuty security agent's RPM signature. You can follow this link to verify. It’s important to select the right region for the environment and add S3 access to your IAM role for EC2.

In the installation step, I encountered a kernel error, so I need to update it. Kernel updates can be critical for your environment, so I strongly recommended do this in your test environment.

After the kernel update, I need to reboot my server. You can use the uname -r command to check the version. After rebooting, I can successfully install the agent.

I’m checking the status of the agent. It seems fine.

This is a very easy step. You can enable it with only a click. If you are using EKS or Fargate, you can enable all of them. For me, runtime monitoring is enough.

After doing this, on the runtime coverage part, I’ve realized my coverage status is unhealthy and my agent does not report anything. So, what is happening here?😦

I forgot step 1 in this link, creating a VPC endpoint to communicate with the agent. You can follow all the steps in the link.

After creating and waiting a little, my agents are healthy and reporting! 🤭

For the ECS runtime coverage, I also see this:

With the script in the GitHub link, I can start the attack simulation successfully. After starting, I can get some findings on the Amazon GuardDuty console.

The last one is related to the ECS runtime monitoring and we got it 🤝! Others already exist in the GuardDuty findings with the old protection methods like malware protection, port scanning, etc. You can see all runtime findings from here. These are the details about the our finding:

 That’s all! Some useful links that I strongly recommend if you want to use this cool feature:

Thanks for reading! Stay safe in the cloud! 🌥