How S3 Object Lock affects the AWS Config, AWS CloudTrail and AWS Control Tower

How S3 Object Lock affects the AWS Config, AWS CloudTrail and AWS Control Tower

When we put a file to a S3 bucket, anyone with sufficient permissions can delete that file. This may be an intended action or an accidental action. However, there can be certain files that we need to be retained for some time without ability for anyone to delete. This may be due to various reasons such as security, compliance or legal. For such scenarios, AWS S3 object lock can help us to achieve this securely without any additional management overhead due to custom solutions.

Published Oct 27, 2024

What is AWS S3 Object Lock?

When we put a file to a S3 bucket, anyone with sufficient permissions can delete that file. This may be an intended action or an accidental action. However, there can be certain files that we need to be retained for some time without ability for anyone to delete. This may be due to various reasons such as security, compliance or legal. For such scenarios, AWS S3 object lock can help us to achieve this securely without any additional management overhead due to custom solutions.
There are different methods that we can use to apply S3 object lock. Out of them two methods are with retention periods that can be specified, and the other method is manual action based. The two Retention modes with specified time are;
  • Governance
  • Compliance
The manual retention mode (manual lock) is known as;
  • Legal Hold

How Object Lock modes work?

Both Governance and Compliance modes keep the S3 Objects for the defined time periods. However, the way these two modes behave is different from each other. The Governance modes is flexible, and more forgiving compared to the Compliance mode. In both modes, you can specify the no. of days that you want the object to be retained. This can be specified either in days or years.
There are few Important points that we need to keep in mind which are common to all these modes;
  • To enable Object lock, you need to have Bucket Versioning enabled for that Bucket.
  • Once you enable the AWS S3 Object Lock, you cannot disable the Object Lock or Suspend the Versioning of the bucket.
  • Retention mode can be applied on Bucket Level as Default Retention Configuration or the Individual Object Level via APIs through AWS CLI, AWS SDK or AWS S3 Rest APIs.
  • If both bucket level and object level retention periods are mentioned, the object level retention period will take priority over bucket level configurations.
  • Individual object level configurations need to be mentioned at the PUT action.
  • Otherwise, the Bucket Level Retention configurations will be applied for the object once its placed inside the bucket.
  • If the Bucket Default retention was not set and the PutObject Action also did not have any retention configuration, the object will not be protected by the AWS S3 Object Lock.
  • A “Retain Until” metadata filed will be added to the metadata of the object which calculates the object retention expiry date from the date of object being uploaded to the S3.
  • Different versions of the same object can have different object lock retention periods.
  • Object lock configurations will be kept regardless of the S3 Bucket class and between any class transitions due to S3 lifecycle policies. Object delete markers will be created but the object will not be deleted due to action of bucket owner or the lifecycle policy.
  • Object Lock configuration for each object will be copied over in a S3 Replication job from your source bucket to the destination bucket, so that the object which are replicated in the destination bucket will have the Object lock configuration in the source bucket.
  • Object lock configurations will be applied to any new object that you put into an S3 bucket with Versioning and Object Lock enabled and configured. If you enabled object lock in an existing bucket and need the existing objects to be protected by the object lock configurations, you need to apply the object lock configurations individually using AWS S3 Batch jobs or re-upload the files to S3.
  • Legal hold will be kept until you manually remove the hold by yourself or anyone with permission to do so. You can do this through the console or through the APIs. Please note that you can’t enable Legal Hold at the moment of object upload through the console. Only after upload completes, you can configure these settings in the console.
  • In order to use the Object Lock with retention configurations, the PUT Object Action needs to provide additional header value known as “Content-MD5”. This header value should contain the MD5 digest (hash) of the file which you are uploading to the S3. AWS S3 will also calculate the digest of the file you upload and will only accept the object if and only if the two digests are same. If you are uploading through the console, this header value will be calculated for you at the time of upload.

Difference Between Governance and Compliance Retention Modes

There are a few differences which we need to be aware of when we are using the Governance or the Compliance retention modes.
  • You can change the retention period (days) for objects which have retention period already configured, only in the Governance mode. In Compliance mode you cannot reduce the retention days but you can increase it.
  • If the retention mode is set to Governance, you can delete the objects by overriding the lock if you have IAM permissions “s3:BypassGovernanceRetention” and the delete request should include “x-amz-bypass-governance-retention:true” in the request header.
  • If the retention mode used is compliance mode, then there is no way to delete the objects which are protected by the Compliance retention configuration. Not even the AWS Root Account can override this.
  • The only way to remove/delete bucket or the objects protected by the Compliance Retention configuration is to delete the AWS Account itself.
As you can see, the Compliance retention mode provides strict security and Governance Retention mode provides a bit more flexibility with security. Out of them compliance mode is the better choice but will not be the ideal choice for every use case.
Object cannot be deleted if the S3 has Object Lock feature configured correctly
Object cannot be deleted if the S3 has Object Lock feature configured correctly
For example, if you don’t understand or know the use case properly, it might be a good idea to start with the Governance Retention mode so that you can get more idea on the behavior and the requirements. Then with the time, change the mode to Compliance.
Compliance retention mode will be really beneficial when you have a organizational standard that you need to follow such as log files stored in S3 need to be kept for longer time periods without anyone's ability to change or delete them.

AWS Control Tower, AWS CloudTrail and AWS Config

As you may already know or read through my other articles, both AWS Control Tower and AWS Config are Governance tools which will help you to archive consolidated compliance throughout your entire AWS virtual data center which can span across multiple AWS Accounts.
If you are using AWS Control Tower, you will be using AWS CloudTrail and AWS Config underneath. Or you might just be using AWS CloudTrail and AWS Config standalone without AWS Control Tower. Regardless of the method, let’s see how these services will get affected if you intend to use AWS S3 Object Lock feature in any of the related S3 buckets with above 2 services.

How S3 Object Lock affects AWS CloudTrail and AWS Config

As mentioned above, when you put an object to Object Lock Enabled S3 with Default Retention enabled (either in compliance or governance modes), you need to provide “Content-MD5” header with the hash value. If this header and value is not provided in the PutObject request, S3 will deny the put action indicating that “Content-MD5” header is not present. Please note that you’ll not be able to see this error in CloudTrail as this is a Data Event. If you need to see or capture this event, you need to have CloudTrail Data Event trail configured with S3 write data events enabled.
When we enable S3 object lock and the Default Retention on S3 buckets where the CloudTrail and Config events are stored, AWS CloudTrail will behave as expected without any issues. However, AWS Config will not be able to deliver the log files anymore due to this. The reason behind this is that AWS CloudTrail sends the “Content-MD5” header when it injects the log files to the S3, therefore CloudTrail logs are accepted by the S3 bucket. AWS Config Log Delivery Channel does not add this header to its PutObject request at the moment. I got this verified with AWS Support on JULY 2024 via a Support Case. Due to this AWS Config log files are rejected by the S3. As per the AWS Support, this will be fixed in the future, but they do not have an ETA.
Since both of these services sends log files through the service themselves using Service-Linked Roles, we do not have an easy way to implement addition of the header to AWS Config Log Delivery Channel.

How will this affect the AWS Control Tower and what to consider

AWS Control Tower used both AWS CloudTrail and AWS Config as the backbone of its infrastructure solution. When you need to create or on-board a new AWS Account to your AWS Control Tower, AWS CloudTrail Organizational Channel will be added to the new Account and AWS Config recorder with the Delivery Channel will be created inside the Account. When this happens, the CloudTrail sub-step will be succeeded, and the AWS Config sub-step will fail due to above mentioned header issue. This is because when AWS Config Delivery Channel is configured, it will do a Write Access Check on S3. Thus, new Account onboarding will get halted for this AWS Control Tower.
If your account onboarding failed, DO NOT update the AWS Control Tower to this current status. If you do, this will cause the AWS Control Tower to go to an Error Drifted status, thus breaking the AWS Control Tower functions. Instead, remove the error-ed AWS Account from onboarding. You can do this by removing Provisioned Service Catalog Product. Only then update the AWS Control Tower if you need to.
AWS Service Catalog Provisioned Products
AWS Service Catalog Provisioned Products
Updating the Control Tower will not cause the Config Header issue to rise as all the AWS Config Delivery Channels are already configured and access check for the AWS Config Delivery Channel will not be checked again.

Workaround

Since this is causing issues when onboarding new accounts to the Control Tower, the only direct workaround is to temporarily disable the AWS S3 object lock default retention. This will not remove the object lock or the retention from the objects which were uploaded while the AWS S3 object lock default retention was enabled or specifically enforced with the “PutObject” request. Those objects will still have the object lock configuration in their metadata. Only the new file which get uploaded within the time frame of the retention disabled will not be protected. Once the new account onboarding is completed, you can re-enable the default retention in the mode you require again.
Disable Object Lock Default Retention
Disable Object Lock Default Retention
However, please note that in this case you’ll have to apply object lock retention configurations for the objects which got uploaded in the time which bucket object lock default retention configurations were disabled.

Conclusion

AWS S3 Object Lock can greatly help you to achieve required security compliance for your project/organization if you are using it in the correct way. Think twice and plan before you enable the Object Lock feature on how you should use it, whether you should use the Governance retention Mode, Compliance retention mode or just the Legal Hold.
Hope this article has helped you in your Cloud Journey. If you wish to read and learn in depth please follow the AWS Official Documentation here.
 

Comments