AWS Logo
Menu
AWS Cost: VPC Flow Logs for Department Chargebacks

AWS Cost: VPC Flow Logs for Department Chargebacks

Learn how to implement precise cost allocation using VPC Flow Logs in AWS. Master setup, tracking, tagging, and reporting for effective department chargebacks.

Dan Sims
Amazon Employee
Published May 28, 2025
Last Modified May 30, 2025
Managing data processing costs in a multi-account AWS environment can be challenging, especially when dealing with shared network services. Transit Gateway flow logs provide a powerful solution for tracking and allocating these costs across different accounts. In this post, we'll explore how to set up Transit Gateway flow logs to Amazon S3, enabling you to effectively monitor and analyze network traffic patterns. This setup forms the foundation for implementing cost allocation strategies and understanding traffic flows across your AWS organization. Through Amazon QuickSight integration, you'll be able to create dynamic reports that visualize data transfer costs in various units (megabytes, gigabytes, terabytes, and petabytes) and analyze usage patterns across different time periods (daily, weekly, monthly, quarterly, and yearly). By following these best practices for flow log configuration, you'll be better equipped to manage your network resources and implement accurate cost chargebacks across your accounts, with the flexibility to view and analyze the data in ways that best suit your organization's needs.
Based on the AWS blog post "Using AWS Transit Gateway Flow Logs to chargeback data processing costs in a multi-account environment" (February 14, 2024), here's a summary of the process for getting Transit Gateway flow logs to S3:
  1. Enable Transit Gateway Flow Logs:
  • Configure the logs to be published to an Amazon S3 bucket
  • If you have multiple transit gateways, enable flow logs for each and send them to a single S3 bucket
  • Avoid using S3 prefixes for the chargeback process
  • Use the default format for log records
  • Set the format to Parquet to reduce storage space and improve query times
  • Consider partitioning flow logs per hour if traffic volume is large
  1. Key Configuration Points:
  • When setting up the destination, choose Amazon S3 (rather than CloudWatch Logs or Kinesis Data Firehose)
  • For multiple transit gateways in different regions, consolidate logs to a single bucket
  • Use the default log record format unless specific fields are needed
  • The Parquet format is recommended for efficiency
  1. Storage Considerations:
  • Create an S3 bucket to store the flow logs
  • Ensure proper permissions are set up for the bucket
  • It may take up to 24 hours for logs to start appearing in the bucket
This setup process is the foundation for enabling transit gateway flow log analysis and can be used for various purposes, including cost allocation and traffic analysis across multiple AWS accounts.
After setting up your Transit Gateway flow logs in Amazon S3 and configuring your Cost and Usage Reports, you'll need to analyze the data to perform accurate cost allocation. The following SQL query demonstrates how to calculate usage percentages and chargeback costs across different time granularities. This query combines data from your Transit Gateway flow logs and CUR reports, allowing you to track data transfer volumes and associated costs by account. The query is designed to work with Amazon QuickSight to create dynamic visualizations that can be adjusted based on your preferred time periods and data measurement units.

SQL to get daily chargeback to account- 2 versions

In the queries there are a few things to change
In daily_totals and tgw_daily_data
In cur_daily_costs

For use to see costs in a Athena query or to use in another BI tool.

For use to see costs when creating a quicksight report- chargeback calculation will be done in quicksight
 

Create Parameters and Calculated Fields in QuickSight

Create Parameter

Name
Select Single Value
After Saving select create Control

Create Control

Name
Select the dropdown option and add the following

Create Parameter

Name
Select Single Value
After Saving select create Control

Create Control

Name
Select the dropdown option and add the following

Create Calculated Field

Name

Create Calculated Field

Name
Formula

Create Calculated Field

Name
Formula
 

Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.

Comments