Finding Available and Cost Effective GPU Instances
Quickly discover what Accelerated Compute instances are available for short-term reservation with Amazon EC2 Capacity Blocks for ML
Kareem Abdol-Hamid
Amazon Employee
Published Apr 4, 2025
In my work with startups, I often have customers urgently needing access to powerful GPU instances (accelerated compute) in short time frames. Customers often feel they aren't able to find instances they need in an appropriate time frame and would like to save on costs where possible. However, there is likely more capacity available than you think!
This guide will provide some quick tips and some suggested code to easily search across AWS for available instances, to simplify decision making, and reduce risk. Specifically we're going to discuss Amazon EC2 Capacity Blocks for ML and how to effectively leverage them for access to short term powerful compute for Training Jobs, PoCs, and more.
Let's dive in.
Often customers want instances in regions they're working in or regions they're familiar with but for many quick training or PoC workloads that is not strictly necessary. What is key is what instances are available and at what price. Similar to the mentality around Spot, this changes the paradigm you work around when acquiring instances.
Think of Capacity Blocks as a hotel reservation, if you are willing stay at a different time range, are open do a variety of rooms, and are open to different locations you will have more options at more cost effective prices.
Capacity blocks are the same. If you consider a variety of regions, different start/end date ranges, and different instance types, you will have more choice over when and how you get access to these instances at more flexible prices.
Note: Prices are set per region, more cost effective options are purely region dependent. See the pricing page for more.
Alright so how do we do this? Go through each region manually and check the prices? To be honest, when I first started receiving customer requests in this area I did. But if I've learned anything at AWS, there's an API for everything.
Note there is not actually an API for everything.
The AWS Boto3 SDK makes it really easy to check CB availability in a region, for multiple instances, across different ranges. So all we need to do is extend that capability to all regions.
Below is a scrappy script I threw together and use constantly to answer this question for customers.
Prerequisites for my script are awscli and python. I've validated it on an Intel Mac system, but it should work for linux and windows similarly.
First create a python environment
python3 -m venv .venvsource .venv/bin/activateThen install boto3
pip install boto3Then copy the code below into a file called cb.py.
Disclaimer time: As noted this script is scrappy. Use at your own risk and review the contents. This script does not create resources but it does search through all regions and look through availability. It works on mac, it requires boto3, and there are likely edge cases where it will fail. Adjust to your need. This is not AWS official guidance but a personal script I find useful
The script takes instance count, duration, start, end, and an order by date and outputs a list of available instances across regions. It then outputs to a CSV
Here's example usage:
python cb.py --duration-hours 24 --instance allHere's the first 1 row of that call at the time of running the script (out of 120):
With this information we can now more effectively make a decision on what instances to use.
Remember you can definitely adjust a script like this to your use case. Want to get a daily report? Put this in a lambda and use events to trigger it daily. Want an api? Use API Gateway to call adhoc for these reports. Really it can be very flexible and applied to your use case.
Now we can feel much more confident in having access to cost effective and powerful compute for our novel use cases!
Happy hunting!
Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.