Select your cookie preferences

We use essential cookies and similar tools that are necessary to provide our site and services. We use performance cookies to collect anonymous statistics, so we can understand how customers use our site and make improvements. Essential cookies cannot be deactivated, but you can choose “Customize” or “Decline” to decline performance cookies.

If you agree, AWS and approved third parties will also use cookies to provide useful site features, remember your preferences, and display relevant content, including relevant advertising. To accept or decline all non-essential cookies, choose “Accept” or “Decline.” To make more detailed choices, choose “Customize.”

AWS Logo
Menu

Deploy any AWS Lambda Layer within a Minute

Creating AWS Lambda layers can be a tedious, manual process prone to errors. Do it within a minute with this bash script.

lambdalambda-layersbash
Qinjie Zhang
Amazon Employee
Published Feb 3, 2025
Last Modified Feb 4, 2025
Comments

Introduction

AWS Lambda layers allow you to bundle your code and dependencies into a single file, which can then be reused across multiple functions. However, deployment of lambda function involves several manual steps, which can be time-consuming and error-prone.
This blog post shares a convenient script in GitHub repo qinjie/aws-create-python-lambda-layer to streamline the process. It is capable of creating a layer by handling both custom source folders and requirements files.

Steps in Creating a Lambda Layer

Here are the steps involved in creating a lambda layer. They are manual steps which are error-prone and time-consuming. It is not suitable for automation and CI/CD workflows either.
  1. Identify the libraries and the files to be included in the layer. Create a requirements.txt file which contains all libraries.
  2. Create a folder structure which matches the runtime requirements, e.g. `python/lib/python3.12/site-packages/` .
  3. Install the libraries to the appropriate folder.
  4. Compress the root folder python as a zip file.
  5. Create a lambda layer using the zip file.

Script to Rescue

The script can be found in GitHub repo qinjie/aws-create-python-lambda-layer .
It is a Python-based tool designed to generate zip files that can be used as AWS Lambda layers. It offers 2 methods for creating a layer:
  1. Using a Source Folder: If you have custom code or files to be included in the layer, you can place them together with a requirements.txt file in a single folder. The script will include this folder in the zip file.
  2. Using a Requirement File: You specify the dependencies in a requirements.txt file. When run, the script will install these dependencies into a resulting zip file.
The script supports customization through environment variables, allowing you to tailor the behavior to suit your specific needs.
  • PYTHON_BASE_VERSION: base version of Python
  • LAYER_NAME: name of the layer to be deployed
  • LAYER_DESCRIPTION: description of the layer to be deployed
If LAYER_NAME and/or LAYER_DESCRIPTION are provided, the script will attempt to deploy the layer using AWS CLI.

How to Use the Script?

Step 1: Prepare Your Environment

Ensure that you have the following installed on your system:
  • Python: The bash script uses Python, specifically virtualenv and pip modules, to create a virtual environment and install python libraries.
  • Other Python Interpreters: To create a Lambda of a particular Python version, we need to have that Python interpreter installed locally too. This is because `virtualenv` use it to replicate a Python virtual environment. For example, to create a layer of python3.13, we need to have python3.13 installed locally.
  • AWS CLI: To publish layers directly to AWS, you'll need the AWS Command Line Interface installed and configured.
Install both virtualenv and optional awscli , and setup AWS CLI credentials by following this guide.
For Ubuntu,
1
sudo apt install python3-virtualenv awscli
For Mac,
1
2
pip3 install virtualenv
brew install awscli
For Mac, use brew to install other Python interpreter locally. For example, following commands install both Python3.12 and Python3.13 on Mac.
1
2
brew install python@3.12
brew install python@3.13

Step 2: Clone or Download the Repository

Clone the GitHub repository or downloading the script directly. The repository contains the script along with example layers in the useful-layers directory.
1
2
git clone https://github.com/qinjie/aws-create-python-lambda-layer
cd aws-create-python-lambda-layer

Step 3: Customization via Environment Variables

You can customize the behavior of our script using the following environment variables:
  • PYTHON_VERSION: Specifies the version of Python to use. Defaults to 3.13.
  • LAYER_NAME: Sets a custom name for your Lambda layer. If it is not provided, the script will not run AWS CLI to deploy the layer.
  • LAYER_DESCRIPTION: Adds a description for your layer (default is "A sample Lambda layer")
1
2
3
export PYTHON_VERSION=3.11
export LAYER_NAME="my-sample-lambda-layer"
export LAYER_DESCRIPTION="Sample description for lambda layer"

Step 4: Package (and Deploy) the Layer

Option 1: Using a Source Folder

If you have custom files that you want to include in your Lambda layer, place them in a folder and run the script with the folder name as an argument. The script will create a zip file named myfolder.zip in the same directory.
  1. Create a folder (e.g., myfolder) containing all the files you wish to include.
  2. Navigate to the directory where our script is located.
  3. Run the script with the name of your folder as an argument.
1
./create_lambda_layer.sh myfolder

Option 2: Using a Requirements File

If you're working with Python dependencies, provide a requirements.txt file listing the libraries. The script will install these dependencies and include them in requirements.zip.
  1. Create a requirements.txt file listing all the dependencies your layer needs.
  2. Navigate to the directory where our script is located.
  3. Run the script without any arguments (since it will read from requirements.txt by default).
1
./create_lambda_layer.sh

Step 5: (Optional) Manual Deploy via AWS Console

Upload the generated zip file through the AWS Management Console.

Conclusion

This blog post demonstrates how developers can utilize a script to streamline their AWS Lambda layer workflow. Whether working with Python libraries or custom files, the tool helps save time and maintain consistency in your projects.
Reference:

Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.

Comments

Comments

Log in to comment