AWS Logo
Menu
Converting PDFs to Images on AWS Lambda ⚡️ with Poppler

Converting PDFs to Images on AWS Lambda ⚡️ with Poppler

A step-by-step guide to packaging native binaries, like Poppler, into an AWS Lambda layer for PDF processing.

Alexis Philippart de Foy
Published Nov 10, 2024

A Simple Plan… That Got Complicated 😱😱😱

I recently started working on building an AI Agent using Amazon Bedrock. As part of its capabilities, one of the agent's tasks was to convert PDF documents into images. My initial plan was to just grab a Node.js library for PDF processing, throw it into an AWS Lambda function, and call it a day. Easy, right?
But as I quickly learned, document processing on serverless architectures brings its own set of challenges. Especially when you need native binaries on Lambda! Many libraries assume you’re on a standard server with all dependencies at your fingertips. In a Lambda environment, however, you’re working with limited permissions, disk space, and custom deployment layers. The solution turned out to be building a custom Lambda layer with Poppler, a powerful PDF processing library that can convert PDFs into images.
In this post, I’ll walk you through the process I followed to:
  1. Package the Poppler utility I needed (pdftocairo) for deployment on Lambda.
  2. Create an AWS Lambda function that uses this utility (node-poppler) to convert PDFs into images.
  3. Set up AWS SAM to streamline deployment.
The final solution is a reusable Lambda layer that you can add to your own document processing projects. I’ve shared the full source code on GitHub so you can dive right in!

What is Poppler?

Poppler is a powerful open-source library for handling PDF documents. It provides a set of tools for viewing, manipulating, and converting PDF files. One of its utilities, pdftocairo, can convert PDF pages directly into images (e.g., PNG, JPEG) which makes it ideal for tasks requiring document-to-image processing.
For my use case, I needed an efficient way to convert PDF files into images so that I could pass those images to AI models for further analysis. While Poppler isn’t inherently serverless-ready, it can be packaged into a custom Lambda layer, allowing us to leverage its capabilities in a serverless environment.
In the following sections, I’ll walk you through the steps to build a Poppler-based Lambda layer, deploy it with AWS SAM, and test it with a sample PDF.

Building Poppler Binaries for the Lambda Layer 🏗️

Since AWS Lambda requires native binaries for ARM64 or x86_64 architecture, we need to build Poppler in a way that fits Lambda's runtime constraints. I’ll be building the Poppler binaries using Docker, which helps ensure compatibility regardless of the host machine’s architecture.

Dockerfile for Building Poppler on Amazon Linux 2

Here’s the Dockerfile I used to build Poppler and package it into a zip file compatible with AWS Lambda.
This Dockerfile builds an Amazon Linux 2 image to package the Poppler PDF utilities and required fonts into a ZIP file for deployment in environments like AWS Lambda. It performs the following steps:
  1. Sets up the Environment: Configures the target platform and package directory.
  2. Installs Dependencies: Installs essential libraries, Poppler utilities (for PDF manipulation), and common fonts (DejaVu and Liberation).
  3. Packages Libraries and Fonts: Copies the Poppler binaries, related libraries, and font files into a custom directory structure.
  4. Configures Fonts: Copies a custom fonts.conf file to define font fallbacks.
  5. Prepares the ZIP for Deployment: Compresses the packaged files into a ZIP, ready to be used in Lambda or similar environments.
The ZIP file is saved to a specified location for easy access and deployment.
Note: Packaging fonts and a fonts config file along with the Lambda layer ensures us that even if the fonts used in the PDFs to be converted are not available on the Lambda runtime, another font will still be used to perform the conversion.

Building the Lambda layer

Let's now create the Poppler layer.
In our scenario we will be building the layer for ARM architecture. You can run the following commands to generate the lambda layer for Poppler.
Note: These commands suppose you have downloaded the source code from Github and you are running those commands from the root directory of the source code project.
In case you wish to build the layer for X86_64 architecture, you can adapt the commands with following parameters:
  • Platform: linux/amd64
  • Target Platform: amd64
  • Image Name: poppler-lambda-layer-x86-64

Deploying with AWS SAM 🚀

With the Poppler layer ready, let’s have a look at the SAM project included in the Github repository. We'll use it to deploy our Lambda function with a layer containing our Poppler utilities.
The SAM project structure is as follows:

The template.yaml file

This is the SAM template that defines the infrastructure for our serverless application. It’s essentially a configuration file that specifies the resources SAM should create and manage. The key components of our APP are:
  • Lambda Function (PdfProcessingFunction): Configured to run our PDF-to-image conversion code, specifying the handler file (index.mjs), runtime, architecture, and timeout.
  • Lambda Layer (PopplerLayer): Points to the release-amd64.zip file in layers/poppler/, making the Poppler binaries available to the Lambda function.

The layers/poppler/release-arm64.zip file

This zip file is the one we have generated previously with the Dockerfile presented earlier.
It. contains the Poppler binaries (pdftocairo and its dependencies) packaged specifically for the ARM architecture. By uploading this file as a Lambda layer, we can access Poppler’s PDF conversion utilities in the Lambda environment.

The src/Function/index.mjs file

This is the main Lambda handler file that contains the application logic for our function. When a request comes in, it will:
  1. Decode a base64-encoded PDF from the incoming request.
  2. Saving the PDF as a temporary file in /tmp.
  3. Use pdftocairo (via node-poppler) to convert the PDF's first page into a PNG image.
  4. Return the PNG image as a base64-encoded response.

Running the deploy commands

To deploy our application, run the following commands:
This will guide you through the deployment steps.

Testing the application 🧪

I order to test the application we will need to send a POST request on our API including a base64 encoded version of a PDF. It will then return the first page of that PDF in PNG format as a base64 encoded string.
Make sure you have a pdf called sample.pdf in your local directory. Add your API endpoint (you can find it in the outputs of your SAM deployment) and run the following command:
You should now see a new file in your local directory called output.png. This file contains an image version of the first page of your sample.pdf file.

Conclusion

Converting PDFs into images was more challenging than I initially expected, but it allowed me to unlock the power of serverless document processing. By building the layer, setting up the Lambda function with SAM, and testing it end-to-end, we now have a reusable solution for converting PDFs to images on AWS Lambda. Hopefully, this guide saves you a few steps if you’re setting up something similar. Happy coding! 🎉
 

Comments