Use AWS Generative AI CDK constructs to speed up app development
Assemble and deploy the infrastructure for a RAG solution using AWS CDK for Python
Abhishek Gupta
Amazon Employee
Published Jun 26, 2024
In this blog, we will use the AWS Generative AI Constructs Library to deploy a complete RAG application composed of the following components:
- Knowledge Bases for Amazon Bedrock: This is the foundation for the RAG solution.
- OpenSearch Serverless collection: It supports the vector search collection type that provides similarity search capability.
- An S3 bucket: This will act as the data source for the Knowledge Base.
- AWS Lambda function (written in Python) along with an API Gateway that uses the RetrieveAndGenerate API to query the knowledge base and generate responses from the information it retrieves.
AWS Cloud Development Kit (AWS CDK) is an open-source software development framework for defining cloud infrastructure in code and provisioning it through AWS CloudFormation. You can use any of these supported programming languages (TypeScript, JavaScript, Python, Java, C#/.Net, and Go) to define reusable cloud components known as constructs, that represent one or more AWS CloudFormation resources and their configuration.
Constructs from the AWS Construct Library are categorized into three levels with each level offers an increasing level of abstraction.
- L1 constructs are the lowest-level construct and offer no abstraction. Each L1 construct maps directly to a single AWS CloudFormation resource.
- L2 constructs provide a higher-level abstraction along with helper methods for most resources that make it simpler to define resources.
- L3 constructs, also known as patterns, are the highest-level of abstraction, and are used to create entire AWS architectures for particular use cases in your application.
Amazon Bedrock is a fully managed service that makes high-performing foundation models (FMs) from leading AI startups and Amazon available for your use through a unified API. As far as AWS CDK is concerned, Bedrock does not support L2 constructs yet.
AWS Generative AI Constructs Library is an open-source extension of the AWS Cloud Development Kit (AWS CDK) and fills the gap by providing L2 and L3 constructs to help developers build generative AI solutions using pattern-based definitions for their architecture.
Let's get started!
Make sure to have the following setup and configured: AWS CDK, Python, Docker.
Also, configure and set up Amazon Bedrock, including requesting access to the Foundation Model(s). The application uses Amazon Titan Text Embeddings model along with the Anthropic Claude 3 Sonnet as the LLM.
Start by cloning the GitHub repo:
Make sure to use the docs folder to add the documents (PDFs etc.) that you want to be used as a source for QnA. These will be automatically uploaded to a S3 bucket that will serve as the data source for Knowledge Base. Here is the list of document types supported by Knowledge Base.
Activate virtual env and install dependencies:
Deploy the stack:
It might take at least five minutes for the deployment to complete. Once done, verify the CloudFormation stack in the AWS Console.
Navigate to the Knowledge Base and sync the data source with the OpenSearch Serverless vector index.
For QnA, you can use the API Gateway endpoint. Send queries via a POST request - make sure you add
/query at the end of the API Gateway URL. I had uploaded the Amazon 2022 shareholder document and asked the following question: "What is Amazon doing in the field of generative AI?"Below is the result returned by the Lambda function based on response from
RetrieveAndGenerate API:Once you are done, don't forget to delete the stack, which in turn will delete all the components:
Here is a quick walkthrough of the different components in the stack - refer to the complete code here.
The Knowledge Base along with the S3 bucket, uploaded document(s), and the data source configuration:
The Lambda function (along with necessary IAM policies) that invokes the RetrieveAndGenerate API - refer to the code here.
Finally, the API Gateway with a
query endpoint:The patterns defined in AWS Generative AI CDK Constructs are high level, multi-service abstractions of AWS CDK constructs that provide well-architected patterns for quickly defining solutions in code to create predictable and repeatable infrastructure. I showed you a Python example, but there is TypeScript support as well.
I also encourage you to check out the samples repository that includes a collection of functional use case implementations to demonstrate the usage of AWS Generative AI CDK Constructs.
Happy Building!
Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.