logo
Build a Streamlit app with LangChain and Amazon Bedrock

Build a Streamlit app with LangChain and Amazon Bedrock

Use ElastiCache Serverless Redis for chat history, deploy to EKS and manage permissions with EKS Pod Identity

AG
Abhishek Gupta
Amazon Employee
Published Jan 12, 2024
Last Modified Jan 22, 2024

It's one thing to build powerful machine learning models and another thing to be able to make them useful. A big part of it is to be able to build applications to expose its features for end users. Popular examples include ChatGPT, Midjourney etc.
Streamlit is an open-source Python library which makes it easy to build web applications for machine learning and data science. It has a set of rich APIs for visual components including several chat elements, making it quite convenient to build conversational agents or chatbots, especially when combined with LLMs (Large Language Models).
And that's the example for this blog post as well - A Streamlit based chatbot, deployed to a Kubernetes cluster on Amazon EKS. But that's not all!
We will use Streamlit with LangChain, which is a framework for developing applications powered by language models. The nice thing about LangChain is that it supports many platforms and LLMs, including Amazon Bedrock (which will be used for our application).
A key part of chat applications is the ability to refer to historical conversation(s) - at least within a certain time-frame (window). In LangChain, this is referred to as Memory. Just like LLMs, you can plug-in different systems to work as the memory component of a LangChain application. This includes Redis, which is a great choice for this use case since it's an high performance in-memory database with flexible data structures. Redis is already a preferred choice for real-time applications (including chat) combined with Pub/Sub and WebSocket. This application will use Amazon ElastiCache Serverless for Redis, an option that simplifies cache management and scales instantly. This was announced at re:Invent 2023, so let's explore while it's still fresh!
To be honest, the application can be deployed on other compute options such as Amazon ECS, but I figured since it needs to invoke Amazon Bedrock, it's a good opportunity to also cover how to use EKS Pod Identity (also announced at re:Invent 2023!!)
GitHub repository for the app - https://github.com/build-on-aws/streamlit-langchain-chatbot-bedrock-redis-memory
Here is a simplified, high-level diagram:
architecture
High level architecture
Let's go!!

Clone the GitHub repository:
1
2
git clone https://github.com/build-on-aws/streamlit-langchain-chatbot-bedrock-redis-memory
cd streamlit-langchain-chatbot-bedrock-redis-memory
Create an ECR repository:
1
2
3
4
5
6
7
export REPO_NAME=streamlit-chat
export REGION=<AWS region>

ACCOUNT_ID=$(aws sts get-caller-identity --query "Account" --output text)

aws ecr get-login-password --region $REGION | docker login --username AWS --password-stdin $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com
aws ecr create-repository --repository-name $REPO_NAME
Create the Docker image and push it to ECR:
1
2
3
docker build -t $REPO_NAME .
docker tag $REPO_NAME:latest $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/$REPO_NAME:latest
docker push $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/$REPO_NAME:latest

Update the app.yaml file:
  • Enter the ECR docker image info
  • In the Redis connection string format, enter the Elasticache username and password along with the endpoint.
Deploy the application:
1
kubectl apply -f app.yaml
To check logs: kubectl logs -f -l=app=streamlit-chat

To access the application:
1
kubectl port-forward deployment/streamlit-chat 8080:8501
Navigate to http://localhost:8080 using your browser and start chatting! The application uses the Anthropic Claude model on Amazon Bedrock as the LLM and Elasticache Serverless instance to persist the chat messages exchanged during a particular session.
chatbot
Streamlit Chatbot

To better understand what's going, you can use redis-cli to access the Elasticache Redis instance from EC2 (or Cloud9) and introspect the data structure used by LangChain for storing chat history.
1
keys *
Don't run keys * in a production Redis instance - this is just for demonstration purposes
You should see a key similar to this - "message_store:d5f8c546-71cd-4c26-bafb-73af13a764a5" (the name will differ in your case).
Check it's type: type message_store:d5f8c546-71cd-4c26-bafb-73af13a764a5 - you will notice that it's a Redis List.
To check List contents, use the LRANGE command:
1
LRANGE message_store:d5f8c546-71cd-4c26-bafb-73af13a764a5 0 10
You should see a similar output:
1
2
3
4
5
6
7
8
1) "{\"type\": \"ai\", \"data\": {\"content\": \" Yes, your name is Abhishek.\", \"additional_kwargs\": {}, \"type\": \"ai\", \"example\": false}}"
2) "{\"type\": \"human\", \"data\": {\"content\": \"Thanks! But do you still remember my name?\", \"additional_kwargs\": {}, \"type\": \"human\", \"example\": false}}"
3) "{\"type\": \"ai\", \"data\": {\"content\": \" Cloud computing enables convenient, on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and released with minimal management effort.\", \"additional_kwargs\": {}, \"type\": \"ai\", \"example\": false}}"
4) "{\"type\": \"human\", \"data\": {\"content\": \"Tell me about Cloud computing in one line\", \"additional_kwargs\": {}, \"type\": \"human\", \"example\": false}}"
5) "{\"type\": \"ai\", \"data\": {\"content\": \" Nice to meet you, Abhishek!\", \"additional_kwargs\": {}, \"type\": \"ai\", \"example\": false}}"
6) "{\"type\": \"human\", \"data\": {\"content\": \"Nice, my name is Abhishek\", \"additional_kwargs\": {}, \"type\": \"human\", \"example\": false}}"
7) "{\"type\": \"ai\", \"data\": {\"content\": \" My name is Claude.\", \"additional_kwargs\": {}, \"type\": \"ai\", \"example\": false}}"
8) "{\"type\": \"human\", \"data\": {\"content\": \"Hi what's your name?\", \"additional_kwargs\": {}, \"type\": \"human\", \"example\": false}}"
Basically, the Redis memory component for LangChain persists the messages as a List and passes its contents as additional context with every message.

To be completely honest, I am not a Python developer (mostly use Go, Java or sometimes Rust), but found Streamlit relatively easy to start with, except for some of the session related nuances.
I figured out that for each conversation, the entire Streamlit app is executed (this was a little unexpected coming from a backend dev background). That's when I moved the chat ID (kind of unique session ID for each conversation) to the Streamlit session state, and things worked.
This is also used as part of the name of the Redis List that stores the conversation (message_store:<session_id>) - each Redis List is mapped to the Streamlist session. I also found the Streamlit component based approach to be quite intuitive and it pretty extensive as well.
I was wondering if there are similar solutions in Go 🤔 If you know of something, do let me know.
Happy building!
 

Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.