logo
Menu
Project Management Dash-Boarding with Bedrock LLMs and RAG

Project Management Dash-Boarding with Bedrock LLMs and RAG

How I built a simple, minimalistic Project Management Dashboard application with Bedrock and Qdrant vector db

retrival-augmented-generationllmgenerative-ai-on-awsamazon-bedrockrag
Tanmay Baxi
Published Sep 4, 2024
Last Modified Sep 5, 2024
After building an AI driven, heavy data analysis tool like Apexon-Compass (Refer to the blog if you have not read it already) with a lot of efforts, I wanted to check if LLMs can do that work better or at least minimalistic way.
In this blog post, I will guide you through the steps I took, to build a dynamic project manage dashboard application. The dashboard is powered by a Retrieval-Augmented Generation (RAG) architecture and Large Language Models (LLMs).

Key Prerequisites

Before diving in, make sure you have the following:
  1. AWS Account with appropriate permissions to use Amazon Bedrock and some decent credits to work with.
  2. Basic knowledge of Python, LangChain and familiarity with AWS services.
  3. Understanding of Docker (optional but recommended).

Understanding the Architecture

Here are the key components of the solution architecture:
  1. AirByte: Key component for data ingestion. As this tool is more of management oriented, we don't need to send it real time. I used AirByte to collect data on nightly basis.
  2. S3 Buckets: Used S3 to park the raw data collected from AirByte.
  3. Lambda and CloudWatch: CloudWatch triggers were used to trigger Lambdas to collect the raw data that will be converted into VectorDB.
  4. LangChain: A very critical framework to build any production grade LLM based applications.
  5. Amazon Bedrock: Model as a Service of AWS.
  6. Qdrant DB: A vector search database optimized for storing and retrieving high-dimensional data such as embeddings.
  7. ECS: A critical component of the architecture to deploy AirByte and Qdrant DB. Also, I used it to deploy the inference point web application built using Streamlit.

Basic Architecture Diagram:

Flow/Architecture Diagram
Note: Drawing this basic diagram was not the intent. It's due to time crunch.

How to set up AirByte on ECS:

  • Create ECS cluster:
  • Create AirByte Scheduler Task Definition:
  • Register the Task Definition:
  • Create AirByte Web App task definition:
  • Register the task definition:
  • Create ECS Service and RDS PGSQL instance:
  • Feel free to set up Autoscaling, Monitoring etc. but as this was POC, I did not set up those things.

How to set up Qdrant in a Container:

  • Create ECS Cluster:
  • Create Task Definition for Qdrant:
  • Create the EFS File System:
  • Note the FileSystemId from the output, as you will need it for the ECS task definition.
  • Create a mount target for the EFS:
  • Create Task Definition:
  • Create the ECS Service:
  • Feel free to set up Autoscaling, Monitoring etc. but as this was POC, I did not set up those things.

Write Lambda function to fetch data and create vectors:

  • AirByte automatically puts data into S3 buckets. You can set up a CloudWatch trigger for new file arrival event and trigger your lambdas:
  • I created 5 different buckets, i.e., GitHub/BitBucket, Jira, SonarQube, Jenkins and Google Sheets. And also setup alerts/triggers accordingly. Because I am not so much hands-on for Terraform and all, I did all the activities manually.
  • Lambda for Jira Data Processing:
    • I am not sharing steps on how to upload heavy libs like LangChain etc. on how to use them in lambda. I am assuming it's easy to google those things. (LOL) - Feel free to put a comment and I will add the location and resources from where I learned it.
  • Prompt:
  • Define the retriever that will interact with Qdrant DB extractor:
  • You can build a Streamlit application and deploy the same on ECS. Again not sharing the code as it will expose my poor front-end skills. I am guessing you are not here to laugh on my frontend skills (LOL). - Feel free to put a comment and I will share the code for it as well.
Footnotes:
  1. Unfortunately my GitHub account has some issues, I have also lost my recovery codes. So not able to give you full access or publish diagram. Will update the same soon.
  2. After getting a good amount AWS Billing - I stopped the overall deployed solution. This solution can easily be built using Ollama - Deployed on EC2 instance as well.
  3. Thanks to AWS Community Builder program which gives AWS credits for certain services I did not go broke (LOL) with the Claude Bills.

Conclusion

  1. Getting actionable insights into data has become very easy with LLMs. Make use of them.
  2. RAG and LLM with Bedrock is extremely easy, fast, extremely powerful.
    • My next task is to explore some things like Bedrock Knowledge base and all to see if that can outperform our own solution. If you want to contribute funds for it - Please contact me ;-)
  3. If you don't have AWS credits like me and don't want to spend a lot of money - use Ollama. Host it on a cheap EC2 machine. It works with CPUs as well. Minimalistic but experiments has to be minimalistic in my thoughts.
  4. Next big wave of change is #AIAgents and #AgenticBots. That will be my next big exploration.
  5. AI won't take your job but if you are not using AI - you will definitely be left behind and will become replaceable.
     

Comments