AI-Powered Medical Report Analyzer with AWS 🏥🔍

Important Disclaimer: The solution described in this post is designed to assist in understanding medical reports but should not replace professional medical advice. Always consult with your healthcare provider for interpretation of medical results and before making any health-related decisions.

The Frustrating Reality of Medical Reports 🥹

Picture this: It's a late afternoon, and you've just received an email with your latest blood test results. Excitement quickly turns to confusion as you open the report, only to be confronted by a dizzying array of numbers, acronyms, and medical jargon.

I found myself in this exact situation last month. There I was, Googling each result separately, trying to decipher what "elevated ALT" meant or why my vitamin D levels were flagged. It felt like I was doing a medical treasure hunt, piecing together information from various sources, all while wondering if I was interpreting everything correctly.

As I sat there, tab after tab open in my browser, a thought struck me: "There has to be a better way to do this". What if we could leverage the power of AI and serverless architecture to simplify this process for patients everywhere?

A Lightbulb Moment: The Birth of MedicalGenie 💡

Inspired by this personal frustration, I decided to embark on a project to make medical report interpretation more accessible and less time-consuming. The goal was simple: create a system where patients could upload their medical reports and receive an easy-to-understand summary, complete with explanations of each result and its potential implications.

In this post, I'll walk you through how I built MedicalGenie, a serverless application that uses AWS Step Functions, Lambda, and Amazon Bedrock to transform complex medical reports into patient-friendly insights. We'll explore:

How to orchestrate a multi-step workflow using AWS Step Functions
Leveraging Amazon Bedrock for AI-powered medical text interpretation
Securely storing and retrieving patient data with DynamoDB
Providing real-time updates to users via AppSync
Deploying the entire solution using AWS SAM

So, grab a cup of coffee (or your beverage of choice), and let's dive into how we can use AWS to turn medical jargon into clear, actionable insights for patients everywhere!

The Solution: Architecting MedicalGenie 🏗️

The heart of my solution is an AWS Step Functions state machine that orchestrates the entire process. Here's a breakdown of the main steps:

Trigger: An S3 event triggers the workflow when a new medical report is uploaded.
AI Analysis: A Lambda function invokes Amazon Bedrock to analyze the report content.
Data Storage: Results are stored in DynamoDB for future reference (this step is optional if you don't need to persist the data).
User Notification: Another Lambda function updates the user interface via AppSync Pub/Sub API.

Let's look at the Step Functions workflow:

This state machine ensures that each step of the process is executed in the correct order, with built-in error handling and retries.

Leveraging Amazon Bedrock for AI-Powered Analysis 🧠

Amazon Bedrock plays a crucial role in our solution by providing access to cutting-edge models from a range of leading companies. I used Claude 3.5 Sonnet from Anthropic to interpret my medical reports.

Claude - at the moment of writing this article - supports JPEG, PNG, GIF, and WebP image formats. The Step Function workflow is triggered by uploading a new file to S3. The first step of the flow is invoking a Lambda function that checks the file type (PDF, JPEG etc..), converts it to PNG where needed, and saves a local copy of the image. After that, we make an API call to Amazon Bedrock specifying the model id, number of output tokens, prompt, image file and media type. I crafted a detailed prompt for Bedrock that includes instructions on how to analyze the medical report.

Here is the function that invokes Bedrock:

The prompt I crafted for Bedrock is designed to guide the AI in providing a comprehensive and patient-friendly analysis of medical reports. Let's break down the key components of the prompt:

Step-by-step analysis and instruction: The prompt begins by providing a clear set of instructions for the AI to follow: a. Examine the image and extract all medical test results. b. For each result, provide detailed information including the test name, patient's result, normal range, explanation of the test, classification of the result, and health recommendations. c. Identify correlations between different test results. d. Provide an overall health summary based on the results. This step-by-step approach ensures that the AI thoroughly analyzes all aspects of the medical report in a systematic manner.
Organized thinking and comprehensive analysis: Before presenting the final output, the AI is instructed to organize its analysis within <medical_interpretation> tags. This step serves to: a. List each test result separately with its details. b. Explicitly classify results as within, above, or below normal range. c. Brainstorm explanations for abnormal results. d. Identify potential correlations between test results. This "thinking" step ensures that all points mentioned in the initial instructions are thoroughly considered and analyzed, helping to catch any oversights and organize the information coherently.
Structured final output: The AI is instructed to present its final analysis in a specific format with three distinct sections: a. Detailed explanation of each test result. b. Explanation of relationships between different test results. c. Concise overview of the overall health picture. This structured output ensures that the final presentation is organized, easy to parse, and covers all aspects of the analysis in a clear and accessible manner.

Bedrock returns a detailed analysis based on this prompt. The response is then passed to the next step in our workflow for storage in DynamoDB and presentation to the user.

This approach allows me to leverage advanced AI models for medical interpretation without needing to train or host our own models or maintain complex ML infrastructure. By carefully crafting my prompt, I ensure that MedicalGenie provides valuable, patient-friendly interpretations of medical reports while maintaining high standards of accuracy and usefulness.

Building the frontend and more.. 🖥️

Using AWS Amplify Gen2, I was able to quickly build and deploy a responsive web application that seamlessly integrates with the serverless backend. Amplify Gen2 is much more than just a frontend framework; it's a full-stack development platform that significantly accelerated my development process.

Here's how Amplify Gen2 benefited the MedicalGenie frontend and backend integration:

Authentication: Amplify automatically set up Amazon Cognito for secure user authentication flows. This allowed me to implement features like user sign-up, sign-in, and password reset with minimal code. It also integrated seamlessly with my frontend, providing a consistent and secure authentication experience.
Storage: Amplify configured Amazon S3 for secure medical report uploads. It handled the complex tasks of setting up appropriate IAM roles and policies, ensuring that users could only access their own files.
API Integration: Amplify created and configured an AppSync API, automatically generating the necessary resolvers and data sources. This simplified the process of connecting our frontend to the backend services. We could define our GraphQL schema, and Amplify took care of creating the corresponding API endpoints and integrating them with backend resources.
Real-time updates: Amplify leveraged AppSync's Pub/Sub functionality to enable real-time updates in the application. It handled the WebSocket connection management and provided a simple API to subscribe to these real-time events in my frontend code. This Pub/Sub model ensured that users received immediate notifications as soon as their medical reports were analyzed, significantly enhancing the responsiveness and user experience of MedicalGenie.
CI/CD Pipeline: Amplify Hosting set up a continuous deployment pipeline, automatically building and deploying my frontend whenever I pushed changes to the repository.

Implementing Real-time Updates with AppSync ⏲️

AppSync's real-time capabilities are the core of MedicalGenie's responsive user experience. By leveraging AppSync's Pub/Sub functionality, I ensure that users receive instant updates as soon as their medical reports are analyzed. Here's a detailed look at how I implemented this feature:

AppSync schema definition: In Amplify data backed, I defined a data model for the "Patient", created custom mutation to define an API request that will modify backend data or trigger a subscription event, and a custom subscription to receive an event when a mutation is triggered. This subscription allows clients to listen for updates to analysis results.
Lambda function for publishing updates: Like I mentioned before, after storing results in DynamoDB, I use a Lambda function to publish a mutation via the AppSync API. This function is triggered as part of the Step Functions workflow.
Frontend subscription: In my Amplify-powered frontend, I subscribe to these updates using the generated GraphQL operations, adding a filter for the logged-in user's ID. Here is a code snippet from the frontend:

This feature allowed me to implement a subscription system where the frontend could subscribe to specific events (like the completion of a medical report analysis) and receive instant updates.

Deploying with AWS SAM 🔥

To simplify deployment and management of the serverless application, I use AWS Serverless Application Model (SAM). Here's a look at template.yaml file:

This template defines Step Functions state machine, Lambda functions and Layers, and necessary AWS IAM roles and policies. It also sets up the S3 trigger that kicks off the workflow when a new medical report is uploaded.

To deploy MedicalGenie using SAM, follow these steps:

Install the AWS SAM CLI and configure your AWS credentials.
Navigate to your project directory and run sam build to build your application.
Run sam deploy --guided for an interactive deployment process.
Follow the prompts to specify your stack name, AWS Region, and other parameters.
Confirm the changes and allow SAM to deploy your application.

After deployment, you can see your resources provisioning updates in AWS CloudFormation Console.

Testing the Application 🧪

To test the medical report analysis system, we'll need to upload a sample medical report to the S3 bucket. You should see the Step Functions workflow trigger automatically. You can monitor the execution in the AWS Console and check the DynamoDB table for the analyzed results. If you've built a frontend using AWS Amplify, you can take advantage of real-time updates by configuring AppSync's Pub/Sub functionality. In the frontend application you subscribe to these AppSync events and when new results are available, your frontend will receive real-time updates, allowing you to display the analyzed report to the user instantly.

This real-time capability ensures that users get immediate feedback as soon as their medical report has been processed, enhancing the overall user experience of MedicalGenie.

Conclusion: From Frustration to Innovation 🚀

What started as a personal frustration with understanding medical reports has evolved into a powerful, serverless solution that has the potential to impact countless users. By harnessing the power of AWS Step Functions, Lambda, and Bedrock, we've created a scalable, cost-effective tool that transforms medical jargon into accessible insights.

Important Disclaimer: MedicalGenie is designed to supplement, not replace, professional medical advice. Always consult with your healthcare provider for interpretation of medical results and before making any health-related decisions.

Next Steps:

Looking ahead, my next exciting step is to incorporate historical data analysis. This feature will allow MedicalGenie to provide even more personalized insights by considering a user's medical history and trends over time. Stay tuned for a future post where we'll dive into implementing this functionality

What healthcare challenge will you tackle next? Share your thoughts and ideas in the comments below. Together, we can make healthcare more accessible, one serverless function at a time! 🏥💻🎉

Site Terms, Privacy, and more.