5 lessons learned in building generative AI application with a workflow

It turns out that I really like Amazon Bedrock. Bedrock combines leading Foundation models (FM) providers in a single service and it brings a smooth development experience when it comes to building Generative AI applications.

Here's an example of how I interact with the Anthropic Claude with Amazon Bedrock API using the AWS Software Development Kit (SDK), in particular Boto3 🐍 in this case:

Now, let's say if I want to use Meta Llama2. I simply need to change the modelId and the model configuration, and I'm good to go:

While this could be my go-to implementation for various use cases, but I wonder if there's another approach.

I'm beyond excited when my colleague, Marcia, announced this integration on Build generative AI apps using AWS Step Functions and Amazon Bedrock post, last year. Now, we have flexibility on building with Amazon Bedrock. I can use AWS SDK to interact with Bedrock APIs or build a visual workflow with AWS Step Functions.

So, I took this idea, built the content and delivered for a session at AWS Innovate AI/ML, titled: "Build and Enrich Your Generative AI Apps with Visual Workflow". Apparently, I truly enjoyed building generative AI applications with AWS Step Functions. It was so exciting that I published a post on LinkedIn (I know, shocking right?):

For those who want to just jump into the code, you can try to clone my repo and deploy all the demos that I used AWS Cloud Development Kit (CDK). I've included all the instructions and additional documentation (thanks Amazon Q! You rawk 🤘🏻).

Here's the repo: https://github.com/donnieprakoso/demo-stepfunctions-bedrock.

Now, in this post, I won't describe the how-to or getting started, rather than the 5 lessons that I've learned in my journey.

Here they are:

So what? Well, first, this is great as I don't need to build everything from scratch. Not that I don't like coding, but I believe the more code I write, the more potential liability I'll have in the future. AWS Step Functions takes care this challenge for me.

One of the integration API is called InvokeModel and as the name implies, you can use this integration to run inference for text models, image models, and embedding models. Which is pretty handy and I heavily used this integration in my session.

Let's say that you want to build an application to generate social media materials, starting from its title, description to the featured image.

Here's the workflow that I built:

Here's the actual output that I copied and pasted from Step Functions execution:

If you're reading until now, it's likely that Step Functions caught your interest. Now, the next question that you might have is, "How can I build the workflow?"

Good question!

Initially, I built the workflow from scratch. You can use Amazon State Language (ASL) to build AWS Step Functions workflow. This is how Amazon State Language looks like for a workflow with one state to call Amazon Bedrock:

It works and weirdly fun. But the truth is, I'm already familiar with AWS Step Functions, as I've been using this for years. However, if you're getting started, building workflow with ASL could be a potential challenge for you.

The best way to do this is using AWS Step Functions Workflow Studio. Workflow Studio aims to help us build workflow without additional learning curve with Amazon State Language.

You can design your workflow, define the configurations for any integrations, including Amazon Bedrock. Plus, everything is done in a visual way (read: drag and drop).

Here's how it looks like:

Then, if you click on the Code tab, you'll find the ASL definition in JSON format. You can download the file and integrate into your infrastructure-as-code (IaC) which I cover next on lesson #4.

Now, once that you're using Visual workflow studio, you can find 300+ integrations and the best part is, including the logic, which exactly what I need in my use case.

One example is that I can use the parallel flow, which helps me to define a workflow that will generate social media posts, but using two different models at the same time. This saves my time a lot because I can produce alternative materials for my social media posts and can decide later which one I'll use.

Following is the screenshot:

And this is great because, especially with the flow integration. This is amazing because it's easier for me to read the workflow. Plus, simply I can easily experiment with various workflows and skip the if...then...elif...else logic, which makes it harder to understand the code.

Building an application isn't complete without integrating into DevOps. In this case, I need to integrate my application with Infrastructure-as-Code (IaC), so I don't need to manually use the AWS Management console to deploy my AWS Step Functions workflow.

At this point, I have my state machine definition in JSON format, which I've downloaded from visual workflow studio. What can I do with this?

Turns out, I can easily load my state machine definition, called app.asl.json and pass it to StateMachine object in AWS Cloud Development Kit (CDK). Have a look at the following snippet:

You can use this function DefinitionBody.from_file() to load the definition body from a file. The coolest part about this is I can decouple the process of making state machine workflow and deploying with CDK.

While building the content, I realized that I have a use case that requires me to connect to external APIs. In my case, I need to extend the functionality of my workflow and integrate with the Hugging Face API.

Fortunately, Step Functions has a really nice integration with Amazon EventBridge API Destinations. In a nutshell, with API destinations, I can invoke any HTTP endpoints as the target of the rule.

Here's how it looks like:

This workflow works by integrating API Destinations into the workflow. Of course, I need to define the API Destinations in Amazon EventBridge.

Following are the details of my API Destinations in Amazon EventBridge. I can configure the endpoint. Also, in this case, I use authorization type API Key and define credentials to integrate with Hugging Face in AWS Secrets Manager.

When I run this state machine, here's the sample output:

This is just a simple example, but with this integration, you can have the flexibility by extending the functionality with external APIs.

These are 5 lessons that I've learned when building visual workflow for generative AI applications using AWS Step Functions with Amazon Bedrock integration. Obviously, there are more lessons such as debugging your workflow to integration with AWS Application Composer.

So, what's next? Well, with all of this reasoning and code that I provided, you can get started building your own. Good idea for weekend project, right?