Getting started with different LLMs on Amazon Bedrock

You've probably found your way here because you're interested in getting started with Large Language Models (LLMs) on Amazon Bedrock. Or maybe you heard Bedrock supports an assortment of different models, like Anthropic's Claude 3, Mistral AI's models, and Meta's Llama 3 models, that can be used side-by-side (and even play each other in video games). That's awesome! Before we get started, let's define what an LLM is, and see briefly how they work. Then we'll talk about how to access a ton of LLMs using Amazon Bedrock, and I'll show you how to get started, how to switch between LLMs, and we will explore some of the features of Amazon Bedrock.

LLMs are a type of Foundation Model (FM) that is trained on vast amounts of text data that allows them to understand language and perform complex tasks. The complex tasks include generating stories, summarizing text, writing code for you, and much, much more. Amazon Bedrock is a managed service that gives you access to several high-performing FMs from several well known AI companies. At a high level, you provide an LLM with some initial text or instructions. We call that a prompt. The model then uses this as a starting point to generate its own response or continuation to a response depending on the scenario. What's cool is that the LLM has the ability to recognize patterns in the data it was trained on and it uses that knowledge to produce a relevant, coherent output. It's not always perfect. Sometimes an LLM will hallucinate and make things up that are not entirely accurate. This is why you should pay close attention to the responses and compare them against what you know to be true.

Now, one of the key aspects of working with LLMs is something called prompt engineering. This is the crafting of effective prompts that let the LLM give you the type of output you're looking for. There are a few techniques like zero-shot learning, where the LLM tackles a task without any examples, and few-shot learning, where you provide a small number of examples to guide the LLM. These methods can help you get more accurate responses from the LLM.

With Amazon Bedrock, you get access to several LLMs from leading AI companies like Anthropic, Cohere, and Amazon's own models, all through a single API. Switching between different LLMs is super easy since you're really just changing a line of code. But we'll get into using Bedrock a bit later. For now, just know that Bedrock gives you the tools to experiment with various LLMs, customize them with your own data, and build secure, responsible AI applications without having to worry about managing any infrastructure yourself.

But what exactly is Amazon Bedrock? Amazon Bedrock is a managed service that gives you access to a bunch of high-performing FMs from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI AI, Stability AI, and even Amazon itself. This access is gained through a single API. What this means is that as a developer, you can easily change a couple lines of code and quickly gain access to a totally different LLM. We're going to get into that a little later in this article. In fact, Bedrock also offers a set of capabilities that you can use to build generative AI applications with security, privacy, and responsible AI in mind. With Bedrock you can experiment with and evaluate different LLMs. This gives you the opportunity to find the perfect fit for your use case. Additionally, you can customize them with your own data using techniques like fine-tuning and Retrieval Augmented Generation (RAG). So how do you get started?

I'm going to assume that you already have an AWS account and jump right into getting access to Amazon Bedrock. First, lets look at the AWS Console.

In the AWS console navigate to Amazon Bedrock. From there click "Get Started" and you will be taken to an overview page. You can see that in Figure 1.

In Figure 1, you'll note several areas called out with the numbers one through six. Area number one is the getting started menu where you can gain an overview of Amazon Bedrock, see some examples, and view details about the model providers, specifics on the LLMs that are available for that model, see what the API request looks like, and access other resources for each provider. Area two is where you can see your base FMs and any custom models you have trained. Area three is the playground area. This is where you can select and test different LLMs inside the AWS console interface. Area four is your safeguards, which currently does watermark detection. This feature is currently in preview, but it will detect whether an image was generated by a Titan Image Generator model on Bedrock. Next, area five is Orchestration. This is where you can gain access to knowledge bases and Agents. Area five is Assessment and Deployment which lets you do model evaluation and control provisioned throughput. The entire menu on the left is subject to change, but at least you are familiar with what's there today.

One of the most important menu items is not seen in Figure 1. Its the Model access menu item and its at the bottom of the menu on the left. I say this is the most important menu because access to each model must be requested before you can use them, and they are not granted by default. Therefore, and administrator will need to go into Model access and manage model access, requesting it for the models you wish to use. You can see this in Figure 2.

At the time of writing this article there are four pricing models:

Everything we do in this article will fall under the On-Demand model, so you will only pay for what you use. You can find more about the pricing models on the Bedrock pricing page.

Since you're here to get started with Bedrock and we are already in the AWS Console, I would feel remise if we didn't talk about playground access. Without coding anything you can test the LLMs you have model access to through chat, text, or images. For example, to chat with Claude 3 Sonnet you would click on Chat and then Select model as seen in Figure 3.

Then you would select the model provider Anthropic from the Category column, Claude 3 Sonnet from the Model column, and then On-demand will be selected automatically from the Throughput column, followed by clicking Apply as seen in Figure 4.

After applying the model you can chat with it as seen in Figure 5.

This is certainly an easy way to get started with the LLM, but why not take it a step further. Let's look at how to interact with the LLM programatically. For our next example I'll use the AWS SDK, specifically Python and Boto3, but you could use other languages if you prefer. You can find out more in our code examples.

Another way to interact with our LLM is through code. For this article I will give you a few examples in Python to illustrate how to get started. When interacting with Amazon Bedrock through code, you will use the Bedrock API. In our examples we will use Amazon Bedrock Runtime.

The following code interacts with Claude 3 Sonnet. I've added comments into the code to explain what each section of code does.

So when we run this code we end up with something like this:

Sweet! We are now interacting with Claude 3 Sonnet. Let's now consider how we can switch between different LLMs.

I mentioned earlier that we are using the Bedrock API to work with our LLMs. Before moving forward we need to make sure that we have model access. For this article I will show you examples for Claude 3 Sonnet and Haiku, Mistral AI, and Amazon Titan. Navigate to the Model access page in the AWS console and ensure that these LLMs show "Access granted" before you try to test the code I am showing. You should see an output similar to Figure 6 if you have access to these LLMs.

With access enabled, lets take that code we used above for Claude 3 Sonnet and lets change it to use Claude 3 Haiku.

To do this all we will change in our code is the Model ID portion of the code as seen below:

We can then run the code again. Our response looks like the following:

That worked and it was easy to switch to a new LLM. This is because Claude 3 Sonnet and Claude 3 Haiku use the same API request format. You can see them on the providers page if you scroll to the bottom. Note in Figure 7 and Figure 8, that the format is the same and the only difference is the modelID.

So now we can work with two different LLMs. Lets add another. Let's now work with Mistral AI. We need to have a look first at the API request format, and then we can modify our code a bit. We can see the format of the API request in Figure 9. Note that the body format is a bit different. Where Claude 3 used messages Mistral AI does not.

Armed with this information we can change the code to reflect the correct request format. I also recommend going to the AWS documentation and looking at the example for request and response. In my testing I've found that the response may vary as well. With the correct request and response formats, here is our new code:

And here is the output now:

Now we have working code to test Claude 3 Sonnet, Claude 3 Haiku, and - Mistral 7B Instruct. In fact, you can change the Model ID between the different Mistral AI models as well and they should work as well. Now lets work with Amazon's Titan model. Again, we need to check the API request as seen in Figure 10.

We should also look at the Titan response format in the AWS Documentation.

Here is our code to work with the Amazon Titan models:

Running our Titan code produces the following output:

Again, you can switch the Titan models around to test different ones.

We've covered a lot of ground in this article. We introduced you to LLMs and Amazon Bedrock, and we showed you how to get started with different LLMs in the AWS console as well as programmatically using Python. Using Amazon Bedrock provides a unified method of accessing several popular LLMs and makes it easy to integrate it into your code projects. As you experiment with the various LLMs you'll be in a better position to select the best LLM for your specific use case. For guidance in this area I recommend reading the article “Choose the best foundational model for your AI applications." There is much more you can do with Amazon Bedrock and I encourage you to spend some time in the Generative AI space on Community.AWS. Keep learning, building, and testing., and as always, "Happy Labbing!"