logo
Menu
Embed request payloads into Amazon Bedrock's InvokeModel API

Embed request payloads into Amazon Bedrock's InvokeModel API

Understanding Bedrock API requests and how to piece them together can be a bit tricky. It certainly was for me, but the individual AWS SDKs provide everything you need! So, here is my (and your) quick reference for Python, JavaScript, C#, Java, PHP, and Go.

Dennis Traub
Amazon Employee
Published Mar 7, 2024
Last Modified Mar 20, 2024
Want to jump directly to the code examples? Here you go: Python, JavaScript, C#, Java, PHP, and Go.

Understanding Amazon Bedrock's API Design

While experimenting with different AI models on Amazon Bedrock, I quickly realized that while Bedrock exposes a unified API, building the actual requests isn't as trivial as I thought.
Even though Bedrock provides a single API that lets you invoke any of the generative AI models, the models themselves expect their own respective parameters and request schemas.
And this makes total sense: A large language model that completes text requires other input than a model that generates images, sound, or embeddings.
But, this comes with a certain challenge.

The Challenge of Diverse AI Model Requirements

The unified part of the API request can be encapsulated in the respective SDKs, offering an intuitive programming experience. But with the many differences across models, and the pace at which new models are being introduced, creating specific types for each and every variation simply doesn't scale.
This means that, when using Bedrock models with any of the AWS SDKs, there is one important aspect to consider. To understand this, let's have a look at the API request structure:

Bedrock's Invoke Model API Request Structure

Bedrock's InvokeModel request looks very similar to any other HTTP request: It contains some headers, the ID of the model you want to use, and a request payload, encapsulated in the body.
The request payload is what Bedrock will forward to the actual AI model, including the prompt plus some additional parameters. Here's a simplified (and inaccurate) schema of the request object:
1
2
3
4
5
6
7
8
9
10
{
"contentType": "application/json",
"accept": "application/json",
// ...
"modelId": "ai21.j2-mid-v1",
"body": {
"prompt": "Hello, how are you today?",
"temperature": 0.5,
}
}
However, due to nature of HTTP, while everything outside the body can directly be added to the request, the body itself must first be transformed into a bytestream. The actual schema of the API request looks like this:
1
2
3
4
5
6
7
{
"contentType": "application/json",
"accept": "application/json",
// ...
"modelId": "ai21.j2-mid-v1",
"body": [PAYLOAD_AS_BYTESTREAM]
}
This means that it is up to the developer, i.e. you and me, to take the payload for the model and turn it into a bytestream before adding it to the API request.
The rest of the post will show you how this works in Python, JavaScript, C#, Java, PHP, and Go:

The code examples

This section contains the code examples, showing exactly how to transform the request payload into a bytestream, and how to embed it into the API request for Amazon Bedrock.

Transforming the payload using Python

In Python it is pretty simple. All you need is to use json.dumps() from the built-in JSON module:
1
2
3
4
5
6
7
8
9
10
11
12
13
import json

payload = {
"prompt": "Hello, how are you today?",
"temperature": 0.5
}

bytestream = json.dumps(payload)

response = bedrock.invoke_model(
modelId: "ai21.j2-mid-v1",
body: bytestream
)

Transforming the payload using JavaScript

Since we're working with JSON, there's not much to do. With JSON.stringify() we can transform the payload into a bytestream. There aren't even any imports required:
1
2
3
4
5
6
7
8
9
10
11
const payload = {
prompt,
temperature: 0.5
};

const bytestream = JSON.stringify(payload);

const command = new InvokeModelCommand({
modelId: "ai21.j2-mid-v1",
body: bytestream,
});

Transforming the payload using C#/.NET

In C# you can use JsonSerializer.Serialize() to create a string representing the JSON object, and AWSSDKUtils.GenerateMemoryStreamFromString() will transform it into a bytestream:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
using Amazon.Util;
using System.Text.Json;

string payloadObject = JsonSerializer.Serialize(new
{
    prompt = "Hello, how are you today?",
    temperature = 0.5
});

MemoryStream bytestream = AWSSDKUtils.GenerateMemoryStreamFromString(payload);

InvokeModelRequest = new InvokeModelRequest()
{
  ModelId = "ai21.j2-mid-v1",
    Body = bytestream
});

Transforming the payload using Java

The AWS SDK for Java v2 provides a method called SdkBytes.fromUtf8String() that transforms the payload into an object of type SdkBytes:
1
2
3
4
5
6
7
8
9
10
11
12
13
import software.amazon.awssdk.core.SdkBytes;

String payload = new JSONObject()
.put("prompt", "Hello, how are you today?")
.put("temperature", 0.5)
.toString();

SdkBytes bytestream = SdkBytes.fromUtf8String(payload);

InvokeModelRequest request = InvokeModelRequest.builder()
.modelId("ai21.j2-mid-v1")
.body(bytestream)
.build();

Transforming the payload using PHP

PHP provides a built-in method, json_encode(), that transform the payload into a bytestream. Just like in JavaScript, it works out-of-the-box and you don't need any imports:
1
2
3
4
5
6
7
8
9
10
11
$body = [
'prompt' => $prompt,
'temperature' => 0.5,
];

$bytestream = json_encode($body);

$result = $this->bedrockRuntimeClient->invokeModel([
'modelId' => 'ai21.j2-mid-v1',
'body' => $bytestream,
]);

Transforming the payload using Go

In Go, you can use json.Marshal() to transform the payload into a bytestream:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import (
"encoding/json"
// ...
)

bytestream, err := json.Marshal(struct {
    Prompt      string  `json:"prompt"`
    Temperature float64 `json:"temperature,omitempty"`
}{
    Prompt:      "Hello, how are you today?",
    Temperature: 0.5,
})

input := &bedrockruntime.InvokeModelInput{
    ModelId: aws.String("ai21.j2-mid-v1"),
    Body: bytestream,
}
And that's it for now!
I haven't tried this with any other languages yet, but will update this post once I have. Let me know in the comments if you know how to do this in Kotlin, Rust, C++, or - who knows - maybe even with the AWS SDK for SAP ABAP 😅
If you enjoyed this post or learned something new, please hit the like button or let me know what you think in the comments below. If you want to learn more, have a look at the following resources:

Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.

Comments