Process Amazon Bedrock's Response Stream with JavaScript

Process Amazon Bedrock's Response Stream with JavaScript

Learn how to use Claude 3 through Amazon Bedrock's InvokeModelWithResponseStream API and process the response with the AWS SDK for JavaScript.

Dennis Traub
Amazon Employee
Published Mar 18, 2024
Last Modified Mar 20, 2024
Here's how you can process Amazon Bedrock's response stream with the AWS SDK for JavaScript v3, using Anthropic Claude 3 Haiku as an example.
No fluff, just code. Let's go!

The Step-by-step Guide

Step 1: Install the required packages

Step 2: Import the packages at the top of your script

Step 3: Create an instance of the Amazon Bedrock Runtime client

Step 4: Prepare the payload with your prompt

Step 5: Invoke Claude with the payload and wait for the API to respond

Step 6: Decode and process the chunks of the response stream

The stream contains different chunk types, allowing us to extract the message along with some additional information:
  • The "message_start" chunk contains the role the model has attached to the message.
  • The "content_block_delta" chunk contains the actual message parts.
  • The "message_stop" chunk contains some metrics, like token count.
The individual chunks can be processed inside a for await ... of loop:

The complete script:

And that's it!
Learned something new? Like this post or let me know in the comments.
If you want to learn more, have a look at the following resources:

Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.