Customizing AI Behavior: System prompts and inference parameters with Java and Bedrock's Converse API

[ Jump directly to the code 💻 or switch to the JavaScript version ]

Welcome to Part 3 of my tutorial series on Amazon Bedrock's Converse API!

In previous parts, we covered the basics of sending requests to the API and implementing conversational turns. If you haven't gone through those tutorials yet, I recommend starting with Part 1 to get a solid foundation.

In this tutorial, we'll explore how to customize the behavior of AI models using system prompts and inference parameters.

Note: The examples in this edition use Java. I've also prepared a JavaScript edition, and many more examples in Python, C#, etc.

Series overview

This series guides you through Amazon Bedrock's Converse API:

In Part 1: Getting Started, you learned how to send your first request.
In Part 2: Conversational AI, I'll show you how to implement conversational turns.
In Part 3: Customizing AI Behavior (this post), we'll configure the model with a system prompt and additional inference parameters.

Future posts will cover extracting invocation metrics and metadata, sending and receiving model-specific parameters and fields, processing model responses in real-time, the new tool-use feature, and more.

Let's dive in and start building! 💻

Step-by-step: Customize the AI response with system prompts and inference parameters

Prerequisites

Before you begin, ensure all prerequisites are in place. You should have:

An AWS account
The AWS CLI installed and configured with your credentials
A Java Development Kit (JDK) version 17 or later and a build tool like Apache Maven installed
Requested access to the model you want to use

Step 1: Set up a new Java project

If you haven't done so already, create a new Java project using Maven and add the Bedrock Runtime and STS dependencies to your pom.xml file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
...
<properties>
    <aws.sdk.version>2.26.9</aws.sdk.version>
</properties>

<dependencies>
    <dependency>
        <groupId>software.amazon.awssdk</groupId>
        <artifactId>bedrockruntime</artifactId>
        <version>${aws.sdk.version}</version>
    </dependency>
    <dependency>
        <groupId>software.amazon.awssdk</groupId>
        <artifactId>sts</artifactId>
        <version>${aws.sdk.version}</version>
    </dependency>
</dependencies>
...

Note: Replace the aws.sdk.version with the latest version of the AWS SDK for Java.

Step 2: Create an instance of the Bedrock Runtime client

Create an instance of the BedrockRuntimeClient, specifying the AWS region where the model is available:

1
2
3
4
5
6
7
8
9
10
11
12
import ... // You can find all required imports in the complete example below.

public class BedrockCustomized {
    public static void main(String[] args) {
        BedrockRuntimeClient client = BedrockRuntimeClient.builder()
                .credentialsProvider(DefaultCredentialsProvider.create())
                .region(Region.US_EAST_1)
                .build();

        // ...
    }
}

Step 3: Specify the model ID

Specify the ID of the model you want to use.

In this example, we'll use Claude 3 Haiku:

1
String modelId = "anthropic.claude-3-haiku-20240307-v1:0";

You can find the complete list of models supporting the Converse API and a list of all available model IDs in the documentation.

Step 4: Prepare a message to send

Prepare a message with your input text and the USER role:

1
2
3
4
5
6
String inputText = "What is 'rubber duck debugging'.";

Message message = Message.builder()
        .content(ContentBlock.fromText(inputText))
        .role(ConversationRole.USER)
        .build();

Step 5: Define a system prompt and additional inference parameters

A system prompt is a special type of message that provides additional context or instructions to the AI model. Unlike user messages, which represent the user's input, a system prompt is used to guide the model's behavior and set expectations for its responses.

In this example, we'll use a system prompt to instruct the model to respond in rhymes:

1
String systemPrompt = "You will always respond in rhymes.";

In addition to the system prompt, we can also specify inference parameters to further customize the model's behavior. Inference parameters allow us to control various aspects of the generated response, such as its length and randomness.

In this example, we will configure the following additional parameters:

1
2
3
4
InferenceConfiguration config = InferenceConfiguration.builder()
                .maxTokens(100)
                .temperature(0.5F)
                .build();

These parameters will limit the response to a maximum of 100 tokens and maintain a balance between creativity and coherence with a temperature of 0.5.

By combining the system prompt and inference parameters, you can fine-tune the model's behavior to suit your specific use case. Experiment with different values to observe their impact on the generated responses.

Step 6: Send the request

Send the message, inference configuration, and system prompt to the model using the Bedrock Runtime client's converse() method:

1
2
3
4
5
6
ConverseResponse response = client.converse(request -> request
        .modelId(modelId)
        .messages(message)
        .inferenceConfig(config)
        .system(SystemContentBlock.fromText(systemPrompt))
);

The converse method sends the conversation to the specified model and returns its response.

Print out the model's response:

1
2
String responseText = response.output().message().content().get(0).text();
System.out.println(responseText);

Let's run the program

With the code complete, let's run it and see the AI engage in a multi-turn conversation!

Here's the full code for reference:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeClient;
import software.amazon.awssdk.services.bedrockruntime.model.*;

public class BedrockCustomized {
    public static void main(String[] args) {
        BedrockRuntimeClient client = BedrockRuntimeClient.builder()
                .credentialsProvider(DefaultCredentialsProvider.create())
                .region(Region.US_EAST_1)
                .build();

        var modelId = "anthropic.claude-3-haiku-20240307-v1:0";

        Message message = Message.builder()
                .role(ConversationRole.USER)
                .content(ContentBlock.fromText("What is 'rubber duck debugging'?"))
                .build();

        String systemPrompt = "You will always respond in rhymes.";

        InferenceConfiguration config = InferenceConfiguration.builder()
                .maxTokens(100)
                .temperature(0.5F)
                .build();

        ConverseResponse response = client.converse(request -> request
                .modelId(modelId)
                .messages(message)
                .inferenceConfig(config)
                .system(SystemContentBlock.fromText(systemPrompt))
        );

        String responseText = response.output().message().content().get(0).text();
        System.out.println(responseText);
    }
}

To run it:

Save the code in a file named BedrockCustomized.java
Compile and run the Java application using your preferred IDE or command-line tools.

If everything is set up correctly, you should see the model's response printed in the console:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
Rubber duck debugging, a clever way,
To solve problems without delay.
You talk to a duck, real or toy,
And the solution you will enjoy.

By explaining the issue out loud,
Your mind becomes less clouded.
The duck listens, never judges,
And the answer often just nudges.

It's a technique that's quite profound,
When you're stuck, it helps you rebound.
So next time you're in a bind,
Grab a duck and let your thoughts unwind.

If you encounter any errors, double check that you have:

Configured your AWS credentials
Requested access to the model you are using
Installed the required dependencies
Specified the correct region and model ID

Next steps

Congratulations on influencing the model's responses with a system prompt and additional inference parameters.

Ready for more? Here are some ideas to keep exploring:

Experiment with different models and compare their responses. Here's the list of all models supporting the Converse API.
Challenge yourself to rewrite this program in another language. Here are examples in Python, JavaScript, C#, and more.
Dive deeper into the Converse API in the Amazon Bedrock User Guide.

In future posts you will learn how to extract invocation metrics and metadata from the model response, how to send and receive model-specific parameters and fields, how to retrieve, process, and print the model response in real-time, and a lot more.

I'd love to see what you build with Amazon Bedrock! Feel free to share your projects or ask questions in the comments.

Thanks for following along and happy building! 💻🤖

Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.

Select your cookie preferences

Site Terms, Privacy, and more.