Build a Tool Use-Based Agent Loop with Amazon Bedrock

Introduction

This article is part of a series on tool use with Amazon Bedrock. In part 1, I provided a quick tutorial on the Amazon Bedrock Converse API. In part 2, I introduced how to use tools with the Converse API. In this article, I’ll walk through building a simple agent loop to orchestrate several tools to complete a user request. I'll even throw the model a curveball to demonstrate error handling and planning. This is meant to be a trivial example, but should hopefully illustrate how agent loops could be applied to more significant use cases.

The Converse API provides a consistent way to access large language models (LLMs) on Amazon Bedrock. It supports turn-based messages between the user and the generative AI model. It also provides a consistent format for tool definitions for the models that support tool use (aka “function calling”).

Tool use is a technique that allows a large language model to tell the calling application to invoke a function with parameters supplied by the model. The available functions and supported parameters are passed to the model along with a prompt. It's important to note that the large language model does not call a function itself - it just returns JSON and lets the calling application do the rest.

An agent loop is a generative AI design pattern that allows an LLM to solve a multi-step problem by iterating over a series of interactions, calling functions and interpreting their results, until the ultimate goal is achieved.

Why are agent capabilities important? They allow us to use generative AI to help solve more complex problems and perform more advanced tasks. I believe that tool use and agent patterns add a much richer degree of usefulness to LLMs beyond basic text processing capabilities.

The tool use-based agent loop featured in this article follows these steps:

The calling application passes (A) tool definitions and (B) a triggering message to the large language model.
The model generates a tool use request, including the parameters to pass to the tool.
The calling application extracts the parameters from the model’s tool use request and passes them to the corresponding local function to get some sort of result (that local function could then call an external service if necessary).
The calling application passes the tool result back to the model to get a follow-on response.
The model either returns a final response, or requests another tool (goto #3 above).
If too many loops occur, then the process ends without resolution.

Setting up your development environment and AWS account

You’ll want to have the latest AWS SDK and Amazon Bedrock model access configured before proceeding:

Set up the Boto3 AWS SDK and Python: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/quickstart.html
Select an AWS region that supports the Anthropic Claude 3 Sonnet model. I'm using us-west-2 (Oregon). You can check the documentation for model support by region.
Configure Amazon Bedrock model access for your account and region. Example here: https://catalog.workshops.aws/building-with-amazon-bedrock/en-US/prerequisites/bedrock-setup
If you don’t have your own local integrated development environment, you can try AWS Cloud9. Setup instructions here: https://catalog.workshops.aws/building-with-amazon-bedrock/en-US/prerequisites/cloud9-setup

Disclaimers

Large language models are non-deterministic. You should expect different results than those shown in this article.
If you run this code from your own AWS account, you will be charged for the tokens consumed.
I generally subscribe to a “Minimum Viable Prompt” philosophy. You may need to write more detailed prompts for your use case.
Not every model supports all of the capabilities of the Converse API, so it’s important to review the supported model features in the official documentation.

Code walkthrough: using multiple tools within an agent loop

Let’s start by writing a Python script that you can run from the command line. I’ll demonstrate defining tools, function-calling, error handling, and running a loop until a resolution or max loop limit is reached.

Define dependencies and a tool error class

We’re using a custom ToolError class to handle some of the potential things that can go wrong with tool use.

1
2
3
4
import boto3, json, math

class ToolError(Exception):
    pass

Define a function to call Amazon Bedrock and return the response

We’re going to call Anthropic Claude 3 Sonnet using the converse method. We pass it a list of messages and a list of tools. We also set an output token limit and set the temperature to 0 to reduce the variability between calls (During development and testing, it can be preferable to set temperature higher for more variability in responses).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
def call_bedrock(message_list, tool_list):
    session = boto3.Session()

    bedrock = session.client(service_name='bedrock-runtime')
    
    response = bedrock.converse(
        modelId="anthropic.claude-3-sonnet-20240229-v1:0",
        messages=message_list,
        inferenceConfig={
            "maxTokens": 2000,
            "temperature": 0
        },
        toolConfig={ "tools": tool_list }
    )
    
    return response

You can learn more about the Converse method here: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html

Add a function to handle tool use method calls

We’ll implement this function as a simple series of if/elif statements to call basic math functions. Note that we're deliberately skipping the tangent tool so something interesting can happen!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
def get_tool_result(tool_use_block):
    
    tool_use_name = tool_use_block['name']
            
    print(f"Using tool {tool_use_name}")
    
    # Note: We're deliberately excluding tangent so something magical can happen
    if tool_use_name == 'cosine':
        return math.cos(tool_use_block['input']['x'])
    elif tool_use_name == 'sine':
        return math.sin(tool_use_block['input']['x'])
    elif tool_use_name == 'divide_numbers':
        return tool_use_block['input']['x'] / tool_use_block['input']['y'] 
    else:
        raise ToolError(f"Invalid function name: {tool_use_name}")

Add a function to handle LLM responses and determine if a follow-up tool call is needed

The LLM may return a combination of text and tool use content blocks in its response. We’ll look for tooUse content blocks, attempt to run the requested tools, and return a message with a toolResult block if a tool was used.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
def handle_response(response_message):
    
    response_content_blocks = response_message['content']
    
    follow_up_content_blocks = []
    
    for content_block in response_content_blocks:
        if 'toolUse' in content_block:
            tool_use_block = content_block['toolUse']
            
            try:
                tool_result_value = get_tool_result(tool_use_block)
                
                if tool_result_value is not None:
                    follow_up_content_blocks.append({
                        "toolResult": {
                            "toolUseId": tool_use_block['toolUseId'],
                            "content": [
                                { "json": { "result": tool_result_value } }
                            ]
                        }
                    })
                
            except ToolError as e:
                follow_up_content_blocks.append({ 
                    "toolResult": {
                        "toolUseId": tool_use_block['toolUseId'],
                        "content": [  { "text": repr(e) } ],
                        "status": "error"
                    }
                })
        
    
    if len(follow_up_content_blocks) > 0:
        
        follow_up_message = {
            "role": "user",
            "content": follow_up_content_blocks,
        }
        
        return follow_up_message
    else:
        return None

Add a function to run the request/response loop

This function will run a request / response loop until either the LLM stops requesting tool use or a maximum number of loops have run.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
def run_loop(prompt, tool_list):
    MAX_LOOPS = 6
    loop_count = 0
    continue_loop = True
    
    message_list = [
        {
            "role": "user",
            "content": [ { "text": prompt } ]
        }
    ]
    
    while continue_loop:
        response = call_bedrock(message_list, tool_list)
        
        response_message = response['output']['message']
        message_list.append(response_message)
        
        loop_count = loop_count + 1
        
        if loop_count >= MAX_LOOPS:
            print(f"Hit loop limit: {loop_count}")
            break
        
        follow_up_message = handle_response(response_message)
        
        if follow_up_message is None:
            # No remaining work to do, return final response to user
            continue_loop = False 
        else:
            message_list.append(follow_up_message)
            
    return message_list

Define the tools

We’re defining four tools for basic trigonometry functions and a division function. We’ll dive deeper into the tool definition format in a later article in this series.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
tools = [
    {
        "toolSpec": {
            "name": "cosine",
            "description": "Calculate the cosine of x.",
            "inputSchema": {
                "json": {
                    "type": "object",
                    "properties": {
                        "x": {
                            "type": "number",
                            "description": "The number to pass to the function."
                        }
                    },
                    "required": ["x"]
                }
            }
        }
    },
    {
        "toolSpec": {
            "name": "sine",
            "description": "Calculate the sine of x.",
            "inputSchema": {
                "json": {
                    "type": "object",
                    "properties": {
                        "x": {
                            "type": "number",
                            "description": "The number to pass to the function."
                        }
                    },
                    "required": ["x"]
                }
            }
        }
    },
    {
        "toolSpec": {
            "name": "tangent",
            "description": "Calculate the tangent of x.",
            "inputSchema": {
                "json": {
                    "type": "object",
                    "properties": {
                        "x": {
                            "type": "number",
                            "description": "The number to pass to the function."
                        }
                    },
                    "required": ["x"]
                }
            }
        }
    },
    {
        "toolSpec": {
            "name": "divide_numbers",
            "description": "Divide x by y.",
            "inputSchema": {
                "json": {
                    "type": "object",
                    "properties": {
                        "x": {
                            "type": "number",
                            "description": "The numerator."
                        },
                        "y": {
                            "type": "number",
                            "description": "The denominator."
                        }
                    },
                    "required": ["x", "y"]
                }
            }
        }
    }
]

Pass a prompt to start the loop

We’re asking Anthropic Claude to calculate the tangent of 7, then printing the messages that were sent back and forth to get the answer.

1
2
3
4
messages = run_loop("What is the tangent of 7?", tools)

print("\nMESSAGES:\n")
print(json.dumps(messages, indent=4))

Now we’re ready to run the code and review the results. Note that you may observe a different but hopefully similar sequence of events.

Output

While the loop is running, the script prints the tools being called. But it’s not just calling the tangent tool, it’s using a bunch of other tools as well! Let’s investigate further.

1
2
3
4
Using tool tangent
Using tool sine
Using tool cosine
Using tool divide_numbers

Here’s our initial message. Let’s do some trig:

1
2
3
4
5
6
7
8
9
[
    {
        "role": "user",
        "content": [
            {
                "text": "What is the tangent of 7?"
            }
        ]
    },

Claude knows it can use the provided tangent tool, so it requests the tool:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
    {
        "role": "assistant",
        "content": [
            {
                "text": "To calculate the tangent of 7, we can use the \"tangent\" tool:"
            },
            {
                "toolUse": {
                    "toolUseId": "tooluse_WnWkxaxYS7-_5BnHvxpLmg",
                    "name": "tangent",
                    "input": {
                        "x": 7
                    }
                }
            }
        ]
    },

Uh-oh, the tangent tool doesn't work properly! Whatever shall we do? Let's tell Claude the bad news:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
    {
        "role": "user",
        "content": [
            {
                "toolResult": {
                    "toolUseId": "tooluse_WnWkxaxYS7-_5BnHvxpLmg",
                    "content": [
                        {
                            "text": "ToolError('Invalid function name: tangent')"
                        }
                    ],
                    "status": "error"
                }
            }
        ]
    },

But what’s this? Claude knows trigonometric identities!?! Here comes plan B! Claude starts by asking for the sine tool:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
    {
        "role": "assistant",
        "content": [
            {
                "text": "Apologies, it seems the \"tangent\" tool is not available in this environment. Let me calculate the tangent of 7 manually:\n\nThe tangent is defined as tan(x) = sin(x) / cos(x)\n\nTo find sin(7) and cos(7), I can use the sine and cosine tools:"
            },
            {
                "toolUse": {
                    "toolUseId": "tooluse_mXWdxEyNTxui14s9vktNNw",
                    "name": "sine",
                    "input": {
                        "x": 7
                    }
                }
            }
        ]
    },

The sine tool is working properly, so we send the tool result back to Claude:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
    {
        "role": "user",
        "content": [
            {
                "toolResult": {
                    "toolUseId": "tooluse_mXWdxEyNTxui14s9vktNNw",
                    "content": [
                        {
                            "json": {
                                "result": 0.6569865987187891
                            }
                        }
                    ]
                }
            }
        ]
    },

Claude, happily in possession of the sine of 7, now requests the cosine of 7:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
    {
        "role": "assistant",
        "content": [
            {
                "toolUse": {
                    "toolUseId": "tooluse_b0uIDNT6Tnq9ZeV9g4d-5g",
                    "name": "cosine",
                    "input": {
                        "x": 7
                    }
                }
            }
        ]
    },

A quick trip to cosine-land, then we send the result back to Claude:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
    {
        "role": "user",
        "content": [
            {
                "toolResult": {
                    "toolUseId": "tooluse_b0uIDNT6Tnq9ZeV9g4d-5g",
                    "content": [
                        {
                            "json": {
                                "result": 0.7539022543433046
                            }
                        }
                    ]
                }
            }
        ]
    },

And now in a stunning climactic moment, Claude asks for the divide_numbers tool:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
    {
        "role": "assistant",
        "content": [
            {
                "text": "Great, now I have sin(7) = 0.6569865987187891 and cos(7) = 0.7539022543433046.\n\nTo get tan(7), I divide sin(7) by cos(7):"
            },
            {
                "toolUse": {
                    "toolUseId": "tooluse_Ss9CW9ldQvmlGijl31biow",
                    "name": "divide_numbers",
                    "input": {
                        "x": 0.6569865987187891,
                        "y": 0.7539022543433046
                    }
                }
            }
        ]
    },

Lets send this sweet, sweet division result back to Claude:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
    {
        "role": "user",
        "content": [
            {
                "toolResult": {
                    "toolUseId": "tooluse_Ss9CW9ldQvmlGijl31biow",
                    "content": [
                        {
                            "json": {
                                "result": 0.8714479827243188
                            }
                        }
                    ]
                }
            }
        ]
    },

And the grand finale. Tangent accomplished!

1
2
3
4
5
6
7
8
9
    {
        "role": "assistant",
        "content": [
            {
                "text": "The tangent of 7 is 0.8714479827243188."
            }
        ]
    }
]

(Roll credits.)

Conclusion

While this specific example wasn't a great use of tokens, hopefully it illustrated how tools can be combined and orchestrated to solve a multi-step process. I hope you learned something and maybe got some ideas about how you could design agents based on tool use. We'll look at more advanced tool definitions in a later article.

Learn more

Not every model supports every capability of the Converse API, so it’s important to review the supported model features in the official documentation.
Check out the official documentation for tool use / function calling with Amazon Bedrock.
Overview of JSON Schema
You can find more generative AI hands-on activities at the Building with Amazon Bedrock workshop guide.

Continue reading articles in this series about Tool Use / Function Calling:

Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.

Select your cookie preferences

Site Terms, Privacy, and more.