Bridging AI and real-time data with tools

Have you ever wondered why you can chat with an LLM about how to solve math problems but they struggle with actual mathematical calculations? Or why they cannot reliably provide up to date information? LLMs are trained on text and they learn to predict patterns in that text. They don't have a built in calculator, so they can only approximate based on patterns they've seen, leading to errors especially with large numbers or complex calculations. The text they're trained on is also at a point in time, so asking about the current weather or today's stock prices would only be as current as the training data.

Many times, though, we'll need to augment LLM responses with real-time data from external tools and APIs. One way we can do this, is using an LLM feature called tool use (or function calling) to give them their own set of tools to use. Tool use can be applied to fetch real-time data from a database or inventory system, stock prices, weather forecasts, or even your cloud resources.

In this blog post, I'm going to show you how to integrate fetching real-time data from a weather API so you can interact with real-time data using natural language. We'll implement this in Python and using Amazon Bedrock's Converse API, we'll define a tool for our AI to use when requested. This creates a seamless interaction between AI and a custom function, allowing us to combine the strengths of both.

Let's get started!

To work through this example, you'll need a few bits set up first:

Just want the code? Grab it here.

First, we'll set up the system prompt and the tool configuration for our tool.

The system prompt tells the model that it should use a tool in a specific situation -- when providing current weather data, use the Weather_Tool.

Then we define the specification for our weather tool using the Converse API tool definition format. We describe the weather tool and defining the inputs so that the model knows how to interact with it. In this case, the tool expects a latitude and longitude of a location.

Next, we'll implement the tool. This is where we'll make an API call to the Open-Meteo API to retrieve the current weather for a given location.

To send the conversation to Amazon Bedrock, we first create a conversation array with our initial user prompt. In the example code repo, you can see how you might collect this directly from the user interactively.

Then, we make the converse call, sending along the model, conversation, tool config, and our system prompt.

Once we have a response from Amazon Bedrock, we have to process the response. We'll check to see if a tool is being requested (toolUse is returned) and if so, we call the requested tool.

With a result from the tool, the fetch_weather_data function, we then prepare a toolResult to send back to Amazon Bedrock for further processing.

With the toolResult containing the current weather data, we now send this to Amazon Bedrock so we can get the final natural language response from the model incorporating the tool result -- the current weather.

Finally, we're ready to run the code and find the current weather for a location through natural language conversations. Run the following at the command line or in your IDE:

The output we get is:

In this post, you learned why LLMs struggle to make complex calculations and provide real-time information. I showed one approach for augmenting your LLM responses with data from external tools and APIs -- tool use (or function calling). Then, I showed how to implement this in Python with Amazon Bedrock's Converse API.

I hope this has been helpful. If you'd like more like this, smash that like button 👍, share this with your friends 👯, or drop a comment below 💬.