Use Mistral AI to build generative AI applications with Go
Learn how to use Mistral AI on Amazon Bedrock with AWS SDK for Go
Abhishek Gupta
Amazon Employee
Published Aug 9, 2024
Mistral AI offers models with varying characteristics across performance, and cost:
- Mistral 7B - The first dense model released by Mistral AI, perfect for experimentation, customization, and quick iteration.
- Mixtral 8x7B - A sparse mixture of experts model.
- Mistral Large - ideal for complex tasks that require large reasoning capabilities or are highly specialized (Synthetic Text Generation, Code Generation, RAG, or Agents).
Let's walk through how to use these Mistral AI models on Amazon Bedrock with Go, and in the process, also get a better understanding of it's prompt tokens.
Lets start off with a simple example using Mistral 7B.
Refer to Before You Begin section in this blog post to complete the prerequisites for running the examples. This includes installing Go, configuring Amazon Bedrock access and providing necessary IAM permissions.
You can refer to the complete code here
To run the example:
The response may (or may not) be slightly different in your case:
You can refer to the complete code here. Here is a quick walk through:
We start by creating the JSON payload - it's modeled as a
struct (MistralRequest). Also, notice the model ID mistral.mistral-7b-instruct-v0:2<s>refers to the beginning of string token- text for the user role is inside the
[INST]...[/INST]tokens - text outside is the assistant role
In the output logs above, see how the
<s> token is interpretedHere is the
MistralRequest struct with the attributes:InvokeModel is used to call the model. The JSON response is converted to a struct (
MistralResponse) and the text response is extracted from it.Moving on to a simple conversational interaction. This is what Mistral refers to as a multi-turn prompt and we will add the
</s> which is the end of string token.To run the example:
Here is my chat interaction:
You can refer to the complete code here
The code itself is overly simplified for the purposes of this example. But, important part is the how the tokens are used to format the prompt. Note that we are using Mixtral 8X7B (
mistral.mixtral-8x7b-instruct-v0:1) in this example.The beginning of string (
bos) token is only needed once at the start of the conversation, while eos (end of string) marks the end of a single conversation exchange (user and assistant).If you've read my previous blogs, I always like to include a "streaming" example because:
- It provides a better experience from a client application point of view
- It's a common mistake to overlook the
InvokeModelWithResponseStreamfunction (the async counterpart ofInvokeModel) - The partial model payload response can be interesting (and tricky at times)
You can refer to the complete code here
Lets try this out. This example uses Mistral Large - simply change the model ID to
mistral.mistral-large-2402-v1:0. To run the example:
Notice the usage of
InvokeModelWithResponseStream (instead of Invoke):To process it's output, we use:
Here are a parts of the
processStreamingOutput function - you can check the code here. The important thing to understand is how the partial responses are collected together to produce the final output (MistralResponse).Remember - building AI/ML applications using Large Language Models (like Mistral, Meta Llama, Claude, etc.) does not imply that you have to use Python. Managed platforms like Amazon Bedrock provide access to these powerful models using flexible APIs in a variety of programming languages, including Go! Thanks to AWS SDK support, you can use the programming language of your choice to integrate with Amazon Bedrock, and build generative AI solutions.
You can learn more by exploring the official Mistral documentation as well the Amazon Bedrock user guide. Happy building!
Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.