Avoiding the Mega Prompt: Task Based Structured Prompting
Complex requirements can lead to monolithic or "mega prompts" in Generative AI. This post outlines an approach to breaking up long prompts into structured task specific ones.
Randall Potter
Amazon Employee
Published Nov 20, 2024
Last Modified Nov 21, 2024
I'm experienced in Natural Language Processing, but with the recent paradigm shift in Large Language Model (LLM) capabilities, I've found new use cases becoming more and more complex.
Organizations can often attempt to solve these complexities in a number of ways:
- They generate long narrative style prompts.
- They have structured prompts (also long) but they require many edits and often intricate weaving of tasks and guidance.
- They have a mix of numbers 1 and 2.
This has led to prompts that can be up to 400 lines long, becoming what I have termed, the "Mega Prompt."
With that said, in this post I will outline an approach to generating task specific workflows that provide more control over the outcomes and deliver results more aligned with business objectives.
By now organizations have begun to create functional teams to advance prompt engineering with LLMs. As I mentioned above, often this presents itself into three basic styles.
In the next section we will break styles 1 and 2 down and discuss a systematic, repeatable approach to prompt engineering...
"Narrative Style" prompts tell a story, usually as a second- or third-person narrator.
Task: Make a Peanut Butter and Jelly Sandwich
System Prompt:
You are an expert sandwich maker tasked with creating a peanut butter and jelly (PB&J) sandwich. You've made several of these sandwiches before, but find yourself passing your knowledge on to a new "rookie" sandwich maker that will eventually take over your business. Customers have grown to love your PB&J sandwiches and as you've been in business for 30 years, you want to ensure this "rookie" under your mentorship understands the importance of maintaining the company reputation. With that said, you need to teach them how to make a PB&J sandwich in a step-by-step foolproof way. When the rookie tells you they are ready, you will begin your task.
User Prompt: I'm ready!
This prompt is easy to read, paints a clear context for the situation and emphasizes the importance of getting the instructions for making the PB&J sandwich correct. It also instructs the LLM to generate a "fool proof" set of instructions. Now imagine you were faced with a situation where you were the owner of a "catering" business and had to prepare 5 types of sandwiches, 4 appetizers, 5 main course options, and 4 dessert options. This style of prompting could get very long. Especially if you had certain cooking styles, methods of preparation, food allergies to account for, budget constraints, etc.
As you may imagine, this can pose a lot of issues with troubleshooting. You may have to account for things like:
1. Is the narrative well organized?
2. Does it tell a complete story?
3. Was proper grammar used?
4. When presenting "if then, else this" style guidance, does it make sense in context or could it adversely influence other aspects of the task?
Now let's take a look at a structured approach and analyze it as well...
"Structured Style" prompts define what I will call "anchors" to guide the LLM to classify parts of the task it's performing, and then use those classifications as a framework to perform its task well.
Task: Make a Peanut Butter and Jelly Sandwich
System Prompt:
User Prompt: Begin!
This prompt should provide a more stable and repeatable outcome. Many large language models desire a "specific style" of prompting. For example, Anthropic's Claude family of models prefer structured prompts such as the above. You don't have to, but in my experience, the quality of my results increased with prompts of this "structured style."
The structured style demonstrates an improvement in performance and quality with the extra benefit of also helping prompt engineers focus on specific areas of the prompt.
However, as good as this is, it has scaling issues as you look at the catering "what if?" scenario I described above.
For this we need a different approach.
Let's place ourselves in this scenario and ask if we were the trainer or trainee in this situation, how would we approach the tasks at hand. I personally would want to break down the effort into manageable "training modules," each with clearly outlined steps to follow and guidance for how to do it well. Perhaps extra context, etc.? If I received the narrative or structured prompt with everything all once, I would be overwhelmed and potentially have trouble breaking down the tasks.
I've noticed an affect where the LLMs can struggle with this all at once style, and if they don't succeed, it can, again, be an intricate effort to troubleshoot the prompt etc.
Ok with that out of the way, let's look at how, using prompt engineering, we can break the catering scenario's tasks up to achieve better performance, quality, and operational efficiency.
What do we know
- Expert sandwich maker owns a business and is training someone.
- The company is a catering business.
- The sandwich maker finds themselves make a variety of dishes, broken down as:
- 5 types of sandwiches
- 4 appetizers
- 5 main course options
- 4 dessert options.
- The expert also has to train the next person to perform the tasks and do it well while protecting the business' reputation.
It sounds like we have some "overall" things to do and then some very specific things to do. Here's an approach to break this down.
- Group all overarching guidance and context together.
- Break up the catering needs into specific tasks.
- Structure the response format in a consistent manner.
Note: In the following section I will provide examples, while also providing a glimpse into some more advanced prompt engineering techniques.
System Template:
Note: For these prompts I will use a "templating approach." Using the scripting language of your choice, you would replace things like, {prompt} with the final prompt text using string substitution.
Task: Create 4 desserts for the catering event.
Prompt Template:
Good catch! A while back I worked on a project where I had a key person performing a task. The task was rather complex and required multiple groups of instructions. I realized that as I wrote out the instruction sets that the tasks actually matched multiple "job roles." I then created a manager and employee relationship that had what I'll call "sub roles." Each role performed their task and reported back to the manager when done. Right away I experienced a performance and quality gain.
At the end of the prompt construction, we will combine the system prompt with the task prompt and send that to the LLM as one task, in this case "dessert making and training."
If we follow this approach across all the tasks, splitting up 1 task to 1 LLM call we end up with smaller specialized prompts that can focus on one thing at a time. This also allows us to pick individual LLMs for each task. For example, a simple one, may use a smaller and faster model, while a more complex task may need a larger, slower, yet more capable model.
So now we have a prompt that has been specialized to one specific task within the context of a larger effort. We can then allow for specialties across the meal preparation for the catering event.
This solution can allow for smoother...
- Troubleshooting of the prompt's performance.
- Management of prompts (versioning etc.)
- Grading the response against rubrics that are focused on the single task.
- Adherence to latency requirements. More focused, simpler prompts allow for direct knowledge searching within the LLM itself.
- Experimentation with other LLMs per task.
Thank you!
Randall Potter
Senior Solutions Architect and Generative AI Subject Matter Expert at Amazon Web Services
Senior Solutions Architect and Generative AI Subject Matter Expert at Amazon Web Services
Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.