We use essential cookies and similar tools that are necessary to provide our site and services. We use performance cookies to collect anonymous statistics, so we can understand how customers use our site and make improvements. Essential cookies cannot be deactivated, but you can choose “Customize” or “Decline” to decline performance cookies.
If you agree, AWS and approved third parties will also use cookies to provide useful site features, remember your preferences, and display relevant content, including relevant advertising. To accept or decline all non-essential cookies, choose “Accept” or “Decline.” To make more detailed choices, choose “Customize.”
Customize cookie preferences
We use cookies and similar tools (collectively, "cookies") for the following purposes.
Essential
Essential cookies are necessary to provide our site and services and cannot be deactivated. They are usually set in response to your actions on the site, such as setting your privacy preferences, signing in, or filling in forms.
Performance
Performance cookies provide anonymous statistics about how customers navigate our site so we can improve site experience and performance. Approved third parties may perform analytics on our behalf, but they cannot use the data for their own purposes.
Allowed
Functional
Functional cookies help us provide useful site features, remember your preferences, and display relevant content. Approved third parties may set these cookies to provide certain site features. If you do not allow these cookies, then some or all of these services may not function properly.
Allowed
Advertising
Advertising cookies may be set through our site by us or our advertising partners and help us deliver relevant marketing content. If you do not allow these cookies, you will experience less relevant advertising.
Allowed
Blocking some types of cookies may impact your experience of our sites. You may review and change your choices at any time by selecting Cookie preferences in the footer of this site. We and selected third-parties use cookies or similar technologies as specified in the AWS Cookie Notice.
Your privacy choices
We display ads relevant to your interests on AWS sites and on other properties, including cross-context behavioral advertising. Cross-context behavioral advertising uses data from one site or app to advertise to you on a different company’s site or app.
To not allow AWS cross-context behavioral advertising based on cookies or similar technologies, select “Don't allow” and “Save privacy choices” below, or visit an AWS site with a legally-recognized decline signal enabled, such as the Global Privacy Control. If you delete your cookies or visit this site from a different browser or device, you will need to make your selection again. For more information about cookies and how we use them, please read our AWS Cookie Notice.
With 2023 being an year of Proof of concepts (PoCs) , 2024 has been year of production for GenAI workloads. With increasing Production workloads customers have been exploring possibilities to migrate their existing GenAI workloads to AWS GenAI stack. In this post we'll be exploring possible reasons on migrating your genAI workloads to AWS.
In this post we will seek to address following questions :
Common reasons for migrating GenAI workloads to AWS
What are the considerations while migrating your GenAI workload?
Why evaluations are so important?
How to Migrate?
What are possible next steps?
Common reasons for migrating GenAI workloads to AWS
Model Choice - Many customers want to use more than one model / model provider to keep their GenAI workloads agnostic to model providers and keep their workloads upto date with latest model capabilities. Amazon Bedrock provides customers broadest selection of models/ model providers to choose from depending upon cost, relevancy and performance of the model.
Ease of getting started - Hosting data, apps and GenAI services within the same cloud simplifies system maintainability and ability to innovate
Safe, private, and secure - Security is the top priority at AWS - Customer data remains confidential
Cost optimization - Best performance/cost for GenAI services with flexible pricing models Consumption of services (including models on Bedrock) draws from EDPs
Reliability - Hundreds of thousands of customers use AWS for GenAI/ML and trust AWS for their mission critical applications
Customer Obsession - AWS Builder teams and APN Partners are committed to helping customers be successful with generative AI
Considerations while migrating GenAI workload
Don't forget to optimise your prompts - This is crucial , prompt that works well for one model (source model) may not work for the other (target model). Make sure you are optimising your prompts using some prompt optimiser tool. This can be done by Automatic Prompt Optmiser (APO) via Amazon Bedrock
Managing Prompts well - As you scale your GenAI applications , the size of prompts starts becoming large. Its becomes difficult to manage prompts lifecycle difficult hence impacting the reusability, maintainability of the prompts. Amazon Bedrock Prompt Management is a great tool to manage , test , version and optimise your prompts.
Evaluations : Highly important to choose right metrics to evaluate your models against , the benchmarks (GPQA, MMLU etc) may give right direction but may not reflect real world scenarios especially the usecase you are working on. Its highly important to choose metric that reflect your usecases. Using Amazon Bedrock evaluations its possible to evaluate model quickly. LLM-as-a-Judge is a quickest way to evaluate models for qualitative metrics like helpfulness, coherence, readability etc rather than Human evals that can rise evaluation costs high.
Trade-offs between Latency/costs/performance - Not one model fits all sizes , identify for what you would like to optimise your GenAI workload for, must strike right trade-off between all three to achieve your goals.
Why evaluations are important
Benchmarking of new or custom models - Large language models are advancing rapidly, with new models being developed frequently. Evaluation helps the users benchmark progress in capabilities like reasoning, common sense, and factual knowledge.
Understanding task specific strengths & weaknesses - Evaluation sheds light on what tasks large models are good at and where they still struggle. This helps guide research to improve weaknesses.
Comparing models - Standardized evaluations allow for head-to-head comparison of different language model architectures and training approaches. This helps determine which approaches are most promising.
Monitoring biases - Large models risk inheriting harmful societal biases from their training data. Evaluation helps detect biases so they can be addressed.
Feedback for model customization- Results of evaluations provide feedback to application developers on where models need improvement and guide the selection of better training data, architectures, and objectives.
Evaluate real-world applicability using your own data – Evaluating models using customer data aims to indicate how capable models are for real-world deployment and use. Evaluation proxies practical use cases.
User trust & safety - Rigorous testing is important for ensuring large language models behave reliably and safely before being integrated into applications used by millions of people.
for generator_model in generator_models: job_name = f"llmaaj-{generator_model.split('.')[0]}-{evaluator_model.split('.')[0]}-{datetime.now().strftime('%Y-%m-%d-%H-%M-%S')}"
except Exception as e: print(f"✗ Error with {generator_model}: {str(e)}") continue
return evaluation_jobs
# Run model comparison evaluation_jobs = run_model_comparison(GENERATOR_MODELS, EVALUATOR_MODEL)
Deploy using Bedrock Flows
Deploy, maintain and reuse your genAI workflows using Bedrock Flows, that helps manage whole prompt lifecycle with ease. and iterate over versions of your prompts.
Image not found
Flows
Next Steps
-> Engage with AWS specialists to get migration support -> partner Up - Connect with AWS Partner offering GenAI migration accelerators here
