Deploying DeepSeek-R1 Distill Model on AWS using Amazon SageMaker JumpStart

This is Part 2 of our series on how to deploy Deepseek on AWS. This post will focus on deploying on Amazon SageMaker JumpStart. View Part 1 of our series on deploying to Amazon EC2 here.

DeepSeek, a Chinese artificial intelligence (AI) company, has recently garnered significant attention for its innovative AI models that rival leading Western counterparts in performance while being more cost-effective. The company's latest release, launched on 20 Jan 2025, DeepSeek-R-1, matches the capabilities of OpenAI's o1 reasoning model across math, code, and reasoning tasks, but at less than 10% of the cost. Furthermore, DeepSeek-R-1 is completely open-source, enabling developers worldwide to access and implement the model on their own systems, disrupting the LLM landscape.

Hosting DeepSeek-R-1 on AWS offers unparalleled scalability and flexibility, ensuring you can seamlessly leverage its powerful AI capabilities for your specific use case - whether for research, business intelligence, or development projects.

This blog post will guide you through a step-by-step process for hosting DeepSeek-R-1, specifically the Deepseek-R1 14B model, on AWS infrastructure. This deployment will involve deploying the model on Amazon SageMaker JumpStart, enabling you to harness DeepSeek-R-1 AI capabilities within the cloud.

In this tutorial, you will deploy the DeepSeek-R1 14B model on the ml.g6.12xlarge instance type. This is the default instance type for inference hosting endpoint.

Because the default quota for ml.g6.12xlarge for endpoint usage is 0, you will need to raise it.

In this section, you will need to set up your SageMaker AI domain in the region which you desire. Follow this guide for a fuss-free creation of your domain. This can take a while, so be patient!

Image not found

Once the domain has been successfully created, browse to your user profile, then click on Studio. This will bring you to the SageMaker AI Studio console.

Image not found

You will then be brought to SageMaker AI Studio console.

We are finally ready to deploy DeepSeek on Amazon SageMaker JumpStart!

Amazon SageMaker JumpStart is a machine learning (ML) hub with foundation models, built-in algorithms, and prebuilt ML solutions that you can deploy with just a few clicks.

The default notebook requires some modifications before it can successfully deploy the model.

In the first cell, add this code block to upgrade the sagemaker library:

After you've finished using the endpoint, it's important to delete it to avoid incurring unnecessary costs. Uncomment the last code block which we have commented out earlier by removing the "#", and then run the cell.

One reason you would consider using Amazon SageMaker over importing a customized model into Amazon Bedrock is if your target region does not support said feature yet. View the available regions for Amazon Bedrock Custom Model Import here.

Germaine is a Startup Solutions Architect in the AWS ASEAN Startup team covering Singapore Startup customers. She is an advocate for helping customers modernise their cloud workloads and improving their security stature through architecture reviews.