AWS Logo
Menu
Deploying DeepSeek-R1 Distill Llama Models on Amazon Bedrock

Deploying DeepSeek-R1 Distill Llama Models on Amazon Bedrock

Deploying the DeepSeek-R1 Distill Llama models on Amazon Bedrock involves utilizing the Custom Model Import feature, which allows you to integrate your externally fine-tuned models into the Bedrock environment seamlessly. This process enables you to leverage Bedrock's serverless infrastructure and unified API for efficient model deployment.

generativeaiamazonbedrockdeepseek
Manu Mishra
Amazon Employee
Published Jan 27, 2025
Last Modified Jan 30, 2025

Prerequisites

  1. Model Compatibility:
    Ensure your DeepSeek R1 Distill model is based on a supported architecture, such as Llama 2, Llama 3, Llama 3.1, Llama 3.2, or Llama 3.3. Amazon Bedrock supports these architectures for custom model imports.
  2. Model Files Preparation:
    Prepare the necessary model files in the Hugging Face format, including:These files should be stored in an Amazon S3 bucket accessible to your AWS account.> Important: The model is already available in safe tensor format, so we dont need to prepare files seperately.
    • Model weights in .safetensors format.
    • Configuration file (config.json).
    • Tokenizer files (tokenizer_config.json, tokenizer.json, tokenizer.model).

Try out the interactive Jupyter Notebook

Deploy-DeepSeek-R1-On-Amazon-Bedrock.ipynb

Step-by-Step Deployment Guide

1. Install Required Dependencies

Begin by installing the necessary Python packages:

2. Download the DeepSeek R1 Model

Use the Hugging Face Hub to download your specific DeepSeek R1 model. For example, to download the DeepSeek-R1-Distill-Llama-8B model:

3. Upload Model Files to Amazon S3

Upload the downloaded model files to an S3 bucket in your AWS account. Ensure that the bucket is in a region supported by Amazon Bedrock, such as us-east-1 or us-west-2.

4. Import the Model into Amazon Bedrock

Navigate to the Amazon Bedrock console and initiate a new model import job:
  • In the Bedrock console, select "Custom models" and choose "Import model."
  • Provide the S3 URI where your model files are stored (e.g., s3://your-s3-bucket-name/DeepSeek-R1-Distill-Llama-8B/).
  • Follow the prompts to complete the import process.
For detailed instructions, refer to the AWS documentation on importing customized models.

5. Invoke the Model

You can now invoke your model using the Bedrock API:
Replace 'your-account-id' and 'your-model-id' with your specific AWS account ID and model ID, respectively.

Conclusion

By following these steps, you can effectively deploy the DeepSeek R1 DIstill Llama model on Amazon Bedrock, leveraging its serverless infrastructure and unified API for scalable and efficient model inference.
For a visual walk-through and additional insights, you might find the following Git repository and video helpful:
DeepSeekR1onAmazonBedrock
Import Custom Models Into Amazon Bedrock

 Co-Author: Achlesh Soni
 

Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.

13 Comments