
Deploying DeepSeek-R1 Distill Llama Models on Amazon Bedrock
Deploying the DeepSeek-R1 Distill Llama models on Amazon Bedrock involves utilizing the Custom Model Import feature, which allows you to integrate your externally fine-tuned models into the Bedrock environment seamlessly. This process enables you to leverage Bedrock's serverless infrastructure and unified API for efficient model deployment.
Manu Mishra
Amazon Employee
Published Jan 27, 2025
Last Modified Jan 30, 2025
- Model Compatibility:
Ensure your DeepSeek R1 Distill model is based on a supported architecture, such as Llama 2, Llama 3, Llama 3.1, Llama 3.2, or Llama 3.3. Amazon Bedrock supports these architectures for custom model imports. - Model Files Preparation:
Prepare the necessary model files in the Hugging Face format, including:These files should be stored in an Amazon S3 bucket accessible to your AWS account.> Important: The model is already available in safe tensor format, so we dont need to prepare files seperately.- Model weights in
.safetensors
format. - Configuration file (
config.json
). - Tokenizer files (
tokenizer_config.json
,tokenizer.json
,tokenizer.model
).
Begin by installing the necessary Python packages:
Use the Hugging Face Hub to download your specific DeepSeek R1 model. For example, to download the
DeepSeek-R1-Distill-Llama-8B
model:Upload the downloaded model files to an S3 bucket in your AWS account. Ensure that the bucket is in a region supported by Amazon Bedrock, such as
us-east-1
or us-west-2
.Navigate to the Amazon Bedrock console and initiate a new model import job:
- In the Bedrock console, select "Custom models" and choose "Import model."
- Provide the S3 URI where your model files are stored (e.g.,
s3://your-s3-bucket-name/DeepSeek-R1-Distill-Llama-8B/
). - Follow the prompts to complete the import process.
For detailed instructions, refer to the AWS documentation on importing customized models.
You can now invoke your model using the Bedrock API:
Replace
'your-account-id'
and 'your-model-id'
with your specific AWS account ID and model ID, respectively.By following these steps, you can effectively deploy the DeepSeek R1 DIstill Llama model on Amazon Bedrock, leveraging its serverless infrastructure and unified API for scalable and efficient model inference.
For a visual walk-through and additional insights, you might find the following Git repository and video helpful:
Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.