
Deploying DeepSeek-R1 Distill Llama Models on Amazon Bedrock
Deploying the DeepSeek-R1 Distill Llama models on Amazon Bedrock involves utilizing the Custom Model Import feature, which allows you to integrate your externally fine-tuned models into the Bedrock environment seamlessly. This process enables you to leverage Bedrock's serverless infrastructure and unified API for efficient model deployment.
- Model Compatibility:
Ensure your DeepSeek R1 Distill model is based on a supported architecture, such as Llama 2, Llama 3, Llama 3.1, Llama 3.2, or Llama 3.3. Amazon Bedrock supports these architectures for custom model imports. - Model Files Preparation:
Prepare the necessary model files in the Hugging Face format, including:These files should be stored in an Amazon S3 bucket accessible to your AWS account.> Important: The model is already available in safe tensor format, so we dont need to prepare files seperately.- Model weights in
.safetensors
format. - Configuration file (
config.json
). - Tokenizer files (
tokenizer_config.json
,tokenizer.json
,tokenizer.model
).
1
pip install huggingface_hub boto3
DeepSeek-R1-Distill-Llama-8B
model:1
2
3
4
from huggingface_hub import snapshot_download
model_id = "deepseek-ai/DeepSeek-R1-Distill-Llama-8B"
local_dir = snapshot_download(repo_id=model_id, local_dir="DeepSeek-R1-Distill-Llama-8B")
us-east-1
or us-west-2
.1
2
3
4
5
6
7
8
9
10
11
12
import boto3
import os
s3_client = boto3.client('s3', region_name='us-east-1')
bucket_name = 'your-s3-bucket-name'
local_directory = 'DeepSeek-R1-Distill-Llama-8B'
for root, dirs, files in os.walk(local_directory):
for file in files:
local_path = os.path.join(root, file)
s3_key = os.path.relpath(local_path, local_directory)
s3_client.upload_file(local_path, bucket_name, s3_key)
- In the Bedrock console, select "Custom models" and choose "Import model."
- Provide the S3 URI where your model files are stored (e.g.,
s3://your-s3-bucket-name/DeepSeek-R1-Distill-Llama-8B/
). - Follow the prompts to complete the import process.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import boto3
import json
client = boto3.client('bedrock-runtime', region_name='us-east-1')
model_id = 'arn:aws:bedrock:us-east-1:your-account-id:imported-model/your-model-id'
prompt = "Your input prompt here"
response = client.invoke_model(
modelId=model_id,
body=json.dumps({'prompt': prompt}),
accept='application/json',
contentType='application/json'
)
result = json.loads(response['body'].read().decode('utf-8'))
print(result)
'your-account-id'
and 'your-model-id'
with your specific AWS account ID and model ID, respectively.Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.