Deploy DeepSeek R1 on AWS Bedrock

You've probably heard the buzz about the DeepSeek R1 model – and for good reason! It's been making waves in the AI community for its impressive performance, particularly considering its relatively compact size. This distilled version of the LLaMA 8B model packs a serious punch, making it a prime candidate for a variety of applications.

But how do you actually get your hands on this powerful tool? Today, we'll walk you through the process of deploying the DeepSeek R1 Distilled LLaMA 8B model to Amazon Bedrock, from local setup to testing. We’ll be using a macOS environment, but the steps are easily adaptable to other operating systems. This is your complete guide to getting up and running with DeepSeek R1 on AWS.

Why is DeepSeek R1 in the spotlight?

DeepSeek R1 isn't just another language model; it's designed to be efficient. By "distilling" the larger LLaMA model, it retains much of its performance while significantly reducing size. This means faster inference times and lower resource requirements, making it ideal for deployment on platforms like Amazon Bedrock. Its balance of size and capability makes it a hot topic right now, as businesses and developers seek powerful AI without breaking the bank.

What You’ll Need

Before we dive in, make sure you have these prerequisites covered:

An AWS Account: You'll need an active AWS account with appropriate permissions to access Amazon Bedrock.
AWS CLI: The AWS Command Line Interface (CLI) should be installed and configured with a profile that has the necessary permissions.
Homebrew (macOS): If you’re on macOS, we’ll use Homebrew for package management. If you’re on Linux, adapt with your distro’s package manager (e.g., apt, yum).
Storage Space: Make sure you have approximately 30GB of free storage for the model files.

Step-by-Step Deployment

Let's break down the process into easy-to-follow steps.

Step 1: Install Git Large File Storage (LFS)

Large models, large files. Git LFS helps us manage these efficiently. Let’s install it:

1
2
brew install git-lfs
git lfs install

Step 2: Clone the DeepSeek Model Repository

Time to get the model files. We'll clone them from Hugging Face:

git clone https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B

Step 3: Configure AWS Environment Variables

Set up your AWS environment variables to interact with your account:

1
2
export AWS_REGION=us-east-1
export AWS_PROFILE=your-profile-name

Replace us-east-1 with your desired region and your-profile-name with your AWS profile.

Step 4: Create an S3 Bucket

We need a place to store our model files on AWS. Let’s create an S3 bucket:

aws s3 mb s3://bedrock-deepseek-models-{account-id}-us-east-1

Remember to replace {account-id} with your actual AWS account ID and match the region you set earlier!

Step 5: Upload Model to S3

Now, let's move the model files into your newly created S3 bucket:

aws s3 sync --exclude '.git*' DeepSeek-R1-Distill-Llama-8B s3://bedrock-deepseek-models-{account-id}-us-east-1/DeepSeek-R1-Distill-Llama-8B/

This command copies all the files except those in the .git folder.

Step 6: Import the Model into Amazon Bedrock Console

Now the exciting part – deploying it in Bedrock!

Head to the Console: Navigate to the Amazon Bedrock console within the AWS Management Console and select "Imported models".
Start Import: Click on "Import model" to begin the import process.
Model Details: Configure as follows:
- Model name: deepseek-r1-8B (you can choose a different name).
- Tags: Optional.
- VPC settings: Optional.
Import Job Configuration: This is where we connect to S3:
- Import job name: import_deepseek-r1-8B (or any name that helps you).
- Model import source: Select "Amazon S3 bucket."
- S3 location: Enter the S3 path: s3://bedrock-deepseek-models-{account-id}-us-east-1/DeepSeek-R1-Distill-Llama-8B/
- Service Access Configuration:
  - Choose "Create and use a new service role".
  - Accept the auto-generated service role name
  - Use default encryption settings
Import: Finally, click "Import model" to start the import job.

Image not found

Testing Your Deployed Model

Let's make sure everything's working. Save the following code as test_model.py and customize it with your values.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
import json
import boto3
from botocore.config import Config

# Configuration
REGION_NAME = 'us-east-1'
MODEL_ARN = 'arn:aws:bedrock:us-east-1:{account-id}:imported-model/{model-id}'  # Replace with your model ARN

# Configure retry settings
config = Config(
    retries={
        'total_max_attempts': 10, 
        'mode': 'standard'
    }
)

# Test message
message = "What is the color of the sky?"

# Initialize Bedrock Runtime client
session = boto3.session.Session()
br_runtime = session.client(
    service_name='bedrock-runtime', 
    region_name=REGION_NAME, 
    config=config
)

# Invoke the model
try:
    invoke_response = br_runtime.invoke_model(
        modelId=MODEL_ARN, 
        body=json.dumps({'prompt': message, 'max_gen_len': 2048}), 
        accept="application/json", 
        contentType="application/json"
    )
    
    # Parse and print the response
    invoke_response["body"] = json.loads(invoke_response["body"].read().decode("utf-8"))
    print(json.dumps(invoke_response, indent=4))
except Exception as e:
    print(e)
    print(e.__repr__())

Important:

Replace {account-id}: With your AWS account ID.
Replace {model-id}: With the Model ID provided in the Imported Model section of the Amazon Bedrock console. The MODEL_ARN will look similar to arn:aws:bedrock:us-east-1:529088295222:imported-model/h8t50Wwcd0ux.
Make sure REGION_NAME matches your chosen region
max_gen_len – Specify the maximum number of tokens to use in the generated response. The model truncates the response once the generated text exceeds max_gen_len

Run the script:

1
2
export AWS_PROFILE=your-profile-name
python test_model.py

If all goes well, you should see a JSON response in your console, similar to the one below. This means your model is deployed and responding to requests!

Example Output

Here's an example of what you might see after running the Python script successfully:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
{
    "ResponseMetadata": {
        "RequestId": "2e11ea30-1cd7-47ad-80af-49377d94137f",
        "HTTPStatusCode": 200,
        "HTTPHeaders": {
            "date": "Wed, 29 Jan 2025 07:54:28 GMT",
            "content-type": "application/json",
            "content-length": "2517",
            "connection": "keep-alive",
            "x-amzn-requestid": "2e11ea30-1cd7-47ad-80af-49377d94137f",
            "x-amzn-bedrock-invocation-latency": "5390",
            "x-amzn-bedrock-output-token-count": "512",
            "x-amzn-bedrock-input-token-count": "9"
        },
        "MaxAttemptsReached": true,
        "RetryAttempts": 9
    },
    "contentType": "application/json",
    "body": {
        "generation": " It's not blue because of the atmosphere. Wait, that's a myth, right? So, if the sky wasn't blue, what color would it be? Let me think.\n\nFirst, I remember that the sky appears blue during the day because of Rayleigh scattering. This is when sunlight scatters off molecules and small particles in the atmosphere. Blue light has a shorter wavelength, so it scatters more. That's why the sky looks blue.\n\nBut if the sky wasn't blue, maybe it would look another color. What happens during sunrise or sunset? The sky turns red or orange. That's because the sunlight passes through more atmosphere, causing red light to scatter less and dominate the color we see.\n\nSo, if the sky wasn't blue, it might look more like a red or orange during the day. But wait, that's just at sunrise or sunset. What about the middle of the day? If Rayleigh scattering isn't the main reason, what else could cause color?\n\nAnother thought: if there were no atmosphere, the sky would look black because there's nothing to scatter the light. But Earth's atmosphere is thinner, so it's not completely black. Maybe without the atmosphere, the sky would be a different color.\n\nWait, but the original question said it's not blue because of the atmosphere. So, if the atmosphere wasn't there, the sky wouldn't be blue. But what color would it be? Since the atmosphere scatters blue light, without it, blue light wouldn't scatter as much, so the sky might look different.\n\nI think without the atmosphere, the sky would appear black because there's nothing to scatter the sunlight. But that's if the atmosphere is completely gone. Maybe if there's a different kind of atmosphere, the sky could look another color.\n\nAlternatively, if the atmosphere still exists but doesn't scatter blue light as much, maybe the sky would look more like its true color. But I'm not sure what that color would be. I remember something about the sky appearing violet under certain conditions because of the scattering of blue and red light.\n\nWait, maybe if the atmosphere is clear but not causing Rayleigh scattering, the sky would look white. But I don't think that's right because the sky isn't white; it's blue during the day. So, without the atmosphere causing scattering, maybe the sky would look black.\n\nI'm getting confused. Let me try to break it down. The sky is blue because blue light scatters more. Without the atmosphere, blue light wouldn't scatter",
        "generation_token_count": 512,
        "stop_reason": "length",
        "prompt_token_count": 9
    }
}

What to look for in the response:

HTTPStatusCode: 200: This indicates that your request was successful.
x-amzn-bedrock-invocation-latency: This shows how long it took for the model to generate a response (in milliseconds).
x-amzn-bedrock-output-token-count: The number of tokens in the generated output. The model will stop generating if reaches a max token number for the output.
x-amzn-bedrock-input-token-count: The number of tokens in the input prompt
body.generation: This field contains the actual text generated by the model, answering your question. In this case, the DeepSeek R1 model has provided a thorough explanation of why the sky appears blue and what might happen without the atmosphere. The stop reason "length" indicates that the output was truncated because it reached a max token limit.

Cost Considerations

Here's a breakdown of the estimated costs, keep in mind these are rough estimates based on the us-east-1 region. Your actual costs can vary.

Storage Costs (30GB Model)

S3 Storage: Roughly ~$0.69 per month at $0.023/GB.
Bedrock Model Storage: Approximately ~$2.19/month at $0.0001 per GB-hour.

Inference Costs

Base Rate: Around $0.25 per hour for a ml.g4dn.xlarge instance.
Per-Request:
- Estimated cost per second: ~$0.000069
- Example 2-second inference: ~$0.00014 per request
- Example: 10,000 requests/month with 2-second inference: ~$1.40/month

Additional Costs

One-time import process charge.
Data transfer fees (if applicable).
AWS CloudWatch monitoring, if used.

For the most up-to-date pricing, see the official AWS Bedrock pricing page.

Key Considerations

Security: Always ensure you are using proper AWS credentials and permissions to avoid any unauthorized access.
Model Size: Keep in mind the model is around 30GB in size, which has implications for storage and bandwidth.
Region: Bedrock availability may vary. Check that it's available in your chosen region.
Monitoring: Use AWS CloudWatch to track your model’s performance, costs, and identify potential issues.

Troubleshooting Tips

Permissions: Double-check your AWS credentials and ensure they have the necessary Bedrock permissions.
Git LFS: Make sure Git LFS is initialized and configured correctly.
S3 Naming: Verify that your S3 bucket naming adheres to AWS conventions.
CloudWatch Logs: If you encounter errors, examine CloudWatch logs for more detailed information.

Next Steps

Now that you've deployed your DeepSeek R1 model, what's next?

Bedrock Playground: Test the model further through the Bedrock console's playground.
Permissions: Set up fine-grained model access permissions as needed.
Application Integration: Integrate this model into your applications to leverage its power.
Monitoring: Continually monitor performance and resource usage to optimize.

We hope this guide was useful in deploying DeepSeek R1 on Amazon Bedrock. Happy building!

Select your cookie preferences

Site Terms, Privacy, and more.