Build a UGC Live Streaming App with Amazon IVS: Architecture Overview (Lesson 1.4)
Welcome to Lesson 1.4 in this series where we're looking at building a web based user-generated content live streaming application with Amazon IVS. This entire series is available in video format on the AWS Developers YouTube channel and all of the code related to the sample application used in this series can be viewed on GitHub. Refer to the links at the end of the post for more information.
Todd Sharp
Amazon Employee
Published Dec 12, 2023
This lesson provides an architectural overview of the StreamCat UGC application.
The StreamCat application architecture is divided into two categories: a collection of pre-built resources that are deployed via the AWS Cloud Development Kit (CDK), and several resources that are created on-demand by the application itself via the AWS SDK for JavaScript (v3) when users register a new account in the application. To give you an idea what this looks like, here is an overall architecture diagram that we'll break down further in this lesson.
Image not found
The first category we'll focus on is the pre-built resources.
Image not found
Let's break this down and see what each resource does.
1. VPC
A virtual private cloud is used to restrict access to the Amazon Relational Database Service (Amazon RDS) database instance.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
const vpc = new Vpc(this, 'Vpc', {
vpcName: 'streamcat-vpc',
maxAzs: 3,
subnetConfiguration: [
{
cidrMask: 24,
name: 'private-subnet',
subnetType: SubnetType.PRIVATE_WITH_EGRESS,
},
{
cidrMask: 24,
name: 'private-isolated-subnet',
subnetType: SubnetType.PRIVATE_ISOLATED,
},
{
cidrMask: 24,
name: 'public-subnet',
subnetType: SubnetType.PUBLIC,
}
],
});An interface endpoint is added to the VPC to allow the AWS Lambda function that we'll create later on the ability to utilize the AWS Parameters and Secrets Lambda Extension to retrieve database credentials without the need for an SDK.
1
2
3
4
5
vpc.addInterfaceEndpoint('secrets-manager-endpoint', {
service: InterfaceVpcEndpointAwsService.SECRETS_MANAGER,
privateDnsEnabled: true,
subnets: { subnetType: SubnetType.PRIVATE_WITH_EGRESS }
});A security group is created to allow ingress for the database port within the VPC, as well as SSH access.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
const securityGroup = new SecurityGroup(this, 'VpcSecurityGroup', {
securityGroupName: 'streamcat-vpc-security-group',
vpc: vpc,
});
securityGroup.addIngressRule(
Peer.ipv4(vpc.vpcCidrBlock),
Port.tcp(5432),
'Allow port 5432 from within the VPC'
);
securityGroup.addIngressRule(
Peer.anyIpv4(),
Port.tcp(22),
'Allow SSH'
);2. PostgreSQL DB Instance on Amazon RDS
This database instance is used to persist all of the user data for the application. This instance is associated with the VPC we created above, and placed in a private, isolated subnet.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
const dbInstance = new DatabaseInstance(this, 'PostgresInstance', {
vpc: vpc,
vpcSubnets: { subnetType: SubnetType.PRIVATE_ISOLATED },
instanceType: InstanceType.of(InstanceClass.T3, InstanceSize.MICRO),
engine: DatabaseInstanceEngine.postgres({
version: PostgresEngineVersion.VER_14_7,
}),
port: 5432,
securityGroups: [securityGroup],
instanceIdentifier: 'streamcat-test-db',
databaseName: 'streamcat',
credentials: Credentials.fromSecret(dbCredentialsSecret),
backupRetention: cdk.Duration.days(0),
deleteAutomatedBackups: true,
});
3. Amazon IVS Chat Logging Configuration
To log chat messages, a reusable chat logging configuration is created. This configuration will be associated with every chat room and will log all chat events to the CloudWatch log group created in #4 below.
1
2
3
4
5
6
7
8
const ivsChatLogConfig = new CfnLoggingConfiguration(this, 'IvsChatLogConfig', {
name: 'streamcat-chat-logging-config',
destinationConfiguration: {
cloudWatchLogs: {
logGroupName: chatLogGroup.logGroupName
}
}
});4. CloudWatch Log Group
This log group will contain all chat logs that will be created via the chat logging configuration (#3 above).
1
2
3
4
const chatLogGroup = new LogGroup(this, 'ChatLogGroup', {
logGroupName: 'streamcat-chat-log-group',
removalPolicy: cdk.RemovalPolicy.DESTROY,
});5. Amazon IVS Recording Configuration
This configuration will be associated with every Amazon IVS channel and ensures that live streams are recorded to Amazon Simple Storage Service (Amazon S3) and thumbnails are periodically generated.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
const ivsRecordingConfig = new CfnRecordingConfiguration(this, 'IvsRecordingConfiguration', {
name: 'streamcat-recording-config',
recordingReconnectWindowSeconds: 60,
destinationConfiguration: {
s3: {
bucketName: vodBucket.bucketName,
}
},
thumbnailConfiguration: {
recordingMode: 'INTERVAL',
storage: ['LATEST'],
targetIntervalSeconds: 5,
}
});6. Amazon S3 Bucket
This bucket that will contain the video on-demand (VOD) recorded live stream media and assets.
1
2
3
4
5
6
7
8
9
10
11
12
const vodBucket = new s3.Bucket(this, 'VodStorageBucket', {
bucketName: 'streamcat-vod-storage',
removalPolicy: cdk.RemovalPolicy.DESTROY,
autoDeleteObjects: true,
cors: [{
allowedMethods: [
s3.HttpMethods.GET,
],
allowedOrigins: ['*'],
allowedHeaders: ['*'],
}],
});7. Amazon CloudFront Distribution
This distribution allows the application to expose and serve the assets stored in the VOD bucket.
1
2
3
4
5
6
const vodCfDistribution: Distribution = new Distribution(this, "VodCfDistribution", {
defaultBehavior: {
origin: new S3Origin(vodBucket),
viewerProtocolPolicy: ViewerProtocolPolicy.REDIRECT_TO_HTTPS,
},
});8. Chat Moderation Lambda Function
This function is associated with every Amazon IVS chat room that will be created, and provides basic automated moderation with profanity filters for all chat rooms. We will look at the Lambda function code in a subsequent lesson.
1
2
3
4
5
6
7
8
9
const moderateChatHandler = new Function(this, 'ModerateIvsChatHandler', {
runtime: Runtime.NODEJS_18_X,
code: Code.fromAsset('resources'),
handler: 'index.moderateChat',
layers: [lambdaLayer],
});
moderateChatHandler.addPermission('PermitModerateChatInvoke', {
principal: new ServicePrincipal('ivschat.amazonaws.com'),
});9. Amazon EventBridge Rule
This EventBridge rule is triggered automatically when the stream or recording state changes on our user's Amazon IVS channels. This will allow the application to update the channel's state, and persist information related to VOD assets so that it can be retrieved for playback later on.
1
2
3
4
5
6
7
8
9
10
11
12
13
const streamStateChangeEventRule = new Rule(this, 'StreamStateChangeEventRule', {
description: 'Rule to handle IVS Stream State Changes',
ruleName: 'streamcat-stream-change',
eventPattern: {
source: ['aws.ivs'],
detailType: ['IVS Stream State Change', 'IVS Recording State Change'],
},
targets: [new LambdaFunction(streamStateChangeHandler)],
});
streamStateChangeHandler.addPermission('PermitStreamStateInvoke', {
principal: new ServicePrincipal('events.amazonaws.com'),
sourceArn: streamStateChangeEventRule.ruleArn,
});10. Stream State Change Lambda Function
A Lambda function that handles persisting the Amazon IVS channel and recording state. This function is triggered automatically via the EventBridge rule created in #9.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
const lambdaLayer = new LayerVersion(this, 'LambdaLayer', {
layerVersionName: 'LambdaLayer',
compatibleRuntimes: [
Runtime.NODEJS_18_X,
],
code: Code.fromAsset('./resources/dependencies/nodejs'),
});
// refer to https://docs.aws.amazon.com/systems-manager/latest/userguide/ps-integration-lambda-extensions.html#ps-integration-lambda-extensions-add
// for the proper ARN
const secretsAndParametersLayer = LayerVersion.fromLayerVersionArn(this, 'SecretsAndParametersLayer',
'arn:aws:lambda:us-east-1:177933569100:layer:AWS-Parameters-and-Secrets-Lambda-Extension-Arm64:4'
);
const streamStateChangeHandler = new Function(this, 'StreamStateChangeHandler', {
vpc: vpc,
vpcSubnets: {
subnetType: SubnetType.PRIVATE_WITH_EGRESS,
},
runtime: Runtime.NODEJS_18_X,
code: Code.fromAsset('resources'),
handler: 'index.streamStateChanged',
environment: {
PARAMETERS_SECRETS_EXTENSION_LOG_LEVEL: 'info',
PARAMETERS_SECRETS_EXTENSION_CACHE_ENABLED: 'true',
SECRET_NAME: '/streamcat/db-creds',
},
layers: [lambdaLayer, secretsAndParametersLayer],
architecture: Architecture.ARM_64,
timeout: cdk.Duration.seconds(5),
});
const streamStateChangePolicy = new PolicyStatement({
effect: Effect.ALLOW,
resources: ['*'],
actions: ['secretsmanager:GetSecretValue', 'logs:FilterLogEvents'],
});
streamStateChangeHandler.addToRolePolicy(streamStateChangePolicy);11. AWS Secrets Manager for Database Credentials
A manually created secret will contain the database user credentials.
12. Bastion Host
An Amazon Elastic Compute Cloud (Amazon EC2) instance that is used as a bastion host to allow our local development environment to access the RDS instance.
1
2
3
4
5
6
7
8
9
10
11
const bastionHostInstance = new Instance(this, 'BastionHost', {
vpc: vpc,
vpcSubnets: {
subnetType: SubnetType.PUBLIC,
},
securityGroup: securityGroup,
instanceName: 'streamcat-bastion-host',
instanceType: InstanceType.of(InstanceClass.T3, InstanceSize.MICRO),
machineImage: MachineImage.latestAmazonLinux2(),
keyName: 'id_aws',
});When a user registers for a new account, the following set of resources will be created via the AWS SDK for JavaScript (v3).
Image not found
1. An Amazon IVS Channel
2. An Amazon IVS Chat Room
3. An Amazon IVS Stage
In this lesson, we learned about the pre-built and on-demand resources that are utilized by the StreamCat application. In the next lesson, we'll see an overview of the application's schema.
Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.
Comments
Log in to comment