![Live Streaming from Unity - Real-Time Playback (Part 7)](https://assets.community.aws/a/2dJeIWdljK0NgWJ9ZbN6tuMgY3a/unit.webp)
Live Streaming from Unity - Real-Time Playback (Part 7)
Let's add live stream *playback* in a game built with Unity!
Todd Sharp
Amazon Employee
Published Mar 6, 2024
In this series, we've been focusing on broadcasting from a game built in Unity to an Amazon Interactive Video Service (Amazon IVS) real-time stage. But it's not just broadcasting that is possible with Amazon IVS - playback is also possible. In this post, we'll focus on adding real-time playback to the HUD of a Unity game. This capability provides a really unique way to add managed game chat (audio only streams) or team chat (with audio and video) directly into the game experience.
🐉 Here Be Dragons! 🐉: The method used in this post uses some undocumented functionality to obtain the URL used for subscribing to real-time playback with Amazon IVS. This is likely to change (or not work) in the future, so be warned!
We'll use the same
WebRTC
package that we used for broadcasting for playback, and we'll also require a stage token for playback, so if you've not yet read part 2 in this series, now would be a great time to do that. The main difference for playback is that we'll need to render the incoming frames to the UI, and we'll need to modify the URL that we use to connect based on the contents of the stage token. We'll also need to know the participantId
of the stream that we'd like to subscribe to, so we'll need to construct a way to obtain that. Let's start by getting that participantId
. We'll use the Amazon IVS chat integration that we learned about in part 4 in a different way than we've previously seen. This time we'll use the chat WebSocket connection as a message bus so that we can be notified when another participant's video is available to display.
We're going to walk through the various elements in the
WebRTCPlayback
script below, but you can refer to the final script as a reference.Each participant that connects to an Amazon IVS stage is assigned a
participantId
, and we can use the Amazon IVS integration with EventBridge to get notified when another participant has joined the stage. For this, I've created an AWS Lambda function that is triggered by an EventBridge rule filtered to look for events with the detail-type
of IVS Stage Update
with the an event_name
of Participant Published
or Participant Unpublished
. This rule will trigger the UnityParticipantUpdated
function and here is the SAM yaml
used to create the rule.The
UnityParticipantUpdated
function is also defined in yaml
. This function needs two variables, the UNITY_CHAT_ARN
that we'll need to send a message to the game via the WebSocket connection, and the UNITY_STAGE_ARN
to make sure that we're only notifying the message bus when a participant has joined/left the specific Amazon IVS stage that we're interested in. The event that the AWS Lambda function will receive will have the following format:
The function will check the stage ARN, and if it matches it will utilize the
SendEvent
(docs) method of the IvsChatClient
to send a custom event with the name STAGE_PARTICIPANT_UPDATED
to the chat room.Another (easier, but less "dynamic") way to list stage participants would be to create an AWS Lambda function to list the participants (see ListStageParticipantsCommand). This method would need to be refreshed occasionally as participants enter and leave the stage.
For this demo, we'll add a
Raw Image
in the FPS demo game's HUD that we'll ultimately use to render the live stream. We'll add a child Audio Source
as well for the live stream audio playback. ![HUD playback container HUD playback container](https://assets.community.aws/a/2dJf3dy0OroB7hHA1J0ag86vki7/1-pn.webp)
We'll add a script called
WebRTCPlayback
to the Raw Image
to handle listening for participants and rendering the video. We'll set up a chat connection (see part 4) to listen for events in our
WebRTCPlayback
script. When we receive the publish event that our AWS Lambda function publishes, we'll establish the peerConnection
and connect the live stream for playback. If the event is an 'unpublish' event, we'll clear the render texture and dispose of the peerConnection
.In previous demos, we didn't need to parse the JWT stage token at all - we just passed it along when we established the connection. But for playback, we'll need to get the
whip_url
from the token and use that to get our SDP. Let's create a class to model the stage token.Now we can decode and parse the token by adding the following to our
GetStageToken()
function.We'll declare a variable in our
WebRTCPlayback
script for the RawImage
that we'll use to render the video.The
EstablishPeerConnection()
function renders the live stream to the RawImage
by setting the texture
of the receiveImage
every time a new frame is receivedFinally, in
DoWhip()
we use the participantId
and the whipUrl
from the StageToken
to construct the URL used to obtain the SDP.To test out our playback, we can create a page to broadcast to the Amazon IVS page using the Amazon IVS Web Broadcast SDK, or use this CodePen demo with a manually generated token. Launch the game, then connect to the stage and the remote participant will be rendered in the HUD.
In this post, we learned how to add real-time live stream playback directly inside of our Unity built game.
Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.