Unlocking Live Streaming Analytics: Amazon IVS and Datadog Integration - Part 1

Live streaming has become an essential channel for entertainment, education, and communication. Ensuring a high-quality viewing experience is crucial to keep audiences engaged. Amazon Interactive Video Service (IVS) and Datadog offer a powerful combination to optimize live streaming with analytics. Amazon IVS lets you focus on building your own interactive application and audience experience. With Amazon IVS, you don't need to manage infrastructure or develop and configure components of your video workflows, to be secure, reliable, and cost effective.

In this series of articles, we'll explore three important use cases to gather insights into live stream video analytics.

To address these different use cases, we need to gather data on what is happening from ingest to playback of a live stream. The workflow starts when a streaming encoder sends a video stream to an IVS ingest point over one of the supported protocols: RTMP, RTMPS, or SRT. The output playback URL can then be distributed to viewers globally and viewed through the Amazon IVS Player. During a live streaming event, we can monitor the health of streams with Amazon IVS stream health metrics. These metrics are emitted from the service and provide statistics that give us insight into the quality and performance of our live video streams. Playback metrics are client side telemetry captured from the Amazon IVS player and help us understand the end-user viewing experience. 

Monitoring the health of our live streams using IVS stream health metrics and playback metrics will provide us with a holistic view of the quality of experience (QoE). In this first article, we will focus on how to capture playback metrics using Amazon IVS Player events and state updates. When a viewer interacts with the Amazon IVS Player, the player emits events based on the actions they have taken. Using these events, we will demonstrate how to calculate key metrics and how to send them to Datadog on a recurring basis. In the second article in this series, we will visualize the metrics in Datadog to better understand a live streaming experience.

In this first article, we will focus on calculating metrics that can be derived from the Amazon IVS Player SDK and sending them to Datadog. The second article will cover how to graph the metrics collected to create a dashboard to assess playback quality. The final article will cover how to monitor your live streams using Amazon IVS stream health metrics.

The code for this article is available on Github.

What you will learn

This tutorial consists of four parts

Here’s an overview of what our solution looks like at a high level.

We will be integrating the Amazon IVS Player for Web and Datadog RUM Browser SDK into a web app. The Amazon IVS Player emits event data, as mentioned earlier, which we can use to calculate various QoE metrics. Using the RUM Browser SDK, we can then programmatically generate events at the application level and send these metrics to Datadog as part of the events’ metadata on a recurring basis.

Datadog RUM's backend uses an event-driven architecture, with the client-side SDK recording events locally and uploading them in batches. The intake isolates traffic by data type, processes events, and sends them to a main queue. A reducer enriches parent events (Sessions and Views) with child event data (Actions, Errors, Resources, Long Tasks). As Datadog defines it:

Sessions can remain in the loop for up to 4 hours before timing out, while Views are reduced until a new one becomes active. Once closed, events are stored in blob storage, and dedicated services query this data for Datadog's web and mobile apps or alerting purposes. Note, sessions and Views can be queried while still being enriched and do not need to expire to be visible to users or Monitors.

Once these metrics are in Datadog, we can then create graphs and dashboards to help us understand the quality of the video playback experience.

To integrate the Amazon IVS Player into our application follow the getting started guide so that we can start playing a live stream in our web application. The steps are summarized below.

Add HTML elements to help style the page and a <video> element to play the live stream.

Add in the <script> tag for the Amazon IVS Player, which will add an IVSPlayer object to the global context. We will then write JavaScript code to initiate the IVS Player and play a live stream.

We will be creating three separate JavaScript files including config.js to set some test live streams for playback. The ivs.js file will contain code to initialize the IVS Player. Finally, the qos_sdk.js file will contain logic for calculating our quality of experience metrics.

To focus on the most relevant parts of this solution, we won’t be going over the CSS code so download main.css from Github to style the HTML. The complete HTML will look as follows.

Next, create config.js to hold our playback URLs. We will programmatically create a dropdown from these.

Next, create ivs.js to initialize the IVS Player using the IVS Player object. We then attach the IVS player to the <video> element on our page to display the live stream.

There is also code to populate a dropdown on the page with the playback URLs we just included in config.js. When one of the playback URLs is selected from the dropdown, it uses the IVS player to load and play the URL.

Open index.html and we should now see a dropdown to select from one of four test live streams. The drop down displays a unique substring within each playback URL. This unique substring is also a substring that is present in the channel ARN. The channel ARN is a unique identifier for a channel in Amazon IVS, which holds the configuration information for a live stream. By doing this, we can later associate the playback URL to a channel.

Select one of these playback channels and click the play button to start playing a live stream. With the setup of the IVS Player complete, let’s move on to adding it to the Datadog RUM Browser SDK.

The Datadog RUM Browser SDK will enable our application to programmatically send custom data to Datadog. Using what is known as RUM custom actions, we can send our own metrics to Datadog calculated from Amazon IVS Player events on a regular basis such as every minute. This metric data can then be graphed in Datadog. Follow the instructions in the RUM Browser Monitoring Setup guide to get setup with the Datadog RUM Browser SDK before proceeding further.

After going through the setup guide, our HTML should now contain script tags for the Amazon IVS Player and Datadog RUM Browser SDK. Inclusion of the Datadog RUM Browser SDK script tag adds a global variable named DD_RUM to our window object. The DD_RUM object provides us with methods to initialize the Datadog RUM Browse SDK and send custom data to Datadog.

We will also add some HTML code to display the player and add a drop down so that we can play different live streams. In this sample app, we will be populating the dropdown with IVS test streams. We can then use the dropdown to play different test streams so that we can mimic what an end user might do while viewing different streams. This will let us display analytics on a per channel basis which we will get to later.

With the HTML added to display our live stream and the script tags added, we can now work on calculating our quality of experience metrics. These metrics will help us understand and measure the video playback quality experience from the viewer’s perspective.

Once we start playback of a live stream, the Amazon IVS Player emits a number of events and player states as it streams a live channel. Events can be emitted either because a user triggered it (e.g. by clicking pause on the player) or automatically by the IVS Player in response to network conditions to optimize playback.

You can attach a listener to the IVS player to capture some of these events and calculate important playback and QoE metrics through a process called ‘player instrumentation’. For the purposes of measuring QoE metrics, the table below outlines the important ‘IVS Player Event’, IVS Player State updates, and their contribution to calculating a metric.

Using these events and state updates and built-in IVS Player methods, we will calculate the following metrics

Let’s first define and initialize all the QoE variables we will be using for tracking and reporting all of these different QoE metrics. We also initialize an instance of the Datadog RUM Browser SDK using window.DD_RUM. Recall earlier that when we added the <script> tag for Datadog RUM Browser SDK, it added a global variable named DD_RUM to our window object. Datadog lets us instrument custom events, with a lot of flexibility to add all the metadata we need, so the ddRUM object will later be used to attach these metrics to events and send them off to Datadog.

Create a function called initializeQoS, which will be the main entry point in our code. This function attaches event listeners to the IVS Player for all of the events listed in the table earlier. The initializeQoS function also contains a function setInterval(), which will send playback summary events every minute. Once a playback summary is sent, it resets the QoE variables so that they can be calculated for the next minute.

For reference, here’s the complete code. In the rest of this section, we will explain what each part of the code is doing.

In the first part of this function, we create the PlayerState and PlayerEventType variables to make tracking the IVS Player states and events easier.

We also parse the unique channel identifier from the playback URL using a custom function called getChannelWatched(). The implementation for this function is as follows and should be placed outside the initializeQoS() function.

Next, we create a callback function to record the time when the player tells us when it is initialized. We will save that timestamp in the variable `lastInitializedTime`. This timestamp will be useful later in calculating the time to video metric or the time it takes for the player to get the first frame of video.

Next, we create a callback function when the IVS Player tells us when it is ready. It is used to reset/initialize all of the tracking variables.

Next, we attach a listener to tap into buffering events from the IVS Player. We will use this state update to keep track of the number of buffering events during a playback session. If before this state, the player was playing a video, then we need to adjust the variable that tracks the playing time. We also increment the number of times the player buffered in the last minute. 

Finally, we call a custom function called setPlayerStateVariables. We call this function to update the variable used to track the player’s last state and the timestamp when this function was last called. Doing this will be useful in the calculation of the other QoE metrics as you will see soon. Here’s the implementation for this function. This utility function should be placed outside of the initializeQoS function.

Next, we create a callback function to calculate the startup latency and record the playback stream quality/rendition when the IVS Player starts playing a video. Similar to before, we also call setPlayerStateVariables to keep track of the player’s previous state.=

Next, we create a callback function to calculate two QoE metrics related to playing time and buffering time if the IVS Player goes idle. The first metric, playingTimeMsInLastMinute, calculates how much time was spent playing the video in the last minute based on the current time and the last time the player state was updated. In the previous callback functions, we had been calling the setPlayerStateVariable function to keep track of the last time the player state was updated. The second metric, bufferingTimeMsInLastMinute, tells us how much time was spent buffering in the last minute.

Next, we create a callback function to calculate the time the IVS Player spent playing a video since the last time it changed its state. It then adds this time to the playingTimeMsInLastMinute variable. This variable tracks how much time the IVS Player spent playing a video in the last minute.

Next, when the IVS Player emits an event indicating that it encountered an error, we do two things. We can keep track of the number of errors that have occurred in the last minute and we can send the detailed error information to Datadog. The Datadog RUM Browser SDK provides us with an API to send browser errors to Datadog in a more natural way. The benefit is that this appears in a separate error event in Datadog, with a full deobfuscated stack trace and many out-of-the-box attributes. It also provides more advanced features such as Error Tracking, which groups individual errors based on their similarity for faster triaging and troubleshooting. This helps bring more attention to this critical data point that could affect the live streaming experience for users. To use this API, we create a new Error object and can provide the error code, a string indicating the source of the error, and a message associated with the error. We can then call the addError method using the DD_RUM object to send this error information to Datadog.

Next, we will create a callback function to keep track of the rendition or video quality during a live stream. This can happen either  because the user explicitly selects a resolution in the player or because the IVS Player automatically adapts to changing network conditions to maintain uninterrupted playback. The end user’s bandwidth could have decreased for example, necessitating a step down in the video rendition. In this callback function, we save both the previous video rendition and the current video rendition.

At the end of this callback function, we send off a quality change event to Datadog using a custom sendQualityChangedEvent() function. We will cover the implementation of that function in the next section.

Finally, to wrap up the initializeQoS function, we send in a custom event a summary containing all the metrics we have just calculated in all our previous event listeners to Datadog. These metrics will be sent from our custom sendPlaybackSummaryEventIfNecessary function. We will show how this function is implemented in the next section. Using the setInterval function, we will send these metrics off to Datadog every minute. Once the metrics are sent, we reset all of our metrics tracking variables to zero so that we can recalculate them for the next minute.

Now that we have covered how to add event and state update listeners to the IVS Player to calculate the metrics, we will cover how they are sent to Datadog.

In the last section we used the custom sendPlaybackSummaryEventIfNecessary function within the initializeQoS function to send the calculated metrics to Datadog. In this sendPlaybackSummaryEventIfNecessary function, we create a JSON object that contains a summary of all of our QoE metrics. We first make some adjustments to the playing time and buffering time variables, making sure to capture any uncounted time if the IVS Player was in either the playing or buffering state. Additionally, we will include the number of frames that have been dropped so far during the playback session. We can get this using the getDroppedFrames() method exposed by the IVS Player.

We also include the live latency metric in this JSON object. The live latency can be retrieved using the getLiveLatency method provided by the IVS Player. Live latency tells us the time it takes for a video frame to be sent IVS to the player. We multiply it by 1,000 since it is returned to us in seconds and we want to display it in milliseconds to be consistent with our other metrics.

Additionally, we include a userId and sessionId in the JSON object. The userId is a random UUID that is set as the very first session ID of the user that played a given live stream using the IVS Player. It remains the same across different playback sessions unless explicitly reset. We use UUID as our userId just as an example. Instead of UUID, we could instead use another unique identifier associated with the users of our application.

A sessionId represents a unique identifier for each instance of the IVS Player when it is playing a live stream. If an end user for example has several browser windows open each with a separate instance of the IVS Player playing a live stream, each of these would count as a separate play session and have a unique session ID. The session ID is retrieved using the getSessionId method from the IVS Player. Associating all of these metrics to a given user ID or session ID will be important for how we graph our metrics in Datadog later.

Finally, we add the ddRUM object to set the user ID associated with our RUM session using the setUser method. This will enable us to more easily query and filter for the user associated with this metrics data in Datadog. We can also optionally add other user attributes using `setUser` to make it easier to filter users based on certain attributes

Note: Datadog RUM Browser SDK generates its own session ID. However, it does not match the granularity we need as multiple play sessions from the IVS Player might be merged together under a single Datadog RUM session. Therefore, we are passing the session ID provided by the IVS Player as a custom attribute to the events we send to Datadog.

Next, we create another JSON structure to send a QUALITY_CHANGED event to Datadog whenever the IVS Player switches to a different rendition (change in quality, bitrate). This payload is sent to Datadog using the custom sendQualityChangedEvent function.

This function builds a JSON payload that includes the previous video rendition, the new video rendition, the name of the rendition (e.g. 480p), the channel being watched, and whether it was a step up or step down in video quality. We then call our custom pushPayload() function, which wraps a call to the ddRUM object to send this JSON payload to Datadog.

The JSON object for the video quality change event and all of the previously calculated metrics are then sent to Datadog as a custom action. A custom action is a way to track when a user interacts with the IVS Player with a custom JSON payload. We simply invoke the addAction method on the global DD_RUM object, passing in the name of the action as the first parameter and a JSON object as the second parameter. The name of the action will be PLAYBACK_SUMMARY and QUALITY_CHANGED. The name PLAYBACK_SUMMARY is for the JSON object containing all of our calculated QoE metrics in the last minute, while the name QUALITY_CHANGED is for the JSON object containing the information

As more events are sent every minute, we will have a large amount of data in Datadog that we can then graph. The next article will cover how we can visualize this data in Datadog and create a comprehensive dashboard to tell us the quality and health of the playback experience for viewers.

In addition to the data we computed ourselves and attached to the custom action sent to Datadog, the Datadog RUM Browser SDK also tracks and collects a lot of data points to help us get finer context when troubleshooting, e.g. information about the device, the browser, the Internet connection, or the geography the user is in, see the Datadog documentation for more details.

Let’s run a quick test. Open index.html, select a channel to play from a dropdown. If you are using Chrome, also open the developer tools console so we can display debug output. Notice in the console, we print out the player state as it changes, and calculations for the metrics that we will send to Datadog.

After viewing the stream for one minute, we should see confirmation that our JSON payload for our PLAYBACK_SUMMARY event was sent to Datadog.

Leveraging Amazon IVS and Datadog enables live streaming providers to optimize their video streaming analytics and ensure a high-quality viewing experience. By calculating key metrics and sending them to Datadog, providers can gain valuable insights into playback health and viewer engagement. In the next article, we will explore how to visualize these metrics by creating a comprehensive dashboard to assess playback quality, enabling data-driven decisions to further enhance the live streaming experience.

Tony Vu is a Senior Partner Engineer at Twitch. He specializes in assessing partner technology for integration with Amazon Interactive Video Service (IVS), aiming to develop and deliver comprehensive joint solutions to our IVS customers. Tony enjoys writing and sharing content on LinkedIn.

Jaiganesh Girinathan is a Senior Edge Specialist Solutions Architect focused on content delivery networks and edge computing capabilities with AWS. He has worked with several media customers globally over the last two decades, helping organizations modernize & scale their platforms. He is passionate about building solutions to address key customer needs. Outside of work, you can usually find Jaiganesh star gazing!

Riley Beadles is a Senior Customer Success Engineer at Amazon IVS. He combines his technical expertise with a customer centric mindset to ensure customers of IVS achieve their desired implementation with IVS. Riley enjoys maximizing value, while reducing effort.

Maël Lilensten is a Senior Product Manager at Datadog, in charge of the RUM, Tracing, and Logs SDK for mobile devices and smart TVs. He aims at building products for companies to monitor the performance of their frontend applications. As part of his scope, he’s working closely with streaming and broadcasting companies to break data silos and consolidate observability and video QoS on a single Datadog instance.

Jason Mimick is a Senior Partner Solutions Architect at AWS supporting Datadog working closely with the product, engineering, marketing, and sales teams daily. He’s spent the last 25+ years in tech, working for companies like Microsoft, Intersystems, and MongoDB before finding a home at AWS, Jason now focuses on enabling product development and sales success for partners and customers across all industries.

David Thornton is a Senior Customer Success Engineer at Amazon IVS. With 20 years in systems and network engineering, he now focuses on helping IVS customers reach their live streaming goals. When not working, you'll find David reading or trying to be better at video games.