We use essential cookies and similar tools that are necessary to provide our site and services. We use performance cookies to collect anonymous statistics, so we can understand how customers use our site and make improvements. Essential cookies cannot be deactivated, but you can choose “Customize” or “Decline” to decline performance cookies.
If you agree, AWS and approved third parties will also use cookies to provide useful site features, remember your preferences, and display relevant content, including relevant advertising. To accept or decline all non-essential cookies, choose “Accept” or “Decline.” To make more detailed choices, choose “Customize.”
Customize cookie preferences
We use cookies and similar tools (collectively, "cookies") for the following purposes.
Essential
Essential cookies are necessary to provide our site and services and cannot be deactivated. They are usually set in response to your actions on the site, such as setting your privacy preferences, signing in, or filling in forms.
Performance
Performance cookies provide anonymous statistics about how customers navigate our site so we can improve site experience and performance. Approved third parties may perform analytics on our behalf, but they cannot use the data for their own purposes.
Allowed
Functional
Functional cookies help us provide useful site features, remember your preferences, and display relevant content. Approved third parties may set these cookies to provide certain site features. If you do not allow these cookies, then some or all of these services may not function properly.
Allowed
Advertising
Advertising cookies may be set through our site by us or our advertising partners and help us deliver relevant marketing content. If you do not allow these cookies, you will experience less relevant advertising.
Allowed
Blocking some types of cookies may impact your experience of our sites. You may review and change your choices at any time by selecting Cookie preferences in the footer of this site. We and selected third-parties use cookies or similar technologies as specified in the AWS Cookie Notice.
Your privacy choices
We display ads relevant to your interests on AWS sites and on other properties, including cross-context behavioral advertising. Cross-context behavioral advertising uses data from one site or app to advertise to you on a different company’s site or app.
To not allow AWS cross-context behavioral advertising based on cookies or similar technologies, select “Don't allow” and “Save privacy choices” below, or visit an AWS site with a legally-recognized decline signal enabled, such as the Global Privacy Control. If you delete your cookies or visit this site from a different browser or device, you will need to make your selection again. For more information about cookies and how we use them, please read our AWS Cookie Notice.
This blog post was co-authored by AWS Senior Solutions Architects Jaiganesh Girinathan and Jason Mimick; Amazon IVS Customer Success Engineers Riley Beadles and David Thornton; and Datadog Senior Product Manager Maël Lilensten.
Live streaming has become an essential channel for entertainment, education, and communication. Ensuring a high-quality viewing experience is crucial to keep audiences engaged. Amazon Interactive Video Service (IVS) and Datadog offer a powerful combination to optimize live streaming with analytics. Amazon IVS lets you focus on building your own interactive application and audience experience. With Amazon IVS, you don't need to manage infrastructure or develop and configure components of your video workflows, to be secure, reliable, and cost effective.
In this series of articles, we'll explore three important use cases to gather insights into live stream video analytics.
Operational Analytics: Proactively monitor stream health in real-time to identify and address playback issues before they impact viewers. For instance, if video startup latency, or the time it takes to deliver the first frame of video to a viewer, spikes above historical averages, you can promptly investigate and address the issue to prevent potential viewer drop-off.
Playback Health Analytics: Reactively investigate viewer-reported issues and diagnose them to keep viewers engaged. For example, a surge in support tickets about network errors can help you pinpoint whether the issue lies with individual viewers or broader network conditions. This information can in turn be provided to your support teams to properly manage viewer expectations about time to resolution.
Business Analytics: Gain insights into viewer engagement and demographics to inform strategic business decisions. For instance, analyzing viewer locations and peak viewing times can help you tailor live streaming content to your audience. Additionally, understanding whether viewers prefer mobile or desktop devices can shape your content and distribution strategies.
To address these different use cases, we need to gather data on what is happening from ingest to playback of a live stream. The workflow starts when a streaming encoder sends a video stream to an IVS ingest point over one of the supported protocols: RTMP, RTMPS, or SRT. The output playback URL can then be distributed to viewers globally and viewed through the Amazon IVS Player. During a live streaming event, we can monitor the health of streams with Amazon IVS stream health metrics. These metrics are emitted from the service and provide statistics that give us insight into the quality and performance of our live video streams. Playback metrics are client side telemetry captured from the Amazon IVS player and help us understand the end-user viewing experience.
Image not found
Monitoring the health of our live streams using IVS stream health metrics and playback metrics will provide us with a holistic view of the quality of experience (QoE). In this first article, we will focus on how to capture playback metrics using Amazon IVS Player events and state updates. When a viewer interacts with the Amazon IVS Player, the player emits events based on the actions they have taken. Using these events, we will demonstrate how to calculate key metrics and how to send them to Datadog on a recurring basis. In the second article in this series, we will visualize the metrics in Datadog to better understand a live streaming experience.
In this first article, we will focus on calculating metrics that can be derived from the Amazon IVS Player SDK and sending them to Datadog. The second article will cover how to graph the metrics collected to create a dashboard to assess playback quality. The final article will cover how to monitor your live streams using Amazon IVS stream health metrics.
How to incorporate the Amazon IVS Player for Web SDK into a web application and play a live stream
How to calculate quality of experience (QoE) metrics using Amazon IVS Player events
How to incorporate Datadog’s Real User Monitoring (RUM) Browser SDK to send these events to Datadog
Solution Overview
This tutorial consists of four parts
Part 1 - Setting up the Amazon IVS Player for Web SDK
Part 2 - Incorporating the Datadog RUM Browser SDK
Part 3 - Calculating key QoE metrics to inform you of what is going on when viewers watch a live stream
Part 4 - Sending playback and QoE metrics to Datadog
Here’s an overview of what our solution looks like at a high level.
Image not found
Overview of how data flows from the Amazon IVS Player SDK to Datdog
We will be integrating the Amazon IVS Player for Web and Datadog RUM Browser SDK into a web app. The Amazon IVS Player emits event data, as mentioned earlier, which we can use to calculate various QoE metrics. Using the RUM Browser SDK, we can then programmatically generate events at the application level and send these metrics to Datadog as part of the events’ metadata on a recurring basis.
Datadog RUM's backend uses an event-driven architecture, with the client-side SDK recording events locally and uploading them in batches. The intake isolates traffic by data type, processes events, and sends them to a main queue. A reducer enriches parent events (Sessions and Views) with child event data (Actions, Errors, Resources, Long Tasks). As Datadog defines it:
A RUM Session represents the journey of a user browsing an application. It contains high-level information about the user (browser, device, geo-location), and aggregates all RUM events (Views, Actions like clicks and scrolls, Errors, etc.) collected during the user journey with a unique session.id attribute
A RUM View event is generated whenever a user visits a page within an application. While the user remains on the same page, other events (Actions, Errors, etc.) are linked to the related RUM View with the view.id attribute
Sessions can remain in the loop for up to 4 hours before timing out, while Views are reduced until a new one becomes active. Once closed, events are stored in blob storage, and dedicated services query this data for Datadog's web and mobile apps or alerting purposes. Note, sessions and Views can be queried while still being enriched and do not need to expire to be visible to users or Monitors.
Once these metrics are in Datadog, we can then create graphs and dashboards to help us understand the quality of the video playback experience.
Part 1 - Setting up the Amazon IVS Player for Web
To integrate the Amazon IVS Player into our application follow the getting started guide so that we can start playing a live stream in our web application. The steps are summarized below.
Create index.html
Add HTML elements to help style the page and a <video> element to play the live stream.
Add Amazon IVS Player SDK
Add in the <script> tag for the Amazon IVS Player, which will add an IVSPlayer object to the global context. We will then write JavaScript code to initiate the IVS Player and play a live stream.
Add in additional <script> tags
We will be creating three separate JavaScript files including config.js to set some test live streams for playback. The ivs.js file will contain code to initialize the IVS Player. Finally, the qos_sdk.js file will contain logic for calculating our quality of experience metrics.
To focus on the most relevant parts of this solution, we won’t be going over the CSS code so download main.css from Github to style the HTML. The complete HTML will look as follows.
Next, create ivs.js to initialize the IVS Player using the IVS Player object. We then attach the IVS player to the <video> element on our page to display the live stream.
There is also code to populate a dropdown on the page with the playback URLs we just included in config.js. When one of the playback URLs is selected from the dropdown, it uses the IVS player to load and play the URL.
(function (IVSPlayer) { // Initialize player const player = IVSPlayer.create(); player.attachHTMLVideoElement(videoPlayer);
// Setup stream and play player.setAutoplay(true);
// === populating the drop down of channels === var mySelect = $("#playback_url"); var playbackUrls = config.playbackUrls.split(","); $.each(playbackUrls, function (val, text) { mySelect.append( $("<option></option>") .val(text) .html(text.substring(text.lastIndexOf("channel") + 8)) ); });
Open index.html and we should now see a dropdown to select from one of four test live streams. The drop down displays a unique substring within each playback URL. This unique substring is also a substring that is present in the channel ARN. The channel ARN is a unique identifier for a channel in Amazon IVS, which holds the configuration information for a live stream. By doing this, we can later associate the playback URL to a channel.
Select one of these playback channels and click the play button to start playing a live stream. With the setup of the IVS Player complete, let’s move on to adding it to the Datadog RUM Browser SDK.
Image not found
Sample app with Amazon IVS Player SDK playing a live stream
Part 2 - Incorporating the Datadog Real User Monitoring Browser SDK
The Datadog RUM Browser SDK will enable our application to programmatically send custom data to Datadog. Using what is known as RUM custom actions, we can send our own metrics to Datadog calculated from Amazon IVS Player events on a regular basis such as every minute. This metric data can then be graphed in Datadog. Follow the instructions in the RUM Browser Monitoring Setup guide to get setup with the Datadog RUM Browser SDK before proceeding further.
After going through the setup guide, our HTML should now contain script tags for the Amazon IVS Player and Datadog RUM Browser SDK. Inclusion of the Datadog RUM Browser SDK script tag adds a global variable named DD_RUM to our window object. The DD_RUM object provides us with methods to initialize the Datadog RUM Browse SDK and send custom data to Datadog.
We will also add some HTML code to display the player and add a drop down so that we can play different live streams. In this sample app, we will be populating the dropdown with IVS test streams. We can then use the dropdown to play different test streams so that we can mimic what an end user might do while viewing different streams. This will let us display analytics on a per channel basis which we will get to later.
<html> <head> <scriptsrc="https://player.live-video.net/1.28.0/amazon-ivs-player.min.js"></script> <scriptsrc="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script> <linkrel="stylesheet"href="./css/main.css" /> <script> (function (h, o, u, n, d) { h = h[d] = h[d] || { q: [], onReady: function (c) { h.q.push(c); }, }; d = o.createElement(u); d.async = 1; d.src = n; n = o.getElementsByTagName(u)[0]; n.parentNode.insertBefore(d, n); })( window, document, "script", "https://www.datadoghq-browser-agent.com/us5/v5/datadog-rum.js", "DD_RUM" ); window.DD_RUM.onReady(function () { window.DD_RUM.init({ clientToken: "ADD YOUR DATADOG RUM APP CLIENT TOKEN", // Replace with your Datadog RUM app client token applicationId: "ADD YOUR DATADOG RUM APP ID", // Replace with your Datadog RUM app ID // `site` refers to the Datadog site parameter of your organization // see https://docs.datadoghq.com/getting_started/site/ site: "us1.datadoghq.com", service: "my-ivs-player-app", // Declare a service name for easier search in Datadog env: "dev", // Specify a version number to identify the deployed version of your application in Datadog version: "1.0.0", sessionSampleRate: 100, sessionReplaySampleRate: 0, startSessionReplayRecordingManually: true, trackUserInteractions: true, trackResources: true, trackLongTasks: true, defaultPrivacyLevel: "mask-user-input", allowFallbackToLocalStorage: true, }); }); </script> </head>
With the HTML added to display our live stream and the script tags added, we can now work on calculating our quality of experience metrics. These metrics will help us understand and measure the video playback quality experience from the viewer’s perspective.
Part 3 - Calculating key quality of experience (QoE) and playback metrics to inform you of what is going on when viewers watch a live stream.
Once we start playback of a live stream, the Amazon IVS Player emits a number of events and player states as it streams a live channel. Events can be emitted either because a user triggered it (e.g. by clicking pause on the player) or automatically by the IVS Player in response to network conditions to optimize playback.
You can attach a listener to the IVS player to capture some of these events and calculate important playback and QoE metrics through a process called ‘player instrumentation’. For the purposes of measuring QoE metrics, the table below outlines the important ‘IVS Player Event’, IVS Player State updates, and their contribution to calculating a metric.
IVS Player Event or State
Description
INITIALIZED
This event is emitted when the player is first created. Used to calculate the time it takes to play the first frame of video in combination with the READY and PLAYING states. Also known as time to video (TTV).
READY
This state is emitted when the player is ready to start the playback, used to initialize/reset all the tracking variables.
PLAYING
When the user requests for a channel to be played. The plugin will capture the current playback position, channel watched, startup latency in addition to the user and session ID.
BUFFERING
This state is emitted during playback when the player’s buffer is not filled in time and playback has stopped. Used to keep track of the number of buffering events during a playback session.
IDLE
If the previous state was "PLAYING", it calculates the time spent in the playing state since the last update and adds it to the playingTimeMsInLastMinute variable.
ENDED
If the previous state was "PLAYING", it calculates the time spent in the playing state since the last update and adds it to the playingTimeMsInLastMinute variable.
ERROR
Keeps a count of the number of errors that occurred in the current playback session. This is sent in the final summary once playback has ended.
QUALITY_CHANGED
Keeps track of current playback stream quality and emits the metric along with the information on the previous quality level and the newer quality level, rendition name and bitrate and direction in terms of whether the quality moved “UP” or “DOWN”
Using these events and state updates and built-in IVS Player methods, we will calculate the following metrics
Time to Video is the time it takes for the player to get the first frame of video and play it. The lower the time to video the better. This data point is crucial for engaging users and preventing them from abandoning the video.
Playback Latency Percentiles measures the distribution of delays between the time a video is sent from Amazon IVS servers and the time it is received and displayed by the Amazon IVS Player. This metric reports the 50th, 75th, 90th, and 95th percentile values of the playback latency distribution, providing insights into the higher end of the latency range experienced by viewers. Unlike an average, these percentile values are not skewed by outliers and better represent the typical and worst-case latency experienced by most viewers.
Unique Users by Region will show a color coded map of unique users by region. Darker regions indicate more users while lighter regions indicate less users. Hovering over a region will show a popup of the number of users viewing from that region. The Datadog RUM Browser SDK collects the end viewer’s country by default among other default attributes.
Unique Users by Rendition provides insights into the viewing experience of your live stream audience. It tracks the number of distinct users watching at each available video quality or "rendition" of the stream. This metric can help us understand the distribution of users across different bitrates and resolutions.
Errors Per Minute Per Channel is the number of errors that occur during video playback per minute of watched video by Amazon IVS channel. The IVS Player emits three possible error types.
Rebuffering Percentage measures the proportion of a viewer's total watch time that was spent waiting for video segments to download. This metric captures the average amount of time a viewer had to pause playback and wait for the video stream to buffer, relative to their total viewing time. A higher rebuffering percentage indicates more interruptions and a poorer viewing experience, as viewers had to spend a greater fraction of their session waiting for the video to resume playing.
Average playback time measures the average duration viewers spend watching the live stream, providing insights into audience engagement and content popularity.
Unique users is the number of distinct users that have accessed the live stream, providing insights into the overall audience size and reach of the content. To achieve this, recall that the data we sent to Datadog in the previous article was enriched with a custom generated unique user ID.
Unique User Count Per Top 10 Channels tracks the number of distinct users that have accessed the live stream on a specific Amazon IVS channel for the top 10 channels. This metric helps provide insights into the overall audience size and engagement for a particular channel.
Unique Users by Client Platform tracks the number of distinct users viewing the live stream, categorized by the operating system or device type (e.g. iOS, Android, web browser). This will help us understand the distribution of our audience across different client platforms, allowing us to prioritize features that would enhance the viewing experience for the most popular platforms.
Percent Time Buffering by Channel Watched measures the percentage of time viewers spend waiting for the video to buffer during playback, broken down by the specific channel they are watching. This metric helps identify any issues with video delivery or encoding that may be causing increased buffering for viewers on certain channels, allowing you to troubleshoot and optimize the experience.
Let’s first define and initialize all the QoE variables we will be using for tracking and reporting all of these different QoE metrics. We also initialize an instance of the Datadog RUM Browser SDK using window.DD_RUM. Recall earlier that when we added the <script> tag for Datadog RUM Browser SDK, it added a global variable named DD_RUM to our window object. Datadog lets us instrument custom events, with a lot of flexibility to add all the metadata we need, so the ddRUM object will later be used to attach these metrics to events and send them off to Datadog.
// === Define and initialize QoS event work variables === // timing control and auxiliary variables var hasBeenPlayingVideo = false; var lastReadyTime = -1; // milliseconds since Epoch, UTC, for computing startupLatencyMsOfThisSession var lastInitializedTime = -1; // milliseconds since Epoch, UTC, for computing timeToVideoMs var lastPlayerState = ""; var lastPlayerStateUpdateOrPlaybackSummaryEventSentTime = -1; // milliseconds since Epoch, UTC, for computing playing/bufferingTimeMsInLastMinute var lastPlaybackStartOrPlaybackSummaryEventSentTime = -1; // milliseconds since Epoch, UTC, for the timing of sending playback summary events
// payload of events var userId = ""; // unique UUID of each device if localStorage is supported, otherwise set to sessionId of each playback session var sessionId = ""; // unique UUID of each playback session var startupLatencyMsOfThisSession = 0; var timeToVideoMs = 0; var playingTimeMsInLastMinute = 0; var bufferingTimeMsInLastMinute = 0; var bufferingCountInLastMinute = 0; var lastQuality = undefined; // the latest rendition being played // === Define and initialize QoS event work variables === // store whether this is a live channel var isLive = false; // store the channel name var channelWatched = "";
Create a function called initializeQoS, which will be the main entry point in our code. This function attaches event listeners to the IVS Player for all of the events listed in the table earlier. The initializeQoS function also contains a function setInterval(), which will send playback summary events every minute. Once a playback summary is sent, it resets the QoE variables so that they can be calculated for the next minute.
For reference, here’s the complete code. In the rest of this section, we will explain what each part of the code is doing.
setUserIDSessionID(player); startupLatencyMsOfThisSession = 0; playingTimeMsInLastMinute = 0; bufferingTimeMsInLastMinute = 0; bufferingCountInLastMinute = 0; errorCountInLastMinute = 0; lastQuality = undefined; // === Send off playback end event and reset QoS event work variables === });
player.addEventListener(PlayerState.BUFFERING, function () { console.log("Player State - BUFFERING");
// === Update QoS event work variables === if (lastPlayerState == "PLAYING") { // PLAYING -> BUFFERING (can only happen in the middle of a playback session) playingTimeMsInLastMinute += Date.now() - lastPlayerStateUpdateOrPlaybackSummaryEventSentTime; bufferingCountInLastMinute += 1; }
setPlayerStateVariables("BUFFERING"); // === Update QoS event work variables === });
player.addEventListener(PlayerState.PLAYING, function () { console.log("Player State - PLAYING");
if (startupLatencyMsOfThisSession == 0) { // the very beginning of a playback session lastPlaybackStartOrPlaybackSummaryEventSentTime = Date.now(); startupLatencyMsOfThisSession = Date.now() - lastReadyTime; console.log( "startupLatencyMsOfThisSession: " + startupLatencyMsOfThisSession );
if (lastQuality === undefined) { lastQuality = player.getQuality(); } } else { if (lastPlayerState == "BUFFERING") { // BUFFERING -> PLAYING (in the middle of a playback session) bufferingTimeMsInLastMinute += Date.now() - lastPlayerStateUpdateOrPlaybackSummaryEventSentTime; } }
setPlayerStateVariables("PLAYING"); // === Send off playback start event and update QoS event work variables === });
player.addEventListener(PlayerState.IDLE, function () { console.log("Player State - IDLE");
We also parse the unique channel identifier from the playback URL using a custom function called getChannelWatched(). The implementation for this function is as follows and should be placed outside the initializeQoS() function.
1 2 3 4 5 6 7 8
functiongetChannelWatched(playbackUrl) { var myIndex1 = playbackUrl.indexOf("channel.") + 8; var myIndex2 = playbackUrl.indexOf(".m3u8"); var channelName = playbackUrl.substring(myIndex1, myIndex2); console.log("playbackUrl ", playbackUrl); console.log("Channel name :", channelName); return channelName; }
Next, we create a callback function to record the time when the player tells us when it is initialized. We will save that timestamp in the variable `lastInitializedTime`. This timestamp will be useful later in calculating the time to video metric or the time it takes for the player to get the first frame of video.
setUserIDSessionID(player); startupLatencyMsOfThisSession = 0; playingTimeMsInLastMinute = 0; bufferingTimeMsInLastMinute = 0; bufferingCountInLastMinute = 0; errorCountInLastMinute = 0; lastQuality = undefined; // === Send off playback end event and reset QoS event work variables === });
Next, we attach a listener to tap into buffering events from the IVS Player. We will use this state update to keep track of the number of buffering events during a playback session. If before this state, the player was playing a video, then we need to adjust the variable that tracks the playing time. We also increment the number of times the player buffered in the last minute.
1 2 3 4 5 6 7 8 9 10 11 12 13 14
player.addEventListener(PlayerState.BUFFERING, function () { console.log("Player State - BUFFERING");
// === Update QoS event work variables === if (lastPlayerState == "PLAYING") { // PLAYING -> BUFFERING (can only happen in the middle of a playback session) playingTimeMsInLastMinute += Date.now() - lastPlayerStateUpdateOrPlaybackSummaryEventSentTime; bufferingCountInLastMinute += 1; }
setPlayerStateVariables("BUFFERING"); // === Update QoS event work variables === });
Finally, we call a custom function called setPlayerStateVariables. We call this function to update the variable used to track the player’s last state and the timestamp when this function was last called. Doing this will be useful in the calculation of the other QoE metrics as you will see soon. Here’s the implementation for this function. This utility function should be placed outside of the initializeQoS function.
Next, we create a callback function to calculate the startup latency and record the playback stream quality/rendition when the IVS Player starts playing a video. Similar to before, we also call setPlayerStateVariables to keep track of the player’s previous state.=
if (lastQuality === undefined) { lastQuality = player.getQuality(); } } else { if (lastPlayerState == "BUFFERING") { // BUFFERING -> PLAYING (in the middle of a playback session) bufferingTimeMsInLastMinute += Date.now() - lastPlayerStateUpdateOrPlaybackSummaryEventSentTime; } }
setPlayerStateVariables("PLAYING"); // === Send off playback start event and update QoS event work variables === });
Next, we create a callback function to calculate two QoE metrics related to playing time and buffering time if the IVS Player goes idle. The first metric, playingTimeMsInLastMinute, calculates how much time was spent playing the video in the last minute based on the current time and the last time the player state was updated. In the previous callback functions, we had been calling the setPlayerStateVariable function to keep track of the last time the player state was updated. The second metric, bufferingTimeMsInLastMinute, tells us how much time was spent buffering in the last minute.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
player.addEventListener(PlayerState.IDLE, function () { console.log("Player State - IDLE");
setPlayerStateVariables("IDLE"); // === Update QoS event work variables === });
Next, we create a callback function to calculate the time the IVS Player spent playing a video since the last time it changed its state. It then adds this time to the playingTimeMsInLastMinute variable. This variable tracks how much time the IVS Player spent playing a video in the last minute.
1 2 3 4 5 6 7 8 9 10 11 12 13
player.addEventListener(PlayerState.ENDED, function () { console.log("Player State - ENDED");
// === Update QoS event work variables === if (lastPlayerState == "PLAYING") { // PLAYING -> ENDED playingTimeMsInLastMinute += Date.now() - lastPlayerStateUpdateOrPlaybackSummaryEventSentTime; }
setPlayerStateVariables("ENDED"); // === Update QoS event work variables === });
Sending IVS Player errors to Datadog
Next, when the IVS Player emits an event indicating that it encountered an error, we do two things. We can keep track of the number of errors that have occurred in the last minute and we can send the detailed error information to Datadog. The Datadog RUM Browser SDK provides us with an API to send browser errors to Datadog in a more natural way. The benefit is that this appears in a separate error event in Datadog, with a full deobfuscated stack trace and many out-of-the-box attributes. It also provides more advanced features such as Error Tracking, which groups individual errors based on their similarity for faster triaging and troubleshooting. This helps bring more attention to this critical data point that could affect the live streaming experience for users. To use this API, we create a new Error object and can provide the error code, a string indicating the source of the error, and a message associated with the error. We can then call the addError method using the DD_RUM object to send this error information to Datadog.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
player.addEventListener(PlayerEventType.ERROR, function (err) { console.warn("Player Event - ERROR:", err);
Next, we will create a callback function to keep track of the rendition or video quality during a live stream. This can happen either because the user explicitly selects a resolution in the player or because the IVS Player automatically adapts to changing network conditions to maintain uninterrupted playback. The end user’s bandwidth could have decreased for example, necessitating a step down in the video rendition. In this callback function, we save both the previous video rendition and the current video rendition.
// === Send off quality change event and update QoS event work variables === let newQuality = player.getQuality(); if (lastQuality === undefined) { lastQuality = newQuality; log(`Quality initialized to "${lastQuality.name}".`); } elseif (lastQuality.bitrate != newQuality.bitrate) { log( `Quality changed from "${lastQuality.name}" to "${newQuality.name}".` ); sendQualityChangedEvent(lastQuality, newQuality); lastQuality = newQuality; } // === Send off quality change event and update QoS event work variables === });
At the end of this callback function, we send off a quality change event to Datadog using a custom sendQualityChangedEvent() function. We will cover the implementation of that function in the next section.
Finally, to wrap up the initializeQoS function, we send in a custom event a summary containing all the metrics we have just calculated in all our previous event listeners to Datadog. These metrics will be sent from our custom sendPlaybackSummaryEventIfNecessary function. We will show how this function is implemented in the next section. Using the setInterval function, we will send these metrics off to Datadog every minute. Once the metrics are sent, we reset all of our metrics tracking variables to zero so that we can recalculate them for the next minute.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
// === Send off a QoS event every minute === setInterval(function () { if ( lastPlaybackStartOrPlaybackSummaryEventSentTime != -1 && Date.now() - lastPlaybackStartOrPlaybackSummaryEventSentTime > 60000 ) { sendPlaybackSummaryEventIfNecessary(player);
Part 4 - Sending playback and QoE metrics to Datadog
Now that we have covered how to add event and state update listeners to the IVS Player to calculate the metrics, we will cover how they are sent to Datadog.
In the last section we used the custom sendPlaybackSummaryEventIfNecessary function within the initializeQoS function to send the calculated metrics to Datadog. In this sendPlaybackSummaryEventIfNecessary function, we create a JSON object that contains a summary of all of our QoE metrics. We first make some adjustments to the playing time and buffering time variables, making sure to capture any uncounted time if the IVS Player was in either the playing or buffering state. Additionally, we will include the number of frames that have been dropped so far during the playback session. We can get this using the getDroppedFrames() method exposed by the IVS Player.
We also include the live latency metric in this JSON object. The live latency can be retrieved using the getLiveLatency method provided by the IVS Player. Live latency tells us the time it takes for a video frame to be sent IVS to the player. We multiply it by 1,000 since it is returned to us in seconds and we want to display it in milliseconds to be consistent with our other metrics.
// Send playback QoS summary (PLAYBACK_SUMMARY) event functionsendPlaybackSummaryEventIfNecessary(player) { if (lastPlayerState == "PLAYING") { // collect the uncounted time in the PLAYING state playingTimeMsInLastMinute += Date.now() - lastPlayerStateUpdateOrPlaybackSummaryEventSentTime; } elseif (lastPlayerState == "BUFFERING") { // Collect the uncounted time in the BUFFERING state bufferingTimeMsInLastMinute += Date.now() - lastPlayerStateUpdateOrPlaybackSummaryEventSentTime; }
if (playingTimeMsInLastMinute > 0 || bufferingTimeMsInLastMinute > 0) { var myJson = {}; myJson.metric_type = "PLAYBACK_SUMMARY";
Additionally, we include a userId and sessionId in the JSON object. The userId is a random UUID that is set as the very first session ID of the user that played a given live stream using the IVS Player. It remains the same across different playback sessions unless explicitly reset. We use UUID as our userId just as an example. Instead of UUID, we could instead use another unique identifier associated with the users of our application.
A sessionId represents a unique identifier for each instance of the IVS Player when it is playing a live stream. If an end user for example has several browser windows open each with a separate instance of the IVS Player playing a live stream, each of these would count as a separate play session and have a unique session ID. The session ID is retrieved using the getSessionId method from the IVS Player. Associating all of these metrics to a given user ID or session ID will be important for how we graph our metrics in Datadog later.
Finally, we add the ddRUM object to set the user ID associated with our RUM session using the setUser method. This will enable us to more easily query and filter for the user associated with this metrics data in Datadog. We can also optionally add other user attributes using `setUser` to make it easier to filter users based on certain attributes
Note: Datadog RUM Browser SDK generates its own session ID. However, it does not match the granularity we need as multiple play sessions from the IVS Player might be merged together under a single Datadog RUM session. Therefore, we are passing the session ID provided by the IVS Player as a custom attribute to the events we send to Datadog.
if (typeofStorage !== "undefined") { if (!localStorage.getItem("ivs_qos_user_id")) { localStorage.setItem("ivs_qos_user_id", sessionId); } userId = localStorage.getItem("ivs_qos_user_id"); } else { log("Sorry! No web storage support. Use Session ID as User Id"); userId = sessionId; } ddRum.setUser({ id: userId }); }
Next, we create another JSON structure to send a QUALITY_CHANGED event to Datadog whenever the IVS Player switches to a different rendition (change in quality, bitrate). This payload is sent to Datadog using the custom sendQualityChangedEvent function.
This function builds a JSON payload that includes the previous video rendition, the new video rendition, the name of the rendition (e.g. 480p), the channel being watched, and whether it was a step up or step down in video quality. We then call our custom pushPayload() function, which wraps a call to the ddRUM object to send this JSON payload to Datadog.
The JSON object for the video quality change event and all of the previously calculated metrics are then sent to Datadog as a custom action. A custom action is a way to track when a user interacts with the IVS Player with a custom JSON payload. We simply invoke the addAction method on the global DD_RUM object, passing in the name of the action as the first parameter and a JSON object as the second parameter. The name of the action will be PLAYBACK_SUMMARY and QUALITY_CHANGED. The name PLAYBACK_SUMMARY is for the JSON object containing all of our calculated QoE metrics in the last minute, while the name QUALITY_CHANGED is for the JSON object containing the information
As more events are sent every minute, we will have a large amount of data in Datadog that we can then graph. The next article will cover how we can visualize this data in Datadog and create a comprehensive dashboard to tell us the quality and health of the playback experience for viewers.
In addition to the data we computed ourselves and attached to the custom action sent to Datadog, the Datadog RUM Browser SDK also tracks and collects a lot of data points to help us get finer context when troubleshooting, e.g. information about the device, the browser, the Internet connection, or the geography the user is in, see the Datadog documentation for more details.
Let’s run a quick test. Open index.html, select a channel to play from a dropdown. If you are using Chrome, also open the developer tools console so we can display debug output. Notice in the console, we print out the player state as it changes, and calculations for the metrics that we will send to Datadog.
After viewing the stream for one minute, we should see confirmation that our JSON payload for our PLAYBACK_SUMMARY event was sent to Datadog.
Image not found
Amazon IVS Player sending QoE event data to Datadog
Conclusion
Leveraging Amazon IVS and Datadog enables live streaming providers to optimize their video streaming analytics and ensure a high-quality viewing experience. By calculating key metrics and sending them to Datadog, providers can gain valuable insights into playback health and viewer engagement. In the next article, we will explore how to visualize these metrics by creating a comprehensive dashboard to assess playback quality, enabling data-driven decisions to further enhance the live streaming experience.
About the Authors
Tony Vu is a Senior Partner Engineer at Twitch. He specializes in assessing partner technology for integration with Amazon Interactive Video Service (IVS), aiming to develop and deliver comprehensive joint solutions to our IVS customers. Tony enjoys writing and sharing content on LinkedIn.
Jaiganesh Girinathan is a Senior Edge Specialist Solutions Architect focused on content delivery networks and edge computing capabilities with AWS. He has worked with several media customers globally over the last two decades, helping organizations modernize & scale their platforms. He is passionate about building solutions to address key customer needs. Outside of work, you can usually find Jaiganesh star gazing!
Riley Beadles is a Senior Customer Success Engineer at Amazon IVS. He combines his technical expertise with a customer centric mindset to ensure customers of IVS achieve their desired implementation with IVS. Riley enjoys maximizing value, while reducing effort.
Maël Lilensten is a Senior Product Manager at Datadog, in charge of the RUM, Tracing, and Logs SDK for mobile devices and smart TVs. He aims at building products for companies to monitor the performance of their frontend applications. As part of his scope, he’s working closely with streaming and broadcasting companies to break data silos and consolidate observability and video QoS on a single Datadog instance.
Jason Mimick is a Senior Partner Solutions Architect at AWS supporting Datadog working closely with the product, engineering, marketing, and sales teams daily. He’s spent the last 25+ years in tech, working for companies like Microsoft, Intersystems, and MongoDB before finding a home at AWS, Jason now focuses on enabling product development and sales success for partners and customers across all industries.
David Thornton is a Senior Customer Success Engineer at Amazon IVS. With 20 years in systems and network engineering, he now focuses on helping IVS customers reach their live streaming goals. When not working, you'll find David reading or trying to be better at video games.
Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.
