Live Streaming from Unreal Engine to Amazon IVS

This blog post was co-authored by Daniel Gonzales, Senior Worldwide Partner Solutions Architect, and Patrick Palmer, Principal Partner Solutions Architect, both at AWS.

In this blog post, we will explore the integration between Unreal Engine and Amazon Interactive Video Service (IVS), demonstrating how you can stream live content from Unreal Engine directly to Amazon IVS.

What is Amazon IVS?

Amazon IVS is a fully managed live streaming service that empowers developers to build low-latency and real-time streaming applications. It leverages the same video technology that powers Twitch, enabling developers to tap into the same scalable and global infrastructure for delivering live streaming video to audiences worldwide. With Amazon IVS, developers can focus on creating engaging content and innovating their applications rather than managing the underlying video technology.

Why live stream from Unreal Engine?

Imagine being able to unleash Unreal Engine's powerful graphics and rendering capabilities beyond the confines of your desktop computer with some of the following use cases.

Use Case	Description
Live Streaming of Gameplay	Engage the gamer’s community around their skills and experiences by enabling them to live stream their gameplay.
Real-time Sharing of Game Development Processes	Game development companies can share behind-the-scenes game development processes in real-time, allowing their audience to witness it firsthand.
Exclusive Sneak Peeks of Upcoming Titles	Game studios can build excitement and anticipation among their fan base by offering exclusive sneak peaks of upcoming gaming titles
Live Streaming Unreal Engine Tutorials and Workshops	Allows Unreal Engine experts to teach a mass audience how to use the software to create gaming or other content. This provides an interactive and engaging learning experience for the audience.

As you can see, there are a wide range of potential applications for live streaming Unreal Engine content. Whether you're a game developer, content creator, or a creative business looking to leverage the power of live streaming; this article will walk you through the process to get started with Unreal Engine and Amazon IVS. Let's dive in!

Prerequisites

Before we get started, let’s make sure we have the necessary tools and resources in place.

Unreal Engine. You will need to have Unreal Engine version 5.4.3 or later installed.
AWS Account. We will be leveraging the power of Amazon Web Services (AWS) to create the backend resources needed to live stream from Unreal Engine to Amazon IVS.

Once the prerequisites are in place, we follow the steps to harness live streaming:

Part 1 - Create an Unreal Editor project and Install Pixel Streaming Plugin
Part 2 - Create an IVS Stage on AWS
Part 3 - Write code to forward the media stream from Unreal Editor to IVS
Part 4 - Test it out

Solution Overview

At a high level, our solution will forward a media stream containing the contents of Unreal Engine to Amazon IVS for delivery to end viewers. We will install and use the Pixel Streaming Plugin in Unreal Engine which exposes a WebRTC media stream. We will stand up a WebRTC bridge server on our local machine to intercept the media stream. In order to intercept, our WebRTC bridge server will use a Go program to establish a connection to a locally running server that exposes a websocket endpoint of the Pixel Streaming Plugin. Once the media stream is intercepted, it will be forwarded to an Amazon IVS stage for delivery to integrated web and mobile apps using the Amazon IVS Broadcast SDK. With the Amazon IVS Broadcast SDK, developers can easily incorporate live streaming playback capabilities into their web and mobile apps across different platforms.

Streaming from Unreal Engine to Amazon IVS architecture overview

Create an Unreal Editor project and Install Pixel Streaming Plugin

From your Unreal Editor, create a new project. In this example, we will use the first-person shooter template.

Starting a new project will take you to the main Unreal Editor window. From here, let’s install the Pixel Streaming Plug-in. The Pixel Streaming Plug-in allows you to live stream the content from Unreal Editor directly to web browsers. The plug-in captures the rendered frames from the Unreal Editor and converts it into a media stream that is sent over the internet using the WebRTC protocol. Leveraging WebRTC's real-time communication capabilities, the plugin creates a media stream that it distributes to web browsers for viewing via a signaling server. However, instead of using the signaling server, we will write some custom code to forward that stream directly to Amazon IVS.

From your Unreal Editor window, click Edit -> Plugins.

Finding the plugins selection in the menu

In the pop up window, search for “Pixel Streaming” and select the Pixel Streaming plugin. You will then be prompted to restart UE

Searching for the Pixel Streaming plugin

After restarting, you should see Pixel Streaming as a selectable item in your UE toolbar.

Pixel Streaming in your Unreal Editor toolbar

Click on it and then select “Launch Signaling Server”.

This local server runs on your workstation, handling the initial connection and negotiation between the Unreal Engine application and the client's web browser. It also captures the rendered output from the Unreal Engine viewport, encodes it, and packages it as a media stream for transmission over WebRTC. We will write some additional code to intercept this media stream and forward it to Amazon IVS.

Next, click the Pixel Streaming Plugin in the toolbar again and select “Stream Level Editor”. This instructs the plugin to create a media stream solely from the gameplay viewport. You can also choose to stream the full level editor which will stream the gaming viewport and all the UI controls within the editor.

Finally, to start gameplay and create the media stream, click on the play button in the toolbar. This will create a media stream that we can send to IVS.

Starting gameplay and creating the media stream

Create an Amazon IVS Stage

Now that we have the Pixel Streaming Plugin setup and a media stream being sent, we will need to create an Amazon IVS Stage. Forwarding the media stream from the Pixel Streaming Plugin to the stage is necessary for delivering it at scale to viewers. Up to 25K concurrent viewers will be able to join the Stage to watch the stream with latency under 300ms.

From the Amazon IVS console in your AWS account, select Amazon IVS Stage and click Get Started.

Give the stage a name like “unreal-engine-demo”. We could optionally record each individual participant in our live stream, but for now leave that off and click “Create Stage”.

Once the stage has been created, it will present you with additional configuration information and a preview area to view your live stream directly in the console. Later on, we will be able to view our Unreal Engine gameplay here. Scroll down on the configuration page and click "Create token".

Name your token “unreal-engine-demo” and click create token. We give our token both publish and subscribe capabilities which allows whoever has that token to both send video to the stage and receive video from the stage.

Note, participant tokens can also be created programmatically using the AWS SDK. Now we will write the code to intercept the media stream created by the Pixel Streaming Plugin and forward it to the Amazon IVS stage we created.

Forwarding the media stream from Unreal Engine to a Stage

To forward the stream, we will create a small Go application that will run locally on our machine to forward the media stream from the Pixel Streaming Plug-in to the Amazon IVS stage we just created. The complete code is provided below for reference. The Go application runs on our local machine and connects to the WebSocket endpoint exposed by launching the signaling server from the Unreal Engine Pixel Streaming toolbar. It then establishes a WebRTC peer connection to stream audio and video data via the WHIP protocol to our stage. WHIP stands for WebRTC-HTTP Ingestion Protocol. Amazon IVS provides its own WHIP endpoint to enable this functionality. We will use the Pion WebRTC library to handle the WebRTC-related functionality required to set up a connection between Unreal Engine and Amazon IVS.

Let’s review this code piece by piece to understand what’s going on.

In the first few lines, we specify that the program should not be compiled for JavaScript environments and declares itself as the main package. We import various standard Go libraries for tasks like JSON processing, command-line flag parsing, HTTP operations, and time management. We also import the Pion WebRTC library to enable us to handle the WebRTC related functionality. The Websocket package is also imported so we can establish a websocket connection with the signaling server.

Next, we define the structure for JSON messages we will receive when connecting to the Pixel Streaming endpoint. We will use this to manage the communication between our Go program and the Pixel Streaming endpoint

Next, we write the main logic of our program responsible for setting up a WebSocket connection to the Pixel Streaming signaling server and managing the WebRTC communication. Let’s break this down into a few different parts.

Command-line flag parsing: First we define command line flags for the URL we want to connect to, an origin URL for the request that is passed in the HTTP header, and a participant token. The URL we want to connect to will default to http://localhost since our signaling server is running locally and is listening on port 80 for websocket connections. The origin will be http://localhost as well since we will be running this Go program locally. Finally, we also specify a participant token to authenticate with the IVS stage we just created. This can be thought of as the password that enables us to send audio and video from Unreal Engine to IVS.
Websocket connection: Next, we establish a websocket connection to the URL specified from above. We also set up a deferred function to close the connection when we exit the program.
Initial communication: After connecting to the websocket URL, we set up a for loop that continually parses incoming JSON messages from the websocket connection. The different type of messages we handle include
- config: Updates peer connection with configuration information
- offer: Also known as a WebRTC offer, we create a session with our IVS stage (more on this later), set up a peer connection with the signaling server from the Pixel Streaming Plugin (more on this later), and send an answer. This offer and answer process allows the our Go program and the signaling server to negotiate and agree on the parameters of the connection.
- iceCandidate: We use this to add ICE candidates to the peer connection. ICE stands for Interactive Connectivity Establishment. ICE candidates are network addresses that represent potential connection points for a device. Without ICE candidates, we wouldn’t be able to find the best connection point between our Go program and the signaling server. ICE candidates are also used to overcome network address translation (NAT) and firewall traversal issues that can prevent direct connections between peers. Here, both our Go program and our signaling server are running locally, so that won’t be an issue here.
- playerCount: Prints the number of connected players or the number of participants that have joined the stage
- streamerList: We use this JSON message to subscribe to the first available streamer. In this case our “streamer” is the media stream provided by the Pixel Streaming Plugin running in Unreal Engine.

Next, once we get a WebRTC offer from the websocket connection to the UE Pixel Streaming endpoint, we connect to the Amazon IVS stage we created earlier. We’ll call this function createStagesSession. A stage session just represents a period of activity when a participant joins a stage. In this case, our Go program is the participant joining the stage.

In this function, we set up a WebRTC connection for streaming audio and video to the IVS stage. After creating a peer connection, we add audio and video tracks, generate a WebRTC offer, and send it to the Amazon IVS WHIP endpoint. Using the participant token provided via the command line, we authenticate with the stage, create send-only transceivers for audio and video, and manage the WebRTC signaling process. It sends the offer to the IVS Stages server, receives an answer, and sets it as the remote description for the peer connection. The function returns the created audio and video tracks, which can then pass to the Unreal Pixel Streaming Plugin to write the media stream.

Using the audio and video tracks created earlier, we can then set up a WebRTC peer connection to communicate with Unreal Engine. We will name this function createUnrealPeerConnection. This function configures the media engine with H.264 video and Opus audio codecs, creates a peer connection, adds audio/video transceivers, and sets up a data channel. The H.264 video codec and Opus audio codec are industry-standard codecs used for efficient video and audio compression, respectively. The audio/video transceivers are responsible for handling the encoding, decoding, and transmission of the audio and video streams using these codecs. These codecs and transceivers help ensure high-quality, low-latency live streaming from the Unreal Engine Pixel Streaming plugin to IVS.

The function then establishes event handlers for ICE connection state changes, ICE candidates (sending them over WebSocket), and incoming media tracks. The incoming media tracks come from the Pixel Streaming Plugin. When tracks are received, it continuously reads from them and writes the data to the corresponding audio or video tracks we created earlier from our IVS stage. In this function, we essentially create a bridge between the Unreal Engine WebRTC media stream and the Amazon IVS stage to forward the audio/video.

With the code done, open your terminal and change to the directory with your code. Run the following to install the dependencies and start the Go program. Replace YOUR_PARTICIPANT_TOKEN_HERE with the participant token you created earlier in the IVS console

> go run ./main.go -token YOUR_PARTICIPANT_TOKEN_HERE

After running it, you should see output like the following

To witness your Unreal Engine gameplay streaming in action, go back to the IVS console for your stage and generate an additional participant token (you can name it whatever you wish). This will allow us to join the stage from our web browser to view the live stream with the content from Unreal Engine. With the token in hand, navigate to the IVS Real-Time Streaming Web Sample. This publicly accessible web sample is a straightforward web application that integrates the Amazon IVS Broadcast SDK for Web, allowing it to display the video and playback the audio that was transmitted to the stage. Simply paste in the participant token and you should see the Unreal Engine feed directly in your browser.

You can also see what the live stream looks like directly in the IVS console.

Unreal Engine live stream in the IVS console

As you play the game from Unreal Engine, take note of the delay between registering an action and when you see it in your web browser. This real-time latency, with a delay under 300 milliseconds, is a key feature of IVS real-time streaming.

Conclusion

In this article, we delved into the integration between Unreal Engine and Amazon IVS, enabling live streaming of content from Unreal Engine to Amazon IVS. By harnessing Unreal Engine's advanced graphics and rendering capabilities with the scalable and real-time latency capabilities of Amazon IVS, developers can immerse their audiences into the game development process and expand the reach of gaming globally.

As you try out this integration, we encourage you to experiment and discover innovative ways to harness the combined strengths of Unreal Engine and Amazon IVS to elevate your live streaming and gaming experiences for your audience.

About the authors

Tony Vu is a Senior Partner Engineer at Twitch. He specializes in assessing partner technology for integration with Amazon Interactive Video Service (IVS), aiming to develop and deliver comprehensive joint solutions to our IVS customers. Tony enjoys writing and sharing content on LinkedIn.

Daniel Gonzales is a Senior Worldwide Partner Solutions Architect for M&E and Games at AWS. He has spent over twenty years leading technical transformations, from content creation and distribution to audience engagement, and continues working with customers to achieve their business objectives.

Patrick Palmer is a Principal Partner Solutions Architect for Media and Entertainment at AWS. He has led in the entertainment 3D technology space for 25+ years, developing cutting-edge technologies and helping entertainment industry customers to navigate their greatest challenges.

Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.

Site Terms, Privacy, and more.