Learn how to set up an AWS IoT Greengrass managed edge MQTT 5.0 broker for integrating existing on-premise workloads

AWS IoT Greengrass is an open source Internet of Things (IoT) edge runtime and cloud service that helps you build, deploy and manage IoT applications on your devices. Depending on your individual application architecture, you may require a messaging system for communication between different edge components. Greengrass offers messaging options for different use-cases. This post explains the deployment and configuration of an MQTT 5.0 broker, that can be utilized both from within components on a Greengrass core device and other system components on the network.

After reading this post, you will be able to deploy a secure edge broker, that leverages AWS IoT Things for authentication and authorization. We will walk through the process of generating client credentials and utilizing the AWS IoT Device SDK v2 for python for interacting with the edge broker.

Imagine you are working on a project that aims to improve manufacturing production effectiveness by applying Machine Learning (ML) to infer optimal production parameters. At a given point in time, you will be looking for an automated way to integrating those recommendations into a control system at a plant. For this use case, you will require an architecture that allows the deployment of business logic to the edge, enabling interaction with other local components.

A common industry approach for for establishing messaging between loosely coupled IoT components is the usage of the Message Queuing Telemetry Transport (MQTT) protocol. The MQTT protocol is a lightweight, publish-subscribe messaging protocol, that was designed for machine-to-machine communication.

For the sake of keeping this guide simple, I created two small demo applications, that communicate over such a broker. This pattern is specifically interesting, if you want to integrate with components that do not run on the same Greengrass core device. If you are looking for simple communication between components on a single core device, take a look at Greengrass' interprocess communication (IPC) library

You can follow along with this post by either deploying AWS IoT Greengrass and the sample applications on physical hardware or in a virtual environment. Ensure that your compute units hosting the IoT Clients/Applications are connected to the same network, effectively allowing local MQTT connectivity. Since our IoT clients will perform a dynamic cloud-based discovery of the local MQTT endpoints, we will require connectivity to the internet in order to reach the AWS IoT Greengrass APIs.

Our sample application requires communication between system components that run in and outside our Greengrass core device. To facilitate the communication between the two system components, we decided to deploy an MQTT broker to the Greengrass core device. This broker allows Pub/Sub communication as defined in the MQTT standard.

Now that we understand the desired outcome, let's explore the required steps for achieving the above described architecture. As a prerequisite, we require a deployed Greengrass core device. If you don't have an active core device up and running, follow these instructions for installing Greengrass on your device.

First, we need to deploy a couple of components to the Greengrass core device, that are required to run the MQTT broker. Finally, we need to create certificates for our clients and integrate the AWS IoT Device SDK into our application to send and receive messages in our application.

Create a Greengrass Deployment to your core device that includes the following three Components. Ensure to overwrite the default configuration of the Auth component, to include your tailored security policy.

This component includes a modified version of the EMQX MQTT broker. EMQX implements the MQTT 5.0 protocol and supports its new features like session and message expiration intervals, user properties, shared subscriptions, topic aliases, and more. Note that this component requires Docker to be installed on the Greengrass core device, when deploying to a Linux environment.

The Auth component is used to authenticate client devices connecting to the broker. Furthermore, it also authorizes client device actions. This allows policy enforcement, controlling which client can subscribe or publish to certain topics.

The IPDetector component allows easy discovery of the broker endpoint address and port for your client devices. It automatically discovers the core device's network connectivity information and registers the MQTT broker endpoint information in the AWS IoT Greengrass cloud service.

The above described components can be deployed to your edge device through the AWS Management Console, or by using the following AWS CLI command.

Now it's time to create two AWS IoT Things for the clients that will connect to our broker. The following commands create a certificate, attach an IoT Policy to it and associate the AWS IoT Thing with your Greengrass core device. Run the following commands twice. Don't forget to overwrite the THING_NAME variable after completing the creation of the first client.

Now that we have deployed the broker to the edge device and created our client certificates, it's time to publish and subscribe some messages. To demonstrate this in action, we use a simple Python application, that is using the aws-iot-device-sdk-python-v2, which provides a MQTT 5.0 client and useful discovery utilities for dynamically looking up the edge broker endpoint information from the AWS IoT Greengrass cloud service. The sample application allows our first demo client to subscribe to the topic sample/topic. On a different device on the network, we will be able to use the sample application to publish to the same topic.

Start with installing the aws-iot-device-sdk-python-v2 on your machines. Distribute the previously generated certificate, key and AmazonRootCA file to your devices. Copy the python code for our sample applications to your machines. You can find the code at the end of this article.

On a device, connected to the same network as your Greengrass core device, run the following command to subscribe to the sample/topic topic.

You should observe, that our application successfully connected to the broker.

On the other device, run the following command to publish to the same topic.

Now you should see in the current terminal of thingB, that 10 messages have been published to the sample/topic topic.

If you now switch to the terminal of thingA, you should observe, that we have received the messages for topic sample/topic.

While the dynamic registration of the broker endpoint to the AWS IoT Greengrass cloud service is really convenient, there is one caveat to look out for. Running the EMQX component on a Linux based environment requires Docker to be installed on your core device. Docker will for most installations use its bridge network, which creates local network interfaces on your core device. Deployed containers will receive private IP addresses from that network, which allows them to communicate with other containers that are running on the same host. Systems that are not running on the Docker host, and hence are not connected to the local container network, will not be able to communicate over that network.

The IPDetector component registers two endpoints at the AWS IoT Greengrass cloud service. One that is using an IP address from the internal Docker network and another one that is using the IP address of the default interface of the core device. When connecting to the broker from another device, we must not use the Docker IP, since that address will not be reachable by the other system.

To overcome this hurdle, I extend the endpoint discovery by checking the network CIDR of a discovered endpoint and skipping it, if it’s a local Docker network address. You might want to adapt this logic to be more sophisticated for your own use cases.

With that, we have successfully deployed an MQTT 5.0 broker at the edge using AWS IoT Greengrass. You should have a running broker setup on your core device by now. If this article was helpful for you, I would appreciate a like and feel free to leave a comment.