Home

Awesome

Microsoft Azure IoT Protocol Gateway

Azure IoT protocol gateway is a framework for protocol adaptation that enables bi-directional communication with Azure IoT Hub. It is a pass-through component that bridges traffic between connected IoT devices and IoT Hub. The protocol gateway can be deployed in Azure using Cloud Services worker roles. It can also be deployed in on-premises environments.

The Azure IoT protocol gateway provides a programming model for building custom protocol adapters for variety of protocols. It includes an MQTT protocol adapter to demonstrate the concepts and to enable customizations of the MQTT v3.1.1 protocol. Please note that IoT Hub natively supports the MQTT v3.1.1 protocol and the MQTT protocol adapter should be considered only if specific customizations are required. The protocol gateway programming model also allows you to plug in custom components for specialized processing such as custom authentication, message transformations, compression/decompression, or encryption/decryption of traffic between the devices and IoT Hub.

This document describes how to get started using the Azure IoT protocol gateway including general overview, deployment, and setup instructions.

For guidance on customizations and extensibility of the protocol gateway, please refer to the Azure IoT Protocol Gateway - Developer Guide.

How to Build and Run

Prerequisites

The protocol gateway requires an Azure IoT hub to connect to and an Azure Storage account used for persisting state. You can also use the Azure Storage Emulator for the console or cloud hosts running locally.

To build the protocol gateway you need Microsoft Visual Studio 2013 (or later) and Microsoft Azure SDK 2.6 installed on your development machine. For deployment of the protocol gateway you need Microsoft Azure PowerShell installed.

Running the Console Host

The Azure IoT protocol gateway repository contains reference implementations of hosts in the ‘/host’ folder. The console app in this folder hosts a protocol gateway that connects to IoT Hub and starts listening for devices to connect (Note: by default the gateway listens on port 8883 for MQTTS and uses port 5671 for AMQPS traffic). In order to run the console sample please follow these steps:

  1. Provide the IoT hub connection string in the IotHubClient.ConnectionString setting in the app settings configuration file host\ProtocolGateway.Host.Console\appSettings.config.user. You can use the connection string for the device policy in the Shared access policies settings of IoT Hub.
  2. Run build.cmd or build the solution from Visual Studio. Both the build.cmd and the solution file are in the root folder of the repository
  3. Start the Microsoft Azure Storage Emulator
  4. As administrator, run the compiled binary (host\ProtocolGateway.Host.Console[Debug or Release]\Gateway.Host.Console.exe) If you run this from Visual Studio, then Visual Studio needs to be run as administrator. Note: If the required network ports are not enabled for communication with IoT Hub, you might be prompted to open the firewall ports.

Running an End-to-End Test

There is also an end-to-end test which exercises the following:

Please follow these steps in order to run the end-to-end test for the protocol gateway:

  1. Update the test configuration file (test\ProtocolGateway.Tests\appSettings.config.user) by providing the following settings:
    • IotHubClient.ConnectionString - connection string for your IoT hub that will be used for the test. For this test specifically you need to provide the connection string for the device policy in the Shared access policies settings of the IoT hub. Note: It is recommended to use an IoT hub instance specifically created for the tests. The test will register a device in the IoT hub and will use it for to exchange messages between the device and the simulated app back end.
    • End2End.DeviceName - identity of the device that will be used during the test. If the device name is not specified, a new device will be created for the test.
    • If you want to run the test against an already running protocol gateway, uncomment the End2End.ServerAddress setting in the test configuration file and provide the IP-address of the protocol gateway you want to use. Otherwise, the test will start a new in-process protocol gateway. If you want to connect to a running protocol gateway on the same machine you can use 127.0.0.1 as the IP address. (Note: by default, the gateway listens on port 8883 for MQTTS and uses port 5671 for AMQPS traffic).
  2. In Visual Studio, run the unit test EndToEndTests.BasicFunctionalityTest in the ProtocolGateway.Tests project. This test is expected to pass if the setup has been successful.

Deploying a Cloud Services Host

Please follow these steps to deploy the Cloud Services host in Azure Cloud Services worker roles:

  1. Open the solution in Visual Studio.
  2. Right-click on host/ProtocolGateway.Host.Cloud project and choose Package.
  3. Open command line and set current directory to <repo root>\host.
  4. Execute the following command: PowerShell -File deploy.ps1 <cloud service name> <storage account name> <location> <tls cert path> <tls cert password> <IoT Hub connection string> [-SubscriptionName <subscription name>] [-VmCount <VM count>]
    • <IoT Hub connection string> can contain any credentials as the default Device Identity Provider (SasTokenDeviceIdentityProvider) will override them with the ones provided by the device upon connection.
    • Please note that based on your local configuration you might need to change the PowerShell execution policy if you are receiving an execution error.
  5. You will be prompted to sign-in to your Azure account. Your account will be used for the deployment session.
  6. Wait for the script execution to complete.
  7. To verify the successful deployment, perform an end-to-end test as described in Running an End-to-End Test. Please note that you need to supply the public IP address of the Cloud Service in the End2End.ServerAddress setting.

Deploying a Service Fabric Host

The Service Fabric host runs the protocol gateway in a Service Fabric Cluster using standard logging and configuration techniques for that style of deployment. The following deployment is perfomed on a local Service Fabric cluster but may be adjusted to deploy to production / remote clusters.

  1. Update the IoT Hub client configuration file (ProtocolGateway.Host.Fabric.FrontEnd\PackageRoot\Config\FrontEnd.IoTHubClient.json) by providing the following settings:
    • ConnectionString - connection string for your IoT hub that will be used for the test. For this test specifically you need to provide the connection string for the device policy in the Shared access policies settings of the IoT hub. Note: It is recommended to use an IoT Hub instance specifically created for the tests. The test will register a device in the IoT hub and will use it to exchange messages between the device and the simulated app back end.
  2. Update the Azure Storage configuration file (ProtocolGateway.Host.Fabric.FrontEnd\PackageRoot\Config\FrontEnd.AzureState.json) by providing the following settings:
  1. Open the solution in Visual Studio.
  2. Right-click on host/ProtocolGateway.Host.Fabric project and choose Package.
  3. Open PowerShell and set current directory to <repo root>\host\ProtocolGateway.Host.Fabric\Scripts.
  4. Execute the following commands

Import-Module "$ModuleFolderPath\ServiceFabricSdk.psm1" Connect-ServiceFabricCluster -ConnectEndpoint 'localhost:19000' Test-ServiceFabricClusterConnection ./Deploy-FabricApplication.ps1 -ApplicationPackagePath '..\pkg\Release' -UseExistingClusterConnection

  1. You can see the deployed service through the Service Fabric Local Cluster Manager.

Diagnostics and Monitoring

Logging

The protocol gateway uses EventSource and Semantic Logging Application Block for logging (see https://msdn.microsoft.com/en-us/library/dn440729(v=pandp.60).aspx for details). By default, diagnostic events of all levels (including verbose) are logged to the console output in the Console host and to SLABLogsTable Azure Storage Table in the Cloud hosts.

In addition, the Cloud sample is also configured to collect Windows Event Log events and crash dumps. Please see https://msdn.microsoft.com/en-us/library/azure/dn186185.aspx for details on configuring Windows Azure Diagnostics.

Performance Counters

The protocol gateway emits the following performance counters under the “Azure IoT protocol gateway” performance counter category:

For the Cloud sample, Windows Azure Diagnostics is configured to collect protocol gateway as well as the default (processor and memory) performance counters. See https://msdn.microsoft.com/en-us/library/azure/dn535595.aspx for details.

Configuration Settings

The protocol gateway provides a number of configuration settings to help fine tune its behavior. Note that the default configuration provides a good starting point. Usually there is no need to adjust it, unless a specific behavior (like retransmission) is required. The Console sample derives the configuration settings from the appSettings.config.user file, while the Cloud sample derives them from the cloud service configuration.