Home

Awesome

Serverless Reference Architecture: IoT Backend

The serverless IoT backend reference architecture is a general-purpose, event-driven, IoT data processing architecture that uses AWS Lambda. This architecture is ideal for workloads where IoT data needs to be ingested, custom rules applied and prepared for further analysis such as Machine Learning.

This architecture and sample implementation provides a framework for ingesting IoT messages utilizing AWS IoT Core and AWS Lambda.

In this demo humidity sensor application, IoT soil-moisture sensors deliver messages to the IoT backend. IoT rules check if humidity is under a set threshold and invokes a Lambda function if that's the case. The Lambda function sends an alert email notification. All of the IoT data is stored in S3 for downstream processing.

Architectural Diagram

Reference Architecture - IoT Backend

Application Components

Note downstream Data processing components in the architecture diagram - AWS IoT Analytics, Machine Learning, Amazon Kinesis Data Streams, and Amazon DynamoDB are not included in this deployment.

IoT Device/Thing

A soil moisture IoT sensor simulates a message with timestamp and soil humidity, and sends the message to an IoT Topic using the MQTT protocol.

IoT Core

AWS IoT Core is a managed cloud service that lets connected devices easily and securely interact with cloud applications and other devices. All the messages from the sensor are delivered to an IoT topic. IoT rules act on incoming data based on the condition defined. In this example, one IoT Rule is configured to save all messages to a S3 bucket. Another IoT rule checks if humidity in the message is less than certain threshold (set to 35 in this example) and invokes an AWS Lambda function . The AWS Lambda function parses the message to obtain device name and humidity, and sends an alert using an Amazon SNS Topic.

Downstream Processing

As mentioned in the Note, downstream processing is not implemented as part of this deployment and is shown as art of possible. The data ingested in AWS IoT Core is stored in a S3 bucket. This data can be analyzed using services like Amazon Sagemaker to train a machine learning model or can be analyzed using AWS IoT Analytics.

Running the Example

You can use the provided AWS CloudFormation template to launch a stack that demonstrates the Lambda file processing reference architecture. Details about the resources created by this template are provided in the CloudFormation Template Resources section of this document.

Deploying the template

** Important ** You can deploy the template in the following regions: us-east-1 or eu-west-1.

Change the SNSEmail address parameter to your own email and run the command on your terminal to deploy the CloudFormation template.

aws cloudformation deploy --template-file iot-backend.yaml --stack-name iot-backend --capabilities CAPABILITY_NAMED_IAM --parameter-overrides SNSEmail=test@amazon.lu --region eu-west-1

or you can directly go to CloudFormation console and upload the template there.

Seeing the IoT messages and alerts

After you successfully deploy the stack, you will observe the following:

  1. As part of the template, an Amazon EC2 instance will be provisioned and it will start sending messages to the IoT Core automatically. To see the messages, go to IoT Core console, then click "Test" and put # (without the quotes) in "Subscription topic" field which will start accepting messages from all topics. Keep every other field as is. Click "Subscribe to topic".
  2. You will see the messages flowing from the devices to the IoT Core in the console. Messages will look like below:
    {
        "name": "soilSensor3",
        "humidity": 29,
        "timeStampEpoch": 1598536072677,
        "timeStampIso": "2020-08-27T13:47:52.677721"
    }
  1. You will get an email with subject line “AWS Notification - Subscription Confirmation” to confirm that you want to get the notifications, in case the humidity sensor devices record data under a certain threshold. Confirm the subscription from the email.
  2. You should start getting emails for messages where humidity is below the threshold (< 35). Notification email will look like: Attention:Humidity Under Threshold Humidity 32 under threshold for device name soilSensor2
  3. To stop getting emails, go to the email you received, in step 3, and click sns-opt-out at the bottom of that email.
  4. To cleanup the resources after you are done, please perform the following steps.

Cleaning Up

To delete all the resources created in this example, please follow the following steps:

  1. Delete the s3 RuleFilterBucket: Go to S3 and your bucket will be with name like iot-backend-rulefilterbucket-xxxxxxxxxxx, delete all the contents of the bucket and then delete the bucket itself.
  2. Delete the IoT Policy: Go to the IoT Core > Secure > Policies, select serverless-iot-backend-policy and delete it.
  3. Delete the IoT Thing: Go to the IoT Core > Manage > Things, select serverless-iot-backend-thing and delete it.
  4. Finally, you should go to CloudFormation console and select the stack with name iot-backend and click the delete button. This will delete all the other resources created in this example.

CloudFormation Template Resources

Resources

The provided template creates the following resources: