Home

Awesome

Amazon ECS FireLens Examples

Sample logging architectures for FireLens on Amazon ECS and AWS Fargate.

Contributing

We want examples of as many use cases in this repository as possible! Submit a Pull Request if you would like to add something.

ECS Log Collection

Basic FireLens examples

AWS for Fluent Bit init tag examples

An init tag is distributed with each release, it adds useful features for ECS customers.

Multiline Examples

Monitoring Fluent Bit

Fluent Bit Examples

Fluentd Examples

Splitting an applications logs into multiple strings

Artifacts for the blog Splitting an application’s logs into multiple streams: a Fluent tutorial

Setup for the examples

Before you use FireLens, familiarize yourself with Amazon ECS and with the FireLens documentation.

In order to use these examples, you will need the following IAM resources:

Here is an example inline policy with S3 access for FireLens:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject"
      ],
      "Resource": [
        "arn:aws:s3:::examplebucket/folder_name/config_file_name"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetBucketLocation"
      ],
      "Resource": [
        "arn:aws:s3:::examplebucket"
      ]
    }
  ]
}

Using the Examples

You must update each Task Definition to reflect your own needs. Replace the IAM roles with your own roles. Update the log configuration with the values that you desire. And replace the app image with your own application image.

Additionally, several of these examples use a custom Fluent Bit/Fluentd configuration file in S3. You must upload it to your own bucket, and change the S3 ARN in the example Task Definition.

If you are using ECS on Fargate, then pulling a config file from S3 is not currently supported. Instead, you must create a custom Docker image with the config file.

Dockerfile to add a custom configs:

FROM amazon/aws-for-fluent-bit:latest
ADD extra.conf /extra.conf

Then update the firelensConfiguration options in the Task Definition to the following:

"options": {
    "config-file-type": "file",
    "config-file-value": "/extra.conf"
}

License Summary

This sample code is made available under the MIT-0 license. See the LICENSE file.