Home

Awesome

logspout-fluentd

Logspout module for forwarding logs to fluentd and/or fluentbit. What's different about this module is that it uses the fluentd goland log driver to actually forward logs in the most efficient way similar to how the fluentd log driver works and with the same parameters it accepts. These parameters are configurable via environment variables.

Usage

This module works by acting as a fluentd forwarder. You can configuer the fixed prefix tag via the TAG_PREFIX environment variable and the dynamic tag suffix by another TAG_SUFFIX_LABEL environment variable that should point to a docker label.

You can run logspout the following way:

>> docker run --rm --name="logspout" \
			-v /var/run/docker.sock:/var/run/docker.sock \
			-e TAG_PREFIX=docker \
			-e TAG_SUFFIX_LABEL="com.mycompany.service" \
			-e FLUENTD_ASYNC_CONNECT="true" \
			-e LOGSPOUT="ignore" \
			<REGISTRY>/<CUSTOM_LOGSPOUT>:<VERSION> \
				./logspout fluentd://<FLUENTD_IP>:<FLUENTD_PORT>

Parameters required:

Optional environment variables:

The other fluentd specific environment variables include the list below and their explanation can be better read from the fluentd log driver website.

Configure Logspout to receive forwarded messages, something like this:

<source>
  type forward
  port 24224
  bind 0.0.0.0
</source>

# Assuming environment variable "TAG_PREFIX" is set to docker
<match docker.**>
  # Handle messages here.
</match>

Build and Run Locally


# Build logspout locally with custom fluentd module using Dockerfile
>> docker build --build-arg LOGSPOUT_VERSION=master -t mycustomlogspout .

# Example to run custom built logspout locally:
>> docker run --rm --name="logspout" \
			-v /var/run/docker.sock:/var/run/docker.sock \
			-e TAG_PREFIX=docker \
			-e TAG_SUFFIX_LABEL="com.mycompany.service" \
			-e FLUENTD_ASYNC_CONNECT="true" \
			-e LOGSPOUT="ignore" \
			mycustomlogspout \
				./logspout fluentd://<FLUENT_IP>:24224

Optional Testing


# Run standalone fluent-bit instance
>> docker run -ti -p 24224:24224 \
        fluent/fluent-bit:1.2 /fluent-bit/bin/fluent-bit \
            -i forward://0.0.0.0:24224 -o stdout


# Send sample log from test container, you should view the log entry
# captured in logspout and a similar log entry in fluent-bit.
>> docker run -i \
    --log-driver=json-file \
    --log-opt max-size=8m \
    --log-opt max-file=3 \
    --log-opt tag="docker" \
    -l com.mycompany.service="test" \
    alpine echo hello world

Demo