Awesome

ansible-role-fluentd

Configures fluentd.

Notes for FreeBSD

The sysutils/rubygem-fluentd port does not patch the gem to include fluentd_plugin_dir, or /usr/local/etc/fluentd/plugin, by default (as of version 0.12.14_1). If you manage local plugins in the directory, you need to set fluentd_flags.

fluentd_flags: "-p {{ fluentd_plugin_dir }}"

See Add a Plugin Via /etc/fluent/plugin for more details.

Note that fluentd_plugins_to_install does not use fluentd_plugin_dir.

Requirements

None

Role Variables

variable	description	default
`fluentd_user`	the user of `fluentd`	`{{ __fluentd_user }}`
`fluentd_group`	the group of `fluentd`	`{{ __fluentd_group }}`
`fluentd_extra_groups`	list of extra group names (preferred) or a comma-separated string of group names (deprecated, for backward-compatibility) that `fluentd` user belongs to	`[]`
`fluentd_package_name`	package name of `fluentd`	`{{ __fluentd_package_name }}`
`fluentd_config_dir`	path to config directory	`{{ __fluentd_config_dir }}`
`fluentd_config_file`	path to `fluent.conf`	`{{ __fluentd_config_file }}`
`fluentd_config_fragment_dir`	path to `conf.d` directory	`{{ fluentd_config_dir }}/conf.d`
`fluentd_service_name`	service name of `fluentd`	`{{ __fluentd_service_name }}`
`fluentd_plugin_dir`	path to directory where local plugins reside	`{{ fluentd_config_dir }}/plugin`
`fluentd_flags`	optional command line flags for the service	`{{ __fluentd_flags }}`
`fluentd_gem_bin`	path to `fluent-gem`	`{{ __fluentd_gem_bin }}`
`fluentd_plugins_to_install`	list of plug-in names to install	`[]`
`fluentd_plugins_to_create`	list of plug-ins to create (see below)	`[]`
`fluentd_certs_dir`	path to directory where cert files reside	`{{ __fluentd_config_dir }}/certs`
`fluentd_configs`	dict of config fragments, see below	{}
`fluentd_ca_key`	content of `ca_key.pem`	""
`fluentd_ca_cert`	content of `ca_cert.pem`	""
`fluentd_ca_private_key_passphrase`	the passphrase of `ca_key.pem`	""
`fluentd_buffer_path`	path to file-based buffer directory	`/var/spool/fluentd`
`fluentd_unix_pipe_dir`	path to directory where `AF_UNIX` pipe should be created	`{{ __fluentd_unix_pipe_dir }}`
`fluentd_log_dir`	path to directory where `fluentd` can write logs. Set `None` to disable	`/var/log/{{ fluentd_service_name }}`
`fluentd_log_file`	path to log file	`{{ fluentd_log_dir }}/{{ fluentd_service_name }}.log`
`fluentd_system_config`	a string that is enclosed by `<system>` tag in `fluentd.conf`. use `	` in yaml to set multiple lines of system-wide configurations
`fluentd_pid_dir`	path to PID directory	`"{{ __fluentd_pid_dir }}"`
`fluentd_pid_file`	path to PID file	`"{{ __fluentd_pid_file }}"`

Note that although the role provides fluentd_log_dir and fluentd_log_file, you need to configure fluentd to log to fluentd_log_file. The role does NOT configures fluentd to log to the file. See Example Playbook for how.

`fluentd_plugins_to_create`

This variable is list of dict. The dict is described below. The role creates plug-ins listed in this variable under fluentd_plugin_dir.

Key	Description	Mandatory?
name	file name of the plug-in	yes
content	content of the plug-in, must be valid ruby code	yes
state	state of the plug-in, either `present` or `absent`	yes

`fluentd_configs`

Key is the name of the config fragment file. the key has a hash described below. The role creates a configuration fragment of config under fluentd_config_fragment_dir.

key	value
enabled	bool, create the config if true, remove if false
config	the configuration

Debian

Variable	Default
`__fluentd_user`	`td-agent`
`__fluentd_group`	`td-agent`
`__fluentd_package_name`	`td-agent`
`__fluentd_service_name`	`td-agent`
`__fluentd_config_dir`	`/etc/td-agent`
`__fluentd_config_file`	`{{ fluentd_config_dir }}/td-agent.conf`
`__fluentd_bin`	`/usr/sbin/td-agent`
`__fluentd_gem_bin`	`/usr/sbin/td-agent-gem`
`__fluentd_unix_pipe_dir`	`/var/tmp/fluentd`
`__fluentd_flags`	`""`
`__fluentd_pid_dir`	`/var/run/td-agent`
`__fluentd_pid_file`	`{{ fluentd_pid_dir }}/td-agent.pid`

FreeBSD

Variable	Default
`__fluentd_user`	`fluentd`
`__fluentd_group`	`fluentd`
`__fluentd_package_name`	`rubygem-fluentd`
`__fluentd_service_name`	`fluentd`
`__fluentd_config_dir`	`/usr/local/etc/fluentd`
`__fluentd_config_file`	`{{ fluentd_config_dir }}/fluent.conf`
`__fluentd_bin`	`/usr/local/bin/fluentd`
`__fluentd_gem_bin`	`/usr/local/bin/fluent-gem`
`__fluentd_unix_pipe_dir`	`/var/tmp/fluentd`
`__fluentd_flags`	`""`
`__fluentd_pid_dir`	`/var/run/fluentd`
`__fluentd_pid_file`	`{{ fluentd_pid_dir }}/fluentd.pid`

OpenBSD

Variable	Default
`__fluentd_user`	`_fluentd`
`__fluentd_group`	`_fluentd`
`__fluentd_package_name`	`rubygem-fluentd`
`__fluentd_service_name`	`fluentd`
`__fluentd_config_dir`	`/etc/fluentd`
`__fluentd_config_file`	`{{ fluentd_config_dir }}/fluent.conf`
`__fluentd_bin`	`/usr/local/bin/fluentd23`
`__fluentd_gem_bin`	`/usr/local/bin/fluent-gem`
`__fluentd_unix_pipe_dir`	`/var/tmp/fluentd`
`__fluentd_flags`	`""`
`__fluentd_pid_dir`	`/var/run/fluentd`
`__fluentd_pid_file`	`{{ fluentd_pid_dir }}/fluentd.pid`

RedHat

Variable	Default
`__fluentd_user`	`td-agent`
`__fluentd_group`	`td-agent`
`__fluentd_package_name`	`td-agent`
`__fluentd_service_name`	`td-agent`
`__fluentd_config_dir`	`/etc/td-agent`
`__fluentd_config_file`	`{{ fluentd_config_dir }}/td-agent.conf`
`__fluentd_bin`	`/usr/sbin/td-agent`
`__fluentd_gem_bin`	`/usr/sbin/td-agent-gem`
`__fluentd_unix_pipe_dir`	`/var/tmp/fluentd`
`__fluentd_flags`	`""`
`__fluentd_pid_dir`	`/var/run/td-agent`
`__fluentd_pid_file`	`{{ fluentd_pid_dir }}/td-agent.pid`

Dependencies

reallyenglish.language-ruby (OpenBSD, Ubuntu, and RedHat)

Example Playbook

- hosts: localhost
  roles:
    - ansible-role-fluentd
  vars:
    fluentd_extra_groups: tty,bin
    fluentd_flags: "{% if ansible_os_family == 'FreeBSD' %}-p {{ fluentd_plugin_dir }}{% elif ansible_os_family == 'Debian' %}-p {{ fluentd_plugin_dir }}{% elif ansible_os_family == 'RedHat' %}{% elif ansible_os_family == 'OpenBSD' %}--daemon /var/run/fluentd/fluentd.pid --config /etc/fluentd/fluent.conf -p /etc/fluentd/plugin{% endif %} --log {{ fluentd_log_file }}"
    fluentd_system_config: |
      log_level debug
      suppress_config_dump
    fluentd_plugins_to_install:
      - fluent-plugin-redis
      - fluent-plugin-secure-forward
    fluentd_plugins_to_create:
      - name: example.rb
        content: "{{ lookup('file', 'files/example.rb') }}"
        state: present
    fluentd_ca_cert: |
      -----BEGIN CERTIFICATE-----
      MIIDIDCCAggCAQEwDQYJKoZIhvcNAQEFBQAwTTELMAkGA1UEBhMCVVMxCzAJBgNV
      BAgMAkNBMRYwFAYDVQQHDA1Nb3VudGFpbiBWaWV3MRkwFwYDVQQDDBBTZWN1cmVG
      b3J3YXJkIENBMB4XDTcwMDEwMTAwMDAwMFoXDTIxMDQxNTE3MDE1OVowTTELMAkG
      A1UEBhMCVVMxCzAJBgNVBAgMAkNBMRYwFAYDVQQHDA1Nb3VudGFpbiBWaWV3MRkw
      FwYDVQQDDBBTZWN1cmVGb3J3YXJkIENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A
      MIIBCgKCAQEAsubDxMCBata8T8++x85nlsEyXT+fEXseZAln+RffZJqqdsJtbLmc
      /a7u40IefQBShm3itba0dMTsPnG8rrvLtkz+3TSuN6wBTR+iM5/vBlu9Z8b983c9
      HZ/RkDzhlucqzDSIstYbpLDefiw78ME2kpzIbDpmsudzYQCK1XHr7eIVog9pSjJS
      UEvj967lFX1T+ajpib/cqOSjjmjCbA91Pdo+il8iTHSo+SFgG3mIVDNnztD9nBUi
      96sWblnEDlzSH8MV9AOwM/FxI2HQ+giNQnc2NXt7q5wHBNG7fQ19+IiQ7tHjyvke
      eQ3POmo00+NSVcKObX29e6PCqiD14OrnLQIDAQABoxAwDjAMBgNVHRMEBTADAQH/
      MA0GCSqGSIb3DQEBBQUAA4IBAQB6t43EHkd2d8P25h1QNE1136VwC+OHg5ThPmXU
      wexuLY35iyWDs8AdZm2wu+9L0TAd3pqqY6qJW5JblTTGFZ3372oe2043EpcGVxxF
      /ov7VtlcD/Govu8IDkKhzojDrcKEoZCaI2zLaJpDZx5O/Zki0+fAOSNA+9HuLxUv
      mzCgQMazeUYkvvqxnm64Skw07xQ/g8JvjmFxfOz1LqMjY/M5TOXHUTZJFwpZVwHD
      8yt/MboSjEItdGb9qQlDfajYNYbodbbhurDXmwXRJU10uUK3RekKWOGKKoRhEOJ1
      SCi2iwz7n1N+bqXB8nLDOOr/zL1a4zev/KpMRhQNYMFswzUe
      -----END CERTIFICATE-----
    fluentd_ca_private_key_passphrase: password
    fluentd_ca_key: |
      -----BEGIN RSA PRIVATE KEY-----
      Proc-Type: 4,ENCRYPTED
      DEK-Info: AES-256-CBC,CB41DBD602640F0131B108162FBDA4C1

      eUVHk/0/haiey+uTvUVjLMG1uKqXEKzqbhuna3k+dPuOTOYPPrAArNfVgXS3K+rV
      kjsJUGLxwseC2Q9krbOx0tHoY25elMMGW/G46tds5CRJ5quoQIFXfuD8TBBSDEAI
      QFlU8PgUx+6KhEy63xjM1c9Y8CsgJqwpxWO6p6lAvJfVvuzUGBdEc7WvkDzmpGZk
      aNggxKJnd1qy/YNe1gbYpXPrn7y001s3CJtKs3goMJ6baMWfaUBDx6EBx7lJgL/r
      MqCWT+3q2McK3HcqEwK2O4oimlJy2pGj/SDESCPRGmKcYz6MLCZMoEjRDRWgZln8
      L9maXurGCIKsiBz61vNFGZtmu/dPJaJISPF3s4RTh7mst4SgWeNWK88dn6XAygkd
      yHOSM0l5y4YEqxiyPof7uAZBd276yMFp0mO8Hf61rtrTmUz/3nE4rsTTmGI60FIk
      PbzxZbWkZRcWNnDJE9ijuoqQ4W1ZGx5YkhNb5Al/Bap2rZ8ksEtpJswkGTm/j16i
      oSDpgu2ArSnxllbd7PWug/suCoullXlOJuOg3U5zrVASdAcUTK5872ObE40LzVPI
      4PRDYmYjvAIml2MjsXM1qlaN/qr8vnoRb+wZB0c3Jqj+eIVSPkypBcRxeeSp8akT
      Sdt7JmROdSgH7Sv9zdgJreMDofkQns6RN3TpEWnnDhZrE1vWQxF2DhNxeGThouqr
      QCdSv2DDPM6PgJEjFSXAYOKdeO+s22kzUv87fI+ubuU8gesSL7uAELSVHaXe/zKw
      9dGWVVIYfZ7F61aBXYqz598N2EqUv2rviQTWeVJ6Xggr/O7s6N7U6rn+1ptFcUTl
      BrPOdQws3Syc2NE2qov1A1QQ2uQQcy0l7bEEOuc1jxLKUMl5+ZWl94iZ5tg+3zG8
      QMf74rel6ELi1zVGcmwAROgt0AlOzQ7X8iiOjM1E9ZkplnQWaXbXRcZKlkYtnVHg
      Vk/bs2Fs2+qHpNR0m4nNzeWxcm5z/wN2xgJFRPyiOJDSMYfwnNPM3TEOg72RUtBi
      PT8Is8vRb7pb6JjT20OScxymNgOFYRAkcKQn2vVrrA7CWJpA7xeNbcoLM99bSOl7
      upgzyIb5jK8NzUwCv1kjCkESO3tGkfHPLbfXFzj966PEsR4Cyr5Mh8MiPk8p9VQV
      of27ZcwGCwiR7spMTKAQWEComDegZoN2pYELJL78Cb36p4mfu6pi9Ka6XeBoO7Hp
      zPFY9HPVjHHUAYZcFvLlMoaZz+VzjeATxAiSmD/iuWu9aJ77653cCzcicUOVM2T2
      nu1mMNm2EcPLXgZ4MqDLcwmYDV+GKdR2ilVdlbjKquf8rqzGkxXco2rDBOuVGMfk
      Cyc8QO5+Ym+0PPJOTWA6x9cTdCwSg2XfZoy7pRhiENTC7I78KZMa/NT+3Jpnksve
      pBA23tg7gTCi9f1tS+QjMJfz6pavK+0XcHS+4FEqg4B5l4f5pT9FGp3miI5z5uzv
      0gpi2SA4Q15HQAHGmzLtLCbuaVmivXqswvwktYbJ15Fhw8hcDhIg6pJZa2vO93gG
      ZGq87OydiIkk0pZxdabkpGxpbkKiIwK2+zFWDu3x604pR4b+rAMgrpEseD6TjLgr
      -----END RSA PRIVATE KEY-----
    fluentd_configs:
      example_input:
        enabled: true
        config: |
          <soruce>
            @type example
          </soruce>
      listen_on_5140:
        enabled: true
        config: |
          <source>
            @type syslog
            port 5140
            bind 127.0.0.1
            tag syslog
            format /^(?<time>[^ ]*\s*[^ ]* [^ ]*) (?<ident>[a-zA-Z0-9_\/\.\-]*)(?:\[(?<pid>[0-9]+)\])?(?:[^\:]*\:)? *(?<message>.*)$/
            time_format %b %d %H:%M:%S
          </source>
          <match syslog.**>

            @type null
          </match>

License

Copyright (c) 2016 Tomoyuki Sakurai <tomoyukis@reallyenglish.com>

Permission to use, copy, modify, and distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

Author Information

Tomoyuki Sakurai tomoyukis@reallyenglish.com