Ansible Role: Fluentbit

This role is designed to install and configure Fluentbit.

Requirements

• This role works with Ansible version 2.9 or higher.

Role Variables

This role aims to maintain the same default configuration as a manual installation of Fluentbit. For more details on each option, check the Fluentbit documentation.

Variables for Installation

Variable	Default Value	Description
fluentbit_prerequisites	['apt-transport-https', 'curl', 'gnupg']	Packages to install before Fluentbit.
fluentbit_apt_key_path	"/usr/share/keyrings/fluentbit-keyring.gpg"	Path for the APT keyring storing the Fluentbit key.
fluentbit_apt_key_url	https://packages.fluentbit.io/fluentbit.key	URL for the Fluentbit APT key.
fluentbit_apt_repos_url	"https://packages.fluentbit.io/{{ ansible_distribution	lower }}/{{ ansible_distribution_release
fluentbit_apt_repos_component	main	APT repository component.
fluentbit_pkg_name	fluent-bit	Name of the Fluentbit APT package.
fluentbit_pkg_version	""	Specific version of the package to install.
fluentbit_pkg_version_hold	"{{ fluentbit_pkg_version	default(False)
fluentbit_svc_name	fluent-bit	Service name for starting/stopping the daemon.
fluentbit_apt_cleanup_legacy	false	Remove old keys and sources if set to true.
fluentbit_apt_key_legacy_id	F209D8762A60CD49E680633B4FF8368B6EA0722A	Old GPG key ID to remove.
fluentbit_naming_cleanup_legacy	false	Remove old services and configs with the td-agent name.
fluentbit_pkg_name_legacy	td-agent-bit	Legacy package name to be removed.
fluentbit_svc_name_legacy	td-agent-bit	Legacy service name to be removed.
fluentbit_conf_directory_legacy	/etc/td-agent-bit/	Legacy config directory to be removed.

Variables for General Configuration

Variable	Default Value	Description
fluentbit_svc_flush	5	Time to flush in seconds and nanoseconds.
fluentbit_svc_grace	5	Grace time in seconds.
fluentbit_svc_daemon	"off"	Specify if Fluentbit runs as a daemon (should be Off for Systemd).
fluentbit_svc_logfile	""	Path for optional log file; logs to stdout if not specified.
fluentbit_svc_loglevel	info	Verbosity level for logging.
fluentbit_svc_parsers_file	["parsers.conf"]	Paths for parsers configuration files.
fluentbit_svc_plugins_file	["plugins.conf"]	Paths for plugins configuration files.
fluentbit_svc_streams_file	[]	Paths for stream processors configuration files.
fluentbit_managed_parsers_enable	"{{ ((_fluentbit_parsers	length) or (_fluentbit_mlparsers
fluentbit_managed_parsers_file	"{{ fluentbit_conf_directory }}/managed-parsers.conf"	File for defining custom parsers if needed.
fluentbit_svc_http	{}	Configuration for the built-in HTTP server.
fluentbit_svc_storage	{}	Configuration for storage/buffer settings.
fluentbit_svc_limit_open_files	Undefined	Set LimitNOFILE for the systemd service if defined.

For configuring fluentbit_svc_http, just use the option names without the HTTP_ prefix. For example:

fluentbit_svc_http:
  server: On
  listen: 0.0.0.0
  port: "{{ fluentbit_monitoring_port }}"

This applies the same way for fluentbit_svc_storage:

fluentbit_svc_storage:
  path: /var/log/flb-storage/
  sync: full
  checksum: "off"

Other Variables

Variable	Default Value	Description
fluentbit_dbs_path	""	Path for a directory created by the role (e.g., for input DB files).

Variables for Log Processing

Variable	Default Value	Description
_fluentbit_inputs	"{{ lookup('template', 'get_vars.j2'), template_vars=dict(var_type='input') }}"	List of dictionaries for log inputs.
_fluentbit_filters	"{{ lookup('template', 'get_vars.j2'), template_vars=dict(var_type='filter') }}"	List of dictionaries for log filters.
_fluentbit_outputs	"{{ lookup('template', 'get_vars.j2'), template_vars=dict(var_type='output') }}"	List of dictionaries for log outputs.
_fluentbit_parsers	"{{ lookup('template', 'get_vars.j2'), template_vars=dict(var_type='parser') }}"	List of dictionaries for log parsers.
_fluentbit_mlparsers	"{{ lookup('template', 'get_vars.j2'), template_vars=dict(var_type='mlparser') }}"	List of dictionaries for multiline log parsers.

Usually, you should not change these variables. They are built using other variables.

• _fluentbit_inputs, _fluentbit_filters, _fluentbit_outputs, _fluentbit_parsers, and _fluentbit_mlparsers use regex to collect relevant variables.

Example Definitions

For example, defining inputs like this:

fluentbit_nginx_input:
  - name: tail
    path: /var/log/nginx/access.log
  - name: tail
    path: /var/log/nginx/error.log

fluentbit_kernel_input:
  name: tail
  path: /var/log/kern.log

Will generate the corresponding [INPUT] sections in Fluentbit configuration.

If you face issues with duplicate keys in Ansible dictionaries, you can prefix them with numbers. For example:

fluentbit_add_context_filter:
  - name: record_modifier
    match: '*'
    0__record: hostname ${HOSTNAME}
    1__record: product Awesome_Tool

Will become:

[FILTER]
  name record_modifier
  match *
  record hostname ${HOSTNAME}
  record product Awesome_Tool

Dependencies

None

Example Playbook

In group_vars/all.yml:

fluentbit_kernel_input:
  name: tail
  path: /var/log/kern.log

fluentbit_env_filter:
  name: record_modifier
  match: '*'
  record: "env {{ env }}"

fluentbit_central_output:
  name: forward
  match: '*'
  host: "{{ logs_server_address }}"
  port: "{{ log_forward_port }}"

In group_vars/web.yml:

fluentbit_nginx_input:
  - name: tail
    path: /var/log/nginx/access.log
  - name: tail
    path: /var/log/nginx/error.log

In playbook.yml:

- hosts: web
  gather_facts: True
  become: yes
  roles:
    - bimdata.fluentbit

License

BSD

Author Information

BIMData.io