artem_shestakov.fluentbit

Ansible Fluent-bit Role

This is a simple role to install and configure Fluent-bit.

Role Variables

Configuration

Name	Description	Default
fluentbit_config_path	Path to the configuration and parsers file	/etc/fluent-bit
fluentbit_flush	Time to flush records in seconds.nanoseconds. This tells the engine when to flush the records from input plugins to output plugins.	5
fluentbit_grace	Time to wait on exit in seconds. This is the time the engine will wait before shutting down.	5
fluentbit_daemon	Whether Fluent Bit should run in the background. Allowed values are: yes, no, on, off.	off
fluentbit_dns_mode	Main protocol for the DNS resolver, can be overridden for specific plugins.	UDP
fluentbit_log_file	Optional path for a log file. By default, logs go to standard error (stderr).
fluentbit_log_level	Level of detail for logging.	info
fluentbit_http_server	Enable built-in HTTP Server.	off
fluentbit_http_listen	Interface for the HTTP Server to listen on when enabled.	0.0.0.0
fluentbit_http_port	TCP Port for the HTTP Server.	2020
fluentbit_coro_stack_size	Size of coroutine stack in bytes. Should be larger than the system's page size. Don’t set it too low to avoid stack overflow.	24576
fluentbit_scheduler_cap	Maximum retry time in seconds (supported from v1.8.7).	2000
fluentbit_scheduler_base	Base for exponential backoff (supported from v1.8.7).	5
fluentbit_inputs	Defines a source related to an input plugin.	Name: forward Listen: 0.0.0.0 Port: 24224
fluentbit_outputs	Defines a destination records should go to after matching a Tag.	Name: srdout Match: '*'

Pipeline

fluentbit_inputs - List of input sources. Each input is a collection of key/value pairs. Default value:

fluentbit_inputs:
  - Name: forward
    Listen: 0.0.0.0
    Port: 24224

fluentbit_outputs - List of output destinations. Each output is a collection of key/value pairs. Default value:

fluentbit_outputs:
  - Name: stdout
    Match: '*'

fluentbit_filters - List of filters. Each filter is a collection of key/value pairs. Default value is empty ([]). Example:

---
- name: Deploy fluent-bit service
  hosts: "{{ lookup('env', 'TARGET') }}"
  become: true
  roles:
    - role: artem_shestakov.fluentbit
      version: "v1.0.0"
  vars:
    fluentbit_inputs:
      - Name: forward
        Listen: 0.0.0.0
        Port: 24224
      - Name: syslog
        Tag: haproxy
        Path: /var/lib/haproxy/dev/log
        Unix_Perm: "0666"
    fluentbit_filters:
      - Name: record_modifier
        Match: "*"
        Record: hostname ${HOSTNAME}
      - Name: parser
        Match: "*"
        Key_Name: data
        Parser: syslog-rfc3164-local
    fluentbit_outputs:
      - Name: es
        Match: *
        Host: 192.168.2.3
        Port: 9200
        Index: my_index
        Type: my_type
    fluentbit_parsers:
      - Name: syslog-rfc3164-local
        Format: regex
        Regex: ^\<(?<pri>[0-9]+)\>(?<time>[^ ]* {1,2}[^ ]* [^ ]*) (?<ident>[a-zA-Z0-9_\/\.\-]*)(?:\[(?<pid>[0-9]+)\])?(?:[^\:]*\:)? *(?<message>.*)$
        Time_Key: time
        Time_Format: '%b %d %H:%M:%S'
        Time_Keep: On

fluentbit_parsers - List of parsers. Each parser is a collection of key/value pairs. Default value is empty ([]). Example:

fluentbit_parsers:
  - Name: named-capture-test
    Format: regex
    Regex: /^(?<date>[a-zA-Z]+ \d+ \d+\:\d+\:\d+) (?<message>.*)/m
  - Name: docker
    Format: json
    Time_Key: time
    Time_Format: "%Y-%m-%dT%H:%M:%S %z"

fluentbit_multiline_parsers - List of multiline parsers. Each parser includes a list of rules. Each rule has three elements: state_name, regex_pattern, and next_state. Default value is empty ([]). Example:

fluentbit_multiline_parsers:
  - name: multiline-regex-test
    type: regex
    flush_timeout: 1000
    rules:
      - state_name: start_state
        regex_pattern: /([a-zA-Z]+ \d+ \d+\:\d+\:\d+)(.*)/
        next_state: cont
      - state_name: cont
        regex_pattern: /^\s+at.*/
        next_state: cont