Ansible Role: Logstash

This Ansible Role installs Logstash on RedHat/CentOS and Debian/Ubuntu systems.

By default, this role includes a syslog grok pattern. If you wish to add additional filters, please check the documentation for logstash_configuration_files and logstash_filter_files.

For example, you can create a file called 13-myapp.conf with your grok filter and restart Logstash to apply it. You can test your grok regex using the Grok Debugger.

Requirements

While other setups are possible, this role is designed to work with Elasticsearch as the backend for storing log messages.

Role Variables

Below are the available variables along with their default values (see defaults/main.yml):

• logstash_listen_port_beats: 5044
  This is the port on which Logstash will listen for beats.

• logstash_version: 2.3
  This sets the default major version of Logstash to install.

• logstash_configuration_files:

  • 01-lumberjack-input.conf
  • 30-lumberjack-output.conf

• logstash_filter_files:

  • 10-syslog.conf
  • 11-nginx.conf
  • 12-apache.conf
  • 14-solr.conf
  • 15-drupal.conf

• logstash_elasticsearch_host: localhost
  This is the list of configuration and filter files to install. By default, it includes inputs/outputs and variable filter files. You can create your own configuration files in your templates and files directories. Then, make a list in vars/main.yml and include this role to set up Logstash with your configuration. See the Example Playbook below.

• logstash_elasticsearch_hosts:

  • http://localhost:9200
    The hosts where Logstash will send logs to Elasticsearch.

• logstash_ssl_dir: /etc/pki/logstash

• logstash_ssl_certificate_file: logstash-forwarder-example.crt

• logstash_ssl_key_file: logstash-forwarder-example.key
  These are the SSL settings for Logstash to accept requests from logstash-forwarder on remote hosts. Security note: For production servers, create your own key/certificate pair instead of using the default! You can generate the files using OpenSSL with a command like:
  openssl req -x509 -batch -nodes -days 3650 -newkey rsa:2048 -keyout logstash.key -out logstash.crt.
  Keep in mind that filebeat and logstash may not work with self-signed certificates unless the full chain of trust is set up on your server.

• logstash_local_syslog_path: /var/log/syslog

• logstash_monitor_local_syslog: true
  Set this to indicate whether to add the local syslog configuration to Logstash. Use false if you monitor syslog in another way or don't need it. Other local logs can be configured in /etc/logstash/conf.d.

• logstash_enabled_on_boot: yes
  Set this to no if you don’t want Logstash to start automatically when the system boots.

• logstash_install_plugins:

  • logstash-input-beats
    This is a list of Logstash plugins to be installed.

• logstash_install_dir: /usr/share/logstash
  Path where Logstash will be installed. The previous fixed path "/opt/logstash" has been changed to default to "/usr/share/logstash".

Other Notes

If you notice high CPU usage from a Logstash process, and you have another app running on port 80 (like in Ubuntu with upstart), the logstash-web process might be stuck trying to start on port 80 and failing because of the restart flag in /etc/init/logstash-web.conf.

To fix this, you can either add a limit to the respawn statement or disable the logstash-web service in your playbook, like this:

- name: Ensure logstash-web process is stopped and disabled.
  service: name=logstash-web state=stopped enabled=no

Example Playbook

- hosts: search
  vars_files:
    - vars/main.yml

  roles:
    - geerlingguy.elasticsearch
    - geerlingguy.logstash

License

MIT / BSD

Author Information

This role was created in 2014 by Jeff Geerling, author of Ansible for DevOps.