ARCHIVED

This project is no longer being updated.

You can still use it and modify it to fit your needs, including using Elasticsearch 8.x.

For different ways to get started, you may want to try:

• Starting a free trial on Elastic Cloud, our hosted service.
• Checking out Elastic Cloud on Kubernetes (ECK) for launching the stack using Kubernetes.
• Reading our guide on Running the Elastic Stack on Docker.
• Exploring the Elastic Stack Terraform provider.

ansible-elasticsearch

THIS ROLE IS FOR 7.x & 6.x, but it may still work with 8.x (see note).

This is an Ansible role for 7.x/6.x Elasticsearch. Tests were conducted on the following platforms:

• Ubuntu 16.04
• Ubuntu 18.04
• Ubuntu 20.04
• Debian 8
• Debian 9
• Debian 10
• CentOS 7
• Amazon Linux 2

IMPORTANT CHANGES

About Multi-instance Support

• If you have just one instance but want to upgrade from an older version of ansible-elasticsearch, follow the upgrade procedure.
• If you are installing multiple Elasticsearch instances on one host (with different ports, directories, and config files), do not update to ansible-elasticsearch >= 7.1.1. Instead, follow this workaround.
• For multiple instance use cases, we recommend using Docker containers with our official images (https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html).

Removing MAX_THREAD Settings

Ansible-elasticsearch 7.5.2 is removing the customization option for the maximum number of threads due to issues discovered since the removal of multi-instance support in ansible-elasticsearch 7.1.1. This option may be reintroduced in a future release if it remains relevant.

Configuration File Changes

Ansible-elasticsearch 7.5.2 has updated configuration files.

• /etc/default/elasticsearch|/etc/sysconfig/elasticsearch: The new template reflects the configuration in Elasticsearch >= 6.x, removing unused parameters.
• /etc/elasticsearch/jvm.options: The new template aligns with Elasticsearch >= 6.x.
• /etc/elasticsearch/log4j2.properties:
  • The template log4j2.properties.j2 is removed as it was static and did not customize any Ansible variable.
  • New deployments will get the default log4j2.properties from Elasticsearch without any overrides.
  • WARNING: For upgrades, if this file was managed by previous versions of ansible-elasticsearch, it will now be unmanaged. You can update it to version 7.5 by retrieving it here and using it with es_config_log4j2 Ansible variable.

OSS Distribution Removal for Versions >= 7.11.0

Starting from Elasticsearch 7.11.0, OSS distributions will not be available anymore due to recent license changes.

The Ansible role will fail if oss_version is set to true and es_version exceeds 7.11.0.

Refer to the blog post Doubling down on open, Part II for details.

Overriding Configuration Files in ansible-elasticsearch

You can now use your own versions of configuration files with the following Ansible variables:

• es_config_default: "elasticsearch.j2": Replace with your custom template for /etc/default/elasticsearch|/etc/sysconfig/elasticsearch.
• es_config_jvm: "jvm.options.j2": Replace with your custom template for /etc/elasticsearch/jvm.options.
• es_config_log4j2: "": Set this variable to your template path for a custom /etc/elasticsearch/log4j2.properties.

Dependencies

This role depends on the json_query filter, which requires jmespath on the local machine.

Usage

Create your Ansible playbook with your tasks, and include the elasticsearch role. The repository must be accessible within the playbook.

ansible-galaxy install elastic.elasticsearch,v7.17.0

Then create a YAML playbook that adds the elasticsearch role:

- name: Simple Example
  hosts: localhost
  roles:
    - role: elastic.elasticsearch
  vars:
    es_version: 7.17.0

This will install Elasticsearch 7.17.0 on a single node 'node1' at 'localhost'.

Note: The default version of Elasticsearch is described in es_version. You can change this in your playbook to install a different version.

While testing has only been conducted with specific 7.x and 6.x versions (found in the script), this role should still work with other versions most of the time.

This role also uses Ansible tags. Use the --list-tasks flag for more task information.

Testing

This playbook uses Kitchen for CI (Continuous Integration) and local testing.

Requirements

• Ruby
• Bundler
• Docker
• Make

Running Tests

• Make sure you clone the repository to elasticsearch, not ansible-elasticsearch.
• If you don't have a Gold or Platinum license for testing, you can use trial versions by adding -trial to the PATTERN.
• Specify VERSION=7.x if some suites fail.

Install Ruby dependencies with bundler:

make setup

To test features with a license, export the ES_XPACK_LICENSE_FILE variable:

export ES_XPACK_LICENSE_FILE="$(pwd)/license.json"

To run tests:

$ make verify

To list available test suites:

$ make list

To clean everything after testing:

$ make destroy-all

Basic Elasticsearch Configuration

All configuration parameters for Elasticsearch are supported. Use the configuration map es_config which will be serialized into the elasticsearch.yml file.

Here’s an example of applying configuration parameters to an Elasticsearch instance:

- name: Elasticsearch with custom configuration
  hosts: localhost
  roles:
    - role: elastic.elasticsearch
  vars:
    es_data_dirs:
      - "/opt/elasticsearch/data"
    es_log_dir: "/opt/elasticsearch/logs"
    es_config:
      node.name: "node1"
      cluster.name: "custom-cluster"
      discovery.seed_hosts: "localhost:9301"
      http.port: 9201
      transport.port: 9301
      node.data: false
      node.master: true
      bootstrap.memory_lock: true
    es_heap_size: 1g
    es_api_port: 9201

To ensure a successful cluster formation, configure these critical settings:

• es_config['http.port']
• es_config['transport.port']
• es_config['discovery.seed_hosts']
• es_config['cluster.initial_master_nodes']
• es_config['network.host']

Multi Node Server Installations

Applying the elasticsearch role multiple times on a host installs multiple nodes.

Here’s an example of a three-server setup, where the master node is set first:

- hosts: master_node
  roles:
    - role: elastic.elasticsearch
  vars:
    es_heap_size: "1g"
    es_config:
      cluster.name: "test-cluster"
      cluster.initial_master_nodes: "elastic02"
      discovery.seed_hosts: "elastic02:9300"
      http.host: 0.0.0.0
      http.port: 9200
      node.data: false
      node.master: true
      transport.host: 0.0.0.0
      transport.port: 9300
      bootstrap.memory_lock: false
    es_plugins:
     - plugin: ingest-attachment

- hosts: data_node_1
  roles:
    - role: elastic.elasticsearch
  vars:
    es_data_dirs:
      - "/opt/elasticsearch"
    es_config:
      cluster.name: "test-cluster"
      cluster.initial_master_nodes: "elastic02"
      discovery.seed_hosts: "elastic02:9300"
      http.host: 0.0.0.0
      http.port: 9200
      node.data: true
      node.master: false
      transport.host: 0.0.0.0
      transport.port: 9300
      bootstrap.memory_lock: false
    es_plugins:
      - plugin: ingest-attachment

- hosts: data_node_2
  roles:
    - role: elastic.elasticsearch
  vars:
    es_config:
      cluster.name: "test-cluster"
      discovery.seed_hosts: "elastic02:9300"
      http.host: 0.0.0.0
      http.port: 9200
      node.data: true
      node.master: false
      transport.host: 0.0.0.0
      transport.port: 9300
      bootstrap.memory_lock: false
    es_plugins:
      - plugin: ingest-attachment

To run it:

ansible-playbook -i hosts ./your-playbook.yml

Installing X-Pack Features

• Role mappings: Define in YAML as detailed here.

es_role_mapping:
  power_user:
    - "cn=admins,dc=example,dc=com"

• Users: Define as YAML under native and file keys.

es_users:
  native:
    kibana4_server:
      password: changeMe
      roles:
        - kibana4_server

• Roles: Define as YAML similarly under native and file.

es_roles:
  file:
    admin:
      cluster:
        - all

• X-Pack License: The license can be set directly or loaded from a file.

es_xpack_license: "{{ lookup('file', playbook_dir + '/files/' + es_cluster_name + '/license.json') }}"

If you need a trial license, set es_xpack_trial to true.

Important Note on Native Realm Configuration

For native users, ensure you set:

• es_api_basic_auth_username - Admin username
• es_api_basic_auth_password - Admin password

These can be a user with admin permissions or the default "elastic" superuser.

Additional Configuration Options

Other available parameters include:

• oss_version: Default false. Set to true for oss release (for version <7.11.0).
• es_xpack_trial: Default false. Set to true to enable the trial.
• es_version: (e.g. "7.17.0").
• es_api_host: Default is "localhost".
• es_plugins: List of plugins to install.

For proxy settings, use:

• es_proxy_host
• es_proxy_port

Notes

• Ensure the user/group exists on the server.
• The role is idempotent; running it multiple times without changes should not alter the state.
• For X-Pack tests, a license file with security enabled is needed; set the ES_XPACK_LICENSE_FILE environment variable to the license file's path before testing.

IMPORTANT NOTES RE PLUGIN MANAGEMENT

• Changing the Elasticsearch version will remove all plugins and reinstall the listed ones.
• Listing no plugins will remove currently installed ones.
• To reinstall plugins, set es_plugins_reinstall to true.

For any questions on usage, please ask the community at https://discuss.elastic.co/c/elasticsearch.