slapper.ansible_victoriametrics_cluster_role

Ansible Role: Install VictoriaMetrics in Cluster Mode

This Ansible role helps you install or upgrade VictoriaMetrics in a cluster setup. It's based on the standalone version. Thanks to @dreamteam-gg for inspiration.

It has been tested on Centos 7 and Centos 8, but it should work on other Linux distributions with slight modifications.

Requirements

No special requirements!

Ansible Inventory

To use this role, create an inventory file with three groups:

vmstorage (for storage nodes)
vmselect (for selection nodes)
vminsert (for insertion nodes)

More details can be found here.

[vmstorage]
vmstorage01.example.com
vmstorage02.example.com
vmstorage03.example.com
vmstorage04.example.com

[vminsert]
vminsert01.example.com
vminsert02.example.com

[vmselect]
vmselect01.example.com
vmselect02.example.com

Role Variables

Here are the default variables of the role. You can change them at the role or playbook level.

---
## Default settings for all VictoriaMetrics nodes
victoriametrics_repo_url: "https://github.com/VictoriaMetrics/VictoriaMetrics"
victoriametrics_download_url: "{{ victoriametrics_repo_url }}/releases/download/{{ victoriametrics_version }}/victoria-metrics-{{ victoriametrics_version }}-cluster.tar.gz"
victoriametrics_version: "v1.34.6"
victoriametrics_system_user: "victoria"
victoriametrics_system_group: "victoria"

## Variables for vmstorage nodes
victoriametrics_vmstorage_data_dir: "/usr/local/bin/victoria-storage"
victoriametrics_vmstorage_retention_period: "24"
victoriametrics_vmstorage_memory_allowed_percent: "60"  # Default is 60
victoriametrics_vmstorage_service_args: ""  # Add extra options here

## Variables for vmselect nodes
victoriametrics_vmselect_cache_dir: "/usr/local/bin/victoria-cache"
victoriametrics_vmselect_memory_allowed_percent: "60"
victoriametrics_vmselect_service_args: ""  # Add extra options here

## Variables for vminsert nodes
victoriametrics_vminsert_service_args: ""  # Add extra options here
victoriametrics_vminsert_memory_allowed_percent: "60"

# Variables for service's listen address
victoriametrics_vmstorage_listen_address: "{{ ansible_default_ipv4.address }}"
victoriametrics_vmselect_listen_address: "{{ ansible_default_ipv4.address }}"
victoriametrics_vminsert_listen_address: "{{ ansible_default_ipv4.address }}"

Example Playbook for Initial Installation

---
- hosts: vmstorage,vminsert,vmselect
  become: yes

  roles:
   - ansible-victoriametrics-cluster-role

Example Playbook for Cluster Update

Before updating your cluster, always check for new releases here.
To update your cluster without issues, ensure at least one node of each type is running. Read more here.
To update, just change the victoriametrics_version.
Use Ansible's serial function to update one node at a time. More information can be found in the Ansible documentation.
A simple task to gather facts from the nodes is included before the update:

---
- hosts: vminsert,vmselect,vmstorage
  become: true
  gather_facts: true

  tasks:
    - name: Gather facts for Victoria nodes
      setup:

- hosts: vminsert,vmselect,vmstorage
  gather_facts: false
  become: true
  serial: 1

  vars:
    - victoriametrics_version: "v1.40.0"

Extras

To use VictoriaMetrics, you'll need an HTTP load balancer. I used Haproxy. Here’s a simple configuration for your vminsert and vmselect nodes. If you add more nodes, remember to update your Haproxy configuration.

#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
    log          127.0.0.1 local2
    log-send-hostname
    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     40000
    user        haproxy
    group       haproxy
    daemon

    stats socket /run/haproxy.sock mode 666 level admin
    ssl-default-bind-ciphers PROFILE=SYSTEM
    ssl-default-server-ciphers PROFILE=SYSTEM

#---------------------------------------------------------------------
# Common defaults 
#---------------------------------------------------------------------
defaults
    mode                    http
    log                     global
    option                  httplog
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    maxconn                 30000

#---------------------------------------------------------------------
# Frontend for vminsert and vmselect
#---------------------------------------------------------------------
frontend vminsert
    bind 0.0.0.0:8480
    default_backend vminsert_nodes

frontend vmselect
    bind 0.0.0.0:8481
    default_backend vmselect_nodes

#---------------------------------------------------------------------
# Backend settings
#---------------------------------------------------------------------
backend vminsert_nodes
    balance roundrobin
    server vminsert01 10.10.10.100:8480 check
    server vminsert02 10.10.10.101:8480 check

backend vmselect_nodes
    balance roundrobin
    server vmselect01 10.10.10.102:8481 check
    server vmselect02 10.10.10.103:8481 check