wtanaka.apache-kafka

This is an Ansible role to install Apache Kafka and manage it using Monit.

Requirements:

• You need to have Java installed. You can use this Ansible role for that.
• You also need Zookeeper. Implement it using this Ansible role.

Example Playbook

To install a simple default setup, include the role in your playbook like this:

- hosts: servers
  roles:
    - role: wtanaka.apache-kafka

You can configure the role directly in the playbook:

- hosts: servers
  roles:
    - role: wtanaka.apache-kafka
      kafka_env_variables:
        KAFKA_HEAP_OPTS: "-Xmx512m -Xms512m"

You can also use variables in your host_vars or group_vars files.

For a complete list of configuration options, check the defaults/main.yml file.

Configuration Options

kafka_version: "0.10.0.1"
kafka_scala_version: "2.11"
kafka_mirror: 'http://archive.apache.org/dist/kafka'
kafka_install_chdir: "/opt"
kafka_user: "kafka"
kafka_monit_conf_dir: "/etc/monit/conf.d"
kafka_init_wrapper_prefix: "kafka-"
kafka_pid_prefix: "/var/run/kafka-"
kafka_config_prefix: "server-"
kafka_log_prefix: "/var/log/kafka-"

kafka_brokers:
- broker_id: 0
  listeners: "PLAINTEXT://:9092"
- listeners: "PLAINTEXT://:9093"
  file_basename: kafka-9093
  log_dirs: /var/lib/kafka/broker-9093

kafka_config_contents:
  num.network.threads: 3
  num.io.threads: 8
  socket.send.buffer.bytes: 102400
  socket.receive.buffer.bytes: 102400
  socket.request.max.bytes: 104857600
  num.partitions: 1
  num.recovery.threads.per.data.dir: 1
  log.retention.hours: 168
  log.segment.bytes: 1073741824
  log.retention.check.interval.ms: 300000
  zookeeper.connect: "localhost:2181"
  zookeeper.connection.timeout.ms: "6000"

kafka_more_config: {}

kafka_env_variables:
  KAFKA_HEAP_OPTS: "-Xmx1G -Xms1G"

Configuration Details

• kafka_monit_enabled: Default is true. Set to false if you don’t want to create a Monit configuration file or start the Monit service.
• kafka_user: Default is kafka. This is the OS user that the Kafka process will run as, and the role will ensure that this user exists.

License

GPLv2

Author Information

You can find more about the author at wtanaka.com.