AlexeySetevoi.clickhouse

概述 版本 洞察

ansible-clickhouse

构建状态 构建状态 最新 GitHub 标签 Ansible Galaxy

简单的 ClickHouse 服务器部署和管理角色。 欢迎任何问题和 PR。

角色变量

F: 你可以指定特定版本（或使用 * 表示最新版本）。请注意，不支持降级。

clickhouse_version: "19.11.3.11"

F: 你可以管理监听端口

clickhouse_http_port: 8123
clickhouse_tcp_port: 9000
clickhouse_interserver_http: 9009

F: 你可以在默认上添加监听 IP：

clickhouse_listen_host_custom:
  - "192.168.0.1"

F: 你可以管理 TTL 查询日志：

clickhouse_query_log_ttl: 'event_date + INTERVAL 7 DAY DELETE'

F: 你可以管理 TTL 查询线程日志：

clickhouse_query_thread_log_ttl: 'event_date + INTERVAL 7 DAY DELETE'

F: 或者你可以直接指定 IP，例如监听所有 IPv4 和 IPv6 地址：

clickhouse_listen_host:
  - "::"

F: 你可以创建自定义配置文件：

clickhouse_profiles_custom:
 my_custom_profile:
   max_memory_usage: 10000000000
   use_uncompressed_cache: 0
   load_balancing: random
   my_super_param: 9000

允许任何普通的 k-v。转换为 XML

<profiles>
    <!-- 设置配置文件。 -->
    <!-- 默认配置文件。 -->
        <default>
            <max_memory_usage>10000000000</max_memory_usage>
            <load_balancing>random</load_balancing>
            <use_uncompressed_cache>0</use_uncompressed_cache>
        </default>
        <readonly>
            <readonly>1</readonly>
        </readonly>
        <!-- 默认配置文件结束。 -->
        <!-- 自定义配置文件。 -->
        <my_custom_profile>
            <max_memory_usage>10000000000</max_memory_usage>
            <load_balancing>random</load_balancing>
            <use_uncompressed_cache>0</use_uncompressed_cache>
            <my_super_param>9000</my_super_param>
        </my_custom_profile>
        <!-- 自定义配置文件结束。 -->
</profiles>

F: 你可以创建自定义用户：

clickhouse_users_custom:
      - { name: "testuser",
          password_sha256_hex: "f2ca1bb6c7e907d06dafe4687e579fce76b37e4e93b7605022da52e6ccc26fd2",
          networks: "{{ clickhouse_networks_default }}",
          profile: "default",
          quota: "default",
          dbs: [ testu1 ] ,
          comment: "经典用户，使用普通密码"}
      - { name: "testuser2",
          password: "testplpassword",
          networks: "{{ clickhouse_networks_default }}",
          profile: "default",
          quota: "default",
          dbs: [ testu2 ] ,
          comment: "经典用户，使用十六进制密码"}
      - { name: "testuser3",
          password: "testplpassword",
          networks: { 192.168.0.0/24, 10.0.0.0/8 },
          profile: "default",
          quota: "default",
          dbs: [ testu1,testu2,testu3 ] ,
          comment: "经典用户，具有多个数据库和多个自定义网络的密码"}
      - { name: "testuser4",
          ldap_server: "example_ldap_server",
          networks: { 192.168.0.0/24, 10.0.0.0/8 },
          profile: "default",
          quota: "default",
          dbs: [ testu1,testu2,testu3 ] ,
          comment: "使用 ldap_server 定义的外部认证"}

F: 你可以管理自己的配额：

clickhouse_quotas_custom:
 - { name: "my_custom_quota", intervals: "{{ clickhouse_quotas_intervals_default }}",comment: "默认配额 - 仅计数" }

配额对象是简单的字典：

 - { duration: 3600, queries: 0, errors: 0,result_rows: 0,read_rows: 0,execution_time: 0 }

F: 你可以创建数据库： 默认数据库状态 - 存在

clickhouse_dbs_custom:
      - { name: testu1 }
      - { name: testu2 }
      - { name: testu3 }
      - { name: testu4, state: absent }
      - { name: testu5, state: present }
      - { name: testu6, state: absent, cluster: testu6 }
      - { name: testu7, state: present, cluster: testu7 }
      - { name: testu8, state: absent, cluster: testu8, engine: Lazy(3600) }
      - { name: testu9, state: present, cluster: testu9, engine: Lazy(3600) }

F: 你可以通过 ODBC 创建字典：

clickhouse_dicts:
          test1:
            name: test_dict
            odbc_source:
              connection_string: "DSN=testdb"
              source_table: "dict_source"
            lifetime:
              min: 300
              max: 360
            layout: hashed
            structure:
              key: "testIntKey"
              attributes:
                - { name: testAttrName, type: UInt32, null_value: 0 }
          test2:
            name: test_dict
            odbc_source:
              connection_string: "DSN=testdb"
              source_table: "dict_source"
            lifetime:
              min: 300
              max: 360
            layout: complex_key_hashed
            structure:
              key:
                attributes:
                  - { name: testAttrComplexName, type: String }
              attributes:
                - { name: testAttrName, type: String, null_value: "" }

F: 从主机上移除 ClickHouse 的标志（默认禁用）

clickhouse_remove: no

F: 你可以管理 Kafka 配置

# 全局配置
clickhouse_kafka_config:
  auto_offset_reset: smallest
  debug: cgrp
# 主题级别配置
clickhouse_kafka_topics_config:
  topic1:
    retry_backoff_ms: 250
    fetch_min_bytes: 100000
  topic2:
    retry_backoff_ms: 300
    fetch_min_bytes: 120000

F: 你可以管理 LDAP 服务器配置

clickhouse_ldap_servers:
  # 使用 ldapwhoami 进行调试 -H '<host>' -D '<bind_dn>' -w <password>
  example_ldap_server:
    host: "ldaps.example.com"
    port: "636"
    bind_dn: "EXAMPLENET\\{user_name}"
    verification_cooldown: "300"
    enable_tls: "yes"
    tls_require_cert: "demand"

F: 你可以管理 LDAP 外部用户目录

clickhouse_ldap_user_directories:
  - server: "example_ldap_server"
    roles:
      - "ldap_user"
    role_mapping:
      base_dn: "ou=groups,dc=example,dc=com"
      attribute: "CN"
      scope: "subtree"
      search_filter: "(&amp;(objectClass=group)(member={user_dn}))"
      prefix: "clickhouse_

F: 你可以管理 Merge Tree 配置。有关可用参数的列表，请参阅 MergeTree 表设置。

clickhouse_merge_tree_config:
  max_suspicious_broken_parts: 5
  parts_to_throw_insert: 600

示例剧本

包含如何使用角色的示例（例如，传递参数的变量）总是对用户友好的：

  - hosts: clickhouse_cluster
    remote_user: root
    vars:
      clickhouse_users_custom:
          - { name: "testuser",
              password_sha256_hex: "f2ca1bb6c7e907d06dafe4687e579fce76b37e4e93b7605022da52e6ccc26fd2",
              networks: "{{ clickhouse_networks_default }}",
              profile: "default",
              quota: "default",
              dbs: [ testu1 ] ,
              comment: "经典用户，使用普通密码"}
          - { name: "testuser2",
              password: "testplpassword",
              networks: "{{ clickhouse_networks_default }}",
              profile: "default",
              quota: "default",
              dbs: [ testu2 ] ,
              comment: "经典用户，使用十六进制密码"}
          - { name: "testuser3",
              password: "testplpassword",
              networks: { 192.168.0.0/24, 10.0.0.0/8 },
              profile: "default",
              quota: "default",
              dbs: [ testu1,testu2,testu3 ] ,
              comment: "经典用户，具有多个数据库和多个自定义网络的密码"}
      clickhouse_query_log_ttl: 'event_date + INTERVAL 7  DELETE'
      clickhouse_query_thread_log_ttl: 'event_date + INTERVAL 7  DELETE'
      clickhouse_dicts:
          test1:
            name: test_dict
            odbc_source:
              connection_string: "DSN=testdb"
              source_table: "dict_source"
            lifetime:
              min: 300
              max: 360
            layout: hashed
            structure:
              key: "testIntKey"
              attributes:
                - { name: testAttrName, type: UInt32, null_value: 0 }
          test2:
            name: test_dict
            odbc_source:
              connection_string: "DSN=testdb"
              source_table: "dict_source"
            lifetime:
              min: 300
              max: 360
            layout: complex_key_hashed
            structure:
              key:
                attributes:
                  - { name: testAttrComplexName, type: String }
              attributes:
                - { name: testAttrName, type: String, null_value: "" }
      clickhouse_dbs_custom:
         - { name: testu1 }
         - { name: testu2, state:present }
         - { name: testu3, state:absent }
      clickhouse_clusters:
        your_cluster_name:
          shard_1:
              - { host: "db_host_1", port: 9000 }
              - { host: "db_host_2", port: 9000 }
          shard_2:
              - { host: "db_host_3", port: 9000 }
              - { host: "db_host_4", port: 9000 }       
      clickhouse_zookeeper_nodes:
        - { host: "zoo_host_1", port: 2181 }
        - { host: "zoo_host_2", port: 2181 }
        - { host: "zoo_host_3", port: 2181 }
    roles:
      - ansible-clickhouse

生成宏：在文件 host_vars\db_host_1.yml 中

clickhouse_macros:
  layer: 01
  shard: "your_shard_name"
  replica: "db_host_1"

增强集群安全性。您可以使用额外设置配置集群，这些设置启用

  • HTTPS 端口
  • TLS 加密 TCP 端口
  • 数据复制的 HTTPS
  • 数据复制的凭证
  • 分布式查询的秘密验证
  • ZooKeeper ACL
- hosts: clickhouse_cluster
  become: true
  roles:
    - ansible-clickhouse
  vars:
    # 使用 HTTPS 而不是普通 HTTP
    clickhouse_https_port: 8443
    # TLS 加密用于原生 TCP 协议（需要 `clickhouse-client --secure`）
    clickhouse_tcp_secure_port: 9440
    # 节点之间的 TLS 加密
    clickhouse_interserver_https: 9010
    # 在数据复制过程中用于验证节点的凭证
    clickhouse_interserver_http_credentials:
      user: "internal"
      password: "supersecretstring"
    # 用于验证集群中节点的秘密，用于分布式查询
    clickhouse_distributed_secret: "supersecretstring2"
    # 密码保护 ClickHouse 使用的 zookeeper 路径
    clickhouse_zookeeper_identity:
      user: "zoo_user"
      password: "secretzoostring"
    # OpenSSL 设置
    clickhouse_ssl_server:
      certificate_file: "/etc/clickhouse-server/server.crt"
      private_key_file: "/etc/clickhouse-server/server.key"
      dh_params_file: "/etc/clickhouse-server/dhparam.pem"
      verification_mode: "none"
      load_default_ca_file: "true"
      cache_sessions: "true"
      disable_protocols: "sslv2,sslv3"
      prefer_server_ciphers: "true"
    clickhouse_clusters:
      your_cluster_name:
        shard_1:
          - host: "db_host_1"
            port: 9440
            secure: true
          - host: "db_host_2"
            port: 9440
            secure: true
        shard_2:
          - host: "db_host_3"
            port: 9440
            secure: true
          - host: "db_host_4"
            port: 9440
            secure: true
    clickhouse_zookeeper_nodes:
      - host: "zoo_host_1"
        port: 2181
      - host: "zoo_host_2"
        port: 2181
      - host: "zoo_host_3"
        port: 2181

F: 你可以单独调用阶段（从剧本、外部角色等）：

标记 操作
install 仅安装软件包
config_sys 仅配置系统配置（users.xml 和 config.xml）
config_db 仅添加和移除数据库
config_sys 仅重新生成字典
config config_sys + config_db

许可证

BSD

作者信息

ClickHouse 由 ClickHouse, Inc. 提供。

角色由 AlexeySetevoi 制作。

亲爱的贡献者，感谢你们。

关于项目

Role for installation and managing ClickHouse

role clickhouse database sql
安装
ansible-galaxy install AlexeySetevoi.clickhouse
GitHub 仓库
167 星标
109 分叉
21 未解决问题
许可证
Unknown
下载
4.3k
拥有者