nifi
Ansible Role: NiFi
An Ansible Role that installs NiFi on Linux. By default, it installs NiFi in a way that makes upgrading painless.
Requirements
Requires at least Java 8.
Role Variables
See defaults/main.yml
for all variables and how to specify them. For a deeper dive, the NiFi System Administrator’s Guide is a great resource.
The following specifies where to download (or look for existing) binaries (tarballs), where to install NiFi, and a home directory which will be symbolically linked to the specified release. Also, a centralized config directory to store files that need not be changed (to avoid copying during upgrades). You can add more artbitrary key/value pairs to this dict and those directories will be created. This might be useful if you need extra directories for things like custom nars, drivers, etc.
nifi_config_dirs:
binaries: /tmp
install: /opt/nifi/releases
home: /opt/nifi/releases/current
external_config: /opt/nifi/config_resources
run: {{ nifi_config_dirs.home }}/run
logs: {{ nifi_config_dirs.home }}/logs
By default, this is the directory structure that will be created:
|--opt/
|--nifi/
|--releases/
|--current -> nifi-1.14.0/
|--nifi-1.14.0/
|--nifi-1.13.2/
|--config_resources/
|--archive/
|--authorizations.xml
|--content_repository/
|--custom_nars/
|--database_repository/
|--flow.xml.gz
|--flowfile_repository/
|--provenance_repository/
|--state/
|--users.xml
Any key/value pair from a config file can be added to the following dicts. Dict names correspond to file names. The current config options for these files can be found here.
nifi_properties:
bootstrap:
nifi_env:
logback:
login_identity_providers:
state_management:
authorizers:
zookeeper:
Dependencies
None.
Example Playbooks
These assume you have hash_behaviour=merge
set in your config. If not, please also include the default dict key/values from defaults/main.yml
.
Basic single node NiFi instance:
- hosts: nifi_servers
become: yes
roles:
- triviadata.nifi
Basic 3 node NiFi cluster using embedded Zookeeper:
- hosts: nifi_servers
become: yes
roles:
- triviadata.nifi
vars:
nifi_properties:
# HTTP properties must be unset for HTTPS to work
nifi.web.http.host: ""
nifi.web.http.port: ""
nifi.web.https.host: "{{ ansible_fqdn }}"
nifi.web.https.port: 8443
nifi.cluster.is.node: true
nifi.cluster.node.address: "{{ ansible_fqdn }}"
nifi.cluster.node.protocol.port: 11443
nifi.cluster.flow.election.max.candidates: 3
nifi.cluster.load.balance.host: "{{ ansible_fqdn }}"
nifi.cluster.load.balance.port: 6342
nifi.state.management.embedded.zookeeper.start: true
nifi.zookeeper.connect.string: nifi_server1:2181,nifi_server2:2181,nifi_server3:2181
login_identity_providers:
/loginIdentityProviders/provider/identifier: single-user-provider
/loginIdentityProviders/provider/class: org.apache.nifi.authentication.single.user.SingleUserLoginIdentityProvider
authorizers_user_group_providers: 0
authorizers:
/authorizers/authorizer/identifier: single-user-authorizer
/authorizers/authorizer/class: org.apache.nifi.authorization.single.user.SingleUserAuthorizer
state_management:
/stateManagement/cluster-provider/property[@name="Connect String"]: "{{ nifi_properties['nifi.zookeeper.connect.string'] }}"
# Assuming nifi_server1 = 192.168.1.10, nifi_server2 = 192.168.1.11, nifi_server3 = 192.168.1.12
# we have Ansible automatically set the myid file on each host to last octet of the node's IP address
# and we set the 'X' of the zookeeper['server.X'] keys to those same numbers.
zookeeper_myid: "{{ ansible_default_ipv4.address.split('.')[-1] }}"
zookeeper:
server.10: nifi_server1:2888:3888
server.11: nifi_server2:2888:3888
server.12: nifi_server3:2888:3888
Secure single node NiFi instance with LDAP:
- hosts: nifi_servers
become: yes
roles:
- triviadata.nifi
vars:
nifi_properties:
# HTTP properties must be unset for HTTPS to work
nifi.web.http.host: ""
nifi.web.http.port: ""
nifi.web.https.host: "{{ ansible_fqdn }}"
nifi.web.https.port: 8443
nifi.security.keystore: /path/to/keystore.jks
nifi.security.keystoreType: JKS
nifi.security.keystorePasswd: keystorePassword
nifi.security.keyPasswd: keyPassword
nifi.security.truststore: /path/to/truststore.jks
nifi.security.truststoreType: JKS
nifi.security.truststorePasswd: truststorePassword
login_identity_providers:
/loginIdentityProviders/provider/identifier: ldap-provider
/loginIdentityProviders/provider/class: org.apache.nifi.ldap.LdapProvider
/loginIdentityProviders/provider/property[@name="Authentication Strategy"]: SIMPLE
/loginIdentityProviders/provider/property[@name="Manager DN"]: cn=nifi,ou=people,dc=example,dc=com
/loginIdentityProviders/provider/property[@name="Manager Password"]: password
/loginIdentityProviders/provider/property[@name="Referral Strategy"]: FOLLOW
/loginIdentityProviders/provider/property[@name="Connect Timeout"]: 10 secs
/loginIdentityProviders/provider/property[@name="Read Timeout"]: 10 secs
/loginIdentityProviders/provider/property[@name="Url"]: ldap://hostname:port
/loginIdentityProviders/provider/property[@name="User Search Base"]: OU=people,DC=example,DC=com
/loginIdentityProviders/provider/property[@name="User Search Filter"]: sAMAccountName={0}
/loginIdentityProviders/provider/property[@name="Identity Strategy"]: USE_DN
/loginIdentityProviders/provider/property[@name="Authentication Expiration"]: 12 hours
authorizers_identifiers:
- file-user-group-provider
- ldap-user-group-provider
- composite-user-group-provider
authorizers:
/authorizers/userGroupProvider[1]/identifier: file-user-group-provider
/authorizers/userGroupProvider[1]/class: org.apache.nifi.authorization.FileUserGroupProvider
/authorizers/userGroupProvider[1]/property[@name="Users File"]: "{{ nifi_config_dirs.external_config }}/users.xml"
/authorizers/userGroupProvider[1]/property[@name="Initial User Identity 1"]: cn=John Smith,ou=people,dc=example,dc=com
/authorizers/accessPolicyProvider/identifier: file-access-policy-provider
/authorizers/accessPolicyProvider/class: org.apache.nifi.authorization.FileAccessPolicyProvider
/authorizers/accessPolicyProvider/property[@name="User Group Provider"]: file-user-group-provider
/authorizers/accessPolicyProvider/property[@name="Authorizations File"]: "{{ nifi_config_dirs.external_config }}/authorizations.xml"
/authorizers/accessPolicyProvider/property[@name="Initial Admin Identity"]: cn=John Smith,ou=people,dc=example,dc=com
/authorizers/authorizer/identifier: managed-authorizer
/authorizers/authorizer/class: org.apache.nifi.authorization.StandardManagedAuthorizer
/authorizers/authorizer/property[@name="Access Policy Provider"]: file-access-policy-provider
Secure 3 node NiFi cluster with LDAP using embedded zookeeper:
- hosts: nifi_servers
become: yes
roles:
- triviadata.nifi
vars:
nifi_properties:
# HTTP properties must be unset for HTTPS to work
nifi.web.http.host: ""
nifi.web.http.port: ""
nifi.web.https.host: "{{ ansible_fqdn }}"
nifi.web.https.port: 8443
nifi.security.keystore: /path/to/keystore.jks
nifi.security.keystoreType: JKS
nifi.security.keystorePasswd: keystorePassword
nifi.security.keyPasswd: keyPassword
nifi.security.truststore: /path/to/truststore.jks
nifi.security.truststoreType: JKS
nifi.security.truststorePasswd: truststorePassword
nifi.cluster.protocol.is.secure: true
nifi.cluster.is.node: true
nifi.cluster.node.address: "{{ ansible_fqdn }}"
nifi.cluster.node.protocol.port: 11443
nifi.cluster.flow.election.max.candidates: 3
nifi.cluster.load.balance.host: "{{ ansible_fqdn }}"
nifi.cluster.load.balance.port: 6342
nifi.state.management.embedded.zookeeper.start: true
nifi.zookeeper.connect.string: nifi_server1:2181,nifi_server2:2181,nifi_server3:2181
login_identity_providers:
/loginIdentityProviders/provider/identifier: ldap-provider
/loginIdentityProviders/provider/class: org.apache.nifi.ldap.LdapProvider
/loginIdentityProviders/provider/property[@name="Authentication Strategy"]: SIMPLE
/loginIdentityProviders/provider/property[@name="Manager DN"]: cn=nifi,ou=people,dc=example,dc=com
/loginIdentityProviders/provider/property[@name="Manager Password"]: password
/loginIdentityProviders/provider/property[@name="Referral Strategy"]: FOLLOW
/loginIdentityProviders/provider/property[@name="Connect Timeout"]: 10 secs
/loginIdentityProviders/provider/property[@name="Read Timeout"]: 10 secs
/loginIdentityProviders/provider/property[@name="Url"]: ldap://hostname:port
/loginIdentityProviders/provider/property[@name="User Search Base"]: OU=people,DC=example,DC=com
/loginIdentityProviders/provider/property[@name="User Search Filter"]: sAMAccountName={0}
/loginIdentityProviders/provider/property[@name="Identity Strategy"]: USE_DN
/loginIdentityProviders/provider/property[@name="Authentication Expiration"]: 12 hours
authorizers_identifiers:
- file-user-group-provider
- ldap-user-group-provider
- composite-user-group-provider
authorizers:
/authorizers/userGroupProvider[1]/identifier: file-user-group-provider
/authorizers/userGroupProvider[1]/class: org.apache.nifi.authorization.FileUserGroupProvider
/authorizers/userGroupProvider[1]/property[@name="Users File"]: "{{ nifi_config_dirs.external_config }}/users.xml"
/authorizers/userGroupProvider[1]/property[@name="Initial User Identity 1"]: cn=John Smith,ou=people,dc=example,dc=com
# Use the full DN of the node certificates here
/authorizers/userGroupProvider[1]/property[@name="Initial User Identity 2"]: CN=nifi_server1.example.com, O=ExampleLLC, L=Saint Louis, ST=Missouri, C=US
/authorizers/userGroupProvider[1]/property[@name="Initial User Identity 3"]: CN=nifi_server2.example.com, O=ExampleLLC, L=Saint Louis, ST=Missouri, C=US
/authorizers/userGroupProvider[1]/property[@name="Initial User Identity 4"]: CN=nifi_server3.example.com, O=ExampleLLC, L=Saint Louis, ST=Missouri, C=US
/authorizers/accessPolicyProvider/identifier: file-access-policy-provider
/authorizers/accessPolicyProvider/class: org.apache.nifi.authorization.FileAccessPolicyProvider
/authorizers/accessPolicyProvider/property[@name="User Group Provider"]: file-user-group-provider
/authorizers/accessPolicyProvider/property[@name="Authorizations File"]: "{{ nifi_config_dirs.external_config }}/authorizations.xml"
/authorizers/accessPolicyProvider/property[@name="Initial Admin Identity"]: cn=John Smith,ou=people,dc=example,dc=com
/authorizers/accessPolicyProvider/property[@name="Node Identity 1"]: CN=nifi_server1.example.com, O=ExampleLLC, L=Saint Louis, ST=Missouri, C=US
/authorizers/accessPolicyProvider/property[@name="Node Identity 2"]: CN=nifi_server2.example.com, O=ExampleLLC, L=Saint Louis, ST=Missouri, C=US
/authorizers/accessPolicyProvider/property[@name="Node Identity 3"]: CN=nifi_server3.example.com, O=ExampleLLC, L=Saint Louis, ST=Missouri, C=US
/authorizers/authorizer/identifier: managed-authorizer
/authorizers/authorizer/class: org.apache.nifi.authorization.StandardManagedAuthorizer
/authorizers/authorizer/property[@name="Access Policy Provider"]: file-access-policy-provider
state_management:
/stateManagement/cluster-provider/property[@name="Connect String"]: "{{ nifi_properties['nifi.zookeeper.connect.string'] }}"
# Assuming nifi_server1 = 192.168.1.10, nifi_server2 = 192.168.1.11, nifi_server3 = 192.168.1.12
# we have Ansible automatically set the myid file on each host to last octet of the node's IP address
# and we set the 'X' of the zookeeper['server.X'] keys to those same numbers.
zookeeper_myid: "{{ ansible_default_ipv4.address.split('.')[-1] }}"
zookeeper:
server.10: nifi_server1:2888:3888
server.11: nifi_server2:2888:3888
server.12: nifi_server3:2888:3888
License
GPLv3
Author Information
This role was created in 2018 by cavemandaveman.
ansible-galaxy install triviadata/ansible-nifi