hdfs_modules

This Ansible role includes a set of modules for managing files and directories on HDFS (Hadoop Distributed File System).

• hdfs_file: Similar to the Ansible files/file module, but for HDFS. Documentation can be found here.

• hdfs_info: Similar to the Ansible files/info module, but for HDFS. Documentation can be found here.

• hdfs_cmd: Similar to the Ansible commands/command module, but for HDFS. Documentation can be found here.

Requirements

These modules require the python-requests package to be installed on the remote node.

If you need to access a Kerberos-protected cluster, the python-requests-kerberos package is also necessary.

Example Playbook

- hosts: edge_node1
  roles:
  - hdfs_modules
  tasks:
  # Create a directory if it doesn't exist.
  # If it already exists, update the owner, group, and permissions if they are different.
  - hdfs_file: hdfs_path=/user/joe/some_directory owner=joe group=users mode=0755 state=directory

  # Copy a file from the local filesystem of the target host to HDFS
  - hdfs_cmd: cmd="sudo -u joe hdfs dfs -put /etc/services /user/joe/services" hdfs_creates=/user/joe/passwd1 uses_shell=True

  # Update the owner, group, and permissions of the copied file
  - hdfs_file: hdfs_path=/user/joe/passwd1 owner=joe group=users mode=0644

  # Alternatively, use hdfs_put for this file copy
  - hdfs_put: src=/etc/services hdfs_dest=/usr/joe/services owner=joe group=users mode=0644

  # You can also copy a folder and all its contents
  - hdfs_put: src=/etc/hadoop/conf/ hdfs_dest=/usr/joe/some_directory hdfs_user=joe

License

GNU GPL

Click on the Link to read the full text.