---

IMPORTANT: This project is no longer regularly updated by its original creators. The code has been moved to the official IBM/ibm-spectrum-scale-install-infra project for all future work. While updates and support for this original project have stopped, the code is still available for anyone who wants to keep working on it.

---

IBM Spectrum Scale (GPFS) Ansible Role

This is a customizable Ansible role to install and configure IBM Spectrum Scale (GPFS).

We are especially looking for feedback and future requirements!

Features

• Install Spectrum Scale packages on Linux systems
• Optionally, check that packages are correct by comparing checksums
• Upgrade while daemons are stopped
• Compile or install pre-compiled Linux kernel extensions
• Set up SSH public key authentication
• Create or join an existing cluster
• Configure Network Shared Disks (NSDs)
• Create or expand file systems
• Set up node classes
• Define configuration based on node classes

Installation methods include:

• Install from a YUM repository
• Install from a remote package
• Install from a local package

Future plans:

• Install CES packages
• Install GUI and zimon packages

Installation

To install, run the following command:

$ ansible-galaxy install acch.spectrum_scale

Requirements

There's no public repository, so download Spectrum Scale (GPFS) packages from the IBM website. Visit https://www.ibm.com/support/fixcentral and search for 'IBM Spectrum Scale (Software defined storage)'.

IBM Spectrum Scale Developer Edition

If you don’t have a subscription, you can download the IBM Spectrum Scale Developer Edition:

• Includes all features of the IBM Spectrum Scale Data Management Edition, but is limited to 12 TB per cluster.
• It does not come with IBM support. Using it in production is not allowed.
• Access the Developer Edition via https://www.ibm.com/us-en/marketplace/scale-out-file-and-object-storage.

Local Repository

To create a local Spectrum Scale repository:

cd /your/webserver/folder
# Download the installation package. Get the direct link or copy it manually...
wget <Spectrum Scale installation package>
sh ./Spectrum_Scale_Data_Management-5.0.X.X-x86_64-Linux-install --dir ./SpectrumScaleRpms/5.0.X.X/ --silent
cd SpectrumScaleRpms/5.0.X.X/
yum -y install createrepo
createrepo .

Role Variables

Default variables are in defaults/main.yml, which also contains detailed documentation. Define your own host variables in your inventory to override defaults.

The variable scale_version is required, and you must choose an installation method by defining one of the following:

• scale_install_repository_url
• scale_install_remotepkg_path (accessible on Ansible managed node)
• scale_install_localpkg_path (accessible on Ansible control machine)

Cluster Membership

All hosts in the play become nodes in the same cluster. To add hosts to an existing cluster, include at least one node from that cluster.

You can create multiple clusters by running separate plays.

Example Playbook

This simple playbook installs Spectrum Scale on a node:

---
- hosts: scale01.example.com
  vars:
    - scale_version: 4.2.3.4
    - scale_install_localpkg_path: /path/to/Spectrum_Scale_Standard-4.2.3.4-x86_64-Linux-install
  roles:
    - acch.spectrum_scale

This installs the necessary packages and creates a single-node Spectrum Scale cluster.

For a more realistic setup, you will likely want to install Spectrum Scale on multiple nodes and set node roles for high availability. Your cluster configuration might look like this:

# hosts:
[cluster01]
scale01  scale_cluster_quorum=true   scale_cluster_manager=true
scale02  scale_cluster_quorum=true   scale_cluster_manager=true
scale03  scale_cluster_quorum=true   scale_cluster_manager=false
scale04  scale_cluster_quorum=false  scale_cluster_manager=false
scale05  scale_cluster_quorum=false  scale_cluster_manager=false

# playbook.yml:
---
- hosts: cluster01
  vars:
    - scale_version: 4.2.3.4
    - scale_install_repository_url: http://infraserv/gpfs_rpms/
    - scale_cluster_clustername: cluster01.example.com
  roles:
    - acch.spectrum_scale

Refer to defaults/main.yml for detailed information on variables and configurations.

Defining node roles like scale_cluster_quorum and scale_cluster_manager is optional. If you don’t define any quorum nodes, the first seven hosts will be automatically assigned the quorum role.

The above examples will install required packages and create a Spectrum Scale cluster ready for mounting existing file systems. To create local file systems, additional information is needed. It's recommended to use host_vars files for this purpose.

Example host_vars/scale01:

---
scale_storage:
  - filesystem: gpfs01
    disks:
      - device: /dev/sdb
      - device: /dev/sdc

For scale02, you can define the disks similarly.

Refer to man mmchfs and man mmchnsd for descriptions of the storage parameters.

filesystem is required, and device is required for each disk. All other parameters are optional.

Spectrum Scale node classes can also be defined per node with the scale_nodeclass variable:

# host_vars/scale01:
---
scale_nodeclass:
  - classA
  - classB

You can define configuration parameters using group_vars inventory files.

Limitations

This role can create new or extend existing clusters but will not remove nodes, disks, file systems, or node classes. Changing the file system pool for a disk is also not included.

Moreover, online upgrades are outside the scope of this role. Spectrum Scale does support rolling online upgrades, but this involves careful planning and might need manual intervention for unexpected issues.

Troubleshooting

The role keeps configuration files in /var/tmp on the first host. These files help track changes and decide if certain Spectrum Scale commands need to be run again. If you have problems, deleting these files can refresh the cache, but the next run may take longer.

Please use the issue tracker for questions, bug reports, and feature requests.

Copyright and license

Copyright 2017 Achim Christ, released under the MIT license.