CVMFS

Install and configure CernVM-FS (CVMFS), particularly for Galaxy servers.

Requirements

On Enterprise Linux (ansible_os_family == "RedHat"), it is assumed that you have enabled Extra Packages for Enterprise Linux (EPEL) for CVMFS's dependencies. If you need to enable EPEL, geerlingguy.repo-epel can easily do this for you.

Role Variables

All variables are optional. However, if unset, the role will essentially do nothing. See the defaults and example playbook for examples.

Galaxy Client

Other than cvmfs_role as described below, Galaxy administrators will most likely only need to set the galaxy_cvmfs_repos_enabled variable (disabled by default), which automatically configures the CVMFS client for galaxyproject.org CVMFS repositories.

The value of galaxy_cvmfs_repos_enabled can be either config-repo or any value that evaluates to true (or false to explcititly disable, although this is the default). Using config-repo is recommended since it causes the role to only install a minimal configuration needed to mount the cvmfs-config.galaxyproject.org CVMFS repository, and then uses CVMFS' Config Repository support to obtain the configs for the other galaxyproject.org CVMFS repositories. This ensures you will always have up-to-date configs for all galaxyproject.org CVMFS repositories.

Setting galaxy_cvmfs_repos_enabled to config-repo overrides the value of cvmfs_config_repo since there can be only one default config repo configured on the client.

Setting galaxy_cvmfs_repos_enabled to any other truthy value will causes the role to create a static configuration where the full configurations for each galaxyproject.org CVMFS repository is installed on the target host. This option is retained for legacy purposes.

You can override the defaults for Galaxy's cvmfs_keys, cvmfs_server_urls, and cvmfs_repositories by prepending galaxy_ to the variable names. See the defaults for details.

If galaxy_cvmfs_repos_enabled is not set, full configuration of non-Galaxy repositories can be performed using the set of variables described below.

Client or shared client/server variables

The complex (list of dict) variables have the following syntaxes:

cvmfs_keys:
  - path: 'absolute path to repo key.pub'
    owner: 'user owning key file (default: root)'
    key: |
      -----BEGIN PUBLIC KEY-----
      MIIBIjAN...

cvmfs_server_urls:
  - domain: 'repo parent domain'
    urls:
      - 'repository URL'

cvmfs_repositories:
  - repository: 'repo name'
    stratum0: 'stratum 0 hostname'
    owner: 'user owning repository (default: root)'
    key_dir: 'path to directory containing repo keys (default: /etc/cvmfs/keys)'
    server_options:
      - KEY=val
    client_options:
      - KEY=val

For Stratum 0 / Release Managers, you can automatically prune older snapshots using the prune_snapshots_time, a hash having keys that correspond to the cron module options. If prune_snapshots_time is unset, then snapshots are not automatically pruned.

cvmfs_repositories:
  - repository: repo.example.org
    owner: user1
    prune_snapshots_count: 20
    prune_snapshots_time:
       special_time: daily

The per-repository prune_snapshots_count option defaults to the value of cvmfs_stratum0_prune_snapshots_count in defaults/main.yml if unset.

Server variables

Dependencies

None.

Example Playbook

Configure all hosts as CVMFS clients with configurations for the Galaxy CVMFS repositories:

- name: CVMFS
  hosts: all
  vars:
    cvmfs_role: client
    galaxy_cvmfs_repos_enabled: config-repo
  roles:
    - geerlingguy.repo-epel
    - galaxyproject.cvmfs

Create a Stratum 1 (mirror) of the Galaxy CVMFS repositories and configure clients to prefer your Stratum 1 (assuming you have configured hosts in groups cvmfsclients and cvmfsstratum1servers):

- name: CVMFS
  hosts: cvmfsclients:cvmfsstratum1servers
  vars:
    cvmfs_srv_device: /dev/sdb
    galaxy_cvmfs_repos_enabled: true
    # override the default
    galaxy_cvmfs_server_urls:
      - domain: galaxyproject.org
        urls:
          - "http://cvmfs.example.org/cvmfs/@fqrn@"
          - "http://cvmfs1-psu0.galaxyproject.org/cvmfs/@fqrn@"
          - "http://cvmfs1-iu0.galaxyproject.org/cvmfs/@fqrn@"
          - "http://cvmfs1-tacc0.galaxyproject.org/cvmfs/@fqrn@"
          - "http://cvmfs1-mel0.gvl.org.au/cvmfs/@fqrn@"
          - "http://cvmfs1-ufr0.galaxyproject.eu/cvmfs/@fqrn@"
  roles:
    - galaxyproject.cvmfs

Create your own CVMFS infrastructure. Run once without keys (new keys will be generated on repo creation):

- name: CVMFS
  hosts: cvmfsstratum0servers
  vars:
    cvmfs_numfiles: 4096
    cvmfs_server_urls:
      - domain: example.org
        urls:
          - "http://cvmfs0.example.org/cvmfs/@fqrn@"
    cvmfs_repositories:
      - repository: foo.example.org
        stratum0: cvmfs0.example.org
        key_dir: /etc/cvmfs/keys/example.org
        server_options:
          - CVMFS_AUTO_TAG=false
          - CVMFS_GARBAGE_COLLECTION=true
          - CVMFS_AUTO_GC=false
        client_options:
          - CVMFS_NFILES=4096
      - repository: bar.example.org
        stratum0: cvmfs0.example.org
        key_dir: /etc/cvmfs/keys/example.org
  roles:
    - galaxyproject.cvmfs

Once keys have been created, add them to cvmfs_keys and run the same as above but hosts: all and cvmfs_keys defined as:

- name: CVMFS
  vars:
    cvmfs_keys:
      - path: /etc/cvmfs/keys/example.org/foo.example.org.pub
        key: |
          -----BEGIN PUBLIC KEY-----
          MIIBIjAN...
      - path: /etc/cvmfs/keys/example.org/bar.example.org.pub
        key: |
          -----BEGIN PUBLIC KEY-----
          MIIBIjAN...

License

MIT

Author Information

Nate Coraor Helena Rasche

View contributors on GitHub