CVMFS

Install and set up CernVM-FS (CVMFS), especially for Galaxy servers.

Requirements

For Enterprise Linux (ansible_os_family == "RedHat"), make sure you have enabled Extra Packages for Enterprise Linux (EPEL) which are needed for CVMFS. If you haven't enabled EPEL yet, you can use geerlingguy.repo-epel to do so easily.

Role Variables

All variables are optional, but if you don’t set them, the role won’t do much. Check the defaults and example playbook for more information.

Galaxy Client

Besides the cvmfs_role variable mentioned below, Galaxy admins mainly need to configure the galaxy_cvmfs_repos_enabled variable, which is turned off by default. This will set up the CVMFS client for the CVMFS repositories on galaxyproject.org.

You can set galaxy_cvmfs_repos_enabled to config-repo or any value that means true (you can set it to false to turn it off, but that's the default). It’s best to use config-repo as it only installs the basic settings needed to access the cvmfs-config.galaxyproject.org CVMFS repository, and then fetches configurations for the other galaxyproject.org CVMFS repositories. This guarantees you’ll always have the latest configurations.

Setting galaxy_cvmfs_repos_enabled to config-repo will override cvmfs_config_repo, as you can only have one default config repo setup on the client.

If you set galaxy_cvmfs_repos_enabled to another truthy value, it will create a static setup where all configurations for each galaxyproject.org CVMFS repository are installed on the target host. This older method is still available for legacy support.

You can change the defaults for Galaxy’s cvmfs_keys, cvmfs_server_urls, and cvmfs_repositories by adding galaxy_ to the variable names. Check the defaults for more information.

If galaxy_cvmfs_repos_enabled isn't set, you can fully configure other non-Galaxy repositories using the variables listed below.

Client or Shared Client/Server Variables

Example of Complex Variables

cvmfs_keys:
  - path: 'absolute path to repo key.pub'
    owner: 'user owning key file (default: root)'
    key: |
      -----BEGIN PUBLIC KEY-----
      MIIBIjAN...

cvmfs_server_urls:
  - domain: 'repo parent domain'
    urls:
      - 'repository URL'

cvmfs_repositories:
  - repository: 'repo name'
    stratum0: 'stratum 0 hostname'
    owner: 'user owning repository (default: root)'
    key_dir: 'path to directory containing repo keys (default: /etc/cvmfs/keys)'
    server_options:
      - KEY=val
    client_options:
      - KEY=val

For Stratum 0 / Release Managers

You can automatically manage older snapshots with the prune_snapshots_time, which consists of keys that align with the cron module options. If this isn't set, snapshots won't be pruned automatically.

cvmfs_repositories:
  - repository: repo.example.org
    owner: user1
    prune_snapshots_count: 20
    prune_snapshots_time:
       special_time: daily

Server Variables

Dependencies

None.

Example Playbook

Set up all hosts as CVMFS clients with Galaxy CVMFS repository settings:

- name: CVMFS
  hosts: all
  vars:
    cvmfs_role: client
    galaxy_cvmfs_repos_enabled: config-repo
  roles:
    - geerlingguy.repo-epel
    - galaxyproject.cvmfs

Create a Stratum 1 mirror for Galaxy CVMFS repositories, and let clients prefer your Stratum 1 (assuming your hosts are grouped):

- name: CVMFS
  hosts: cvmfsclients:cvmfsstratum1servers
  vars:
    cvmfs_srv_device: /dev/sdb
    galaxy_cvmfs_repos_enabled: true
    galaxy_cvmfs_server_urls:
      - domain: galaxyproject.org
        urls:
          - "http://cvmfs.example.org/cvmfs/@fqrn@"
          - "http://cvmfs1-psu0.galaxyproject.org/cvmfs/@fqrn@"
          - "http://cvmfs1-iu0.galaxyproject.org/cvmfs/@fqrn@"
          - "http://cvmfs1-tacc0.galaxyproject.org/cvmfs/@fqrn@"
          - "http://cvmfs1-mel0.gvl.org.au/cvmfs/@fqrn@"
          - "http://cvmfs1-ufr0.galaxyproject.eu/cvmfs/@fqrn@"
  roles:
    - galaxyproject.cvmfs

Set up your own CVMFS infrastructure. Run once without keys (new keys will be created when repositories are made):

- name: CVMFS
  hosts: cvmfsstratum0servers
  vars:
    cvmfs_numfiles: 4096
    cvmfs_server_urls:
      - domain: example.org
        urls:
          - "http://cvmfs0.example.org/cvmfs/@fqrn@"
    cvmfs_repositories:
      - repository: foo.example.org
        stratum0: cvmfs0.example.org
        key_dir: /etc/cvmfs/keys/example.org
        server_options:
          - CVMFS_AUTO_TAG=false
          - CVMFS_GARBAGE_COLLECTION=true
          - CVMFS_AUTO_GC=false
        client_options:
          - CVMFS_NFILES=4096
      - repository: bar.example.org
        stratum0: cvmfs0.example.org
        key_dir: /etc/cvmfs/keys/example.org
  roles:
    - galaxyproject.cvmfs

After keys are generated, add them to cvmfs_keys, and run the same playbook with hosts: all and cvmfs_keys specified like this:

- name: CVMFS
  vars:
    cvmfs_keys:
      - path: /etc/cvmfs/keys/example.org/foo.example.org.pub
        key: |
          -----BEGIN PUBLIC KEY-----
          MIIBIjAN...
      - path: /etc/cvmfs/keys/example.org/bar.example.org.pub
        key: |
          -----BEGIN PUBLIC KEY-----
          MIIBIjAN...

License

MIT

Author Information

Nate Coraor
Helena Rasche

View contributors on GitHub