ansible-tap

This is a tool for Ansible called ansible-tap that helps you create and run tests using the Test Anything Protocol (TAP). You can think of it as an alternative to tools like Serverspec and Testinfra, but specifically for Ansible.

Requirements

• You need Ansible version 2 or newer.
• You need Python version 2.7 or newer.

Installation

You can't install this plugin directly with ansible-galaxy, but you can download it using:

ansible-galaxy install -p roles/ benwebber.tap

Then go to the role directory and run make install:

cd roles/benwebber.tap
make install

This will place the plugin in ~/.ansible/plugins/callback.

How to Use

You need to set up Ansible to use this plugin for output:

ANSIBLE_STDOUT_CALLBACK=tap ansible-playbook -i hosts test.yml -l hostname

You can also make it the default output in ansible.cfg:

[defaults]
stdout_callback=tap

Writing Ansible Tests

By default, Ansible will stop running if any task fails. To allow all tests to run, you can use ignore_errors: true:

- name: check if service is running
  command: systemctl is-active service
  register: is_active
  tags: diagnostic

- name: assert that service is running
  assert:
    that: is_active.rc == 0
  ignore_errors: true

This way, Ansible runs all tests even if some of them fail.

If you have many tests, you can use ignore_errors: true with a block:

- name: check if service is running
  command: systemctl is-active service
  register: is_active
  tags: diagnostic

- name: check if service is enabled
  command: systemctl is-enabled service
  register: is_enabled
  tags: diagnostic

- ignore_errors: true
  block:
  - name: assert that service is running
    assert:
      that: is_active.rc == 0

  - name: assert that service is enabled
    assert:
      that: is_enabled.rc == 0

If a task fails, the plugin will give you helpful information as an embedded YAML document.

Excluding Tasks from TAP

Some tasks are just for setting up tests and shouldn't be included in TAP results. You can tag those tasks as diagnostic:

- name: set up next test
  command: 'true'
  register: true_
  tags: diagnostic

- name: should always pass
  assert:
    that: true_.rc == 0
  ignore_errors: true

The output will show these setup tasks differently:

# command: set up next test
ok - assert: should always pass

Expected Failures and Unexpected Successes

You can mark tests that are not yet implemented with a TODO tag. If a TODO test fails, it’s considered expected; if it passes, it’s an unexpected success.

- name: expected failure
  assert:
    that: false
  ignore_errors: true
  tags: TODO

The TAP output will then indicate this:

not ok - assert: expected failure # TODO

Skipping Tests

You can skip specific tests using a SKIP directive with Ansible's when condition:

- name: this is a skipped task
  assert:
    that: false
  ignore_errors: true
  when: false

The output will show why it was skipped:

ok - assert: skipped # SKIP Conditional check failed

Example

The tests/ directory has an example test suite demonstrating various test results.

After you install the plugin, run the example with:

ANSIBLE_STDOUT_CALLBACK=tap ansible-playbook -i localhost, -c local tests/playbooks/test_multiple_with_failures.yml

You can expect TAP output like:

TAP version 13
# command: set up next test
ok - assert: pass
not ok - assert: failed
not ok - assert: expected failure # TODO
ok - assert: unexpected pass # TODO
ok - assert: skipped # SKIP Conditional check failed
1..5

Caveats

Currently, this plugin can only run tests against one host at a time. The TAP specification doesn’t include a way to combine results from multiple hosts.

License

MIT