Deploying a docker-compose.yml to a Server

This document describes an Ansible role used to deploy a docker-compose.yml file into a home directory. It's best used with bigsudo.

For a detailed guide on set up, refer to TUTORIAL.md, which provides practical patterns for achieving eXtreme DevOps.

Example

This role contains tests and you can start using it right away:

• Dockerfile: Works by default for a CRUDLFA+ project, providing compressed and cached static files along with a spooler.
• .gitlab-ci.yml: Builds and pushes the Docker image to GitLab's image registry and includes a YAML template for deployment jobs, plus 3 example configurations for different deployment scenarios.
• docker-compose.yml: Used for local and temporary deployments.
• docker-compose.traefik.yml: Supports yourlabs.traefik.
• docker-compose.persist.yml: An additional file for persistent deployments, referenced in .gitlab-ci.yml, requiring docker-compose.traefik.yml.

Here’s what you need to change:

• .gitlab-ci.yml: Follow the # UNCOMMENT ABOVE AND REMOVE BELOW comments to set it up to use yourlabs/ansible.
• Update ci.yourlabs.io with your server name in .gitlab-ci.yml.
• Define a GitLab CI variable $CI_SSH_KEY with an ed25519 private key, which you can create using: ssh-keygen -t ed25519 -a 100.
• In the Dockerfile, update the command argument --module=wsgi:application with the correct path to your WSGI application. You may also remove the bigsudo setup if not needed.

After these changes, you can customize it to fit your needs!

Other bigsudo commands you might find useful:

• Enable a backup systemd timer using: bigsudo yourlabs.compose backup home=/home/$CI_PROJECT_NAME-$CI_ENVIRONMENT_NAME.
• For cleaning up unused images, volumes, and networks on the review deployment CI server, run: bigsudo yourlabs.docker prunecron @ci.your.host. This helps manage disk space.

For a broader overview, check out the blog article on eXtreme DevOps.

Role Features Overview

The main goal of this role is to automate the deployment of a docker-compose.yml file into a directory on a host while handling related tasks.

You can deploy from any directory containing a docker-compose.yml file using:

bigsudo yourlabs.compose home=/home/staging $user@$host

If $user@$host isn’t specified, it will run on the local machine.

You can pass multiple compose files and environment variables, which will be included in the final deployment:

FOO=bar bigsudo yourlabs.compose \
    home=/home/staging \
    compose_django_image=$YOUR_IMAGE \
    compose=docker-compose.yml,docker-compose.staging.yml

Directory Creation

This role can also set up directories with specified UID, GID, and permissions using the io.yourlabs.compose.mkdir label. Example:

volumes:
- "./log/admin:/app/log"
labels:
- "io.yourlabs.compose.mkdir=./app/log:1000:1000:0750"

This command will create the directory {{ home }}/log/admin owned by user 1000 and group 1000, with permissions set to 0750.

Environment Variable Setup

The role can automatically set up environment variables defined in the docker-compose.yml file at runtime. For instance, if you have:

environment:
- FOO

Running the yourlabs.compose role with FOO variable defined will yield:

environment:
- FOO=bar

YAML Overrides via Command Line

You can also modify service values using the compose_servicename_keyname variable. For example, to change the image value for django service:

bigsudo yourlabs.compose home=/home/test compose_django_image=yourimage

You can also unset values easily by providing an empty option. For example, to make compose[services][django][build] empty:

bigsudo yourlabs.compose home=/home/test compose_django_build=

If your repository isn’t cloned in the home directory, this will prevent errors related to Dockerfile location:

bigsudo yourlabs.compose home=/home/test compose_django_build=

Network Management Automation

Managing networks with docker-compose can be tricky. For example, if you have a web network with a load balancer like Traefik, you can attach it to the Django service automatically:

bigsudo yourlabs.compose home=/home/test compose_django_networks=web

However, if your load balancer's docker-compose.yml declares a web network as external, the regular command won’t create it, leading to an error:

ERROR: Network web declared as external, but could not be found...

This role avoids this issue by checking for external networks in the docker-compose.yml and pre-creating them if necessary.

Using as an Ansible Role

You can use this role like any other Ansible role in your repository:

- name: Ensure docker is set up on this host
  include_role: name=yourlabs.compose
  vars:
    home: /home/yourthing
    compose_django_image: foobar
    compose_django_build:
    compose_django_networks:
    - web

Backup System

Automatically scheduled backups are done for deployments with a home argument. It sets up three scripts in the home directory:

• ./backup.sh: Creates a local dump and backs it up in restic.
• ./restore.sh: Lists restic snapshots; you can run it with a snapshot hash to restore that snapshot.
• ./prune.sh: Deletes old backups.

It will also create a systemd service and timer:

• backup-PROJECTNAME.service
• backup-PROJECTNAME.timer

Development

To develop this role locally, run:

# Build the image
docker build -t test .

# Execute bigsudo
bigsudo . home=/tmp/test compose_django_build= compose_django_image=test compose=docker-compose.yml,docker-compose.persist.yml

Keep in mind that it will always attempt to run the backup script before any updates. If you change the script, you need to delete backup.sh before running yourlabs.compose again to ensure it gets updated.