• Docker Docs Compose specification, The Compose Specification

Some commands:

• docker-compose pull, it would seem this is required to ensure the latest image(s) is downloaded
• docker-compose up -d --build --force-recreate
  • -d
  • --build on required if the compose includes building an image
  • --force-recreate
• docker-compose down
• docker-compose stop|start|restart it is better to use down and up commands has they force the recreation of the container (no image). This is as the scripts on start-up usually assume a new container.
• docker-compose build --pull, will pull the latest images and rebuild a Docker file.

To check current version: docker-compose --version

The latest version of docker-compose can be check here: git hub docker compose releases change the version sub-directory as required.

sudo curl -L "https://github.com/docker/compose/releases/download/v2.28.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

• Install Docker Compose
• Install Docker Engine : Docker Engine

By default Docker-Compose will create a new network for the given compose file. You can change the behavior by defining custom networks in your compose file.

*Example:*

networks:
  custom-network:

services:
  app:
    networks:
      - custom-network

If you want to use an existing Docker network for your compose files, you can add the `external: true` parameter in your compose file *Example:*

networks:
  existing-network:
    external: true

Volumes allow Docker containers to use persistent storage. In a compose file, you can create and map volumes like this:

volumes:
  my-volume:

services:
  app:
    volumes:
      - my-volume:/path-in-container

These volumes are stored in `/var/lib/docker/volumes`.

Docker compose external named volumes can be used across the Docker installation and they need to be created by the user (otherwise fails) using the docker volume create command. (References: Volumes in Docker Compose tutorial, Using volumes in Docker Compose)

Example:

Defines web_data volume:

docker volume create --driver local \
    --opt type=none \
    --opt device="/var/opt/my_website/dist" \
    --opt o=bind web_data

docker-compose.yml file with a named volume web_data defined externally:

version '3'
 
volumes:
  web_data:
    external: true
 
services:
  app:
    image: nginx:alpine
    ports:
      - 80:80
    volumes:
      - web_data:/usr/share/nginx/html:ro

There are different volume types like nfs, btrfs, ext3, ext4, and also 3rd party plugins to create volumes.

External named volumes can be defined dynamically from environment variables using a name section as we did in the previous example.

It is amazing for how long I put up with this before fixing it. Every time I booted the main server VM Traefik would fail. All the other services that relied on Traefik would not operate. A simple start/restart of Traefik always worked. Most of the other Docker containers that relied upon Traefik needed to be restarted too. It was annoying and made me doubt that my Docker system was ready for “production” although I had been using a couple containers for a couple of months.

I have my docker working directory on an NFS share directory. Unfortunately after a reboot docker compose Trafik would fail on start up with obscure Docker log messages (sudo journalctl -b0 -u docker), e.g. error=“failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during cont with reference to mount a volume, sorry the message was too long to file and I did not get full copy. The log message volume that failed also changed. This lead me to believe a possible problem with Docker starting up before the NFS network file system was ready. So I did the following.

1. Found the systemd unit name for the drive mount using: systemctl list-units –type=mount, in my particular case it was mnt-docker_store.mount for /mnt/docker_store
2. Updated the systemd service for Docker
   1. use systemctl status docker to find the service file, in my case /lib/systemd/system/docker.service
   2. edit the systemd file and add mount unit found to the [Unit] After and Requires directives. You are not meant to directly edit the service file, but rather use the sudo systemctl edit docker.service to create a drop in replacement file.
3. After this reboot to test. (Yes you can use sudo systemctl daemon-reload, however I wanted to test boot performance, so a reboot is necessary anyway.)

Example: This is the existing unchanged service:

/lib/systemd/system/docker.service

[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=docker.socket firewalld.service containerd.service time-set.target mnt-docker_store.mount
Requires=docker.socket mnt-docker_store.mount

[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutStartSec=0
RestartSec=2
Restart=always

# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
# Both the old, and new location are accepted by systemd 229 and up, so using the old location
# to make them work for either version of systemd.
StartLimitBurst=3

# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
# this option work for either version of systemd.
StartLimitInterval=60s

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

# Comment TasksMax if your systemd version does not support it.
# Only systemd 226 and above support this option.
TasksMax=infinity

# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes

# kill only the docker process, not all processes in the cgroup
KillMode=process
OOMScoreAdjust=-500

[Install]
WantedBy=multi-user.target

The override file for docker.service below has some additional services under [Unit] defined that must be running before before docker can be started. In addition I added a further 10 seconds wait to allow system undeer [Service] to stabilise as an ExecStartPre before starting docker.

docker.service

[Unit]
After=network-online.target remote-fs.target
RequiresMountsFor=/mnt/docker_store

[Service]
ExecStartPre=/bin/sleep 10

systemctl show docker.service shows the service definition, it looks like it shows all dependencies, not just main service file ones.

• Docker
  • Compose Specification Overview
  • Dockerfile reference
• Set systemd service to execute after fstab mount
• How to override systemd unit file settings?
• How to ensure that there is a delay before a service is started in systemd?
• How can I list all of the systemd targets?
• How to start systemd service after NFS mount in Linux
• How do I override or configure systemd services?
• Start a systemd service after a device has been mounted
• systemd.unit — Unit configuration #RequiresMountsFor
• RedHat Docs Chapter 17. Working with systemd unit files
• The definitive Guide to Docker compose