Docker-Compose [KPTree Wiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
{{tag>linux docker compose docker-compose "docker compose" container network nftables iptables nft}}
======Docker-Compose======
    *Docker Docs [[https://docs.docker.com/compose/compose-file/|Compose specification]], [[https://github.com/compose-spec/compose-spec/blob/master/spec.md#version-and-name-top-level-elements|The Compose Specification]]

Some commands:
  ***''docker-compose pull'', it would seem this is required to ensure the latest image(s) is downloaded**
  *''%%docker-compose up -d --build --force-recreate%%''
    * ''-d''
    * ''%%--build%%'' on required if the compose includes building an image
    * ''%%--force-recreate%%''
  *''docker-compose down''
  *''docker-compose stop|start|restart'' it is better to use down and up commands has they force the recreation of the container (no image). This is as the scripts on start-up usually assume a new container.
  *''%%docker-compose build --pull%%'', will pull the latest images and rebuild a Docker file.
=====Installation / Update=====
To check current version: ''%%docker-compose --version%%''

The latest version of docker-compose can be check here: [[https://github.com/docker/compose/releases | git hub docker compose releases]] change the version sub-directory as required.
<code bash>
sudo curl -L "https://github.com/docker/compose/releases/download/v2.26.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
</code>

  *[[https://gdevillele.github.io/compose/install/|Install Docker Compose]]
  *Install Docker Engine : [[https://docs.docker.com/engine/install/|Docker Engine]]

=====Networking=====
By default Docker-Compose will create a new network for the given compose file. You can change the behavior by defining custom networks in your compose file.

====Create and assign custom network====

*Example:*
<code yaml>
networks:
  custom-network:

services:
  app:
    networks:
      - custom-network
</code>

====Use existing networks====
If you want to use an existing Docker network for your compose files, you can add the `external: true` parameter in your compose file
*Example:*
<code yaml>
networks:
  existing-network:
    external: true
</code>


=====Volumes=====
Volumes allow Docker containers to use persistent storage. In a compose file, you can create and map volumes like this:
<code yaml>
volumes:
  my-volume:

services:
  app:
    volumes:
      - my-volume:/path-in-container
</code>

These volumes are stored in `/var/lib/docker/volumes`.

====Docker external named volumes====
Docker compose external named volumes can be used across the Docker installation and they need to be created by the user (otherwise fails) using the docker volume create command. (References: [[http://sefidian.com/2021/11/05/volumes-in-docker-compose-tutorial/|Volumes in Docker Compose tutorial]], [[https://devopscell.com/docker/docker-compose/volumes/2018/01/16/volumes-in-docker-compose.html|Using volumes in Docker Compose]])

Example:

Defines web_data volume:
<code bash>
docker volume create --driver local \
    --opt type=none \
    --opt device="/var/opt/my_website/dist" \
    --opt o=bind web_data
</code>


docker-compose.yml file with a named volume web_data defined externally:
<code yaml>
version '3'
 
volumes:
  web_data:
    external: true
 
services:
  app:
    image: nginx:alpine
    ports:
      - 80:80
    volumes:
      - web_data:/usr/share/nginx/html:ro
</code>

There are different volume types like nfs, btrfs, ext3, ext4, and also 3rd party plugins to create volumes.

External named volumes can be defined dynamically from environment variables using a name section as we did in the previous example.
=====NFS File System not ready=====
It is amazing for how long I put up with this before fixing it.  Every time I booted the main server VM Traefik would fail.  All the other services that relied on Traefik would not operate.  A simple start/restart of Traefik always worked. Most of the other Docker containers that relied upon Traefik needed to be restarted too.  It  was annoying and made me doubt that my Docker system was ready for "production" although I had been using a couple containers for a couple of months.

I have my docker working directory on an NFS share directory.  Unfortunately after a reboot docker compose Trafik would fail on start up with obscure Docker log messages (''sudo journalctl -b0 -u docker''), e.g. ''error="failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during cont'' with reference to mount a volume, sorry the message was too long to file and I did not get full copy. The log message volume that failed also changed.  This lead me to believe a possible problem with Docker starting up before the NFS network file system was ready.  So I did the following.
  - Found the systemd unit name for the drive mount using: ''systemctl list-units --type=mount'', in my particular case it was ''mnt-docker_store.mount'' for ''/mnt/docker_store''
  - Updated the systemd service for Docker
    - use systemctl status docker to find the service file, in my case ''/lib/systemd/system/docker.service''
    - edit the systemd file and add mount unit found to the [Unit] After and Requires directives.  You are not meant to directly edit the service file, but rather use the ''sudo systemctl edit docker.service'' to create a drop in replacement file.
  - After this reboot to test.  (Yes you can use ''sudo systemctl daemon-reload'', however I wanted to test boot performance, so a reboot is necessary anyway.)
Example: This is the existing unchanged service: ++++/lib/systemd/system/docker.service|
<code>[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=docker.socket firewalld.service containerd.service time-set.target mnt-docker_store.mount
Requires=docker.socket mnt-docker_store.mount

[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutStartSec=0
RestartSec=2
Restart=always

# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
# Both the old, and new location are accepted by systemd 229 and up, so using the old location
# to make them work for either version of systemd.
StartLimitBurst=3

# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
# this option work for either version of systemd.
StartLimitInterval=60s

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

# Comment TasksMax if your systemd version does not support it.
# Only systemd 226 and above support this option.
TasksMax=infinity

# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes

# kill only the docker process, not all processes in the cgroup
KillMode=process
OOMScoreAdjust=-500

[Install]
WantedBy=multi-user.target
</code>++++

The override file for docker.service below has some additional services under [Unit] defined that must be running before before docker can be started.  In addition I added a further 10 seconds wait to allow system undeer [Service] to stabilise as an ''ExecStartPre'' before starting docker.
++++docker.service|
<code>[Unit]
After=network-online.target remote-fs.target
RequiresMountsFor=/mnt/docker_store

[Service]
ExecStartPre=/bin/sleep 10</code>
++++

''systemctl show docker.service'' shows the service definition, it looks like it shows all dependencies, not just main service file ones.

References:
  * [[https://unix.stackexchange.com/questions/246935/set-systemd-service-to-execute-after-fstab-mount|Set systemd service to execute after fstab mount]]
  * [[https://unix.stackexchange.com/questions/398540/how-to-override-systemd-unit-file-settings|How to override systemd unit file settings?]]
  * [[https://stackoverflow.com/questions/43001223/how-to-ensure-that-there-is-a-delay-before-a-service-is-started-in-systemd:|How to ensure that there is a delay before a service is started in systemd?]]
  * [[https://unix.stackexchange.com/questions/619848/how-can-i-list-all-of-the-systemd-targets|How can I list all of the systemd targets?]]
  * [[https://www.golinuxcloud.com/start-systemd-service-after-nfs-mount/|How to start systemd service after NFS mount in Linux]]
  * [[https://askubuntu.com/questions/659267/how-do-i-override-or-configure-systemd-services|How do I override or configure systemd services?]]
  * [[https://superuser.com/questions/930798/start-a-systemd-service-after-a-device-has-been-mounted|Start a systemd service after a device has been mounted]]
  * [[https://www.freedesktop.org/software/systemd/man/systemd.unit.html#RequiresMountsFor=|systemd.unit — Unit configuration #RequiresMountsFor]]
  * [[https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_basic_system_settings/assembly_working-with-systemd-unit-files_configuring-basic-system-settings|RedHat Docs Chapter 17. Working with systemd unit files]]


<- docker_notes:docker|Back ^ docker_notes:index|Start page ^ docker_notes:init|Next ->