Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
docker_notes:docker [2024-01-14 Sun wk02 11:23] baumkpdocker_notes:docker [2024-09-28 Sat wk39 18:34] (current) – [Build Images] baumkp
Line 1: Line 1:
-{{tag>linux docker installation network volume iptables nftables portainer}} +{{tag>linux docker cli installation network volume iptables nftables portainer}}
-Much of this material was originally sourced from: [[https://github.com/xcad2k/cheat-sheets/blob/main/docker/docker.md|xcad2k cheat-sheets/docker/docker.md]] +
  
 ======Docker====== ======Docker======
Line 31: Line 29:
   *[[https://stackoverflow.com/questions/62677013/uninstall-docker-version-installed-via-script|Uninstall docker version installed via script]]   *[[https://stackoverflow.com/questions/62677013/uninstall-docker-version-installed-via-script|Uninstall docker version installed via script]]
  
-=====Docker and iptables=====+=====Docker and iptables on Host=====
 Docker applies iptables on the host machine, see [[https://docs.docker.com/network/iptables/|Docker and iptables]]. Docker applies iptables on the host machine, see [[https://docs.docker.com/network/iptables/|Docker and iptables]].
  
Line 64: Line 62:
  
 ++++ ++++
 +
 +=====DNS and nftable / iptables / netfilter within containers=====
 +Docker has to perform some interesting network filtering both on the container host, as noted above and within containers as outlined here.  It looks like this is required to allow container DNS functionality on Docker containers using bridge networking.
 +
 +The containers DNS (''/etc/resolv.conf'') is assigned to a proxy on 127.0.0.11:53.  //(Note that DNS uses UDP not TCP datagrams.)//
 +
 +Further to this The container netfilter use NAT chains to operate on 127.0.0.11.  See the following nftables info:
 +++++nft list tables|
 +<code>table ip nat</code>
 +++++
 +++++nft list table ip nat|
 +<code># Warning: table ip nat is managed by iptables-nft, do not touch!
 +table ip nat {
 + chain DOCKER_OUTPUT {
 + meta l4proto tcp ip daddr 127.0.0.11 xt match "tcp" counter packets 0 bytes 0 xt target "DNAT"
 + meta l4proto udp ip daddr 127.0.0.11 xt match "udp" counter packets 329 bytes 20249 xt target "DNAT"
 + }
 +
 + chain OUTPUT {
 + type nat hook output priority dstnat; policy accept;
 + ip daddr 127.0.0.11 counter packets 329 bytes 20249 jump DOCKER_OUTPUT
 + }
 +
 + chain DOCKER_POSTROUTING {
 + meta l4proto tcp ip saddr 127.0.0.11 xt match "tcp" counter packets 0 bytes 0 xt target "SNAT"
 + meta l4proto udp ip saddr 127.0.0.11 xt match "udp" counter packets 0 bytes 0 xt target "SNAT"
 + }
 +
 + chain POSTROUTING {
 + type nat hook postrouting priority srcnat; policy accept;
 + ip daddr 127.0.0.11 counter packets 329 bytes 20249 jump DOCKER_POSTROUTING
 + }
 +}
 +/ # </code>
 +++++
 +
 +On VPN setup, at least openvpn, the /etc/resolv.conf is overwritten anyway so the Docker netfilter chains become irrelevant. 
 +
 +On the other hand where Docker bridge network DNS container name resolution is desirable then these netfilter chains must basically remain unadulterated.  So in these cases where I need to used netfilter within the container, the simple solution is to simply add extra chains without using the nft ''flush ruleset'' command first.  These means that if subsequent rule changes are made the container will need to be recreated.   This is effectively only for simple basic filter (input, output & forward) chains only, more complex netfilter nat requirements would need further consideration.
 +  
  
 =====Portainer===== =====Portainer=====
Line 84: Line 122:
   * Then remove the agent container: ''%%docker rm portainer_agent%%''   * Then remove the agent container: ''%%docker rm portainer_agent%%''
   * Then pull the latest portainer/agent: ''%%docker pull portainer/agent%%'', default is latest if version is not specified.   * Then pull the latest portainer/agent: ''%%docker pull portainer/agent%%'', default is latest if version is not specified.
-<code yaml>docker run -d   -p 9001:9001   --name portainer_agent   --restart=always +<code yaml>docker run -d   -p 9001:9001   --name portainer_agent   --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker/volumes:/var/lib/docker/volumes portainer/agent</code> 
--v /var/run/docker.sock:/var/run/docker.sock +
--v /var/lib/docker/volumes:/var/lib/docker/volumes +
-portainer/agent</code>+
  
-=====Build Images===== 
  
 =====Docker CLI===== =====Docker CLI=====
 +Much of this material was originally sourced from: [[https://github.com/ChristianLempa/cheat-sheets/blob/main/docker/docker.md|cheat-sheets/docker/docker.md]]
  
 **Run Containers** **Run Containers**
Line 198: Line 234:
 |''docker volume prune'' | Delete all volumes (not referenced by any container)| |''docker volume prune'' | Delete all volumes (not referenced by any container)|
  
 +====docker ps command====
 +The ''docker ps'' command output is long and often difficult to read on the terminal for this reason.
 +  *''%%docker ps --format 'table {{ .ID }}\t{{.Image}}\t{{ .Names }}'%%''
 +<code>CONTAINER ID   IMAGE                    NAMES</code>
 +  *''%%docker ps -s --format 'table {{ .ID }}\t{{.Status}}\t{{.Image}}\t{{ .Names }}'%%''
 +<code>CONTAINER ID   STATUS                  IMAGE                 NAMES</code>
 +  *''%%docker ps -s --format 'table {{ .ID }}\t{{.Status}}\t{{.Image}}\t{{ .Names }}\t{{.Size}}'%%''
 +<code>CONTAINER ID   STATUS                  IMAGE                 NAMES               SIZE</code>
 +
 +===reference===
 +  *[[https://devdojo.com/bobbyiliev/how-to-change-the-docker-ps-output-format|How to change the docker ps output format]]
 +  *Docker Docs
 +    *[[https://docs.docker.com/reference/cli/docker/container/ls/|docker container ls]]
 +    *[[https://docs.docker.com/storage/storagedriver/#container-size-on-disk|Container size on disk]]
 ====Backup a container==== ====Backup a container====
 Backup docker data from inside container volumes and package it in a tarball archive.\\ Backup docker data from inside container volumes and package it in a tarball archive.\\
Line 250: Line 300:
   - Overlay network, an even more obscure network arrangement I know nothing about.   - Overlay network, an even more obscure network arrangement I know nothing about.
   - None network - no assigned network, container has no external network connectivity   - None network - no assigned network, container has no external network connectivity
 +
 +====network troubleshooting====
 +A lot of containers are setup to be small and hence do not include many, if any of the tools required to diagnose problems.  A small docker image ''netshoot'' includes the most common networking tools and when attached to the same docker network can be used to diagnose the network and containers networks thereon.
 +  *''%%docker run --rm --name netshoot --network proxy -it nicolaka/netshoot /bin/bash%%''
 +
 ====Troubleshooting==== ====Troubleshooting====
   *[[https://github.com/nicolaka/netshoot|netshoot: a Docker + Kubernetes network trouble-shooting swiss-army container]] ''%%docker run --name netshoot --rm -it nicolaka/netshoot /bin/bash%%''   *[[https://github.com/nicolaka/netshoot|netshoot: a Docker + Kubernetes network trouble-shooting swiss-army container]] ''%%docker run --name netshoot --rm -it nicolaka/netshoot /bin/bash%%''