This is an old revision of the document!
Much of this material was originally sourced from: xcad2k cheat-sheets/docker/docker.md
Docker
Docker is a set of platform as a service (PaaS) products that use OS-level virtualization to deliver software in packages called _containers_. The service has both free and premium tiers. The software that hosts the containers is called Docker Engine.
- Project Homepage: [Home - Docker](https://www.docker.com/)
- Documentation: [Docker Documentation | Docker Documentation](https://docs.docker.com/)
- Docker hub: [Docker hub](https://hub.docker.com)
Installation
One click installation script:
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
Note for Debian testing just use sudo apt install docker
, as testing packages the latest docker and the docker repo does not seem to have testing.
Run docker as non root user:
sudo groupadd docker
sudo usermod -aG docker $USER
Install Docker Engine : [Docker Engine](https://docs.docker.com/engine/install/)
Uninstall
Both the install methods actually use the standard package manager to install docker.
dpkg -l|grep docker
to check the docker packages actually installedsudo apt remove docker-ce
- Check the the
var/lib/docker
directory,sudo du -d 1 -h var/lib/docker
rm -rf /var/lib/docker
to remove the docker directory. I try not to store any important files in docker, so this should be low risk for me.
Docker and iptables on Host
Docker applies iptables on the host machine, see Docker and iptables.
A Docker host that is acting or running as a router, e.g. using Traefik, the following may be needed to allow operation.
DNS and nftable / iptables / netfilter within containers
Docker has to perform some interesting network filtering both on the container host, as noted above and within containers as outlined here. It looks like this is required to allow container DNS functionality on Docker containers using bridge networking.
The containers DNS (/etc/resolv.conf
) is assigned to a proxy on 127.0.0.11:53. (Note that DNS uses UDP not TCP datagrams.)
Further to this The container netfilter use NAT chains to operate on 127.0.0.11. See the following nftables info:
On VPN setup, at least openvpn, the /etc/resolv.conf is overwritten anyway so the Docker netfilter chains become irrelevant.
On the other hand where Docker bridge network DNS container name resolution is desirable then these netfilter chains must basically remain unadulterated. So in these cases where I need to used netfilter within the container, the simple solution is to simply add extra chains without using the nft flush ruleset
command first. These means that if subsequent rule changes are made the container will need to be recreated. This is effectively only for simple basic filter (input, output & forward) chains only, more complex netfilter nat requirements would need further consideration.
Portainer
Portainer is a handy web view into the Docker system. It can be loaded on the host as a Docker container as per this docker documentation Install Portainer with Docker on Linux! A lot of the official Portainer web points to the business version. They do allow 5 free business installations with registration. The portainer-ee image is the business image on Docker hub, portainer-ce is the community addition. When downgrading from the business edition to the community edition the database on the volume storage needs to be downgraded or portainer-ce will not start.
The community addition command is:
docker volume create portainer_data docker run -d -p 9443:9443 --name=portainer --restart=always \ -v /var/run/docker.sock:/var/run/docker.sock \ -v portainer_data:/data \ portainer/portainer-ce:latest
Notes:
- The port 8000 (-p 8000:8000) is not normally required on community edition. Further to this Kea dhcp control agent uses port 8000 as default.
- Some browsers give strange message if http access is requested and then require exception for dodgy https certificate.
Portainer agent
Portainer agent allows a remote docker machine to be seen else were via the network. Default port seems to be 9001.
- First stop the agent container:
docker stop portainer_agent
- Then remove the agent container:
docker rm portainer_agent
- Then pull the latest portainer/agent:
docker pull portainer/agent
, default is latest if version is not specified.
docker run -d -p 9001:9001 --name portainer_agent --restart=always \ -v /var/run/docker.sock:/var/run/docker.sock \ -v /var/lib/docker/volumes:/var/lib/docker/volumes \ portainer/agent
Build Images
Docker CLI
Run Containers
COMMAND | DESCRIPTION |
---|---|
docker run IMAGE | Start a new container |
docker run --name CONTAINER IMAGE | Start a new container and set a name |
docker run -p HOSTPORT:CONTAINERPORT IMAGE | Start a new container with mapped ports |
docker run -P IMAGE | Start a new container and map all ports |
docker run -d IMAGE | Start a new container in background |
docker run -v HOSTDIR:TARGETDIR IMAGE | Map local directory and Launch |
Container Management:
COMMAND | DESCRIPTION |
---|---|
docker ps | Show a list of all running containers |
docker ps -a | Show a list of all containers |
docker create IMAGE | Create a new container |
docker start CONTAINER | Start a container |
docker stop CONTAINER | Graceful stop a container |
docker kill CONTAINER | Kill (SIGKILL) a container |
docker restart CONTAINER | Graceful stop and restart a container |
docker pause CONTAINER | Suspend a container |
docker unpause CONTAINER | Resume a container |
docker rename OLD NEW | Rename a container |
docker rm CONTAINER | Destroy a container |
docker rm -f CONTAINER | Destroy a running container |
docker exec -it CONTAINER COMMAND | Run a command in a container |
docker exec -it CONTAINER bash | Run bash shell in a container as default user, usually root |
docker exec -it --user www-data CONTAINER bash | Run bash shell in a container as defined user |
Not that in many simple Alpine containers bash is not the default sh, ash or sh is more likely.
Container Bulk Management
COMMAND | DESCRIPTION |
---|---|
docker stop $(docker ps -q) | To stop all the running containers |
docker stop $(docker ps -a -q) | To stop all the stopped and running containers |
docker kill $(docker ps -q) | To kill all the running containers |
docker kill $(docker ps -a -q) | To kill all the stopped and running containers |
docker restart $(docker ps -q) | To restart all running containers |
docker restart $(docker ps -a -q) | To restart all the stopped and running containers |
docker rm $(docker ps -q) | To destroy all running containers |
docker rm $(docker ps -a -q) | To destroy all the stopped and running containers |
docker pause $(docker ps -q) | To pause all running containers |
docker pause $(docker ps -a -q) | To pause all the stopped and running containers |
docker start $(docker ps -q) | To start all running containers |
docker start $(docker ps -a -q) | To start all the stopped and running containers |
docker rm -vf $(docker ps -a -q) | To delete all containers including its volumes use |
docker rmi -f $(docker images -a -q) | To delete all the images |
docker system prune | To delete all dangling and unused images, containers, cache and volumes |
docker system prune -a | To delete all used and unused images |
docker system prune --volumes | To delete all docker volumes |
Inspect / Troubleshoot Containers:
COMMAND | DESCRIPTION |
---|---|
docker ps | List running containers |
docker ps -a | List all containers, including stopped |
docker logs CONTAINER | Show a container output |
docker logs -f CONTAINER | Follow a container output |
docker top CONTAINER | List the processes running in a container |
docker diff | Show the differences with the image (modified files) |
docker inspect | Show information of a container (json formatted) |
docker stats | Show stats |
docker port CONTAINER | Show mapped port of a container |
Run Commands:
COMMAND | DESCRIPTION |
---|---|
docker attach CONTAINER | Attach to a container |
docker cp CONTAINER:PATH HOSTPATH | Copy files from the container |
docker cp HOSTPATH CONTAINER:PATH | Copy files into the container |
docker export CONTAINER | Export the content of the container (tar archive) |
docker exec CONTAINER | Run a command inside a container |
docker exec -it CONTAINER /bin/bash | Open an interactive shell inside a container (there is no bash in some images, use /bin/sh) |
docker wait CONTAINER | Wait until the container terminates and return the exit code |
Images:
COMMAND | DESCRIPTION |
---|---|
docker images | List all local images |
docker history IMAGE | Show the image history |
docker inspect IMAGE | Show information (json formatted) |
docker tag IMAGE TAG | Tag an image |
docker commit CONTAINER IMAGE | Create an image (from a container) |
docker import URL | Create an image (from a tarball) |
docker rmi IMAGE | Delete images |
docker pull REPO:[TAG] | pull an image/repo from a registry |
docker push REPO:[TAG] | push and image/repo to a registry |
docker search TEXT | Search an image on the official registry |
docker login | Login to a registry |
docker logout | Logout from a registry |
docker save REPO:[TAG] | Export an image/repo as a tarball |
docker load | Load images from a tarball |
docker build DIRECTORY | Build an image from a docker file |
Volumes:
COMMAND | DESCRIPTION |
---|---|
docker volume ls | List all volumes |
docker volume create VOLUME | Create a volume |
docker volume inspect VOLUME | Show information (json formatted) |
docker volume rm VOLUME | Destroy a volume |
docker volume ls --filter="dangling=true“ | List all dangling volumes (not referenced by any container) |
docker volume prune | Delete all volumes (not referenced by any container) |
docker ps command
The docker ps
command output is long and often difficult to read on the terminal for this reason.
docker ps --format 'table {{ .ID }}\t{{.Image}}\t{{ .Names }}'
CONTAINER ID IMAGE NAMES
docker ps -s --format 'table {{ .ID }}\t{{.Status}}\t{{.Image}}\t{{ .Names }}'
CONTAINER ID STATUS IMAGE NAMES
docker ps -s --format 'table {{ .ID }}\t{{.Status}}\t{{.Image}}\t{{ .Names }}\t{{.Size}}'
CONTAINER ID STATUS IMAGE NAMES SIZE
Backup a container
Backup docker data from inside container volumes and package it in a tarball archive.
docker run --rm --volumes-from CONTAINER -v $(pwd):/backup busybox tar cvfz /backup/backup.tar CONTAINERPATH
Restore container from backup
Restore the volume with a tarball archive.
docker run --rm --volumes-from CONTAINER -v $(pwd):/backup busybox sh -c "cd CONTAINERPATH && tar xvf /backup/backup.tar --strip 1"
Volumes
Docker volume types:
- Host Volumes: -v /path/on/host:/path/in/container
- Anonymous Volumes: -v /path/in/container
- Named Volumes: -v name:/path/in/container
Docker controlled volumes are stored at /var/lib/docker/volumes
Networks
Type of networks on Docker:
- Default Bridge (The default separate isolated bridge network.)
- This is the default network used if nothing else is defined
- User defined bridge (DNS name resolution on Docker containers, which default bridge does not.)
docker network create network_named
- Host (Appears on the host machine as if installed there, no separate network.)
- MACVLAN
- MACVLAN (without subVLAN) this create a new ip address on the host machine
docker network create -d macvlan \ --subnet 192.168.1.0/24 \ --gateway 192.168.1.1 \ -o parent=br0 \ network_named
- No host DHCP access so need to specify ip address when creating container (docker cli
--ip 192.168.1.14
). If not specified docker DHCP will assign and could cause clash with host. - May be problem with multiple MACs on common switch port. Need to set promiscuous mode on network, e.g.
sudo ip link set br0 promisc on
.
- MACVLAN with subVLAN - never had to use this yet myself, so do not know much about it, something about trunking on a common link.
- IPVLAN
- IPVLAN on host subnet, this create a new ip address on the host machine, but not with new MAC number, it uses the host MAC number
docker network create -d ipvlan \ --subnet 192.168.1.0/24 \ --gateway 192.168.1.1 \ -o parent=br0 \ network_named
- No host DHCP access so need to specify ip address when creating container (docker cli
--ip 192.168.1.14
). If not specified docker DHCP will assign and could cause clash with host. - May be problem with shared MAC with multiple IP address, but less likely than MACVLAN.
- IPVLAN on separate subnet using the host machine as gateway, but not with new MAC number, it uses the host MAC number
docker network create -d ipvlan \ --subnet 192.168.1.0/24 \ -o parent=br0 -o ipvlan_mode=l3 \ --subnet 192.168.10.0/24 \ network_named
- No host DHCP access so need to specify ip address when creating container (docker cli
--ip 192.168.1.14
). If not specified docker DHCP will assign and could cause clash with host. - May be problem with shared MAC with multiple IP address, but less likely than MACVLAN.
- Overlay network, an even more obscure network arrangement I know nothing about.
- None network - no assigned network, container has no external network connectivity
network troubleshooting
A lot of containers are setup to be small and hence do not include many, if any of the tools required to diagnose problems. A small docker image netshoot
includes the most common networking tools and when attached to the same docker network can be used to diagnose the network and containers networks thereon.
docker run --rm --name netshoot --network proxy -it nicolaka/netshoot /bin/bash
Troubleshooting
- netshoot: a Docker + Kubernetes network trouble-shooting swiss-army container
docker run --name netshoot --rm -it nicolaka/netshoot /bin/bash