A how-to manual: how Docker works, running and debugging containers, building small images, volumes and file ownership, networks, Compose, limits, logs, security, cleanup and troubleshooting, with every example run for real in a lab.
▶Prefer to watch? This short video runs this page's main commands on a real Ubuntu 24.04 machine, with on-screen captions and no voiceover. Click to play, or watch it on YouTube.
docker: runs programs in containers, from images.
docker run -d --name web -p 8080:80 nginx:alpine
docker ps
docker logs web
docker compose up -d
Docker packages a program and everything it needs into an image, and runs that image as a container: an isolated process with its own file system, network and users that shares the host's kernel. It is how most services are now built, tested and shipped, and it is also where most of the surprises live: a container that exits at once, a stop that takes ten seconds, files owned by root, a port that is open although the firewall says closed, a disk filled by logs.
This guide goes from the first docker run to the parts that cause trouble, in the order you meet them: containers, images, builds, volumes, networks, Compose, limits, logs, security and cleanup, then the errors and what each exit code means. The older cheat sheet that stood on this page is replaced by this guide, and its one-line commands are in the cheat sheet section.
Every example ran for real on Ubuntu 24.04 with Docker Engine 29.1.3 (Ubuntu's docker.io 29.1.3-0ubuntu3~24.04.2), Compose 2.40.3 and Buildx 0.30.1, inside a lab with its own Docker daemon, so no real containers were touched. The lab had only two small images (alpine and nginx:alpine) and no registry access, so the commands that need a registry (pull, push, login, search) are shown and marked as not run, and so are the ones that need systemd, a second machine or a firewall change.
What Docker is, which one you have, and the lab
Docker runs programs in containers: isolated processes that share the host's kernel but see their own file system, network, process list and users. A container starts in a fraction of a second because it is a process, not a virtual machine.
The words, in the order you meet them:
image a read-only template: the file system and the default command. Built once, run many times.
container a running (or stopped) copy of an image, plus a thin writable layer on top.
registry a server that stores images (Docker Hub, GitHub Container Registry, your own).
volume storage that lives outside any one container and survives its removal.
network a virtual network that containers join; user-defined ones give name-based DNS.
engine the dockerd daemon. The docker command only talks to it over a socket.
compose a second tool that starts several containers from one YAML file.
Every example below ran for real in a lab: an Ubuntu 24.04 machine with Docker Engine 29.1.3 (package docker.io 29.1.3-0ubuntu3~24.04.2), Compose 2.40.3 and Buildx 0.30.1. The lab is itself a container with its own Docker daemon inside it (so the guide never touched anyone's real containers). Only two small images were loaded into it, alpine and nginx:alpine, so the examples that need a registry download are shown and marked as not run. Commands that need systemd or a second machine are marked the same way.
$ docker --version
Docker version 29.1.3, build 29.1.3-0ubuntu3~24.04.2
$ docker compose version
Docker Compose version 2.40.3+ds1-0ubuntu1~24.04.1
$ docker version --format 'client {{.Client.Version}}, server {{.Server.Version}}'
client 29.1.3, server 29.1.3
$ docker info --format 'storage driver: {{.Driver}}, cgroup: {{.CgroupVersion}}, containers: {{.Containers}}, images: {{.Images}}'
storage driver: overlayfs, cgroup: 2, containers: 0, images: 2
$ docker images
IMAGE ID DISK USAGE CONTENT SIZE EXTRA
alpine:latest 294b683cb724 13MB 3.94MB
nginx:alpine df221db836e1 94.4MB 27.2MB
The first test of any installation is to run something small. alpine is a 13 MB image, and --rm deletes the container when it exits:
$ docker run --rm alpine echo "hello from a container"
hello from a container
$ docker run --rm alpine cat /etc/os-release | head -2
NAME="Alpine Linux"
ID=alpine
$ docker run --rm alpine uname -r
7.0.0-38-generic
$ uname -r
7.0.0-38-generic
The last two commands print the same kernel version. That is the whole idea: a container uses the host's kernel, and brings its own user space (here Alpine's, whatever the host runs).
The other tools you will hear about: Podman (a daemonless, rootless-first engine with a nearly identical docker-style command line; sudo apt install podman), containerd and nerdctl (what Kubernetes uses underneath), and Docker Desktop (a packaged Docker for Windows and macOS that runs Linux in a small virtual machine). This guide covers Docker Engine on Linux. Podman was not run in the lab.
Installing it and who may use it
On Ubuntu there are two common ways to get Docker:
sudo apt install docker.io docker-compose-v2 docker-buildx Ubuntu's own packages (what the lab uses). Updated with the system.
Docker's own apt repository (docker-ce) Newer releases sooner. Follow the current instructions at docs.docker.com/engine/install/ubuntu.
Pick one and not both: the packages conflict. Then check that the daemon runs:
$ sudo systemctl status docker --no-pager | head -5
$ sudo systemctl enable --now docker
(Not run: the lab has no systemd, its daemon is started by hand.)
Permission denied on the socket is the first error everybody meets. The docker command talks to /var/run/docker.sock, which belongs to root and the group docker:
$ ls -l /var/run/docker.sock
srw-rw---- 1 root docker 0 Oct 18 21:24 /var/run/docker.sock
$ id -nG | tr ' ' '\n' | grep -x docker
docker
$ docker ps # run as the user nobody
permission denied while trying to connect to the docker API at unix:///var/run/docker.sock
(exit status 1)
$ sudo usermod -aG docker "$USER" # then log out and in (or run: newgrp docker)
(The third command ran as the user nobody, who is not in the group, so the socket is closed to it.)
THE GROUP docker IS ROOT. Anyone in it can read and change any file on the host. Here is the proof, run by an ordinary user (ubuntu) who has no sudo:
$ id -un
ubuntu
$ cat /etc/shadow
cat: /etc/shadow: Permission denied
(exit status 1)
$ docker run --rm -v /etc/shadow:/s:ro alpine head -c 5 /s
root:
The first read is refused, the second one, through a container, is not. So put people in the docker group only if you would give them root, and look at rootless mode (dockerd-rootless-setuptool.sh install, not run) or Podman where that is not acceptable.
Run containers
docker run creates a container from an image and starts it. The pieces you will use daily:
docker run IMAGE COMMAND run COMMAND in a new container (the image's default command if you leave it out)
-d detached: run in the background and print the container ID
--name web a name you can use instead of the ID (names are unique)
-p 8080:80 publish: host port 8080 goes to container port 80
-e KEY=value set an environment variable (--env-file FILE for many)
--rm delete the container when it exits
-it interactive with a terminal, for a shell
--restart unless-stopped start again after a crash or a reboot
A web server in the background, reachable on port 8080:
$ docker run -d --name web -p 8080:80 nginx:alpine
a2795189f6387a50f98d1e6223fa2fd99008efe873052c7a314c466ef18c12eb
$ curl -sI localhost:8080 | head -2
HTTP/1.1 200 OK
Server: nginx/1.31.6
$ docker ps --format 'table {{.Names}}\t{{.Image}}\t{{.Ports}}'
NAMES IMAGE PORTS
web nginx:alpine 0.0.0.0:8080->80/tcp, [::]:8080->80/tcp
The 0.0.0.0:8080 means every network interface, so the whole network can reach it. On a server, publish on localhost only unless you mean otherwise (and see the firewall note below):
$ docker run -d --name private -p 127.0.0.1:8081:80 nginx:alpine
6b31af4743b67a692b30948a6de9c57c46b82f0e334d2a1a16df9659f5d35ca4
$ docker port private
80/tcp -> 127.0.0.1:8081
$ docker rm -f private
private
Docker writes its own iptables rules for published ports. They are evaluated before ufw and firewalld rules, so a port you publish can be open to the network even if ufw says it is closed. Bind to 127.0.0.1 and put a reverse proxy in front, or filter in the DOCKER-USER chain (see the networking section).
Environment variables and a one-off command:
$ docker run --rm -e GREETING=hello alpine sh -c 'echo "$GREETING from $(hostname)"'
hello from 9377579f2274
$ docker run --rm --env-file /dev/null alpine env | sort | head -4
HOME=/root
HOSTNAME=4d5d2bf1babd
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
A name can only be used once. The second run fails. A published port can only be used once too, and the failed container still exists (in the Created state) until you remove it:
$ docker run -d --name web alpine sleep 60
docker: Error response from daemon: Conflict. The container name "/web" is already in use by container "a2795189f6387a50f98d1e6223fa2fd99008efe873052c7a314c466ef18c12eb". You have to remove (or rename) that container to be able to reuse that name.
Run 'docker run --help' for more information
(exit status 125)
$ docker run -d --name other -p 8080:80 nginx:alpine
82af484610bf28e0874c894c4ea00ff8418b357dd019c56a40490f3322e22a9b
docker: Error response from daemon: failed to set up container networking: driver failed programming external connectivity on endpoint other (77fb766212e19e24a3a695c4570b0a12766b3e3a8542a7772022e5227922535a): Bind for 0.0.0.0:8080 failed: port is already allocated
Run 'docker run --help' for more information
(exit status 125)
$ docker rm -f other
other
An interactive shell needs a terminal (-it). In a script or a pipe there is no terminal, so leave -t off:
$ docker run --rm -it alpine sh
$ echo 'echo shell-works; exit' | docker run --rm -i alpine sh
shell-works
(The first line needs a real terminal and was not run here.) Restart policies decide what happens when the container exits or the daemon restarts:
$ docker run -d --name always --restart unless-stopped alpine sleep 300
5e3c8b207f90c0fb570c28e88c4c9d55a42501d99647c3f17eb942aab12daf5b
$ docker inspect -f '{{.Name}} policy={{.HostConfig.RestartPolicy.Name}} restarts={{.RestartCount}}' always
/always policy=unless-stopped restarts=0
$ docker update --restart no always
always
$ docker inspect -f 'policy={{.HostConfig.RestartPolicy.Name}}' always
policy=no
$ docker rm -f always
always
Manage containers: start, stop, and why stop sometimes takes ten seconds
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a2795189f638 nginx:alpine "/docker-entrypoint.…" 5 seconds ago Up 2 seconds 0.0.0.0:8080->80/tcp, [::]:8080->80/tcp web
$ docker ps -a --format 'table {{.Names}}\t{{.Image}}\t{{.State}}'
NAMES IMAGE STATE
web nginx:alpine running
$ docker stop web
web
$ docker ps -a --format 'table {{.Names}}\t{{.State}}\t{{.Status}}'
NAMES STATE STATUS
web exited Exited (0) Less than a second ago
$ docker start web
web
$ docker restart web
web
$ docker rename web web-old
$ docker rm web-old
Error response from daemon: cannot remove container "web-old": container is running: stop the container before removing or force remove
(exit status 1)
(That last command failed: a running container cannot be removed with plain rm.)
$ docker rm -f web-old
web-old
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
docker stop sends SIGTERM, waits ten seconds, then sends SIGKILL. A program that handles SIGTERM exits at once. A program that does not, stays until the ten seconds are up. The usual culprit is that your program runs as PID 1: the kernel gives PID 1 no default signal handlers, so a signal it does not handle explicitly is ignored. Here is a plain sleep, run as PID 1, then stopped:
$ docker run -d --name slow alpine sleep 300
0fb03a63f6922dc4a04a384b02b870474c53f54a22a56a7867e7cda3f3cfb091
$ s=$(date +%s); docker stop slow >/dev/null; echo "stopped after $(( $(date +%s) - s )) seconds"
stopped after 10 seconds
$ docker inspect -f 'exit code {{.State.ExitCode}}' slow
exit code 137
$ docker rm slow
slow
Exit code 137 is 128 + 9: it was killed with SIGKILL. The fix is a tiny init process as PID 1 that forwards signals and reaps children. Docker has one built in, --init:
$ docker run -d --init --name quick alpine sleep 300
5be05536d8b9a952a64e11054e3e9140e4d4ebd1eff6d545e1881cba5258015c
$ s=$(date +%s); docker stop quick >/dev/null; echo "stopped after $(( $(date +%s) - s )) seconds"
stopped after 0 seconds
$ docker inspect -f 'exit code {{.State.ExitCode}}' quick
exit code 143
$ docker rm quick
quick
(In compose, the same thing is init: true.) Related: use the exec form of CMD in a Dockerfile (CMD ["nginx", "-g", "daemon off;"], as a JSON array). The shell form (CMD nginx) starts /bin/sh -c as PID 1, and the shell does not pass signals on to your program. More in the build section.
Other signals and states:
$ docker run -d --name pz alpine sleep 300
62ab8aee3edc7e607661bd4e1754b39efe83c397350b547666e9ff6f807b1fdb
$ docker pause pz && docker ps --format '{{.Names}} {{.State}}'
pz
pz paused
$ docker unpause pz && docker kill pz
pz
pz
$ docker rm pz
pz
$ docker run -d --name stopme alpine sleep 300
e000c13fecbc32cfbc6382855b1d3de1e4765325edc7b7d9b79f05c53311cdd2
$ docker stop -t 2 stopme
stopme
$ docker container prune -f
Deleted Containers:
e000c13fecbc32cfbc6382855b1d3de1e4765325edc7b7d9b79f05c53311cdd2
Total reclaimed space: 4.096kB
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
Inspect and debug
$ docker run -d --name web -p 8080:80 nginx:alpine
a0359e6817d583471a0e5c6a08d2ac368e0aa6d93a9249d714a6e357747b9532
$ curl -s localhost:8080 >/dev/null; curl -s localhost:8080/missing >/dev/null
$ docker logs web 2>&1 | grep -c 'GET /'
3
$ docker logs web 2>&1 | grep '" 404 ' | sed -E 's/^[0-9.]+ - - \[[^]]*\]/IP - - [DATE]/'
IP - - [DATE] "GET /missing HTTP/1.1" 404 153 "-" "curl/8.5.0" "-"
$ docker logs --since 1m web 2>&1 | grep -c 'GET /'
3
$ docker exec web nginx -v
nginx version: nginx/1.31.6
$ docker exec web ls /etc/nginx/conf.d
default.conf
$ docker exec web sh -c 'echo "I am $(id -un) with PID $$"'
I am root with PID 50
$ docker top web | head -3
UID PID PPID C STIME TTY TIME CMD
root 45754 45731 1 21:32 ? 00:00:00 nginx: master process nginx -g daemon off;
101 45813 45754 0 21:32 ? 00:00:00 nginx: worker process
$ docker stats --no-stream --format 'table {{.Name}}\t{{.PIDs}}\t{{.MemPerc}}'
NAME PIDS MEM %
web 9 0.02%
docker logs shows what the main process wrote to standard output and standard error (nginx's image links its logs to those). A program that writes only to a file inside the container shows nothing in docker logs. -f follows, --tail N limits, --since 10m and --until select a period, -t adds timestamps.
docker exec runs a second command in a running container. Add -it for a shell: docker exec -it web sh (the lab image has sh, not bash; many slim images have no shell at all, see the troubleshooting section).
inspect prints everything as JSON, and -f takes a Go template to pick out one value:
$ docker inspect -f '{{.State.Status}} since {{.State.StartedAt}}' web | cut -c1-14
running since
$ docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' web
172.18.0.2
$ docker inspect -f '{{json .Config.Cmd}} {{json .Config.ExposedPorts}}' web
["nginx","-g","daemon off;"] {"80/tcp":{}}
$ docker inspect -f '{{json .HostConfig.PortBindings}}' web
{"80/tcp":[{"HostIp":"","HostPort":"8080"}]}
$ docker inspect web | head -4
[
{
"Id": "a0359e6817d583471a0e5c6a08d2ac368e0aa6d93a9249d714a6e357747b9532",
"Created": "2026-10-18T21:32:01.156417294Z",
$ docker inspect web | grep -c .
238
What did the container change in its own file system, and how do I get a file in or out:
$ docker exec web sh -c 'echo test > /tmp/note.txt'
$ docker diff web | grep -E ' /tmp| /run'
C /tmp
A /tmp/note.txt
C /run
A /run/nginx.pid
$ docker cp web:/etc/nginx/nginx.conf /tmp/nginx.conf && head -3 /tmp/nginx.conf
user nginx;
worker_processes auto;
$ echo 'copied in' > /tmp/in.txt && docker cp /tmp/in.txt web:/tmp/in.txt && docker exec web cat /tmp/in.txt
copied in
$ docker events --since 5m --until 0s --filter container=web --format '{{.Action}}' | head -4
create
start
kill
stop
In docker diff, A is added, C is changed and D is deleted. Everything the container writes goes into its own thin layer and is lost when the container is removed, unless it is in a volume. docker cp works on stopped containers too, which is how you get a file out of a container that has already crashed.
$ docker rm -f web
web
Images: tags, layers, saving and loading
An image is a stack of read-only layers plus some settings (the default command, environment, exposed ports). The name has up to four parts: registry/repository:tag, for example docker.io/library/nginx:1.27-alpine. Leave out the registry and Docker Hub is assumed; leave out the tag and :latest is assumed.
The tag is only a label that points at one image at a time, and it moves: nginx:latest today is not nginx:latest next month. Pin a version tag (nginx:1.27-alpine) for anything that must be reproducible, or a digest (nginx@sha256:...) for exactly that image.
$ docker images --format 'table {{.Repository}}\t{{.Tag}}\t{{.ID}}\t{{.Size}}'
REPOSITORY TAG IMAGE ID SIZE
nginx alpine df221db836e1 94.4MB
alpine latest 294b683cb724 13MB
$ docker image inspect alpine -f '{{.Os}}/{{.Architecture}}, {{len .RootFS.Layers}} layer'
linux/amd64, 1 layer
$ docker image inspect nginx:alpine -f '{{.Os}}/{{.Architecture}}, {{len .RootFS.Layers}} layers'
linux/amd64, 8 layers
$ docker history --format '{{.CreatedBy}}' alpine
CMD ["/bin/sh"]
ADD alpine-minirootfs-3.24.2-x86_64.tar.gz /…
$ docker history --format '{{.Size}}\t{{.CreatedBy}}' nginx:alpine | head -4 | cut -c1-90
52.4MB RUN /bin/sh -c set -x && apkArch="$(cat …
0B ENV ACME_VERSION=0.4.1
0B ENV NJS_RELEASE=1
0B ENV NJS_VERSION=1.0.1
Giving an image a second name does not copy anything. It is a second tag on the same image ID (this is how you prepare a push to your own registry):
$ docker tag alpine:latest myapp:1.0
$ docker tag alpine:latest registry.example.com/team/myapp:1.0
$ docker images --format 'table {{.Repository}}\t{{.Tag}}\t{{.ID}}' | grep -E 'REPOSITORY|alpine|myapp'
REPOSITORY TAG IMAGE ID
nginx alpine df221db836e1
alpine latest 294b683cb724
myapp 1.0 294b683cb724
registry.example.com/team/myapp 1.0 294b683cb724
$ docker rmi registry.example.com/team/myapp:1.0 myapp:1.0
Untagged: registry.example.com/team/myapp:1.0
Untagged: myapp:1.0
rmi removed the two extra names, and the image itself stays because alpine:latest still points at it ("Untagged" in the output, not "Deleted").
Moving an image to a machine without a registry or without internet: save it to a tar file and load it on the other side.
$ docker save -o /tmp/alpine.tar alpine:latest && ls -l /tmp/alpine.tar | awk '{print $5, "bytes", $9}'
3958784 bytes /tmp/alpine.tar
$ docker rmi alpine:latest
Untagged: alpine:latest
Deleted: sha256:294b683cb724975bec92580e1e685676bd4b50bda910ddb8c51d4cabeaec77e6
$ docker load -i /tmp/alpine.tar
Loaded image: alpine:latest
$ docker run --rm alpine echo "still works"
still works
(docker save | gzip, and docker load, work through a pipe too: docker save app | ssh host docker load.)
The registry commands need a registry, so the lab did not run them. This is the full cycle:
$ docker pull nginx:1.27-alpine # download that tag (docker pull NAME alone means :latest)
$ docker search redis --limit 5 # search Docker Hub (official images have no slash in the name)
$ docker login registry.example.com # asks for a user and a token; stored in ~/.docker/config.json
$ docker push registry.example.com/team/myapp:1.0 # upload
$ docker logout registry.example.com
~/.docker/config.json can hold the login in plain text base64, which is not encryption. Use a credential helper (docker-credential-pass or docker-credential-secretservice) on a shared machine, and use a read-only access token, never your account password, in a script or CI job.
Docker Hub limits anonymous pulls per hour. A server that pulls many images should log in, or pull through a mirror (a registry:2 container with REGISTRY_PROXY_REMOTEURL=https://registry-1.docker.io).
Build your own image: Dockerfile and the layer cache
A Dockerfile is a list of instructions. Each RUN, COPY and ADD creates a layer, and docker build keeps a cache of every layer. The rule that follows from that: put what changes least at the top and what changes most at the bottom, so a small edit rebuilds only the last layers.
The lab only has alpine:latest, so the examples build on it. In a real file pin a version (FROM alpine:3.21). The program is a two-line script:
$ cat app.sh
#!/bin/sh
echo "app $VERSION as $(id -un) on $(hostname)"
$ cat Dockerfile
FROM alpine:latest
ARG VERSION=1.0
ENV VERSION=$VERSION
WORKDIR /app
COPY app.sh .
RUN chmod +x app.sh
USER nobody
CMD ["./app.sh"]
$ docker build --progress=plain -t myapp:1.0 . 2>&1 | grep -E '\[[0-9]/[0-9]\]|naming' | cut -c1-70
#4 [1/4] FROM docker.io/library/alpine:latest@sha256:294b683cb724975be
#6 [2/4] WORKDIR /app
#7 [3/4] COPY app.sh .
#8 [4/4] RUN chmod +x app.sh
#9 naming to docker.io/library/myapp:1.0 done
$ docker run --rm myapp:1.0
app 1.0 as nobody on 5df286462924
Build it again with nothing changed: every step is taken from the cache.
$ docker build --progress=plain -t myapp:1.0 . 2>&1 | grep -E 'CACHED' | cut -c1-70
#6 CACHED
#7 CACHED
#8 CACHED
Now change the script. Docker compares the files that COPY reads, sees the difference, and rebuilds that step and every step after it. The steps above it stay cached:
$ echo 'echo "second line"' >> app.sh
$ docker build --progress=plain -t myapp:1.1 . 2>&1 | grep -E '^#[0-9]+ (\[[0-9]/[0-9]\]|CACHED)' | cut -c1-70
#4 [1/4] FROM docker.io/library/alpine:latest@sha256:294b683cb724975be
#6 [2/4] WORKDIR /app
#6 CACHED
#7 [3/4] COPY app.sh .
#8 [4/4] RUN chmod +x app.sh
$ docker run --rm myapp:1.1
app 1.0 as nobody on fab14781e2e3
second line
A build argument changes the value for one build, and an environment variable at run time changes it for one container:
$ docker build -q --build-arg VERSION=2.0 -t myapp:2.0 . >/dev/null && docker run --rm myapp:2.0
app 2.0 as nobody on 1dd2b87d32cb
second line
$ docker run --rm -e VERSION=9 myapp:1.0
app 9 as nobody on 4b260bb4f4f5
$ docker run --rm --user root myapp:1.0 id -un
root
$ docker run --rm myapp:1.0 id
uid=65534(nobody) gid=65534(nobody) groups=65534(nobody)
The last one replaced the CMD with id. The first one overrode the USER. Everything given after the image name replaces CMD:
ENTRYPOINT ["cmd"] the fixed program. Arguments after the image name are added to it.
CMD ["arg"] the default command, or the default arguments for ENTRYPOINT. Replaced by anything after the image name.
$ cat Dockerfile.entry
FROM alpine:latest
ENTRYPOINT ["echo", "entry:"]
CMD ["default args"]
$ docker build -q -f Dockerfile.entry -t entrydemo . >/dev/null
$ docker run --rm entrydemo
entry: default args
$ docker run --rm entrydemo other args
entry: other args
$ docker run --rm --entrypoint sh entrydemo -c 'echo replaced the entrypoint'
replaced the entrypoint
EXEC FORM VERSUS SHELL FORM. CMD ["sleep", "300"] (a JSON array) runs sleep directly as PID 1. CMD sleep 300 (no brackets) runs /bin/sh -c "sleep 300", and the shell is PID 1. A shell does not forward signals, so docker stop waits ten seconds and kills the container. Compare:
$ cat Dockerfile.shellform
FROM alpine:latest
CMD sleep 300 && echo done
$ docker build -q -f Dockerfile.shellform -t shellform . >/dev/null
$ docker run -d --name sf shellform
b0635e3294146a215745011de18af534373651d32d65b700b6c71fda95f405e8
$ docker exec sf ps -o pid,args
PID COMMAND
1 /bin/sh -c sleep 300 && echo done
7 sleep 300
8 ps -o pid,args
$ docker rm -f sf
sf
(The && makes the shell stay as PID 1 here, and sleep is its child. Without it, some shells replace themselves with the last command, which hides the problem on some images and not on others, so use the exec form.)
THE BUILD CONTEXT. docker build . sends the folder to the daemon, and a COPY . /src copies all of it into the image, including things that must never be there. A .dockerignore file, with the same syntax as .gitignore, leaves them out. Here a folder has a big file and a .env file with a password:
$ cat Dockerfile.ctx
FROM alpine:latest
COPY . /src
CMD ["ls", "-A", "/src"]
$ head -c 20000000 /dev/zero > big.bin; echo 'DB_PASSWORD=hunter2' > .env
$ docker build -q -f Dockerfile.ctx -t ctx . >/dev/null
$ docker run --rm ctx
.dockerignore
Dockerfile
Dockerfile.ctx
Dockerfile.dang
Dockerfile.entry
Dockerfile.leak
Dockerfile.multi
Dockerfile.secret
Dockerfile.shellform
Dockerfile.single
Dockerfile.tmp
app.sh
compose.yaml
data.tgz
out
site
tok.txt
$ docker images ctx --format '{{.Size}}'
13MB
$ cat .dockerignore
big.bin
.git
*.log
.env
$ docker build -q -f Dockerfile.ctx -t ctx . >/dev/null
$ docker run --rm ctx
.dockerignore
Dockerfile
Dockerfile.ctx
Dockerfile.dang
Dockerfile.entry
Dockerfile.leak
Dockerfile.multi
Dockerfile.secret
Dockerfile.shellform
Dockerfile.single
Dockerfile.tmp
app.sh
compose.yaml
data.tgz
out
site
tok.txt
$ docker images ctx --format '{{.Size}}'
13MB
$ rm big.bin .env
The first image carries the 20 MB file and the password, and anyone who can pull the image can read both. Write the .dockerignore before the first COPY . and keep .git, .env, key files, logs and build output in it.
SECRETS DO NOT BELONG IN A LAYER. Anything given with ARG or ENV, or copied and then deleted in a later step, stays in the image history, and docker history shows it:
$ cat Dockerfile.leak
FROM alpine:latest
ARG TOKEN
ENV API_TOKEN=$TOKEN
$ docker build -q -f Dockerfile.leak --build-arg TOKEN=s3cret-token -t leaky . >/dev/null
$ docker history leaky --no-trunc --format '{{.CreatedBy}}' | grep -c 's3cret-token'
2
The fix is a BuildKit secret mount: the file is available for one RUN step and never written into a layer.
$ cat Dockerfile.secret
FROM alpine:latest
RUN --mount=type=secret,id=tok wc -c < /run/secrets/tok > /size.txt
CMD ["cat", "/size.txt"]
$ printf 's3cret-token' > tok.txt
$ docker build -q -f Dockerfile.secret --secret id=tok,src=tok.txt -t safe . >/dev/null
$ docker run --rm safe
12
$ docker history safe --no-trunc --format '{{.CreatedBy}}' | grep -c 's3cret-token' || true
0
(The image can use the secret while it builds, here only to count its bytes, and the token itself is in no layer. Runtime secrets are a different matter: pass them at run time, with a file, an environment variable from your secret store, or a compose secret.)
Other instructions you will meet: LABEL (metadata), EXPOSE (documentation only, it does not publish anything), VOLUME (marks a path that gets an anonymous volume), HEALTHCHECK (a command that decides healthy or not), STOPSIGNAL, SHELL. buildx build --platform linux/amd64,linux/arm64 builds for several CPU types (it needs a builder that supports it, and was not run in the lab).
Multi-stage builds: build with the tools, ship without them
A compiler, a package cache and test files are needed to build a program and not to run it. A multi-stage Dockerfile builds in one stage and copies only the result into a clean final stage. Real builds use a compiler image such as golang or node for the first stage. The lab has no registry, so the build stage here makes a large temporary file to stand for build tools, and the artifact is a small text file:
$ cat Dockerfile.single
FROM alpine:latest
RUN head -c 30000000 /dev/zero > /build-tools.tmp && echo "the program" > /app.txt
CMD ["cat", "/app.txt"]
$ cat Dockerfile.multi
FROM alpine:latest AS build
RUN head -c 30000000 /dev/zero > /build-tools.tmp && echo "the program" > /app.txt
FROM alpine:latest
COPY --from=build /app.txt /app.txt
CMD ["cat", "/app.txt"]
$ docker build -q -f Dockerfile.single -t single . >/dev/null
$ docker build -q -f Dockerfile.multi -t multi . >/dev/null
$ docker run --rm single
the program
$ docker run --rm multi
the program
$ docker images --format 'table {{.Repository}}\t{{.Size}}' | grep -E 'REPOSITORY|single|multi'
REPOSITORY SIZE
single 43MB
multi 12.9MB
Same output, and the multi-stage image is about 30 MB smaller because the tools stayed in the first stage. The first stage is thrown away (it is only a build cache). FROM ... AS build names a stage, COPY --from=build takes files from it, and docker build --target build stops at a stage, which is useful for a test or debug image.
The real-world shape of it (not run, needs the golang image from a registry):
FROM golang:1.23 AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download # cached until the dependencies change
COPY . .
RUN CGO_ENABLED=0 go build -o /app
FROM gcr.io/distroless/static # no shell, no package manager
COPY --from=build /app /app
USER 65532:65532
ENTRYPOINT ["/app"]
Small final images are faster to pull and have fewer packages to patch. Distroless and scratch images have no shell at all, so docker exec -it sh will not work on them: see the troubleshooting section for how to look inside.
Volumes, bind mounts and tmpfs
A container's own layer is deleted with the container. Data that must survive goes in one of three places:
named volume -v data:/data Docker creates and manages it. The right choice for a database.
bind mount -v /host/dir:/data A folder from the host. Good for configuration and for development.
tmpfs --tmpfs /scratch Memory only, gone when the container stops. For temporary or secret data.
$ docker volume create data
data
$ docker run --rm -v data:/data alpine sh -c 'echo saved > /data/note.txt'
$ docker run --rm -v data:/data alpine cat /data/note.txt
saved
$ docker volume ls
DRIVER VOLUME NAME
local data
$ docker volume inspect data -f '{{.Mountpoint}}'
/var/lib/docker/volumes/data/_data
The first container wrote the file and was deleted, and the second one still found it. On the host the data is under /var/lib/docker/volumes, readable by root only. A named volume that is empty gets the contents of the image's folder copied in on first use (a bind mount does not: it hides what the image had there).
BIND MOUNTS show a host folder inside the container. Files you change on the host show up at once, which is how a development setup works. Add :ro to make it read-only for the container:
$ mkdir -p site && echo '<h1>version one</h1>' > site/index.html
$ docker run -d --name web -p 8080:80 -v "$PWD/site":/usr/share/nginx/html:ro nginx:alpine
bda89e3f5c9d776f006b443e5cddc01ddfc2568de24596fd441b7c213b921dbc
$ curl -s localhost:8080
<h1>version one</h1>
$ echo '<h1>version two</h1>' > site/index.html && curl -s localhost:8080
<h1>version two</h1>
$ docker exec web sh -c 'echo x > /usr/share/nginx/html/new.html'
sh: can't create /usr/share/nginx/html/new.html: Read-only file system
(exit status 1)
$ docker rm -f web
web
Use the absolute path form, "$PWD/site" or the full path. A relative name such as -v site:/x would mean a named volume called site, which is a very common mistake.
THE OWNER OF FILES. A container's user ids are the host's user ids, with different names. The default container user is root (uid 0), so files it creates in a bind mount belong to root on the host:
$ mkdir out
mkdir: cannot create directory 'out': File exists
(exit status 1)
$ docker run --rm -v "$PWD/out":/out alpine sh -c 'echo made-by-container > /out/file'
$ ls -ln out
total 8
-rw-r--r-- 1 1000 1000 18 Oct 18 21:32 file
-rw-r--r-- 1 1000 1000 16 Oct 18 21:30 mine
$ echo more >> out/file
$ docker run --rm --user "$(id -u):$(id -g)" -v "$PWD/out":/out alpine sh -c 'echo made-as-me > /out/mine'
$ ls -ln out
total 8
-rw-r--r-- 1 1000 1000 23 Oct 18 21:32 file
-rw-r--r-- 1 1000 1000 11 Oct 18 21:32 mine
$ echo more >> out/mine && cat out/mine
made-as-me
more
The first file is owned by uid 0, so the ordinary user cannot edit it. Running with --user "$(id -u):$(id -g)" (or USER in the Dockerfile) makes the files yours. A named volume avoids the question, and compose has user: for it too.
BACK UP AND RESTORE A VOLUME with a throwaway container that mounts the volume and the current folder:
$ docker run --rm -v data:/data:ro -v "$PWD":/backup alpine tar czf /backup/data.tgz -C /data .
$ tar tzf data.tgz
./
./note.txt
$ docker volume create restored >/dev/null && docker run --rm -v restored:/data -v "$PWD":/backup alpine tar xzf /backup/data.tgz -C /data
$ docker run --rm -v restored:/data alpine cat /data/note.txt
saved
For a database do not copy the files of a running database: dump it (pg_dump, mysqldump) or stop the container first.
tmpfs is memory, with an optional size:
$ docker run --rm --tmpfs /scratch:size=1m alpine sh -c 'df -h /scratch | tail -1 | tr -s " " | cut -d" " -f1-3'
tmpfs 1.0M 0
$ docker run --rm --tmpfs /scratch:size=1m alpine dd if=/dev/zero of=/scratch/big bs=1M count=2
dd: error writing '/scratch/big': No space left on device
2+0 records in
1+0 records out
1048576 bytes (1.0MB) copied, 0.001685 seconds, 593.5MB/s
(exit status 1)
ANONYMOUS VOLUMES pile up. An image with VOLUME in its Dockerfile, or a -v /path with no name, creates a volume with a random name for every container, and docker rm does not remove it unless you add -v:
$ docker volume ls -q | wc -l
2
$ docker run -d --name anon -v /var/lib/thing alpine sleep 60 >/dev/null; docker volume ls -q | wc -l
3
$ docker rm -f anon >/dev/null; docker volume ls -q | wc -l
3
$ docker run -d --name anon2 -v /var/lib/thing alpine sleep 60 >/dev/null; docker rm -fv anon2 >/dev/null; docker volume ls -q | wc -l
3
Two volumes before, three with the first container running, and still three after it was removed: the anonymous volume was left behind. With rm -fv the second container's volume went with it, so the count stayed at three. docker volume prune removes unused anonymous volumes (and with -a also unused named ones, so be careful, see the cleanup section):
$ docker volume rm restored
restored
$ docker volume prune -f | head -1
Deleted Volumes:
$ docker volume ls
DRIVER VOLUME NAME
local data
Networks: how containers find each other
Docker creates a bridge network and gives each container an address on it, and the host forwards published ports to it. Three kinds matter in practice:
bridge (the default) containers can reach each other by IP address only.
user-defined bridge the same, plus DNS: a container can reach another by its name. Use this.
host the container uses the host's network directly. No isolation, no -p needed.
none only a loopback interface.
$ docker network ls --format 'table {{.Name}}\t{{.Driver}}\t{{.Scope}}'
NAME DRIVER SCOPE
bridge bridge local
host host local
none null local
On the DEFAULT bridge, names do not resolve:
$ docker run -d --name a1 alpine sleep 300 >/dev/null
$ docker run -d --name a2 alpine sleep 300 >/dev/null
$ docker exec a1 ping -c1 a2
ping: bad address 'a2'
(exit status 1)
$ docker exec a1 ping -c1 $(docker inspect a2 -f '{{.NetworkSettings.Networks.bridge.IPAddress}}') | head -2
PING 172.18.0.3 (172.18.0.3): 56 data bytes
64 bytes from 172.18.0.3: seq=0 ttl=64 time=0.086 ms
(In Docker 29 the old shortcut {{.NetworkSettings.IPAddress}} is gone: it fails with "map has no entry for key". Name the network, as above, or use the range form from the inspect section.)
By IP it works, but the address changes whenever a container is recreated, so nobody should depend on it. On a USER-DEFINED network, the name works:
$ docker rm -f a1 a2 >/dev/null
$ docker network create appnet
ee90eaf3cd1b56d07e9d9ef33233f9451b740236c2a38d623989f6213c5ddf3f
$ docker run -d --name web --network appnet nginx:alpine >/dev/null
$ docker run -d --name client --network appnet alpine sleep 300 >/dev/null
$ docker exec client sh -c 'ping -c1 web | head -1'
PING web (172.19.0.2): 56 data bytes
$ docker exec client wget -qS -O /dev/null http://web 2>&1 | head -1
HTTP/1.1 200 OK
$ docker network inspect appnet -f '{{range .Containers}}{{.Name}} {{.IPv4Address}}{{"\n"}}{{end}}' | sort | sed '/^$/d'
client 172.19.0.3/16
web 172.19.0.2/16
The client reached the web server on its inside port 80 without any -p. Publishing (-p) is only for traffic that comes from OUTSIDE the host. Two containers on the same network can use any port on each other.
A container can be on several networks, and networks can be joined and left while it runs. That is how you give a front end a network to the database that the outside world cannot see:
$ docker network create backend >/dev/null
$ docker network connect backend client
$ docker inspect client -f '{{range $k, $v := .NetworkSettings.Networks}}{{$k}} {{end}}'
appnet backend
$ docker network disconnect backend client && docker network rm backend
backend
The other modes:
$ docker run --rm --network none alpine ip -o -4 addr | cut -d' ' -f2
lo
$ docker run --rm --network host alpine sh -c "ip -o -4 addr | cut -d' ' -f2 | sort -u | tr '\n' ' '"; echo
br-ee90eaf3cd1b docker0 eth0 lo
$ docker rm -f web client >/dev/null; docker network rm appnet
appnet
(With host networking the container sees the host's interfaces, so the names are the host's (docker0 is the default bridge). Linux only, and a published -p is ignored.)
TWO THINGS THAT BITE ON SERVERS (not run, they change the firewall and the daemon).
1. Published ports go around ufw and firewalld, because Docker inserts its own rules ahead of them. Bind to 127.0.0.1, or filter in the DOCKER-USER chain, which Docker leaves alone and evaluates first:
$ sudo iptables -I DOCKER-USER -i eth0 ! -s 192.168.1.0/24 -j DROP # only the LAN may reach published ports
2. The default bridge uses 172.17.0.0/16 and new networks take the next ranges. If your office or VPN uses one of those, the routes clash. Set the pools in /etc/docker/daemon.json and restart the daemon:
{ "default-address-pools": [ { "base": "10.200.0.0/16", "size": 24 } ] }
Docker Compose: several containers from one file
Compose describes a set of containers, networks and volumes in one YAML file and starts them together. On a current system the command is docker compose (a plugin; Ubuntu's package is docker-compose-v2). The old docker-compose with a hyphen is the retired Python version 1, and the file format is the same for both.
A web server with a health check, and a worker that must wait until the web server is healthy. The web folder is the site/ from the volumes section:
$ cat compose.yaml
name: demo
services:
web:
image: nginx:alpine
ports:
- "127.0.0.1:${WEB_PORT:-8080}:80"
volumes:
- ./site:/usr/share/nginx/html:ro
healthcheck:
test: ["CMD", "wget", "-q", "-O", "/dev/null", "http://localhost/"]
interval: 2s
timeout: 2s
retries: 5
restart: unless-stopped
worker:
image: alpine:latest
command: ["sh", "-c", "while true; do wget -q -O /dev/null http://web/ && echo ok; sleep 2; done"]
environment:
MODE: demo
volumes:
- state:/state
depends_on:
web:
condition: service_healthy
volumes:
state:
$ docker compose config --quiet && echo "the file is valid"
the file is valid
$ docker compose up -d
Network demo_default Creating
Network demo_default Created
Volume demo_state Creating
Volume demo_state Created
Container demo-web-1 Creating
Container demo-web-1 Created
Container demo-worker-1 Creating
Container demo-worker-1 Created
Container demo-web-1 Starting
Container demo-web-1 Started
Container demo-web-1 Waiting
Container demo-web-1 Healthy
Container demo-worker-1 Starting
Container demo-worker-1 Started
$ docker compose ps --format 'table {{.Service}}\t{{.State}}\t{{.Ports}}'
SERVICE STATE PORTS
web running 127.0.0.1:8080->80/tcp
worker running
$ docker inspect demo-web-1 -f 'health: {{.State.Health.Status}}'
health: healthy
$ docker compose logs worker | grep -c ok
2
$ docker compose exec -T web nginx -v
nginx version: nginx/1.31.6
$ curl -s localhost:8080
<h1>version two</h1>
Reading it from the top. name: sets the project name, which prefixes everything Compose creates (demo-web-1, the network demo_default, the volume demo_state). Without it the folder name is used. The two services share a network that Compose creates, so worker reached web as http://web/. ports: publishes only on localhost. The health check runs inside the container; depends_on with condition: service_healthy makes the worker wait for it (plain depends_on only waits for the container to START, not for the program to be ready). exec -T switches off the terminal, which a script needs; without a terminal Compose would otherwise complain.
The value ${WEB_PORT:-8080} is filled from the environment or from a .env file in the same folder, with 8080 as the default:
$ docker compose config | grep -A1 'published'
published: "8080"
protocol: tcp
$ WEB_PORT=9090 docker compose config | grep 'published'
published: "9090"
$ docker compose config --services
web
worker
$ docker compose config | head -6
name: demo
services:
web:
healthcheck:
test:
- CMD
docker compose config prints the file as Compose understands it, after filling in variables and merging override files. It is the first thing to run when a file does not behave as you expect.
Change one thing and run up -d again: only the service that changed is recreated, and the rest are left running:
$ sed -i 's/MODE: demo/MODE: changed/' compose.yaml
$ docker compose up -d
Container demo-web-1 Running
Container demo-worker-1 Recreate
Container demo-worker-1 Recreated
Container demo-web-1 Waiting
Container demo-web-1 Healthy
Container demo-worker-1 Starting
Container demo-worker-1 Started
$ docker compose up -d --scale worker=2
Container demo-web-1 Running
Container demo-worker-1 Running
Container demo-worker-2 Creating
Container demo-worker-2 Created
Container demo-web-1 Waiting
Container demo-web-1 Healthy
Container demo-worker-2 Starting
Container demo-worker-2 Started
$ docker compose ps --format 'table {{.Name}}\t{{.State}}'
NAME STATE
demo-web-1 running
demo-worker-1 running
demo-worker-2 running
Stopping and removing, and what happens to the data:
$ docker compose down
Container demo-worker-1 Stopping
Container demo-worker-2 Stopping
Container demo-worker-1 Stopped
Container demo-worker-1 Removing
Container demo-worker-2 Stopped
Container demo-worker-2 Removing
Container demo-worker-1 Removed
Container demo-worker-2 Removed
Container demo-web-1 Stopping
Container demo-web-1 Stopped
Container demo-web-1 Removing
Container demo-web-1 Removed
Network demo_default Removing
Network demo_default Removed
$ docker volume ls --format '{{.Name}}' | grep demo
demo_state
$ docker compose down -v 2>&1 | tail -1
Volume demo_state Removed
$ docker volume ls --format '{{.Name}}' | grep -c demo || true
0
down removes the containers and the network but keeps the named volumes. down -v removes them too, and with them the data. (The -v said "Volume demo_state Removed", and then nothing called demo is left.) docker compose stop and start keep the containers.
Things you will want in a real file, none of which the lab needed to run: env_file: (a file of variables), user: "1000:1000", init: true (a PID 1 that forwards signals), read_only: true, cap_drop: [ALL], mem_limit: and cpus:, profiles: (services that start only when asked for, docker compose --profile debug up), and compose.override.yaml (merged automatically over compose.yaml, the place for developer-only settings). docker compose pull fetches newer images for the tags in the file, and docker compose up -d --pull always forces it. Neither was run, since the lab has no registry.
Limits: memory, CPU and processes
Without limits one container can use all the memory and CPU of the host. Limits are enforced by the kernel (cgroups), so they hold whatever the program does.
$ docker run --rm --memory 64m --cpus 0.5 --pids-limit 50 alpine sh -c 'echo "memory.max=$(cat /sys/fs/cgroup/memory.max) cpu.max=$(cat /sys/fs/cgroup/cpu.max) pids.max=$(cat /sys/fs/cgroup/pids.max)"'
memory.max=67108864 cpu.max=50000 100000 pids.max=50
A container that goes over its memory limit is killed by the kernel (the OOM killer), and Docker records it. This one tries to hold 50 MB in a 20 MB limit:
$ docker run --name hog --memory 20m --memory-swap 20m alpine sh -c 'x=$(head -c 50000000 /dev/zero | tr "\0" a); echo "allocated ${#x} bytes"'
Killed
(exit status 137)
$ docker inspect hog -f 'exit={{.State.ExitCode}} oom_killed={{.State.OOMKilled}}'
exit=137 oom_killed=true
$ docker rm hog
hog
Exit code 137 and OOMKilled=true: the process was killed, and it printed nothing, because it never got the chance. If a container disappears with 137 and no error in its logs, check this first. --memory-swap equal to --memory means no swap, so the limit is firm.
$ docker run -d --name limited --memory 100m alpine sleep 300
cf187ada4f7d18ffbcfcff31064983f141c4579d20b143a6cc14b5bb3f03b76d
$ docker inspect limited -f 'memory limit: {{.HostConfig.Memory}} bytes'
memory limit: 104857600 bytes
$ docker update --memory 200m --memory-swap 200m limited
limited
$ docker inspect limited -f 'memory limit: {{.HostConfig.Memory}} bytes'
memory limit: 209715200 bytes
$ docker stats --no-stream --format 'table {{.Name}}\t{{.MemUsage}}\t{{.MemPerc}}'
NAME MEM USAGE / LIMIT MEM %
limited 768KiB / 200MiB 0.38%
$ docker rm -f limited
limited
--cpus 0.5 means half of one CPU, as a share of time (cpu.max above shows 50000 of every 100000 microseconds). --cpu-shares only sets a priority for when the CPU is busy. Memory shown inside the container by tools such as free and top is the HOST's memory, not the limit, because they read /proc/meminfo: read the cgroup files, as above, for the truth. A program that sizes itself from the memory it sees (Java before version 10, many others) can be fooled, so set its heap explicitly.
In compose the same limits are mem_limit, cpus and pids_limit on the service.
Logs, and log files that fill the disk
Docker keeps what a container prints to standard output and error. The default driver, json-file, writes it under /var/lib/docker/containers and NEVER rotates it. A chatty container, left alone, fills the disk.
$ docker info -f 'logging driver: {{.LoggingDriver}}'
logging driver: json-file
$ docker run -d --name quiet alpine sleep 300
a42ad4c19ee1ffcf9c985b1222c74692e8683399c91455cc26beec678c28b18b
$ docker inspect quiet -f '{{.HostConfig.LogConfig.Type}} {{.HostConfig.LogConfig.Config}}'
json-file map[]
$ docker run -d --name capped --log-opt max-size=10m --log-opt max-file=3 alpine sleep 300
a713eec11a3d956da943b3def8efb6f4887a374daf4c67d689ab721862aec4b6
$ docker inspect capped -f '{{.HostConfig.LogConfig.Type}} {{.HostConfig.LogConfig.Config}}'
json-file map[max-file:3 max-size:10m]
$ docker rm -f quiet capped
quiet
capped
The first container has no limit. The second keeps at most three files of 10 MB. The setting for the whole host goes in /etc/docker/daemon.json and applies to containers created AFTER the daemon restarts (existing ones keep their old settings and have to be recreated):
{
"log-driver": "json-file",
"log-opts": { "max-size": "10m", "max-file": "3" }
}
(Not run: it needs systemctl restart docker.) In compose, put it on the service:
logging:
driver: json-file
options: { max-size: "10m", max-file: "3" }
What reads the logs: docker logs (works with json-file and journald), the journald driver (log-driver: journald, then journalctl CONTAINER_NAME=web), and drivers that ship them elsewhere (syslog, gelf, fluentd, loki through its plugin). With a driver that does not keep local files, docker logs may show nothing, on purpose.
Find out where the disk space went:
$ docker system df --format 'table {{.Type}}\t{{.TotalCount}}\t{{.Size}}'
TYPE TOTAL SIZE
Images 12 155.1MB
Containers 0 0B
Local Volumes 1 6B
Build Cache 14 34.21MB
Container logs are not counted under the container's own size, so for the log size look at the files themselves, as root: sudo du -sh /var/lib/docker/containers/*/*-json.log (not run in the lab, where the ubuntu user is not root).
Security basics
A container is a process with a restricted view of the system. It is not a virtual machine, and the kernel is shared, so the defaults are not a security boundary against a determined attacker. These settings reduce what a compromised container can do. All of them ran in the lab.
1. DO NOT RUN AS ROOT. The default user is root (uid 0) inside the container. Set USER in the Dockerfile or --user at run time:
$ docker run --rm alpine id -u
0
$ docker run --rm --user 1000:1000 alpine id
uid=1000 gid=1000 groups=1000
2. READ-ONLY ROOT FILE SYSTEM, with a tmpfs for the places that need writing:
$ docker run --rm --read-only alpine touch /x
touch: /x: Read-only file system
(exit status 1)
$ docker run --rm --read-only --tmpfs /tmp alpine sh -c 'touch /tmp/x && echo "wrote to /tmp"'
wrote to /tmp
3. DROP CAPABILITIES. Root inside a container still has a default set of Linux capabilities (chown, setuid, net_raw and others). Take them all away and add back only what is needed:
$ docker run --rm alpine sh -c 'grep CapEff /proc/self/status'
CapEff: 00000000a80425fb
$ docker run --rm --cap-drop ALL alpine sh -c 'grep CapEff /proc/self/status'
CapEff: 0000000000000000
$ docker run --rm --cap-drop ALL alpine chown 1:1 /etc/hostname
chown: /etc/hostname: Operation not permitted
(exit status 1)
$ docker run --rm --cap-drop ALL --cap-add CHOWN alpine chown 1:1 /etc/hostname && echo "allowed with CHOWN"
allowed with CHOWN
4. NO NEW PRIVILEGES stops setuid programs inside from gaining rights:
$ docker run --rm alpine grep NoNewPrivs /proc/self/status
NoNewPrivs: 0
$ docker run --rm --security-opt no-new-privileges alpine grep NoNewPrivs /proc/self/status
NoNewPrivs: 1
5. NEVER --privileged unless you know why. It gives the container nearly all the host's devices and capabilities:
$ docker run --rm alpine sh -c 'ls /dev | wc -l'
15
$ docker run --rm --privileged alpine sh -c 'ls /dev | wc -l'
239
6. NEVER MOUNT /var/run/docker.sock into a container (-v /var/run/docker.sock:/var/run/docker.sock) unless that container is allowed to be root on the host: anything that can talk to the daemon can start a container that mounts the whole host, as shown in the first sections. Many "monitoring" and "CI" examples do it. Prefer a socket proxy that allows only read calls, or rootless mode.
7. KEEP SECRETS OUT OF IMAGES AND OUT OF ENVIRONMENT VARIABLES WHERE YOU CAN. docker inspect shows every environment variable to anyone in the docker group. Use files (a bind mount with :ro, or Compose secrets, which appear under /run/secrets) and the build-time secret mount from the build section.
8. KEEP IMAGES SMALL AND CURRENT. Pin versions, rebuild regularly to pick up patched base images, and scan them (docker scout cves IMAGE, or trivy image IMAGE). Neither tool was run in the lab.
9. USE THE SETTINGS TOGETHER. This is the shape of a locked-down container, and it works for a lot of programs once their writable folders are listed:
$ docker run --rm --user 1000:1000 --read-only --tmpfs /tmp --cap-drop ALL --security-opt no-new-privileges alpine sh -c 'id -u; touch /tmp/ok && echo "tmp is writable"; touch /nope 2>&1 | head -1'
1000
tmp is writable
touch: /nope: Read-only file system
Rootless Docker (the daemon runs as a normal user, set up with dockerd-rootless-setuptool.sh install) and user-namespace remapping (userns-remap in daemon.json) go further, so that root in a container is not root on the host. Both change how volumes and ports behave and were not run.
Cleanup and disk space
Docker never deletes anything by itself. Stopped containers, old images, unused networks and volumes all stay. This is what is there now:
$ docker system df --format 'table {{.Type}}\t{{.TotalCount}}\t{{.Size}}'
TYPE TOTAL SIZE
Images 12 155.1MB
Containers 0 0B
Local Volumes 1 6B
Build Cache 14 34.21MB
$ docker images --format '{{.Repository}}:{{.Tag}} {{.Size}}' | sort | head -20
alpine:latest 13MB
ctx:latest 13MB
entrydemo:latest 12.9MB
leaky:latest 12.9MB
multi:latest 12.9MB
myapp:1.0 13MB
myapp:1.1 13MB
myapp:2.0 13MB
nginx:alpine 94.4MB
safe:latest 12.9MB
shellform:latest 12.9MB
single:latest 43MB
Older versions of Docker left a nameless image (shown as <none>, "dangling") behind every time you rebuilt a tag. In this lab, Docker 29 with its containerd image store did not: building the same tag twice left no <none> image, so docker image prune had nothing to remove here, and you will still meet dangling images on older engines. What does collect is images nobody uses any more. Prune can be limited with a filter, here to images carrying a label, so that only the scratch image goes:
$ cat Dockerfile.tmp
FROM alpine:latest
LABEL purpose=scratch
RUN date +%N > /built-at.txt
$ docker build -q -f Dockerfile.tmp -t scratchimg . >/dev/null
$ docker images --format '{{.Repository}}:{{.Tag}} {{.Size}}' | grep -E 'scratchimg|nginx'
scratchimg:latest 12.9MB
nginx:alpine 94.4MB
$ docker image prune -a -f --filter label=purpose=scratch
Deleted Images:
untagged: scratchimg:latest
deleted: sha256:2f3159edec21ddc61e1ca4ba3a7102f23528d9d2663713730330ba064759055c
deleted: sha256:73ebbce04941aea4700b5abfdfd6d0f8649dc27c242ed0125680f244fb6d1fc2
deleted: sha256:b928cffc8a82fc42688d35acca91f89713d497f0908ab2655de6063560040910
Total reclaimed space: 4.263kB
$ docker images --format '{{.Repository}}:{{.Tag}}' | grep -c scratchimg || true
0
The prune commands, from the most careful to the most dangerous:
docker container prune stopped containers
docker image prune dangling images (no name)
docker network prune networks nobody uses
docker volume prune anonymous volumes nobody uses (with -a: named ones too, which is data loss)
docker builder prune the build cache
docker system prune stopped containers, unused networks, dangling images and the build cache
docker system prune -a the same, and EVERY image that no container uses
$ docker create --name old alpine >/dev/null; docker network create unused >/dev/null
$ docker system prune -f
Deleted Containers:
b76a1843315420e7332c29b244bedde9e72facd80db5dd6e1892c3e2e33fc3fd
Deleted Networks:
unused
Deleted build cache objects:
yflu83brecmdludku5nvy2k2y
lhjogbzfbonnd1cbll082i36q
3fercbm44emke1hliafymcczk
uh4sij7readbx4bbprzsjo05o
Total reclaimed space: 118.9kB
$ docker system df --format 'table {{.Type}}\t{{.TotalCount}}\t{{.Active}}'
TYPE TOTAL ACTIVE
Images 12 0
Containers 0 0
Local Volumes 1 0
Build Cache 11 0
docker system prune -a --volumes (not run) would also have removed alpine and nginx:alpine, because no container used them at that moment, and every volume nobody uses. Read what a prune lists before you confirm it, never run it blindly on a server that holds a database volume, and use filters to be selective: docker image prune -a --filter "until=720h" removes unused images older than 30 days.
When the disk is full the usual places are /var/lib/docker/containers (logs, see the logging section), /var/lib/docker/overlay2 (images and writable layers), /var/lib/docker/volumes (data) and the build cache. docker system df -v lists the worst offenders by name.
Errors and troubleshooting
Start with three commands: docker ps -a (is it running, what exit status), docker logs NAME (what it said), and docker inspect NAME (how it was started).
THE CONTAINER STARTS AND EXITS AT ONCE. A container lives exactly as long as its main process (PID 1). If the process ends, the container is Exited, even when nothing is wrong:
$ docker run -d --name quit alpine
ad8852c44cbbe654b06200562bcfb9bb259d0ad906ad27a5f8e3bd6a6f061a91
$ docker ps -a --filter name=quit --format '{{.Names}} {{.Status}}'
quit Exited (0) 1 second ago
$ docker run -d --name fails alpine sh -c 'echo "boom: config missing" >&2; exit 1'
37d63641051d4aa47e6a36e1b4766ad0d6d8d79883efb99fbe0b57663279c51d
$ docker logs fails
boom: config missing
$ docker inspect fails -f 'exit code {{.State.ExitCode}}'
exit code 1
$ docker rm quit fails
quit
fails
alpine's default command is a shell, with nothing to read, so it exits at once with status 0. The second container printed an error and exited with 1, and docker logs still has the message after the container is gone from docker ps.
THE EXIT CODE TELLS YOU WHICH KIND OF FAILURE:
$ docker run --rm alpine sh -c 'exit 3'; echo "exit=$?"
exit=3
$ docker run --rm alpine nosuchcommand; echo "exit=$?"
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: exec: "nosuchcommand": executable file not found in $PATH
Run 'docker run --help' for more information
exit=127
$ docker run --rm alpine /etc/passwd; echo "exit=$?"
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: exec: "/etc/passwd": permission denied
Run 'docker run --help' for more information
exit=126
$ docker run -d --name kz alpine sleep 300 >/dev/null; docker kill kz >/dev/null; docker wait kz; docker rm kz >/dev/null
137
125 Docker itself failed (a bad option, a name in use, a port in use).
126 The command was found but cannot be run (not executable).
127 The command was not found (a wrong path, or the image has no such program).
137 Killed with SIGKILL (128 + 9): out of memory, or docker stop timed out. Check OOMKilled.
139 Segmentation fault. 143 is SIGTERM (a normal stop).
Any other number is the program's own exit status.
THE IMAGE DOES NOT EXIST, or you are not allowed to pull it (a typo, a private registry without docker login):
$ docker run --rm nosuchimage:1; echo "exit=$?"
Unable to find image 'nosuchimage:1' locally
docker: Error response from daemon: pull access denied for nosuchimage, repository does not exist or may require 'docker login'
Run 'docker run --help' for more information
exit=125
"failed to connect to the docker API" (older versions say "Cannot connect to the Docker daemon ... Is the docker daemon running?") means the daemon is down or the socket is not where the client looks (check systemctl status docker, and the DOCKER_HOST variable, which overrides it):
$ DOCKER_HOST=unix:///nonexistent.sock docker ps; echo "exit=$?"
failed to connect to the docker API at unix:///nonexistent.sock; check if the path is correct and if the daemon is running: dial unix /nonexistent.sock: connect: no such file or directory
exit=1
LOOK INSIDE A CONTAINER THAT HAS NO SHELL. A distroless or scratch image has no sh, so docker exec -it NAME sh fails. Start a helper container that shares the first one's process and network namespaces, with your own tools in it:
$ docker run -d --name app nginx:alpine >/dev/null
$ docker run --rm --pid container:app --network container:app alpine sh -c 'ps | grep "nginx: master"; wget -qO- localhost | head -2'
1 root 0:00 nginx: master process nginx -g daemon off;
38 root 0:00 sh -c ps | grep "nginx: master"; wget -qO- localhost | head -2
45 root 0:00 grep nginx: master
<!DOCTYPE html>
<html>
$ docker rm -f app
app
The helper sees the other container's processes and reaches its ports on localhost. (Recent Docker versions also have docker debug NAME, which does this for you; it needs a Docker subscription and was not run.) To look at its files, docker cp them out (it works without a shell) or docker export NAME | tar -t.
CHANGES TO A FILE ARE NOT SEEN, or old code keeps running. Check, in this order: is the file inside the image (COPY) and not rebuilt (docker build --no-cache, or docker compose build --no-cache)? Is the container recreated (docker compose up -d does it only if the file or image changed, and a bind-mounted config is read by the program only at start, so restart it)? Is another container still holding the port?
A CONTAINER CANNOT REACH THE INTERNET, or a name. Check DNS from inside (docker run --rm alpine nslookup example.com) and the daemon's dns setting. On a machine with a VPN or a corporate resolver, containers often need "dns": ["10.0.0.2"] in /etc/docker/daemon.json. Check the host's forwarding: sysctl net.ipv4.ip_forward must be 1, and a firewall that sets the FORWARD policy to DROP breaks container traffic.
Two more you will meet: "no space left on device" (see the cleanup section), and file permission errors on a bind mount (a user id mismatch, see the volumes section).
The daemon and the host
On Ubuntu the daemon is a systemd service. None of these were run in the lab, which has no systemd (its daemon was started by hand):
$ systemctl status docker --no-pager
$ sudo systemctl enable --now docker
$ sudo journalctl -u docker --since "1 hour ago" # the daemon's own log
$ docker info # versions, storage driver, root dir, cgroup driver, warnings
The daemon reads /etc/docker/daemon.json. It is valid JSON (a comma in the wrong place stops the daemon from starting, and journalctl -u docker says where), and a restart applies it. The settings that matter on servers:
{
"log-driver": "json-file",
"log-opts": { "max-size": "10m", "max-file": "3" },
"live-restore": true,
"default-address-pools": [ { "base": "10.200.0.0/16", "size": 24 } ],
"dns": ["10.0.0.2"],
"data-root": "/srv/docker"
}
live-restore keeps containers running while the daemon restarts (for an upgrade), and data-root moves /var/lib/docker to a bigger disk. The current value of the first one in the lab:
$ docker info -f 'live restore enabled: {{.LiveRestoreEnabled}}'
live restore enabled: false
$ docker info -f 'root dir: {{.DockerRootDir}}'
root dir: /var/lib/docker
What to back up: the compose files and the Dockerfiles (in Git), the data in named volumes (the tar method in the volumes section, or a database dump), and your registry credentials. Images and containers can be rebuilt from those, so a backup of all of /var/lib/docker is rarely worth it and is unsafe while the daemon runs.
Upgrading: with the Ubuntu packages, sudo apt update && sudo apt upgrade docker.io. With live-restore enabled the containers keep running; without it they restart with the daemon, and those with a restart policy come back by themselves. Read the release notes first for anything that removes an option (Docker 29 removed the old .NetworkSettings.IPAddress field, for example).
Practical one-liners
A throwaway shell in an image, with the current folder mounted and the files owned by you:
$ docker run --rm -v "$PWD":/work -w /work --user "$(id -u):$(id -g)" alpine sh -c 'echo made in a container > from-container.txt'
$ ls -ln from-container.txt | awk '{print $3, $4, $9}'; rm from-container.txt
1000 1000 from-container.txt
A quick web server for a folder (read-only), then gone:
$ docker run -d --rm --name quick -p 127.0.0.1:8088:80 -v "$PWD/site":/usr/share/nginx/html:ro nginx:alpine >/dev/null
$ curl -s localhost:8088; docker stop quick >/dev/null
<h1>version two</h1>
A table of what is running, and short formats for scripts:
$ docker run -d --name a1 alpine sleep 300 >/dev/null; docker run -d --name a2 alpine sleep 300 >/dev/null
$ docker ps --format 'table {{.Names}}\t{{.Image}}\t{{.State}}'
NAMES IMAGE STATE
a2 alpine running
a1 alpine running
$ docker ps -q | wc -l
2
$ docker ps --filter name=a1 --format '{{.Names}}'
a1
$ docker inspect -f '{{.Name}}: {{.State.Status}}' $(docker ps -q) | sort
/a1: running
/a2: running
$ docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' a1
172.18.0.2
Run something in every container, and stop or remove sets of them:
$ for c in $(docker ps -q); do docker exec $c hostname; done | wc -l
2
$ docker stop $(docker ps -q) >/dev/null && docker ps -q | wc -l
0
$ docker rm $(docker ps -aq) >/dev/null && docker ps -aq | wc -l
0
Get files out of a container that has already exited, which docker cp does, and see the exact command a container was started with:
$ docker run --name gone alpine sh -c 'echo result > /out.txt'
$ docker cp gone:/out.txt - | tar -xO
result
$ docker inspect gone -f '{{json .Config.Cmd}} user={{.Config.User}} image={{.Config.Image}}'
["sh","-c","echo result > /out.txt"] user= image=alpine
$ docker rm gone
gone
Wait for a container and use its exit status in a script:
$ docker run -d --name job alpine sh -c 'sleep 1; exit 7' >/dev/null
$ docker wait job
7
$ docker rm job
job
Cheat sheet
RUN docker run -d --name web -p 8080:80 nginx:alpine background, named, published
docker run --rm -it alpine sh a throwaway shell
docker run --rm -v "$PWD":/work -w /work IMAGE CMD work on the current folder
LIST docker ps / docker ps -a / docker ps -q
LIFECYCLE docker stop|start|restart|kill|rm [-f] NAME
DEBUG docker logs -f --tail 100 NAME docker exec -it NAME sh
docker inspect -f '{{...}}' NAME docker top NAME docker stats --no-stream
docker cp NAME:/path . docker diff NAME
IMAGES docker images docker pull IMAGE docker tag A B docker rmi IMAGE
docker save -o f.tar IMAGE docker load -i f.tar
BUILD docker build -t name:tag . docker build --no-cache -f Dockerfile.x .
VOLUMES docker volume create|ls|inspect|rm -v name:/path (named) -v "$PWD/dir":/path:ro (bind)
NETWORKS docker network create|ls|inspect|connect|disconnect|rm --network NAME
COMPOSE docker compose up -d down [-v] ps logs -f SERVICE exec -T SERVICE CMD config
LIMITS --memory 512m --cpus 1.5 --pids-limit 100 --restart unless-stopped
SAFE --user 1000:1000 --read-only --tmpfs /tmp --cap-drop ALL --security-opt no-new-privileges
CLEAN docker system df docker system prune docker image prune docker volume prune
Common problems
permission denied while trying to connect to the Docker daemon socket
You are not in the docker group (and the group is root, see the first sections), or you have not logged in again since being added. Use sudo, or newgrp docker.
failed to connect to the docker API (older versions: Cannot connect to the Docker daemon)
The daemon is not running, or DOCKER_HOST points somewhere else. systemctl status docker.
Conflict. The container name "/web" is already in use
A container with that name exists, even a stopped one. docker rm web, or use docker compose up, which recreates it.
Bind for 0.0.0.0:8080 failed: port is already allocated
Another container or a program on the host uses the port. docker ps, and ss -ltnp | grep 8080. The failed container stays in the Created state: remove it.
pull access denied, repository does not exist or may require docker login
A typo in the image name, a private image, or no login. docker login, and check the exact name and tag.
exec: "bash": executable file not found in $PATH
The image has no bash (Alpine and slim images). Use sh, or the helper container method in the troubleshooting section.
The container exits straight away with status 0
Its main process ended. A container needs a long-running foreground process: nginx -g "daemon off;", not a program that backgrounds itself.
Exited (137)
Killed: out of memory (docker inspect NAME -f '{{.State.OOMKilled}}') or stop timed out because PID 1 ignores SIGTERM (use init: true / --init and the exec form of CMD).
Files created in a bind mount belong to root
The container runs as root. Use --user "$(id -u):$(id -g)" or USER in the Dockerfile.
A -v site:/path mount is empty
A name without a slash is a NAMED volume, not a folder. Use "$PWD/site" or ./site in compose.
It works, and a published port is open to the whole network although ufw says closed
Docker's iptables rules come before ufw. Publish on 127.0.0.1, or filter in DOCKER-USER.
My changes to the image are not used
The tag points at the old image, or the build cache is stale. docker build --no-cache, and make sure the container was recreated from the new image.
no space left on device
docker system df, logs without rotation, old images and the build cache. See the cleanup and logging sections.
docker: unknown command: docker compose
The compose plugin is not installed. sudo apt install docker-compose-v2 (Ubuntu), or docker-compose-plugin (Docker's repository).
map has no entry for key "IPAddress"
Docker 29 removed that shortcut in docker inspect. Use {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}.
Tips
- Pin image versions (nginx:1.27-alpine, not nginx or nginx:latest) and rebuild on a schedule, so you get both repeatable builds and security fixes.
- One process per container, and let the container die when the process does. Run several containers instead of several programs.
- Containers are disposable. Keep nothing you need in the container's own layer: put data in a named volume, and the configuration in a bind-mounted file or an environment variable.
- Write the exec form: CMD ["prog", "arg"], and use --init (init: true in compose) so docker stop is fast and zombies are reaped.
- Order the Dockerfile from what changes least to what changes most, copy the dependency list and install first, the source last, and keep a .dockerignore.
- Use multi-stage builds and a small base image. Run as a non-root user, read-only, with capabilities dropped.
- Publish ports on 127.0.0.1 unless the whole network must reach them, and put a reverse proxy in front.
- Put a log size limit on everything (daemon.json), and look at docker system df now and then.
- Use a user-defined network or Compose, so containers reach each other by name, and do not rely on IP addresses.
- Use docker compose config before up, and keep the compose file in Git.
- The docker group is root. Give it out as carefully as sudo.
- docker run --help and docker COMMAND --help are accurate for the version you have, and docker system events shows what the daemon is doing.
