Build an image and debug one that will not start

short · 30 min · Objective 2.6

Task

Build an image with a Dockerfile, then reproduce and diagnose the two failures every container operator meets: a container that exits immediately, and layer caching that makes rebuilds slow. Learn that logs, not exec, is the first tool.

Steps

  1. Write a Dockerfile: FROM, WORKDIR, COPY a dependency manifest, RUN install, COPY the source, USER, ENTRYPOINT/CMD. Build it with podman build -t lab:1 ..
  2. Run it and confirm it works. Note the -p host:container order.
  3. Reproduce the exit-immediately failure: change the CMD to a command that errors. Run detached, see it stop, and get the reason from podman logs -- NOT from exec, which cannot attach to a stopped container.
  4. Confirm exec needs a shell: try podman exec -it on a minimal image with no bash and watch it fail, then retry with sh.
  5. Reproduce the caching problem: edit a source file and rebuild, timing it. Then swap the COPY order so source is copied before dependencies are installed, edit again, and time the much slower rebuild. Explain which layer was invalidated.
  6. Confirm USER took effect: run podman exec ... id and confirm the process is not root.
  7. Pin the base image to a version rather than latest, and explain what breaks without it.

Verify

podman logs lab-broken 2>&1 | tail -3          # the reason it exited
podman inspect lab:1 --format '{{.Config.User}}'   # not root
podman image history lab:1 | head              # layers, largest first
podman run --rm lab:1 true && echo "runs"

Step 3 is the habit to build: a container that exits has usually printed why, and podman logs holds it even after the container has stopped. Reaching for exec on a stopped container is the reflex to unlearn.

Notes

The layer-ordering result in step 5 is worth feeling rather than reading: copying the dependency manifest and installing BEFORE copying source means editing code does not re-run the install. Reversing them turns a five-second rebuild into a five-minute one.