Write an fstab entry that cannot stop the boot

applied · 35 min · Objective 1.3

Task

Break a machine's boot with a bad fstab entry, recover it from emergency mode, and then write the entry so that the same mistake cannot repeat. This is a failure people meet for the first time in production, at which point they are learning emergency mode under pressure.

Steps

  1. Create a loop-backed filesystem and mount it by hand at /mnt/data. Note its UUID with blkid.
  2. Add a correct fstab entry using the UUID, and prove it with mount -a followed by umount and mount -a again.
  3. Now break it: change one character of the UUID. Run mount -a and read the error -- you have just caught the fault while still holding a shell.
  4. Reboot with the broken entry and watch the machine drop to emergency mode. Log in at the console, note that the root filesystem is mounted read-only, and remount it read-write with mount -o remount,rw / before you can edit anything.
  5. Fix the UUID, verify with mount -a, and reboot to confirm.
  6. Break it once more, but this time add nofail to the options. Reboot and confirm the machine comes up normally with the filesystem simply absent.
  7. Add _netdev to an entry for a network filesystem and explain what it changes about ordering.

Verify

findmnt --verify --verbose | tail -5
mount -a && echo "fstab parses and mounts cleanly"
grep -q 'nofail' /etc/fstab && echo "non-essential mounts are nofail"
systemctl list-units --type=mount --state=failed --no-pager | tail -2

findmnt --verify is the check worth adopting permanently: it validates every fstab line without mounting anything, and it reports problems that mount -a would only reveal by failing.

Notes

The habit this lab exists to build is one command long: run mount -a after every fstab edit, before rebooting. The fault then appears while you still have a working shell, rather than at the console of a machine that will not boot.