Find what is holding a mount point open

applied · 30 min · Objective 2.3

Before you start — what this lab needs

A disposable virtual machine with a spare block device or a loop file. You will create and destroy a filesystem, so this must be a machine you can rebuild:

dd if=/dev/zero of=/tmp/lab.img bs=1M count=64
mkfs.ext4 /tmp/lab.img
mkdir -p /mnt/lab && mount -o loop /tmp/lab.img /mnt/lab

Task

This is the applied lab for objective 2.3 and draws on both its lessons -- process inspection and the signals used to stop things.

Reproduce a busy mount point, identify every process responsible, and unmount it cleanly. Then do it again with the culprit being your own shell, which is the case people spend the longest on.

Steps

Confirm the mount with findmnt /mnt/lab, and unmount it once to prove it works cleanly. Mount it again.
Start a process holding a file open on the mount: (sleep 600 < /mnt/lab/holder) & after creating the file with touch.
Attempt umount /mnt/lab and read the exact error.
Identify the holder two ways: lsof +D /mnt/lab and fuser -vm /mnt/lab. Note that both name the PID and the user.
Stop that process and confirm the unmount now succeeds.
Recreate the situation with your own shell as the culprit: cd /mnt/lab then try to unmount from another terminal. Confirm fuser -vm reports your shell, and that leaving the directory releases it.
Learn the escape hatch: umount -l /mnt/lab detaches the mount lazily. Read what that means for the processes still using it, and why it is a last resort rather than a shortcut.

Verify

mount -o loop /tmp/lab.img /mnt/lab
touch /mnt/lab/holder && (sleep 300 < /mnt/lab/holder &)
umount /mnt/lab 2>&1 | grep -qi busy && echo "reproduced"
fuser -vm /mnt/lab 2>&1 | grep -q '[0-9]' && echo "culprit identified"
fuser -km /mnt/lab; sleep 1; umount /mnt/lab && echo "unmounted"
findmnt /mnt/lab || echo "gone"

All four echoes must print in order. If "reproduced" does not print, the background process exited before the unmount attempt -- lengthen the sleep.

Notes

fuser -km kills every process using the mount, which is effective and indiscriminate. On a production host it is the command that turns a stuck unmount into an outage, so identify with -vm first and decide what to stop deliberately.