Solve a staged fault by the method

short · 30 min

Task

Work a planted fault end to end using CompTIA's six steps, deliberately choosing tests that gather information rather than confirm a guess, and changing one thing at a time. The discipline is the deliverable, not the fix.

Steps

  1. Identify. Establish the symptom precisely -- what fails, for whom, since when -- and ask "what changed?": dnf history, git log on /etc, the deployment record. Reproduce the fault so you can tell when it is fixed.
  2. Theory. State a specific, falsifiable cause. Question the obvious first: is the service running, is the disk full, has a certificate expired.
  3. Test. Choose a test that gathers information without changing anything -- read the logs, df -h, systemctl status, ss -tlnp. If the theory is wrong, form a new one rather than tweaking the old.
  4. Plan. Decide the change and its rollback before making it. Back up anything you will edit.
  5. Implement. Make ONE change. If it does not help, undo it before trying the next.
  6. Verify. Reproduce the original symptom and confirm it is gone from the user's perspective, not just that a command returned zero.
  7. Prevent and document. Add the monitor or rotation that stops recurrence, and write symptom, cause, fix and evidence.

Verify

# the fix is proven only by the ORIGINAL symptom being gone:
# (example, for the full-/var fault)
df -h /var | awk 'NR==2 {print $5}'      # back below threshold
systemctl is-active theservice           # if a service fault
curl -sf -o /dev/null -w '%{http_code}\n' http://localhost/   # the user-visible check
# and a written record exists:
test -s ~/incident-notes.md && echo "documented"

The verification must be the user-visible symptom, not an intermediate. A service reporting active is not the same as the page loading; reproduce the original complaint and confirm it is gone.

Notes

Choose tests that produce information. A test that applies the suspected fix conflates diagnosis with treatment, and one wrong theory then wastes the whole session. A test that reads the current state moves you forward even when your theory is wrong -- which, as often as not, it is.