Find configuration drift against a golden config

applied · 45 min · Objective 3.1

Task

Capture a known-good configuration, let the lab drift, then detect every difference automatically. Drift is the fault that explains "it worked yesterday", and detecting it is a two-line script rather than a product.

Steps

  1. Capture the golden configuration on the router, as text, in a form that is stable between runs: { ip -br addr show; ip route show; sudo sysctl net.ipv4.ip_forward; sudo iptables-save; } > /tmp/golden.txt.
  2. Read it once. A golden config you have never read cannot be a reference, because you cannot tell an intended line from an accidental one.
  3. Introduce three changes without writing them down: add a static route, change an MTU, and add a firewall rule. Do them quickly and try to forget the details.
  4. Capture the current state the same way into /tmp/current.txt and diff the two. Every difference is either a change somebody made or a change something made — and you now have to decide which.
  5. For each difference, decide: keep it and update the golden config, or revert it. That decision is the whole of configuration management, and doing it three times makes the point.

Verify

{ ip -br addr show; ip route show; sudo sysctl net.ipv4.ip_forward; sudo iptables-save; } > /tmp/current.txt
diff -u /tmp/golden.txt /tmp/current.txt; echo "diff exit $?"
grep -c . /tmp/golden.txt /tmp/current.txt

The diff must show exactly the three changes you made and nothing else. A diff exit code of 0 means no drift; 1 means drift was found. If the diff shows extra noise — timestamps, counters, ordering — your capture is not stable enough to be a golden config, and fixing that is part of the exercise.

Notes

That last point is the practical trap. A naive capture includes packet counters, uptimes and randomly ordered output, so every diff shows hundreds of lines and people stop reading it. A golden config must capture intent, not state: addresses, routes, rules and settings, with counters and timestamps excluded.

Once the capture is stable, the automation is trivial — run it on a schedule, diff against the stored copy, and alert on a non-empty result. That is what commercial configuration-management tools do, and knowing it is the same diff is useful when you have to justify buying one, or not.

The related exam point is version control. Keeping golden configs in a git repository gives you not only the current reference but the history — who changed what, when, and with what commit message. Correlating a fault with the last commit is the fastest root-cause analysis there is.