Replace an assertion with an automated test

applied · 75 min · Objective 1.5

Task

Apply objective 1.5: take three controls currently recorded as assertions and convert each into a machine-evaluable test, then prove the tests detect a control that has stopped working.

Steps

  1. Pick three controls from lab/control-matrix.csv whose evidence is currently a human assertion.
  2. For each, write a test script in lab/tests/ that queries the real system and exits 0 when the control holds and 1 when it does not. The test must state the scope it checked, not just the verdict.
  3. Run all three and record the output in lab/test-results.csv as control,exit,checked,timestamp.
  4. BREAK one control deliberately -- remove a member from an enforced group, disable encryption on one resource -- and re-run. Record the result.
  5. Restore the control, re-run, and write lab/test-canary.md recording that each test was observed both passing and failing.

Verify

ls lab/tests/ | wc -l
awk -F, 'NR>1 {n++} END {print n" result row(s)"}' lab/test-results.csv
awk -F, 'NR>1 && $2==1 {n++} END {print n+0" observed failing"}' lab/test-results.csv
grep -Eci 'passing and failing|observed fail|canary' lab/test-canary.md

Three tests, each observed BOTH passing and failing. A test never seen to fail is not evidence about the control -- it is evidence about the test, and not even that.

This is an independent study companion for CompTIA SecurityX CAS-005 and is not produced by or endorsed by CompTIA.