Measure what your redactor misses, using values you planted

short · 40 min · Objective 2.4

Task

Write a redactor for the structured values a gateway should never pass on, then measure its miss rate per type against a seeded set of synthetic prompts -- including the free-text identifiers that pattern detectors are known to miss, and decoys it must leave alone.

Steps

  1. Write lab/pii-seed.jsonl: at least 24 prompts, one JSON object per line with id, text and planted (a list of objects with type and value). Types are EMAIL, CARD, NINO, PHONE, NAME (an invented full name in a sentence) and CONTEXT (a phrase that identifies someone with no labelled field, such as the only night-shift pharmacist at the Harrogate site). Plant at least four of each.
  2. Add at least four DECOY prompts: order numbers of sixteen digits that fail the Luhn check, and a version string. These are values the redactor must NOT touch, so over-redaction becomes measurable.
  3. Write lab/redact.py with a function redact(text) that replaces each detected value with a typed placeholder -- [EMAIL], [CARD], [NINO], [PHONE] -- so the sentence still reads: Card 4111 1111 1111 1111 for jo.test@example.com comes out as Card [CARD] for [EMAIL], with everything else left as it was. Accept card candidates only when they pass the Luhn check. When run as a script it redacts standard input to standard output.
  4. Write lab/measure-redaction.py, which runs every seed prompt through redact() and writes lab/redaction-results.csv with header id,type,value,survived, where survived is 1 if the planted value is still present verbatim afterwards.
  5. Write lab/redaction-notes.md with one line per type giving its miss rate and the layer that would catch what this one misses: named-entity recognition, minimisation at source, or an output filter. NAME and CONTEXT will mostly survive. That is the finding, not a failure to hide.

Verify

printf 'Card 4111 1111 1111 1111 for jo.test@example.com\n' | python lab/redact.py | grep -c '[[]CARD[]] for [[]EMAIL[]]'
python - <<'EOF'
import csv, collections
rows = list(csv.DictReader(open("lab/redaction-results.csv")))
tot = collections.Counter(r["type"] for r in rows)
miss = collections.Counter(r["type"] for r in rows if r["survived"] == "1")
for t in sorted(tot):
    print(t, "planted", tot[t], "survived", miss[t])
structured = sum(miss[t] for t in ("EMAIL", "CARD", "NINO", "PHONE"))
ok = len(rows) >= 24 and structured == 0 and tot["DECOY"] > 0 and miss["DECOY"] == tot["DECOY"] and tot["NAME"] > 0
print("PASS" if ok else "FAIL")
EOF
grep -c . lab/redaction-notes.md

The one-line test prints 1, the structured types all show zero survivors, every DECOY survives (it was left alone), and the script prints PASS. NAME and CONTEXT survivors are expected and belong in the notes with their rate. A redactor that also removed the decoys is over-redacting, and the model loses the order numbers users actually need answered.

Notes

Keep redact() -- the applied lab wires it into the gateway, and lesson 15's lab reuses it for the log. The miss rate you measured is what lesson 16 means by auditing a control against a known answer set: without the seed file, "the redactor works" is an opinion.

This is an independent study companion for CompTIA SecAI+ CY0-001 and is not produced by or endorsed by CompTIA.