Catch the leak and the accuracy trap before a fraud model ships

short · 35 min · Objective 1.1

Task

Reproduce the two commonest reasons a model "worked in testing": duplicates that straddle the train/test split, and a headline accuracy that hides a model which never detects the rare class. Generate a synthetic, unbalanced fraud dataset with deliberate duplicate records, measure the leak in a naive split, fix it, and show that an always-legitimate baseline scores about 99% accuracy with a recall of zero.

Steps

  1. Write ~/secai-lab/lab/make-fraud.py. With random.seed(7), generate 2,000 transactions with columns id,amount,hour,merchant_category,country_risk,label, where label is fraud for 1% of rows and legit otherwise. Then copy 100 randomly chosen rows again with new ids, as if a source export had been ingested twice, shuffle, and write lab/fraud.csv.
  2. Write ~/secai-lab/lab/split-check.py. Define a record's content hash as the SHA-256 of every field except id. Do a naive split (first 80% train, rest test) and count test rows whose content hash also appears in train: that is straddling_naive, the leak.
  3. Fix it: deduplicate on the content hash, then split by hash (for example, test when the hash as an integer modulo 5 is 0), so identical records can never land on both sides. Count again as straddling_fixed.
  4. On the fixed test set, score a baseline that answers legit for everything. Compute accuracy, precision (0 when it never predicts fraud), recall and false positive rate.
  5. Write ~/secai-lab/lab/validation.json with the keys rows, duplicates_found, straddling_naive, straddling_fixed, baseline_accuracy, baseline_precision, baseline_recall and baseline_fpr. Add one sentence to lab/validation-notes.md saying which metric you would report to the fraud team, and why.

Verify

cd ~/secai-lab
python - <<'EOF'
import csv, hashlib, json
rows = list(csv.DictReader(open("lab/fraud.csv", encoding="utf-8")))
def key(r):
    return hashlib.sha256("|".join(r[k] for k in sorted(r) if k != "id").encode()).hexdigest()
seen = {}
for r in rows:
    seen[key(r)] = seen.get(key(r), 0) + 1
dups = sum(c - 1 for c in seen.values())
v = json.load(open("lab/validation.json"))
checks = {
    "duplicates_found matches the data": v["duplicates_found"] == dups,
    "the naive split leaked": v["straddling_naive"] > 0,
    "the fixed split is clean": v["straddling_fixed"] == 0,
    "accuracy looks excellent": v["baseline_accuracy"] > 0.95,
    "recall exposes the baseline": v["baseline_recall"] == 0,
}
for name, ok in checks.items():
    print("PASS" if ok else "FAIL", name)
print(len(rows), "rows,", dups, "duplicate(s)")
EOF

Five PASS lines and about 2,100 rows with about 100 duplicates. The naive split typically leaks around thirty duplicates into the test set. An accuracy above 95% next to a recall of zero is the lesson's fraud example: a model that never flags fraud, reported as a success.

Notes

Duplicates are also a security finding, not only a quality one: records that appear many times are the ones most likely to be memorised and repeated, which is where membership inference starts. The data lab returns to deduplication as part of cleansing.

This is an independent study companion for CompTIA SecAI+ CY0-001 and is not produced by or endorsed by CompTIA.