Measure a phishing simulation by reporting rate, not click rate

applied · 55 min · Objective 1.2

Task

Apply objective 1.2: design a simulation programme whose primary measure is the behaviour you actually want, and show why the click rate on its own cannot support the conclusion people draw from it.

Steps

  1. Build lab/phish.csv with header campaign,date,recipients,clicks,reports,first_report_minutes,difficulty and six campaigns over a year.
  2. Vary difficulty deliberately across campaigns and let the click rate fall while difficulty also falls -- the pattern a programme reporting steady improvement often actually has.
  3. Compute click rate, report rate and report-to-click ratio per campaign into lab/phish-metrics.csv.
  4. Write lab/phish-finding.md stating, in one paragraph, why the falling click rate does not support 'our users are improving', and which two measures do.
  5. Design the follow-up: write lab/phish-followup.md specifying what a clicker sees, within how long, and why the programme is not punitive.

Verify

awk -F, 'NR>1 {n++} END {print n" campaign(s)"}' lab/phish.csv
awk -F, 'NR>1 && NF>=3 {n++} END {print n" metric row(s)"}' lab/phish-metrics.csv
grep -Eci 'difficulty|not comparable|confound' lab/phish-finding.md
grep -Eci 'report|time to first' lab/phish-finding.md
grep -Eci 'not punitive|no blame|blameless' lab/phish-followup.md

Six campaigns, metrics computed for each, and the finding naming BOTH the difficulty confound and the reporting measure. A finding that only says "click rate fell" has reproduced the error rather than identified it.

This is an independent study companion for CompTIA SecurityX CAS-005 and is not produced by or endorsed by CompTIA.