Threat-model the lab's retrieval app with all four resources

applied · 80 min · Objective 2.1

Task

Apply the whole of objective 2.1 to one system: draw your lab's retrieval app as a data flow diagram with its AI elements explicit, mark the in-prompt trust boundary, find threats with STRIDE, map them to OWASP and ATLAS, widen the assessment with the MIT AI Risk Repository, inventory the components so a new CVE could be matched to them, and test one threat for real.

Steps

  1. Write lab/dfd.md, one line per element in the form element: <name> -- <what it does>, covering at least the model runner, the gateway, the retrieval app, the vector store, the ingestion step that fills it, the system prompt and template, the documents and the log. Then one line per trust boundary in the form boundary: <name> -- <what crosses it>, including the one that appears on no network diagram: retrieved content and user text crossing into the model's instruction stream.
  2. Create lab/threat-model.csv with header id,element,boundary,stride,threat,owasp,atlas,source,decision,owner,test_result. Walk STRIDE over every element and write at least fifteen threats with source=stride and stride set to one letter of S, T, R, I, D or E. Use boundary=prompt for threats that cross the in-prompt boundary.
  3. Enrich each threat: owasp with its entry identifier from either list (LLM01, ML02 and so on) and atlas with its ATLAS technique identifier where one fits. Leave a field empty rather than forcing a match; an empty field is information.
  4. Open the MIT AI Risk Repository, filter by the causal taxonomy for unintentional, post-deployment risks and by the domains that fit an internal policy assistant, and add at least three rows with source=mit-repo and stride=- for risks STRIDE did not surface, such as staff trusting a wrong answer about their entitlements.
  5. Set decision on every row to one of mitigate, accept, transfer or avoid, and owner to a role.
  6. Create lab/ai-inventory.csv with header component,kind,version,source,cve_checked_on, with kind one of software, model or dataset. List the operating system, the runner, the Python libraries the app and gateway import, the vector store, the chat model, the embedding model and your document set. Search the CVE records on your host for every software row and record the date you checked.
  7. Test the information-disclosure threat against retrieval: ask your gateway, as an ordinary user, one question about salary bands, count the canary in that response and record canary_hits=<n> in the matching row's test_result.

Verify

Run from ~/secai-lab.

python - <<'EOF'
import csv
rows = list(csv.DictReader(open("lab/threat-model.csv", newline="")))
inv = list(csv.DictReader(open("lab/ai-inventory.csv", newline="")))
print(len(rows), "threat row(s)")
print("STRIDE letters covered:", "".join(sorted({r["stride"] for r in rows if r["stride"] in {"S", "T", "R", "I", "D", "E"}})))
print(sum(r["boundary"] == "prompt" for r in rows), "threat(s) at the in-prompt boundary")
print(sum(r["source"] == "mit-repo" for r in rows), "risk(s) from the MIT repository")
print(sum(r["owasp"].startswith(("LLM", "ML")) for r in rows), "mapped to OWASP,",
      sum(r["atlas"].startswith("AML.T") for r in rows), "mapped to ATLAS")
print("bad decision:", [r["id"] for r in rows if r["decision"] not in ("mitigate", "accept", "transfer", "avoid")] or "none")
print("no owner:", [r["id"] for r in rows if not r["owner"].strip()] or "none")
print(sum("canary_hits=" in r["test_result"] for r in rows), "threat(s) tested")
print(len(inv), "inventory row(s),", len({r["kind"] for r in inv}), "kind(s),",
      sum(1 for r in inv if r["kind"] == "software" and not r["cve_checked_on"].strip()), "software row(s) not checked against CVE")
EOF
grep -cE '^(element|boundary):' lab/dfd.md
grep -ciE '^boundary:.*(prompt|instruction)' lab/dfd.md

Eighteen or more threat rows, all six STRIDE letters, at least two threats at the in-prompt boundary, three or more from the MIT repository, no bad decision and no missing owner, one tested threat, an inventory with all three kinds and no unchecked software row, eleven or more element and boundary lines, and at least one boundary that names the prompt. A threat model with no in-prompt boundary has missed the one boundary that makes this an AI system.

Notes

The inventory is the CVE half of the objective made practical. CVE only helps if you know what you are running; with this file, a new advisory against your runner or vector store library takes a minute to match. Keep it current when later labs add components -- lesson 12 adds an agent, and its tools are components too. The MIT rows will mostly have governance or responsible-AI owners rather than security ones, which is what the repository is for.

This is an independent study companion for CompTIA SecAI+ CY0-001 and is not produced by or endorsed by CompTIA.