Find the shadow AI in your own lab and set autonomy limits

applied · 80 min · Objective 4.2

Task

Apply objective 4.2 to the risks that need no adversary. Plant the commonest form of shadow AI in your own lab, a convenience script that calls the model without going through the gateway, and find it by inspecting the system. Practise the organisation-wide discovery sources on synthetic telemetry you write. Choose responses the lesson recommends instead of blocking everything, and set the autonomy level of every capability your lab has or might gain.

Steps

  1. Write ~/secai-lab/tools/quick-summary.py, a short script that sends one question straight to the runner's loopback port, bypassing the gateway, and run it twice. Nothing in the gateway log records those requests, and that is the point.
  2. Find it from the system: search ~/secai-lab for Python files that name the runner's port outside gateway/, and while the script runs, use ss -tnp to see which process holds the connection to that port. If your app calls an embedding model on the runner directly, that is a finding too, to register rather than hide.
  3. Create synthetic organisational telemetry in lab/discovery/: dns.csv (ts,user,query, at least twenty rows, some to invented AI services such as chatpad.example.com), oauth-consents.csv (user,app,scopes, including one invented assistant granted mailbox access) and expenses.csv (user,merchant,amount,description, including one AI subscription bought on a card).
  4. Write lab/shadow-ai-findings.csv with header id,source,service,users,evidence,data_risk,response. source is dns, oauth, expenses or code_scan; response is one of sanction_alternative, restrict, block, coach or register. The quick-summary script is a code_scan finding with quick-summary in its evidence.
  5. Choose responses as the lesson does: the widely used chatbot gets a sanctioned alternative, the mailbox-reading assistant is restricted or blocked, the users found are coached. Then fix the real finding: point quick-summary.py at the gateway's POST $GW/ask (HTTPS on 127.0.0.1:8088 since lesson 13) and rerun the code search.
  6. Create lab/autonomy.csv with header capability,level,reversible,blast_radius,attributable,approval_gate,owner for every capability the lab system has (answering, retrieval, any tool an earlier lab added) and two you might add, such as closing tickets in a ticket file or messaging the on-call person from the rota. level is 1 to 4 from the lesson's scale; blast_radius is the number of records one action can touch.
  7. Add each shadow finding to lab/ai-risk-register.csv under category shadow_ai, and to lab/ai-use-register.csv as proposed where the response is register or sanction_alternative.

Verify

cd ~/secai-lab
RUNNER_PORT=11434   # set to your runner's port
grep -rlE "(127[.]0[.]0[.]1|localhost):$RUNNER_PORT" --include='*.py' --exclude-dir=.venv . | grep -v '^./gateway/' | wc -l
python - <<'EOF'
import csv
f = list(csv.DictReader(open("lab/shadow-ai-findings.csv", encoding="utf-8")))
ok = {"sanction_alternative", "restrict", "block", "coach", "register"}
print(len(f), "findings from", len({r["source"] for r in f}), "sources")
print("bad responses:", [r["id"] for r in f if r["response"] not in ok])
print("answered by blocking:", sum(r["response"] == "block" for r in f), "of", len(f))
print("quick-summary found:", any("quick-summary" in r["evidence"] for r in f))
a = list(csv.DictReader(open("lab/autonomy.csv", encoding="utf-8")))
print(len(a), "capabilities")
print("irreversible above level 2:", [r["capability"] for r in a if r["reversible"] == "no" and int(r["level"]) > 2])
print("level 3+ not attributable:", [r["capability"] for r in a if int(r["level"]) >= 3 and r["attributable"] != "yes"])
print("blast radius not a count:", [r["capability"] for r in a if not r["blast_radius"].isdigit()])
reg = list(csv.DictReader(open("lab/ai-risk-register.csv", encoding="utf-8")))
print("shadow_ai rows in the risk register:", sum(r["category"] == "shadow_ai" for r in reg))
EOF

After the fix in step 5 the file count is 0, or 1 if your app calls the runner directly for embeddings and you registered that. Then: five or more findings from four sources, no bad responses, blocking used for a minority, quick-summary found: True, four or more capabilities with all three lists empty, and two or more shadow_ai rows. Before the fix, the count must have included tools/quick-summary.py; if it never did, the search is not finding your runner's port.

Notes

The script in step 1 is the shape shadow AI usually takes inside an engineering team: not a rogue service but a shortcut, written in good faith, that skips the one place logging and controls live. Ollama listens on 11434 by default; llama.cpp's server uses a different default, so set the variable to whatever your build actually uses. Revert to the snapshot if you would rather not keep the script at all.

This is an independent study companion for CompTIA SecAI+ CY0-001 and is not produced by or endorsed by CompTIA.