Assess your lab's suppliers and map where its data goes

applied · 85 min · Objective 4.3

Task

Apply objective 4.3 to the suppliers your lab really depends on, plus one invented hosted vendor, using the lesson's evaluation areas. Record the provenance of the weights you run. Then build a data-flow and sovereignty record for the lab and check it against the disk: every component that actually holds the restricted document's canary must appear in the record as a destination of restricted data.

Steps

  1. Create lab/vendor-assessment.csv with header supplier,component,area,question,answer,evidence,finding,severity. area is one of assurance_scope, data_terms, sub_processors, model_governance, feature_security, regulatory_position, licence or data_location; severity is none, low, medium or high.
  2. Assess five suppliers across all eight areas: the model runner project, the publisher of your open-weight model, the publisher of your embedding model, the vector store library, and Invented hosted vendor. For local components many answers are 'not applicable: nothing leaves the VM', and the evidence is your isolation proof. Read the model's actual licence for the licence row: open weights is not the same as free for any use.
  3. Create lab/model-provenance.csv with header component,source,licence,digest,recorded_on, one row each for the chat and embedding models. digest is the SHA-256 of the weights file from sha256sum, or, where your runner stores weights as content-addressed files, the 64-character digest in the file's name.
  4. Create lab/data-flows.csv with header flow_id,data,from_component,to_component,location,jurisdiction,crosses_border,retention,reachable_by,classification,transfer_mechanism. Name components by their top-level directory under ~/secai-lab (docs, app, gateway, logs) plus runner and snapshots, the last of which live on the host's disk.
  5. Record at least eight lab flows: question to gateway, gateway to runner, documents to embeddings, embeddings to the vector store, retrieved chunks into the prompt, every request into logs/gateway.jsonl, the whole VM into snapshots, and any copy you keep outside snapshots. Mark the restricted ones restricted.
  6. Add at least three flows for the invented vendor, as if UC-02 moved to it: inference served from another region, prompts logged in the provider's home country, and abuse-monitoring reviewers elsewhere. Set crosses_border to yes and give each a transfer mechanism to confirm with counsel.
  7. Run the verify block. If it lists a component that holds the canary with no restricted flow, you have found something your map missed. Add the flow; do not delete the data to make the check pass.

Verify

cd ~/secai-lab
python - <<'EOF'
import csv, os
va = list(csv.DictReader(open("lab/vendor-assessment.csv", encoding="utf-8")))
AREAS = {"assurance_scope", "data_terms", "sub_processors", "model_governance",
         "feature_security", "regulatory_position", "licence", "data_location"}
for s in sorted({r["supplier"] for r in va}):
    print(s, "- areas missing:", sorted(AREAS - {r["area"] for r in va if r["supplier"] == s}))
print("answers with no evidence:", sum(not r["evidence"] for r in va))
print("bad severity:", sum(r["severity"] not in ("none", "low", "medium", "high") for r in va))
pv = list(csv.DictReader(open("lab/model-provenance.csv", encoding="utf-8")))
hexd = set("0123456789abcdef")
print("provenance digests that are SHA-256:",
      sum(len(r["digest"]) == 64 and set(r["digest"].lower()) <= hexd for r in pv), "of", len(pv))
flows = list(csv.DictReader(open("lab/data-flows.csv", encoding="utf-8")))
dest = {r["to_component"] for r in flows}
restricted_to = {r["to_component"] for r in flows if r["classification"] == "restricted"}
holding = set()
for root, dirs, files in os.walk("."):
    for name in files:
        p = os.path.join(root, name)
        top = os.path.relpath(p).split(os.sep)[0]
        if top == "lab" or top == name:
            continue
        try:
            if os.path.getsize(p) < 200_000_000 and b"CANARY-7Q4X-SECAI" in open(p, "rb").read():
                holding.add(top)
        except OSError:
            pass
print(len(flows), "flows; components holding the canary:", sorted(holding))
print("holding it with no restricted flow:", sorted(holding - restricted_to))
print("snapshots recorded as a destination:", "snapshots" in dest)
cross = [r for r in flows if r["crosses_border"] == "yes"]
print(len(cross), "border crossings; with no transfer mechanism:",
      [r["flow_id"] for r in cross if not r["transfer_mechanism"]])
EOF

Five suppliers, each with no areas missing; zero answers without evidence and zero bad severities; every provenance digest SHA-256; eleven or more flows; the canary found in at least docs and logs and usually the directory holding the vector store; an empty "no restricted flow" list; snapshots ... True; and three or more crossings, none without a mechanism.

Notes

The walk skips lab/ because your own notes may quote the canary, and skips files over 200 MB so that weights stored under ~/secai-lab are not read into memory. It cannot see the snapshots, which live on the host; that is why they must appear in the record by hand. They hold everything the VM holds, the restricted document included, and are easily forgotten.

This is an independent study companion for CompTIA SecAI+ CY0-001 and is not produced by or endorsed by CompTIA.