Write provenance, lineage and integrity records for your corpus, and make the integrity check fire

short · 40 min · Objective 1.2

Task

Make your lab corpus something you can vouch for. Record where every document came from (provenance), what happened to it on the way to the model (lineage), and a hash that shows it has not changed (integrity). Prove the integrity check detects a change. Then follow the data through the pipeline and classify every copy it leaves behind, including the two copies people miss: the vector store and the prompt log.

Steps

  1. Hidden content travels. Add an HTML comment holding the marker HIDDEN-NOTE-31 to docs/oncall-rota.md, re-run python app/ingest.py, and search the store for it with grep -rla HIDDEN-NOTE-31 app/store. Note in ~/secai-lab/lab/data-notes.md whether text a reader never sees reached the store.
  2. Write ~/secai-lab/lab/make-manifest.py, which writes ~/secai-lab/lab/manifest.csv with header path,sha256,bytes,classification,owner,source,licence,collected: one row per file under docs/, the classification and owner taken from its front matter, source set to written for this lab, licence to own work, and collected to today's date.
  3. Write ~/secai-lab/lab/verify-manifest.py, which recomputes every hash and exits 1, listing the paths, if any file changed, is missing, or exists under docs/ without a manifest row. It exits 0 otherwise.
  4. Make the check fire. Copy docs/product-faq.md to lab/faq.bak, append one line to the original, run the verifier, then copy the backup back and run it again. Record both runs in ~/secai-lab/lab/integrity-test.csv with header run,change,exit_code: the first should be 1, the second 0.
  5. Write ~/secai-lab/lab/lineage.csv with header step,input,input_version,output,output_version,tool,date and at least four rows: documents to chunks, chunks to embeddings in the store, retrieved chunks to prompts, and prompts to the gateway log. Use the manifest's own SHA-256 as the corpus version.
  6. Write ~/secai-lab/lab/data-types.csv with header copy,location,data_type,classification,owner,retention and one row per copy: the documents, the vector store, the gateway log, the model weights, the manifest and the evaluation results. data_type is structured, semi-structured or unstructured; location is a path relative to ~/secai-lab or an absolute path. A copy that holds the canary is restricted, whatever you thought it was.

Verify

cd ~/secai-lab
python lab/verify-manifest.py; echo "manifest check exit=$?"
awk -F, 'NR>1 {n++} END {print n+0" file(s) in the manifest"}' lab/manifest.csv
find docs -type f | wc -l
awk -F, 'NR>1 {printf "%s ", $3} END {print ""}' lab/integrity-test.csv
grep -rla HIDDEN-NOTE-31 app/store | wc -l
awk -F, 'NR>1 {print $3}' lab/data-types.csv | sort | uniq -c
python - <<'EOF'
import csv, os
def holds(path, mark="CANARY-7Q4X-SECAI"):
    paths = [path] if os.path.isfile(path) else [os.path.join(d, f) for d, _, fs in os.walk(path) for f in fs]
    for p in paths:
        try:
            if mark in open(p, encoding="utf-8", errors="ignore").read():
                return True
        except OSError:
            pass
    return False
rows = list(csv.DictReader(open("lab/data-types.csv", encoding="utf-8")))
wrong = [r["copy"] for r in rows if holds(os.path.expanduser(r["location"])) and r["classification"] != "restricted"]
print(len(rows), "copies listed; holding the canary but not restricted:", wrong or "none")
EOF

A manifest exit of 0, the same count in the manifest as files under docs/, integrity runs 1 0, a hidden-note count of at least 1 (unless your ingest strips comments, which your notes should then say), all three data types present, and none on the last line. If the vector store or the log appears in the last line, it holds the canary and was classified below it: the lesson's point that a copy needs the classification of its most sensitive content.

Notes

The manifest is what makes a poisoning incident scoped rather than guessed. Provenance says which documents came from where, lineage says which store and prompts they reached, and integrity shows your own copies were not altered after collection. Keep make-manifest.py and verify-manifest.py; the capstone uses both.

This is an independent study companion for CompTIA SecAI+ CY0-001 and is not produced by or endorsed by CompTIA.