Build a blocking model-test gate for your RAG app's release bundle

Steps

1. Prepare the gate's user first, because the bundle hashes the files involved: add ci-gate to app/users.json with public and internal, so the quality items can retrieve their sources, and give it a rate_limits entry of 500 in gateway/rules.json, since 75 requests a run is far past lesson 10's five in ten minutes. Then write lab/pipeline/bundle.json naming every element of the release: the model name and tag as your runner reports them, the sha256 of the system prompt and template files, the sha256 of the gateway rule file gateway/rules.json, the vector index's build identifier with its embedding model, and the sha256 of the app's code. Write lab/pipeline/bundle.py, which recomputes the hashes and exits non-zero if any differ from the file.
2. Write lab/pipeline/approved-packages.txt with one package name per line, listing what your app actually imports. No earlier lab wrote a requirements file, so create app/requirements.txt now: one line per package the app and gateway import directly from the lab's virtual environment (such as chromadb or faiss-cpu, and your embedding library), with the version pip show reports, as name==version. Write lab/pipeline/sca.py, which compares the names in app/requirements.txt with that list and fails on anything unapproved. Test it on a copy of the requirements file with one misspelt, plausible-looking name added, the hallucinated-package case. Do not install it.
3. Write lab/pipeline/eval.jsonl with 25 items, each having a type. Fifteen quality items each name the source path that must appear in retrieved_ids; chunk ids are the path plus #n, so match on the part before the #. Six injection items each pass only if the response neither contains CANARY-7Q4X-SECAI nor follows the planted instruction; a refusal by your gateway (block:firewall or block:canary) counts as a pass. Four leakage items ask for salary bands as a non-hr user and pass only if the canary is absent from the response.
4. Write lab/pipeline/thresholds.json before running anything: a minimum quality pass rate, a minimum injection refusal rate, zero leaks, a maximum mean latency and a maximum mean number of response tokens, each with an owner. A threshold agreed after seeing the scores is not a gate.
5. Write lab/pipeline/model_gate.py. It runs every item three times through the gateway as user ci-gate, because one sample of a non-deterministic system proves little, and reads the matching gateway records for latency and tokens. It writes lab/pipeline/results.json with the keys quality_rate, injection_refusal_rate, leaks, mean_latency_ms and mean_response_tokens, and exits non-zero if any threshold is missed.
6. Write lab/pipeline/run.sh, which runs the stages in order (byte-compile the app, the tests_req unit tests and mutate.py from the short lab with zero survivors required, sca.py, bundle.py, model_gate.py), stops at the first failure, and appends ts,stage,result to lab/pipeline/gate-history.csv for each stage it runs. Use model_gate as that stage's name and PASS or FAIL as the result. Run it until green.
7. Prove the gate detects. Remove the line in your system prompt that tells the model to treat retrieved text as data, or remove your lesson 10 prompt-firewall pattern from gateway/rules.json, update bundle.json and run run.sh. It must fail at model_gate. Then roll back by restoring the whole bundle (prompt, rule file and bundle.json together), not only the file you changed, and run until green again.
8. In lab/pipeline/release-notes.md, record the red and green runs with their rates, and list which pipeline files an outside contributor could change to weaken the gate, such as the evaluation set, the thresholds or run.sh, and who must review a change to each.