Build a pipeline that stops a bad change

applied · 45 min · Objective 4.1

Task

Build a CI pipeline that lints, tests, scans and only then deploys, and prove the property that justifies it: a change that fails any stage cannot reach the deploy step. Add a secret scanner and confirm it blocks a committed credential.

Steps

  1. Write a pipeline definition with ordered stages: lint, test, scan, deploy. Each stage fails the pipeline on a non-zero exit.
  2. Add real checks: shellcheck/ansible-lint in lint, a unit test in test, gitleaks and a container scan in scan.
  3. Commit a good change and watch every stage pass and the deploy run.
  4. Commit a change that fails the test. Confirm the pipeline STOPS at test and the deploy stage never runs. This is the whole value.
  5. Commit a change introducing a hardcoded secret. Confirm gitleaks fails the scan stage and blocks the deploy -- and contrast a pre-commit hook, which would have stopped it reaching the history at all.
  6. Restrict deploy to the main branch with an only/if condition, and confirm a feature branch runs the checks but not the deploy.
  7. Inject a secret into the pipeline the right way -- a masked, protected variable -- and confirm it does not appear in the job log.

Verify

cd ~/lab-ci
# good change: deploy stage recorded
./run-pipeline.sh good 2>&1 | grep -qi 'deploy: ok' && echo "good change deploys"
# failing test: pipeline stops before deploy
./run-pipeline.sh badtest 2>&1 | tee out.txt; grep -qi 'deploy' out.txt && echo "DEPLOYED despite failure -- wrong" || echo "stopped before deploy"
# committed secret: scan blocks it
./run-pipeline.sh secret 2>&1 | grep -qi 'gitleaks\|secret detected' && echo "scan blocked the secret"

The failing-test run NOT reaching deploy is the property the whole pipeline exists for: broken change in, no deployment out. The secret scanner catching a credential in CI is useful; a pre-commit hook catching it before the commit is better, because a key that reached CI is already in the history.

Notes

Shift-left is the theme: a defect caught by a pre-commit hook costs seconds, in CI a pipeline run, in staging a deployment, in production an outage. The pipeline is the backstop; the hook is the earlier and cheaper catch, and a secret that reaches either the history or CI must be rotated regardless.