Build a pipeline that stops a bad change

applied · 45 min · Objective 4.1

Task

Build a CI pipeline that lints, tests, scans and only then deploys, and prove the property that justifies it: a change that fails any stage cannot reach the deploy step. Add a secret scanner and confirm it blocks a committed credential.

Steps

  1. Write a pipeline definition with ordered stages: lint, test, scan, deploy. Each stage fails the pipeline on a non-zero exit. The Verify drives it through ./run-pipeline.sh <branch>: with a plain shell pipeline, that script checks out the branch, runs the stages in order under set -e, and prints <stage>: ok as each one passes; with a real runner, make it trigger the pipeline for that branch and print the job log. Create the three branches the Verify names as you go: good (step 3), badtest (step 4) and secret (step 5).
  2. Add real checks: shellcheck/ansible-lint in lint, a unit test in test, gitleaks and a container scan in scan.
  3. Commit a good change and watch every stage pass and the deploy run.
  4. Commit a change that fails the test. Confirm the pipeline STOPS at test and the deploy stage never runs. This is the whole value.
  5. Commit a change introducing a hardcoded secret. Confirm gitleaks fails the scan stage and blocks the deploy -- and contrast a pre-commit hook, which would have stopped it reaching the history at all.
  6. Restrict deploy to the main branch with an only/if condition, and confirm a feature branch runs the checks but not the deploy.
  7. Inject a secret into the pipeline the right way -- a masked, protected variable -- and confirm it does not appear in the job log.

Verify

cd ~/lab-ci
# good change: deploy stage recorded
./run-pipeline.sh good 2>&1 | grep -qi 'deploy: ok' && echo "good change deploys"
# failing test: pipeline stops before deploy
./run-pipeline.sh badtest 2>&1 | tee out.txt; grep -qi 'deploy: ok' out.txt && echo "DEPLOYED despite failure -- wrong" || echo "stopped before deploy"
# committed secret: scan blocks it, and nothing deploys
./run-pipeline.sh secret 2>&1 | tee out2.txt; grep -qi 'leaks found' out2.txt && ! grep -qi 'deploy: ok' out2.txt && echo "scan blocked the secret"

Each check reads a stage's ok line, not the word "deploy" or "gitleaks": those appear in any run that lists its stages or names its tools, whether the stage passed or not.

The failing-test run NOT reaching deploy is the property the whole pipeline exists for: broken change in, no deployment out. The secret scanner catching a credential in CI is useful; a pre-commit hook catching it before the commit is better, because a key that reached CI is already in the history.

Notes

Shift-left is the theme: a defect caught by a pre-commit hook costs seconds, in CI a pipeline run, in staging a deployment, in production an outage. The pipeline is the backstop; the hook is the earlier and cheaper catch, and a secret that reaches either the history or CI must be rotated regardless.

This is an independent study companion for CompTIA Linux+ and is not produced by or endorsed by CompTIA.