Infrastructure as code, cloud APIs and event triggers

Why this matters

Lesson fourteen covered securing the pipeline that runs infrastructure as code. This lesson is about using the same machinery as a security control — which is the engineering half of the same subject and where most of domain 3's value in a real estate comes from.

Three capabilities matter. Declaring infrastructure in code makes a security configuration reviewable and enforceable rather than remembered. Cloud APIs make the estate queryable, which is what turns "are we compliant" from a survey into a program. And event triggers make response immediate: a configuration that drifts can be corrected in seconds rather than at the next assessment cycle.

The risks arrive with the same machinery. Automation that reacts to events can react to its own effects, can be triggered by an attacker who can generate the event, and can amplify a small mistake across an estate faster than anyone can intervene. Those failure modes are the examinable content.

The lesson

Declarative and imperative approaches, and where each fits a security control

The distinction is about what you write down.

Declarative code states the desired end state: this storage account exists, encrypted, private, with these tags. The tool computes the difference between that and reality and makes the changes required. Running it twice changes nothing the second time, so it is idempotent by construction.

Imperative code states the steps: create the account, then enable encryption, then set the policy. The author owns the ordering and the "is it already done" logic, which means running it twice may do the wrong thing unless the author handled it.

For security controls the preference is strongly declarative, and the reasons are specific:

• The code is the standard. A declarative definition of a compliant resource is the configuration standard from lesson one, in executable form, so the two cannot drift apart.
• Drift is computable. Because the tool knows the desired state, it can report what has diverged — which is the drift detection lesson eighteen called for, available for free rather than built.
• Review is meaningful. A diff of desired state is readable by a reviewer. A diff of procedural steps requires mentally executing them.
• Re-running is a control. Applying the definition on a schedule continuously re-asserts the intended configuration.

Imperative remains right for genuinely sequential operations — a migration, a key rotation with a cutover, an incident response action — where the order is the substance and there is no meaningful "desired state" to declare.

The practical estate uses both, and the examinable judgement is placement: declare the steady state, script the transitions. A scenario describing configuration maintained by a procedural script is usually describing why the estate drifts.

Cloud APIs: authentication, least-privilege roles and pagination that hides findings

Everything in a cloud estate is an API call, which makes the API the security engineer's primary instrument — for inventory, assessment, evidence collection and response.

Authentication follows lesson twenty-six's ordering: workload identity first, short-lived tokens next, long-lived keys last and only with compensating controls. The addition specific to APIs is that an automation identity used for assessment should be read-only, and separate from any identity used for remediation. That separation means a compromise of the widely deployed, frequently used assessment credential cannot change anything — and it is routinely skipped because one role is simpler.

Least privilege for API automation has a practical method: start from nothing, run the automation, read the access-denied errors, and grant exactly what they name. Starting from a broad role and narrowing later does not happen, because the automation works and nobody returns to it.

Pagination deserves its own treatment because of how it fails. Most cloud APIs return results in pages. Code that reads the first page and stops works perfectly in development, where results fit in one page, and silently truncates in production, where they do not. The consequences in a security context are specific and bad:

• An inventory that lists the first 100 resources of 4,000, so 97% of the estate is outside every scope derived from it.
• An assessment that reports zero findings because the findings were on page three.
• An evidence collection that is complete-looking and covers a fraction.

Every one of those produces a confident, wrong, reassuring answer — the worst possible failure shape for a control, and precisely the coverage problem lesson five described arriving through a different door.

The defences are mechanical: always follow continuation tokens to exhaustion; assert on counts, comparing against an independently known total where one exists; and alert when a result set changes size implausibly, since a scope that drops from 4,000 to 100 overnight is a bug, not an improvement.

Throttling is the related failure. APIs rate-limit, and code that does not back off and retry loses data — usually quietly, in the middle of a loop. Retry with exponential backoff, and treat an unrecoverable throttle as an error that fails the run rather than as a gap in the results.

Event triggers: acting on a configuration change the moment it happens

Event-driven automation is what makes cloud security controls immediate. The platform emits an event for every change; a function subscribes; the function evaluates and acts.

What it is genuinely good for:

• Auto-remediation of unambiguous misconfigurations — a storage container made public, a security group opened to the internet, encryption disabled, logging switched off. The exposure window shrinks from days to seconds.
• Enrichment and routing. A finding is created; the function attaches asset ownership from the CMDB, classification, and business context, then routes it to the right team. This is unglamorous and it is what makes findings actionable.
• Real-time policy checks in places policy-as-code cannot reach, such as resources created outside the pipeline.
• Signal generation — recording that something changed, for the detection pipeline in domain 4.

The design decision that matters is remediate or alert. Automatic remediation is right when the condition is unambiguous, the correction is unambiguous, and being wrong is cheap. It is wrong when the resource might be public for a legitimate reason, when the correction could cause an outage, or when the change might be an authorised administrator mid-task. The defensible default is a small set of auto-remediated conditions with an explicit exemption mechanism, and alerting for everything else.

Two further properties. Auto-remediation should record what it did and why, to the same standard as lesson twenty-six requires of any script — an unexplained reversion of an engineer's change destroys trust in the control faster than any outage. And it should notify the owner, because a control that silently undoes people's work teaches them to work around it.

Guarding the trigger itself against loops, storms and self-inflicted denial of service

The specific failure modes of event-driven automation, all of which have caused real incidents.

Loops. A function triggered by a configuration change that itself makes a configuration change re-triggers itself. Without a guard this runs until a quota stops it, generating cost, noise and platform throttling that affects unrelated services. The guards: check whether the change was made by the automation's own identity and exit if so; use a distinct identity so that check is reliable; and set a concurrency limit as a backstop that bounds the damage regardless.

Storms. A single upstream action can generate thousands of events — a bulk deployment, a mass tagging operation, an account-wide change. Automation sized for a steady trickle will either fall over or overwhelm whatever it calls. The guards: concurrency limits, batching, and a circuit breaker that stops the automation and raises an alert when volume exceeds a threshold, on the principle that an unprecedented event rate is more likely a fault than a genuine need to act ten thousand times.

Self-inflicted denial of service. Remediation automation that disables or deletes resources can, given a bad rule or a storm, do so across the estate. The guards are the same as lesson twenty-six's: a dry-run mode that is exercised regularly, a cap on actions per run, and a hard refusal to act on target sets above a threshold without human confirmation.

Attacker-triggered automation. If an attacker can cause the event, they can cause the action. Automation that shuts down an instance on an alert can be weaponised into an outage by generating alerts. This is why containment actions with availability consequences are usually gated behind a human — and it is the same reasoning as lesson twenty-eight applies to playbooks.

Failure of the automation itself. A function that stops running looks identical to a clean estate: no alerts, no remediations, nothing. Monitor the automation's own health — last successful execution, error rate, invocation count — and alert on silence. This is the same freshness requirement lesson nineteen placed on evidence, and silence is the failure signature that all of these controls share.

Drift between code and reality, and treating the console as the exception

The closing discipline, and the one that decides whether infrastructure as code is a control or a deployment convenience.

Drift is the gap between what the code declares and what exists. It arises from console changes during an incident, from a script acting outside the code, from platform-side changes, and from manual fixes that were never fed back.

Why it matters more for security than for operations: the code is the reviewed artefact. If reality differs from the code, then the review, the policy-as-code evaluation and the approval all applied to something that is not running. The assurance chain from lesson fourteen breaks silently at that point.

Handling it:

• Detect continuously, by comparing declared and actual state on a schedule and on change events. Drift on a security-relevant attribute — a network rule, an encryption setting, an identity policy — is an alert, not a report line.
• Decide per instance: correct the environment back to the code, or update the code because the change was right. Both are legitimate; leaving it unresolved is not, because unreconciled drift accumulates until the code is fiction.
• Make the console the exception, with a path. Emergency console access is necessary and should be available, time-limited, logged, and followed by a mandatory reconciliation. Pretending nobody will use the console produces undocumented drift; providing a break-glass path with a reconciliation obligation produces documented drift that gets closed.
• Prevent casual console change with the guardrails from lesson eighteen, so that routine modification goes through the reviewed route and the console is genuinely exceptional.

The measure worth reporting: the number of resources currently drifted, and the age of the oldest drift. Both small numbers, both trend-able, and together they say whether the code still describes the estate — which is the precondition for every other control in this lesson meaning anything.