Troubleshooting a deployment that will not deploy

Why this matters

Troubleshooting is 12% of CV0-004 and it is the domain most improved by having a method, because the questions are scenarios: symptoms are described and you choose the cause or the next step.

Deployment failures are a good place to start because their causes are a small, well-defined set. Unlike a performance problem, which can come from anywhere, a deployment that will not deploy has usually hit one of five things: a quota, a permission, a template fault, a dependency problem, or unavailable capacity. Knowing that list turns an unfamiliar error into a short checklist.

The lesson

Reading the deployment error properly before changing anything

The first discipline, and the one people skip under pressure: read the actual error. Cloud deployment errors are usually specific, and the instinct to start changing things discards the best evidence available.

What to extract before touching anything:

• The failing resource, by logical name and type. In a template deploying forty resources, one failed and the rest may have succeeded.
• The error code, which is usually more precise than the message.
• The phase. Did it fail during validation (before anything was created), during creation, or during rollback? A validation failure is a template problem; a creation failure is an environment problem.
• Whether a rollback occurred, and whether it completed.
• The timestamp, to correlate with other logs and other changes.

Then look in the right place. The deployment tool reports what the platform told it; the platform's activity or audit log records the underlying API call, its parameters and the exact rejection. That second view frequently contains detail the first summarises away — particularly for permission errors, where the audit log names the specific action and resource that was denied.

Two habits worth having: look at the first error, not the last. A single root failure cascades into many dependent failures, and the last message is usually a consequence. And do not re-run immediately. If the deployment is idempotent, a re-run is safe but proves nothing you did not already know; if it is not, it can make the state worse.

Quota, capacity and region availability as the causes that look like bugs

Three environmental causes that produce errors which look like faults in your template.

Quotas and service limits (objective 2.4). You have reached the maximum number of a resource type permitted in that region. Recognisable by errors mentioning limits, quotas or maximums — but sometimes reported only as a generic failure. Remember that quotas are per region and often per instance type, so identical code succeeds in one region and fails in another, and that terminated resources can still count against a quota until fully released. Soft limits can be raised by request, with lead time.

Insufficient capacity. The provider genuinely does not have that instance type available in that zone right now. This is not your quota and not your fault, and it surprises people who assume the cloud is infinite. It is most common with large or specialised instance types and during regional incidents. The remedies are to try a different zone, a different but equivalent instance type, or to use a flexible capacity request that accepts several types. Where capacity must be guaranteed, a capacity reservation is the answer.

Region or zone availability of a service. Not every service exists in every region, and new services roll out gradually. A template that works in one region fails in another because a service or a feature is simply not there. Check the provider's regional availability rather than assuming parity.

The reason these three matter is that they all produce environment-dependent failures — the same code succeeding here and failing there — which people instinctively diagnose as a code problem. A scenario where a deployment works in development and fails in production with no code difference is pointing at this group.

Permissions: the deployment identity that cannot create what the template asks for

The most common single cause, and it has a distinctive property: the error arrives partway through, because permissions are checked per action as each resource is created. So a deployment creates twelve resources successfully and fails on the thirteenth.

What to check, in order:

1. Which identity is deploying? The pipeline's role, a user, or a service principal. It is often not the identity of the person who ran it.
2. What exactly was denied? The audit log names the action and the resource.
3. Is it missing an allow, or hitting a deny? Remember objective 4.2's rule: an explicit deny always wins, and organisation policies and permission boundaries limit rather than grant. A permission that appears present in the role and still fails is very often being cut off above.
4. Is it a resource-based policy? The identity may be permitted, and the target resource's own policy may refuse it.
5. Is a condition failing? Policies restricted by region, tag, network or MFA fail in ways that look like a missing permission.

Two specific cases worth recognising:

• The pass-role problem. Creating a resource that itself assumes a role requires permission both to create the resource and to assign that role. Missing the second produces a confusing denial about a role rather than about the resource being created — and it is deliberate, because without it anyone who can create an instance could grant it administrative rights.
• Service-linked permissions. Some services need permission to act on your behalf, and a missing one fails at an unexpected moment.

The fix is to grant the specific missing permission, scoped to the specific resources — not to attach an administrative policy and move on, which is how estates accumulate the over-broad roles that objective 4.1 later reports as findings.

Template and dependency faults, including the circular reference

Faults in the deployment definition itself.

Syntax and schema errors. Caught at validation, before anything is created. Cheap, and the reason validation runs first in the ladder from objective 2.2.

Invalid values. A name that breaks the service's rules — storage account names must be globally unique and lowercase, bucket names cannot be reused immediately after deletion, some resources cannot be renamed at all. These fail at creation with service-specific messages.

Missing or wrong references. Referring to a resource or output that does not exist, usually after a refactor moved something.

Dependency ordering. Most declarative tools infer dependencies from references. The failures come from implicit dependencies the tool cannot see — objective 2.4's case where an instance's bootstrap expects a secret that nothing in the template connects it to. The signature is a deployment that fails intermittently, or succeeds on a re-run, because the timing happened to work. That signature is worth memorising: intermittent deployment failure is almost always an undeclared dependency.

Circular references. A depends on B and B depends on A. The tool cannot order them and refuses. Usually introduced by two resources referencing each other's identifiers — a security group pair, or a resource and the role that uses it. The fix is to break the cycle: use a separate rule resource rather than inline rules, use a predictable name instead of a reference, or split into two stacks with one-way dependency (objective 2.2).

Immutable property changes. Changing a property that cannot be updated in place forces a replacement. The deployment may succeed and destroy something you needed, which is why the plan output is read before every apply. Where the resource cannot be replaced either — because something depends on it — the deployment fails in a way that requires manual untangling.

Partial deployments, cleaning up, and re-running an idempotent template safely

When a deployment fails midway, something exists and something does not, and the recovery depends on what the tool did next.

Automatic rollback is offered by most provider-native tooling: on failure it deletes what it created, returning to the prior state. Clean, and it has two traps — rollback can itself fail, leaving a stuck state that needs manual intervention; and rollback deletes, so a resource created successfully and carrying data can be destroyed as part of cleaning up a failure elsewhere.

No rollback is the behaviour of most provider-neutral tooling: it stops, records what it created in state, and leaves it. Less dramatic, and recovery is simply to fix the problem and re-run, because the tool knows what exists and will create only what is missing. This is idempotence (objective 2.2) doing the work, and it is why declarative tooling is comfortable to operate.

Practical guidance:

• Fix the cause, then re-run. For an idempotent template this is safe and is the normal path.
• Check state matches reality first if anything was touched by hand during the incident. If they disagree, the next apply will act on a false picture — objective 2.2's drift problem at the worst moment.
• Watch for orphans. A resource created outside state — because the deployment died before recording it — is invisible to the tool and must be imported or deleted manually. It will also cause a name collision on the next run, which is often how it is discovered.
• Mind protected resources. Deletion protection on a database stops a rollback from destroying it, which is usually what you want and will make the rollback fail.
• Never fix it by hand and leave it. If you must intervene manually, reconcile the code and state afterwards, the same day.

The structural preventions are the ones from earlier domains: plan before apply, deploy to a throwaway environment first, and split stacks by blast radius so a failure in one cannot leave another half-built.

What to take into the exam

• Read the first error, not the last, and check the platform audit log for the precise denied action. Note the phase — validation means the template, creation means the environment.
• Quota, capacity and regional availability produce failures that depend on where you deploy, not on the code. Quotas are per-region and per-type; insufficient capacity is the provider's, not your quota.
• Permission failures arrive partway through. Check the deploying identity, explicit denies, boundaries and organisation policies that limit rather than grant, resource-based policies, conditions, and the pass-role case.
• Intermittent deployment failure, or success on re-run, means an undeclared implicit dependency.
• Circular references are broken by separate rule resources, predictable names, or splitting stacks.
• Automatic rollback can delete data and can itself get stuck. Without rollback, fix and re-run — but verify state matches reality first, and watch for orphans created outside state.