Disaster recovery: RPO, RTO and the site you fail over to

Why this matters

Backups answer "can we get the data back". Disaster recovery answers "can we get the service back, and how quickly". They are different questions with different costs, and conflating them is how an organisation ends up with excellent backups and a two-week outage.

The exam tests two things here: whether you can use RPO and RTO correctly to select an architecture, and whether you know what each architecture costs. That second half is what makes the first half a real decision — everyone wants instant recovery until they see the bill for it.

The lesson

RPO and RTO defined by the business, then costed by you

Two numbers, and the distinction is examined constantly.

• Recovery Point Objective (RPO) — how much data you can afford to lose, measured in time. An RPO of one hour means losing up to an hour of work. RPO drives backup and replication frequency.
• Recovery Time Objective (RTO) — how long the service may be down. An RTO of four hours means being back within four hours. RTO drives the recovery architecture.

The memory hook: RPO looks backwards (how far back does the last good copy sit), RTO looks forwards (how long until we are running again).

Two related terms appear alongside them:

• MTD / MTO (maximum tolerable downtime) — the point past which the business suffers unacceptable harm. RTO must be shorter.
• MTTR (mean time to repair) — the observed average, as opposed to the target. Comparing MTTR against RTO tells you whether the target is real.

The essential discipline is that the business sets these numbers and you cost them. Asked in the abstract, every system owner wants zero and zero. The conversation becomes productive when each objective is priced: near-zero RPO means synchronous replication and its write-latency penalty; near-zero RTO means running a second environment continuously.

Set them per workload, not per organisation. The payment system and the internal wiki do not deserve the same architecture, and a blanket standard means either overspending on the wiki or underprotecting payments. Objective 4.4's data classification is the input that makes this tractable.

Both numbers should also come from a business impact analysis — what actually happens per hour of downtime, in revenue, regulatory exposure, contractual penalty and reputation. That is what justifies the spend.

Backup-restore, pilot light, warm standby and active-active as four price points

Four standard patterns, in increasing cost and decreasing recovery time. This mapping is the most reliably examined content in the objective.

Backup and restore. Backups exist elsewhere; infrastructure is built when needed. RTO hours to days, RPO the backup interval. Cheapest by far — you pay only for storage. Right for systems that can tolerate a long outage. With infrastructure as code the rebuild is far faster than it used to be, which has made this viable for more systems than before.

Pilot light. The core — usually the database, replicating continuously — is always running in the recovery region, with everything else defined but not running. On disaster you start the rest and scale up. RTO tens of minutes to hours, RPO seconds to minutes. Cost is the replicating data tier plus storage.

Warm standby. A complete but under-sized copy of the environment runs continuously. On disaster you scale it to full size and redirect. RTO minutes, RPO seconds. Cost is a meaningful fraction of production, continuously. The advantage beyond speed: because it is running, it is continuously proven to work, which a pilot light is not.

Active-active / multi-site. Full capacity in two or more regions, both serving traffic. RTO near zero, RPO near zero. Most expensive — more than double production, because of cross-region data synchronisation — and the most complex, since it requires solving data consistency across regions and the application must tolerate it. Right for systems where downtime is catastrophic.

The examinable move is to read the scenario's RTO/RPO and pick the cheapest pattern that meets it. Picking a more expensive one "to be safe" is wrong in an exam and wrong in practice.

Zones and regions, and the failure domain each one protects against

Getting this right determines whether an architecture survives the thing it was designed for.

• Availability zone. An isolated location within a region — separate power, cooling and networking, close enough for low-latency synchronous replication. Spreading across zones protects against a data centre failure: a power event, a flood, a fire.
• Region. A geographically separate location. Protects against a regional event — a natural disaster, a large-scale provider failure, a regulatory or jurisdictional problem.
• Account or subscription. Not geographic, and a genuine failure domain anyway: it is the boundary crossed by a credential compromise, a mistaken bulk deletion, or a ransomware event. Separating backups by account protects against the failure modes that no amount of geography addresses.

The practical distinction: multi-zone is high availability; multi-region is disaster recovery. Multi-zone is relatively cheap, often automatic in managed services, and should be the default for anything production. Multi-region costs substantially more — in replication, in duplicated infrastructure, and in cross-region data transfer — and is justified by an RTO that a regional recovery cannot meet.

Two things frequently missed:

• Control-plane dependencies. A recovery plan that requires the failed region's management APIs to work is not a plan. Keep the recovery region's ability to operate independent.
• The DR region's quotas. Capacity limits are per-region (objective 2.4), so a region that has only ever run a pilot light may have quotas far below full production scale — and a request to raise them takes days. This is a classic cause of a DR test failing for a reason nobody anticipated.

Failover and failback, and why failback is the half nobody rehearses

Failover is switching to the recovery environment. It gets the attention, the automation and the testing.

Failback is returning to the primary once it is healthy, and it is usually harder:

• The data has moved on. The recovery site has been taking writes, so the primary is stale. Returning means replicating in the opposite direction — and the replication was probably only ever configured one way.
• It needs a second outage, planned, to cut back.
• Reverse replication may not exist as a configured capability, and building it under pressure is exactly the wrong time.
• Configuration drift between sites accumulated during the incident, as emergency fixes were made at the recovery site only.

A DR plan must therefore cover both directions, and the failback procedure deserves the same rehearsal. In some designs the right answer is not to fail back at all: promote the recovery site to primary and treat the old primary as the new recovery site. That avoids a second outage entirely, and it only works if the two sites are genuinely equivalent — which is an argument for symmetry in the design.

The mechanisms for switching traffic are worth naming: DNS with health checks (simple, subject to the caching caveats in objective 2.3), global load balancing (faster and cleaner), and manual redirection (slowest, most controlled). Automatic failover is attractive and carries the risk of split-brain — both sites believing they are primary and both accepting writes — which is why many designs deliberately keep a human decision in the loop.

The DR test that proves the runbook, and the runbook nobody has read since

An untested DR plan is a document, not a capability. The escalating ladder of tests:

1. Walkthrough / tabletop. The team talks through the plan. Cheap, finds missing steps, stale contacts and wrong assumptions. Do this first and often.
2. Component test. Fail over one piece — a database, a service — and measure.
3. Simulation. Execute the plan against the recovery environment without moving production traffic. Produces real timings.
4. Full failover. Actually move production. The only complete proof, and the only one that reveals the dependency nobody documented.
5. Unannounced test. The strongest and the one requiring most organisational confidence.

What tests reliably reveal, which is why they are worth the disruption:

• Runbooks referencing people who left, systems renamed, or consoles redesigned.
• Credentials nobody has, or that live only in the failed environment — the password manager hosted in the region that is down being the classic.
• Undocumented dependencies, especially on shared services in the primary region.
• Quota and capacity limits in the recovery region.
• Recovery times two to five times the estimate.

Keep the runbook executable: specific commands, specific identifiers, specific screens, written for someone tired and under pressure who may not be the person who wrote it. Store it outside the environment it recovers — a runbook readable only from the failed region is the sharpest version of this whole lesson's point — and date it, name its owner, and review it on a schedule.

What to take into the exam

• RPO = data loss tolerated (backwards). RTO = downtime tolerated (forwards). RPO drives replication frequency; RTO drives architecture. Set both per workload, from a business impact analysis.
• The four patterns by price: backup/restore → pilot light → warm standby → active-active. Pick the cheapest that meets the stated objectives.
• Multi-zone is high availability; multi-region is disaster recovery. The account is a failure domain too, and the one ransomware crosses.
• Check DR-region quotas — they are per-region and will fail a test.
• Failback is the unrehearsed half. Reverse replication usually does not exist; promoting the recovery site may beat failing back.
• Automatic failover risks split-brain.
• Store the runbook outside the environment it recovers, and test up the ladder from tabletop to full failover.