Replicate to a recovery site asynchronously and measure the lag

short · 45 min · Objective 4.4

Task

Set up asynchronous replication of a data folder from a "primary site" server to a "recovery site" server, write continuously on the primary, and measure how far behind the recovery copy is at any moment. That lag is the data you would lose if the primary failed now -- your real RPO, as opposed to the one written in a plan.

Steps

  1. On lin-srv, start a writer that creates a file named with the current Unix time in /srv/primary every five seconds.
  2. Schedule rsync -a --delete /srv/primary/ lin-b:/srv/replica/ every minute. Save the crontab line to lab/replica/cron.txt.
  3. Every minute for ten minutes, record the newest file name on each side in lab/replica/lag.csv with header sample_time,primary_newest,replica_newest.
  4. Power off lin-srv without warning, and record the newest file on the replica and the last file the writer created (from the primary's disk after restart) in lab/replica/loss.txt as lost_seconds: <n>.
  5. Record in lab/replica/sync-vs-async.txt what synchronous replication would change about the loss, and what it would cost.

Verify

These checks run in a POSIX shell: Terminal on macOS or Linux, and on Windows Git Bash (it comes with Git for Windows) or WSL. A stock Windows PowerShell or Command Prompt has no awk or grep, so there the first line fails.

grep -Ec 'rsync' lab/replica/cron.txt
awk -F, 'NR>1 {l=$2-$3; if (l>m) m=l; s+=l; n++} END {print "max lag "m" s, mean "int(s/n)" s over "n" readings"}' lab/replica/lag.csv
grep -E '^lost_seconds: [0-9]+' lab/replica/loss.txt
grep -Eic 'zero|latency|distance|slower|every write' lab/replica/sync-vs-async.txt

The lag is up to about a minute and never negative, and the unplanned failure lost a number of seconds of writes within that range. Synchronous replication would have lost none, at the cost of every write waiting for the recovery site to confirm it -- practical only over short distances.

Notes

Replication copies deletions and corruption as faithfully as good data: the --delete flag here mirrors a mistaken deletion to the recovery site within a minute. That is why a recovery site does not replace backups.

This is an independent study companion for CompTIA Server+ SK0-005 and is not produced by or endorsed by CompTIA.