Make retries duplicate the work, then make them safe

short · 45 min · Objective 5.4

Task

At-least-once delivery plus retries means duplicates, and duplicates corrupt any non-idempotent operation. Cause it with a counter, then fix it with an idempotency key.

Steps

  1. Run a queue and a consumer that increments a stored total by the amount in each message. Send ten messages totalling a known value and confirm the total is correct.
  2. Set the visibility timeout SHORTER than the consumer's processing time. Send the ten messages again and record the resulting total in lab/queue/dupes.md -- it will overshoot, and by how much tells you how many were redelivered.
  3. Fix it the wrong way first: lengthen the visibility timeout. Record that it works and state why it is not a fix -- what happens when a single message is unusually slow?
  4. Fix it properly: give each message an idempotency key, have the consumer record processed keys, and ignore repeats. Re-run with the short timeout and confirm the total is correct despite redelivery.
  5. Add a dead-letter path: introduce one message the consumer always fails on, confirm it is moved aside after a retry limit rather than blocking the queue, and record the depth alert you would configure.

Verify

grep -Eci 'visibility timeout' lab/queue/dupes.md
grep -Ec '[0-9]+' lab/queue/dupes.md
grep -Eci 'idempotency key' lab/queue/dupes.md
grep -Eci 'dead.letter' lab/queue/dupes.md

All four non-zero, with an overshoot recorded before the fix and a correct total after. Without the overshoot the visibility timeout was never actually shorter than processing.

This is an independent study companion for CompTIA Cloud+ CV0-004 and is not produced by or endorsed by CompTIA.