Message queues & pub/sub

Do it later

Some work doesn't need to finish before you reply to the user: sending an email, resizing an image, updating a search index, writing analytics. Put a message on a queue and reply right away; background workers take messages off the queue and do the work at their own pace.

• Decoupling: the producer doesn't need the consumer to be up, fast or even known.
• Burst absorption: a spike becomes a longer line instead of an outage.
• Retries: if a worker fails, the message is delivered again.

Queue vs pub/sub

Delivery guarantees

• At-least-once (the usual default): a message may be delivered twice, so consumers must be idempotent (processing twice has the same effect as once, for example by tracking a message ID).
• At-most-once: never duplicated, but may be lost.
• Exactly-once is expensive and only holds within specific systems; in practice, design for at-least-once plus idempotency.
• Messages that keep failing go to a dead-letter queue for inspection.

Try it

A write-heavy app with the primary database at about 88%. Turn on the async write queue: p95 drops because users no longer wait for the database to store their writes. Then, in the Traffic tab, set Reads to 30% and requests to2k. The primary can't keep up even on average, the backlog never drains, and the simulator flags writes that were acknowledged but never stored. A queue smooths bursts; it adds no database capacity.