Distributed Systems Consistency

Quorum Reads & Writes

W + R > N guarantees overlap between read and write sets — tunable consistency, sloppy quorums, hinted handoff, read repair, and anti-entropy in Cassandra and DynamoDB.

⏱ 10 min read

What it is

A quorum is the minimum number of nodes that must respond to a read or write operation for it to be considered successful. In a system with N replicas, a write quorum W and read quorum R are chosen such that W + R > N, guaranteeing that every read set and every write set share at least one common replica. This overlap ensures that a read always includes at least one replica with the most recent write, providing a configurable consistency guarantee without requiring a single central coordinator.

Why it exists

Strong consistency (all replicas must confirm) maximises durability but reduces availability — any single node failure makes writes impossible. Eventual consistency (write to one, propagate later) maximises availability but exposes stale reads. Quorums offer a spectrum between these extremes: by tuning W and R for a fixed N, operators can favour read performance (low R, high W), write performance (low W, high R), or balance both (W = R = (N+1)/2 for the majority quorum). The W + R > N rule is what guarantees freshness when balance is chosen.

When to use

Leaderless replication systems (Cassandra, DynamoDB, Riak) where no single primary handles all writes.
When you need tunable consistency to trade off read vs. write latency.
When you can tolerate some staleness (eventual consistency) but want an option for strong reads.
High-availability scenarios where majority consensus is preferred over strict unanimity.

When not to use

Strict serialisability with concurrent writes requires coordinator-based approaches (e.g., Raft), not just quorums.
Sloppy quorums must be avoided when strong consistency is mandatory — they trade consistency for availability.
Very small clusters (N=2) make quorum configurations degenerate.

Typical architecture

Quorum Configuration (N=3)
═══════════════════════════
Strong consistency:   W=2, R=2 → W+R=4 > N=3  ✓
High write speed:     W=1, R=3 → reads always hit all nodes
High read speed:      W=3, R=1 → writes must hit all nodes
No consistency:       W=1, R=1 → W+R=2 ≤ N   ✗

Write path (W=2, N=3):
  Client──►Coordinator──►Replica1 (ACK)
                       ──►Replica2 (ACK)  → W=2 reached → OK
                       ──►Replica3 (slow) → ignored for now

Read path (R=2, N=3):
  Client──►Coordinator──►Replica1 (v=5)
                       ──►Replica2 (v=5)  → R=2 reached → OK
           Read repair: Replica3 (v=4) is stale → update async

Sloppy Quorum (DynamoDB):
  If target replicas are unreachable, accept writes on any
  available node ("hinted handoff"). Hint stored with data.
  When original node recovers, hints are replayed → eventually
  all replicas converge. Improves availability; breaks
  strict quorum overlap guarantee temporarily.

Pros and cons

Pros

Tunable consistency — operators choose the right W/R trade-off per use case.
No single point of failure — any node can coordinate reads and writes.
Read repair and anti-entropy converge replicas without coordinator overhead.
Sloppy quorums dramatically increase write availability during partial failures.

Cons

W+R > N does not guarantee linearisability with concurrent writes — only strong eventual consistency.
Sloppy quorums sacrifice the freshness guarantee of strict quorums.
Read repair adds read latency (waiting for all R replicas before responding).
Anti-entropy background processes add network and disk I/O overhead.

Implementation notes

Cassandra consistency levels map directly to quorum concepts: ONE sets W=1 or R=1; QUORUM sets W or R to (N/2)+1; ALL requires all replicas. Cassandra additionally supports LOCAL_QUORUM which applies quorum within a single data centre only, avoiding cross-DC latency for writes while accepting cross-DC staleness until asynchronous replication catches up.

Read repair happens in two modes: synchronous read repair (the coordinator waits for all R responses, upgrades stale replicas before returning to the client — increases latency but improves consistency) and background read repair (stale replicas are updated asynchronously after the response is sent). Cassandra uses a read_repair_chance setting to control background repair frequency.

Anti-entropy (Merkle tree-based repair in Cassandra's nodetool repair) compares hash trees of data ranges between replicas. Differing subtrees identify divergent data without transmitting all data. Repair should be scheduled regularly (especially in write-heavy clusters) to prevent unbounded divergence.

Common failure modes

Stale reads despite W+R > N: clock skew or concurrent writes can produce read results that appear stale even with quorum configuration — quorum guarantees overlap, not linearisability.
Hinted handoff storm: if many nodes go offline simultaneously, a large backlog of hinted handoffs accumulates; when nodes recover they can be overwhelmed by replaying all hints.
Anti-entropy neglect: skipping regular repair operations leads to replica divergence, especially after node replacements or network partitions.
Tombstone accumulation in Cassandra: deletes create tombstones that must be collected via compaction within the gc_grace_seconds window; missed repair lets ghost data resurface after tombstone expiry.

Decision checklist

What are your consistency requirements — eventual, strong eventual, or linearisable?
What is your replication factor N? Are W and R values tuned appropriately?
Is sloppy quorum acceptable, or do you need strict quorum overlap at all times?
Is read repair enabled? Synchronous or background? What is the latency impact?
Is anti-entropy repair scheduled and monitored? What is the repair interval?

Example use cases

Cassandra with LOCAL_QUORUM reads and writes for a globally replicated user profile store accepting some cross-region staleness.
Amazon DynamoDB using sloppy quorums internally to maintain high write availability during AZ failures.
Riak using W=1/R=1 for highest throughput counters and W=3/R=3 for critical configuration data — same cluster, different consistency per bucket type.
Voting systems requiring quorum agreement before committing ballot records to prevent double-voting.

Vector Clocks — used during quorum reads to identify and resolve divergent replica versions.
Consensus Algorithms — Raft/Paxos use quorum-based voting internally to achieve linearisable consensus.
CAP Theorem — quorum configuration determines where a system sits in the CA/AP spectrum during partition.