Reliability & Resilience Essential

Exponential Backoff with Jitter

Progressively spacing retry attempts to avoid thundering herd and let overwhelmed systems recover.

⏱ 9 min read

What it is

Exponential backoff is a delay strategy for retry loops where the wait interval between successive attempts grows exponentially with the attempt number: delay = base * 2^attempt. A base of 100 ms yields waits of roughly 100 ms, 200 ms, 400 ms, 800 ms before the fourth attempt. A maximum cap (max_delay) prevents the interval from growing indefinitely — typical caps range from 30 seconds to 2 minutes depending on the operation's acceptable latency budget.

Jitter adds a random component to each backoff interval so that many clients that all started failing at the same moment do not retry at the exact same time. Without jitter, exponential backoff still produces synchronized retry bursts: all 1,000 callers that failed at T=0 will retry at T=100 ms, T=200 ms, etc., in lockstep. Jitter desynchronises these retries, spreading load and giving the target system a better chance of recovering before the next wave.

Why it exists

The thundering herd problem is a well-documented failure mode where a large number of clients simultaneously attempt to reconnect or retry after a shared failure, overwhelming the recovering service and preventing it from stabilising. Classic examples include all application servers reconnecting to a database that just restarted, or all mobile clients retrying API calls after a brief server outage. Without exponential backoff, fixed-interval retries keep the server continuously saturated; without jitter, even exponential backoff produces synchronised bursts that cause the server to oscillate between brief recovery windows and overload spikes.

The AWS SDK blog post "Exponential Backoff and Jitter" (2015) by Marc Brooker popularised several concrete jitter algorithms and demonstrated through simulation that full jitter (fully randomised interval) and decorrelated jitter produce the least contention under high concurrency, while naive exponential without jitter performs no better than fixed intervals at scale.

When to use

Any retry loop where multiple callers may be retrying against the same downstream target simultaneously.
Reconnection logic for persistent connections (WebSocket, database connection pools, message broker consumers).
Scheduled job rescheduling after failure — jobs that fail should not all reschedule at the same next-minute boundary.
Cloud SDK client configuration — AWS, GCP, and Azure SDKs all support configuring custom backoff strategies.
Mobile and IoT clients reconnecting after network loss — especially important when thousands of devices lose connectivity simultaneously (e.g., a cellular base station restarts).

When not to use

Interactive user-facing operations where even 200 ms of added latency is unacceptable — fail fast and surface the error instead.
Operations with a total latency budget smaller than the first backoff interval — use immediate retry (attempt once more) or fail fast.
Background batch work where a dead-letter queue provides better recovery semantics than in-process retries with long backoffs.
Situations where the downstream system is completely down for an extended period — a circuit breaker should open and stop all retry attempts, not rely on exponential backoff alone.

Typical architecture

Backoff Strategy Comparison — 4 Clients, 3 Retry Attempts
══════════════════════════════════════════════════════════

Fixed Interval (100ms) — all clients fire in lockstep
Time(ms):  0    100   200   300   400
Client A:  ×    ×     ×     ×     ✓
Client B:  ×    ×     ×     ×     ✓
Client C:  ×    ×     ×     ×     ✓
Client D:  ×    ×     ×     ×     ✓
           ↑ server hit by 4 simultaneous retries every 100ms

Exponential No Jitter — still synchronised
Time(ms):  0   100   300   700  1500
Client A:  ×    ×     ×     ×    ✓
Client B:  ×    ×     ×     ×    ✓
Client C:  ×    ×     ×     ×    ✓
           ↑ all arrive at same checkpoints

Full Jitter: sleep = random(0, min(cap, base * 2^attempt))
Time(ms):  0   ~80  ~230  ~650
Client A:  ×    ×     ×     ✓
Client B:  ×         ×      ×    ✓
Client C:  ×    ×          ×     ✓
Client D:  ×         ×           ✓
           ↑ spread across window — server recovers

Decorrelated Jitter (AWS):
  sleep = min(cap, random(base, prev_sleep * 3))
  - Further desynchronises by using previous sleep as seed

Pros and cons

Pros

Allows recovering services to stabilise by reducing incoming request rate over time.
Jitter eliminates synchronised retry bursts, the primary cause of thundering herd.
Trivial to implement — a few lines of code in any language; most SDK libraries include it.
Cap prevents runaway latency accumulation in long retry chains.
Composable: can be embedded in any retry policy, reconnect loop, or job scheduler.

Cons

Full jitter increases average wait time compared to deterministic backoff, which may be unacceptable for time-sensitive operations.
Does not prevent retry storms on its own — a circuit breaker is still needed for prolonged outages.
Random intervals make latency profiles harder to reason about and test deterministically.
Backoff cap must be tuned per-use-case — a cap that is too low re-introduces synchronisation; too high adds excessive latency.

Implementation notes

The three most practical jitter algorithms are: full jitter (sleep = random(0, min(cap, base * 2^n))), which is best for minimising server-side load; equal jitter (sleep = min(cap, base * 2^n) / 2 + random(0, min(cap, base * 2^n) / 2)), which guarantees a minimum wait while still randomising; and decorrelated jitter (sleep = min(cap, random(base, prev_sleep * 3))), which is the AWS SDK default and produces the best empirical throughput under high concurrency.

Always cap the backoff with a max_delay to prevent sleeps growing to minutes. A reasonable starting point is base=100ms, cap=30s for microservice HTTP calls and base=1s, cap=5m for background job rescheduling. Respect Retry-After headers on HTTP 429/503 responses by using the server-provided interval as an override to your calculated backoff. Most retry libraries (Resilience4j, Polly, tenacity, go-retry) support custom backoff factories where you can plug in these algorithms directly.

Common failure modes

Missing jitter: pure exponential backoff without randomisation still produces synchronised waves — the single most common implementation mistake.
No cap on max_delay: after enough failures, retries space out to hours, making the system effectively unresponsive without an operator noticing.
Ignoring Retry-After: calculating a local backoff while the server is providing an explicit retry hint, leading to either too-fast or too-slow retries.
Applying backoff to non-retryable errors: sleeping before re-failing on a 400 Bad Request wastes time without benefit.
Backoff without circuit breaker: exponential backoff still allows retries to continue indefinitely during a prolonged outage, just at longer intervals.

Decision checklist

Have you chosen a jitter algorithm (full, equal, or decorrelated) appropriate for your concurrency level?
Is a max_delay cap configured to bound the maximum wait between retries?
Does your implementation respect Retry-After headers from rate-limited upstream services?
Is backoff only applied to retryable error categories, not to permanent failures?
Is a circuit breaker combined with this backoff to terminate retries during sustained outages?
Are the backoff parameters (base, cap, max_attempts) configurable per-dependency rather than hardcoded globally?

Example use cases

AWS SDK default retry strategy uses decorrelated jitter; all AWS service calls from the SDK benefit automatically without any configuration.
A Kubernetes pod's database connection pool reconnects with full jitter after the primary PostgreSQL instance fails over, avoiding a stampede from all pods reconnecting simultaneously.
A mobile app retrying a failed API request after a 503 with decorrelated jitter so that millions of devices coming back online after an outage do not re-DDoS the backend.
A gRPC client configured with Resilience4j's ExponentialRandomBackoffConfig for calls to a rate-limited ML inference service.

Retry Pattern — the outer pattern that backoff decorates.
Circuit Breaker — the complementary pattern that stops all retries once failure rates persist.
Load Shedding — the server-side complement to client-side backoff.