Rate Limiting

What it is

Rate limiting controls the number of requests a client can make to an API or service within a given time window. When a client exceeds its allowed rate, subsequent requests are rejected with HTTP 429 Too Many Requests until the window resets or the client slows down. Rate limiting serves two distinct purposes: it protects the service from being overwhelmed by any single client (even a legitimate one experiencing a bug), and it provides fair resource sharing across all clients in a multi-tenant system.

Five primary algorithms implement rate limiting, each with different characteristics. Fixed window counter counts requests in discrete time windows (e.g., 0–60 seconds). Sliding window log tracks exact timestamps of every request for precise counting but uses significant memory. Sliding window counter approximates a sliding window using two fixed window counters and is the most practical distributed approach. Token bucket allows bursts up to a bucket capacity while enforcing a long-term average rate. Leaky bucket enforces a strictly uniform output rate by queuing and processing requests at a fixed rate.

Why it exists

Without rate limiting, a single misbehaving client can consume all available server resources, causing degraded performance or outright failures for all other clients. A buggy application that enters an infinite retry loop, a web scraper running at maximum speed, or a DDoS attack all present as sudden traffic spikes from specific sources. Rate limiting provides the first line of defense that caps per-client impact before it cascades to system-wide failures.

For public APIs with SLA tiers (free/basic/premium), rate limiting is the enforcement mechanism for those tiers. A free tier limited to 100 requests/minute cannot degrade service for a paid tier at 10,000 requests/minute because the application layer enforces the boundary. Rate limiting also protects downstream dependencies: a database that handles 5,000 queries/second will fail at 50,000; rate limiting the API tier ensures that no combination of clients can drive more than the safe maximum throughput.

When to use

• Public APIs exposed to arbitrary clients — bots, scrapers, misconfigured applications, and DDoS attacks are all real threats.
• Multi-tenant SaaS platforms where a single tenant's traffic must not starve other tenants.
• APIs with tiered subscription plans (free vs. paid) where the tier limits are business-critical.
• Any API that calls a downstream resource with known capacity limits (external APIs, databases, AI inference endpoints).
• Login and authentication endpoints to prevent brute-force and credential stuffing attacks.

When not to use

• Internal service-to-service calls within a trusted network where the caller is always known and controlled — circuit breakers and bulkheads are more appropriate.
• Single-tenant systems with a single known client where per-client rate limiting provides no benefit.
• When server-side throttling (request queuing and backpressure) is more appropriate than rejection — rate limiting rejects excess requests, while throttling queues them.

Typical architecture

  Algorithm Comparison:

  Fixed Window (simple, has boundary burst problem):
  Window [0s-60s]: count=95/100  [61s-120s]: count=0/100
  Burst attack: 100 req at t=59s + 100 req at t=61s = 200 req in 2s

  Sliding Window Counter (best practical choice):
  current_count = prev_window_count × (1 - elapsed/window) + curr_window_count
  → approximates true sliding window with only 2 counters

  Token Bucket (allows bursts):
  bucket_size = 100 tokens   (max burst)
  refill_rate = 10 tokens/s  (sustained rate)
  On request: if tokens > 0: tokens--; allow else: reject
  → Good for: API clients that need burst allowance

  Leaky Bucket (uniform output):
  queue incoming requests; process at fixed rate = 10 req/s
  if queue full: reject
  → Good for: protecting downstream with strict rate requirement

  Distributed Rate Limiting (Redis):
  -- Sliding Window Counter in Redis (Lua script for atomicity)
  local key = "rl:" .. user_id
  local now = tonumber(ARGV[1])
  local window = 60  -- seconds
  local limit = 100
  redis.call("ZADD", key, now, now)          -- add current timestamp
  redis.call("ZREMRANGEBYSCORE", key, 0, now - window * 1000)  -- remove old
  local count = redis.call("ZCARD", key)
  redis.call("EXPIRE", key, window)
  return count <= limit

  HTTP Response (RFC 6585 + industry conventions):
  HTTP/1.1 429 Too Many Requests
  Retry-After: 30
  X-RateLimit-Limit: 100
  X-RateLimit-Remaining: 0
  X-RateLimit-Reset: 1699123456
          

Pros and cons

Implementation notes

Algorithm selection: For most APIs, the sliding window counter implemented with Redis is the best choice: it approximates a true sliding window, uses only 2 counters (O(1) memory), and is accurate within ~0.5% of the ideal algorithm. Use token bucket when your API clients have legitimately bursty access patterns (e.g., an analytics job that queries 500 records at startup then only 5 records/minute ongoing). Use leaky bucket when protecting a downstream with a strict rate requirement (e.g., an external AI API with a hard 10 req/s limit). Avoid sliding window log in production — O(requests) memory does not scale.

Rate limit keys: Choose the right client identifier. API key rate limiting is the most precise — it tracks by authenticated client regardless of IP. IP-based rate limiting is appropriate for unauthenticated endpoints (login) but is unreliable for corporate or mobile clients behind NAT. User-based rate limiting (authenticated user ID) is best for per-user fairness in SaaS products. Implement multiple dimensions: per-IP for unauthenticated, per-API-key for authenticated, and global limits per endpoint as a last resort against total overload. Use separate Redis key namespaces per dimension: rl:ip:{ip}, rl:key:{apikey}:{endpoint}.

Common failure modes

• Redis unavailability: If the rate limiting Redis cluster goes down, should the API allow all traffic (fail open) or reject all traffic (fail closed)? Fail open is safer for availability but vulnerable to abuse during outages. Implement Redis availability monitoring and fail-open with alerting.
• Retry storm amplification: Clients receive 429, retry immediately without backoff, and generate more requests. Always respond with Retry-After and document exponential backoff requirements. Implement progressive penalties for clients that don't respect Retry-After.
• Shared IP rate limit starvation: Corporate users all behind one NAT IP all share an IP-based rate limit. An automated process at one user's desk exhausts the IP limit for 500 colleagues. Use API key authentication to provide per-user limits rather than per-IP for authenticated endpoints.
• Clock skew in distributed counters: If two API instances have clock skew of 1 second and use a 1-second fixed window, they may be in different windows simultaneously. Use server-side Unix timestamps from Redis (REDIS TIME command) to avoid this.

Decision checklist

• Rate limit algorithm has been chosen based on burst tolerance requirements: token bucket for bursty clients, sliding window counter for most use cases.
• Client identifier is appropriate: API key or user ID for authenticated endpoints, IP-based only for unauthenticated (login/registration) endpoints.
• HTTP 429 responses include Retry-After, X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset headers.
• Fail-open behavior on Redis unavailability is acceptable for the use case; monitoring and alerting is in place.
• Client SDK or documentation provides retry logic with exponential backoff for 429 responses.
• Separate rate limits are defined per endpoint for expensive operations (e.g., 100/min for GET, 20/min for POST that triggers heavy computation).

Example use cases

• SaaS API tiers: A REST API enforces 100 requests/min for free tier API keys, 1,000/min for basic, and 10,000/min for enterprise. Each tier's limit is checked in Redis using the API key as the rate limit key. Exceeding the limit returns 429 with Retry-After: 60.
• Login endpoint protection: A login endpoint applies IP-based rate limiting of 10 attempts per 15 minutes. After 5 failed attempts, the account is also temporarily locked for that IP. This prevents credential stuffing attacks that cycle through password lists at high speed.
• AI inference API protection: An API that calls an expensive AI model backend applies per-user token bucket limiting: 50 requests burst capacity, 5 requests/minute sustained. Power users can make 50 rapid requests when needed (e.g., batch processing) without permanently consuming the 5/min average allocation.

Related patterns

• Request Throttling — the server-side complement: instead of rejecting excess requests, throttling queues and slows them down.
• Circuit Breaker — rate limiting protects a server from its clients; circuit breakers protect a client from a failing server.
• CDN Architecture — CDN-level rate limiting and WAF rules can enforce coarse-grained limits before requests reach the application.

Further reading

• Stripe Engineering — Rate Limiters and Algorithms
• Redis — Rate Limiting with Redis
• Figma Engineering — An Alternative Approach to Rate Limiting