Caching Strategies

What it is

Caching is the practice of storing copies of frequently accessed data in a fast-access storage layer (in-process memory, a distributed cache like Redis, or a CDN edge node) to reduce the latency and load of retrieving that data from the authoritative source (a database, an API, or a file system). A cache hit serves the data immediately from fast storage; a cache miss fetches from the origin and optionally populates the cache for subsequent requests. The cache-to-origin latency ratio is typically 100–1000×, making caching the single most impactful performance optimization available to most systems.

Several distinct caching patterns exist, each with different consistency and complexity tradeoffs. Cache-aside (lazy loading) is the most common: the application checks the cache first, handles misses by loading from the database, and writes the result to the cache. Read-through delegates both the cache check and the origin load to a caching library or proxy, keeping application code simpler. Write-through writes to both cache and database synchronously on every write, keeping the cache always consistent. Write-behind (write-back) writes to the cache immediately and asynchronously persists to the database, optimizing write latency at the cost of potential data loss.

Why it exists

Database queries are expensive: they require network round trips, disk I/O for uncached pages, CPU for query planning and execution, and lock acquisition. For read-heavy workloads, a cache serves the same data at microsecond latency compared to millisecond database latency, and without consuming any database CPU or I/O resources. This means a small Redis cluster can absorb traffic that would otherwise require many expensive database read replicas. Cache hit ratios of 90–99% are achievable for stable reference data, translating to a 10–100× reduction in database load.

Caching is also essential for external API calls. Third-party APIs have rate limits, latency unpredictability, and cost per call. Caching their responses for minutes or hours reduces both cost and latency while providing resilience against API outages. CDN caching extends this to static assets and even dynamic API responses, delivering content from edge nodes geographically close to users with latencies under 5ms regardless of origin location.

When to use

• Data is read far more frequently than it changes and the system can tolerate brief staleness.
• Computing or fetching the data is expensive (complex database queries, external API calls, CPU-intensive computations).
• The same data is accessed by many users or processes concurrently — cache hit ratio will be high.
• You need to absorb traffic spikes without proportionally scaling the database or origin service.
• You are serving static or semi-static assets (HTML, CSS, JS, images, API responses) to geographically distributed users.
• Session data needs to be shared across horizontally scaled application instances.

When not to use

• Data changes on every request or is unique to each user/request (e.g., personalized real-time feeds, financial account balances).
• Strict consistency is required and stale reads would cause incorrect system behavior (inventory levels, financial transactions).
• The cache hit ratio would be too low to justify the operational overhead (data with near-uniform random access across millions of unique keys).
• Sensitive data that must not be stored outside the primary database for compliance or security reasons (PII, credentials).

Typical architecture

  Pattern 1: Cache-Aside (Lazy Loading)
  App ──► Cache (Redis) ── HIT ──► return value
          │
          MISS
          │
          ▼
          DB ──────────────────────► return value
          │                          + populate cache
          └──► cache.set(key, val, TTL=300s)

  Pattern 2: Write-Through
  Write request ──► App
                    ├──► Cache.set(key, newVal)   [sync]
                    └──► DB.update(key, newVal)   [sync]
  Result: Cache always consistent with DB

  Pattern 3: Write-Behind (Async)
  Write request ──► App
                    ├──► Cache.set(key, newVal)   [sync, immediate]
                    └──► Queue ──► DB.update()    [async, ~100ms later]
  Result: Low write latency, risk of data loss on crash

  Multi-Level Cache (L1 / L2 / CDN):
  User ──► CDN Edge (L3, TTL=3600s, global)
            │  MISS
            ▼
           App ──► L1 In-Process (Caffeine/Guava, TTL=30s)
                    │  MISS
                    ▼
                   L2 Redis (TTL=300s)
                    │  MISS
                    ▼
                   DB

  Eviction Policies:
  LRU:  evict least recently accessed item
  LFU:  evict least frequently accessed item
  TTL:  evict after fixed time regardless of access
  FIFO: evict oldest inserted item (rare)
          

Pros and cons

Implementation notes

Eviction and TTL tuning: TTL is the most important parameter — too short and cache hit rate drops; too long and stale data accumulates. Start with empirical measurement: P95 time between updates for the data type. For product catalogs updated hourly, TTL=5 minutes is appropriate. For user profile data updated rarely, TTL=1 hour is fine. Use LRU eviction for general caches where access patterns follow a power law (20% of keys receive 80% of traffic). Use LFU for caches where frequency of access is a better predictor of future access than recency (e.g., global top-K popular items). Use TTL-only eviction when data expires independently of access pattern (session tokens, rate limit counters).

Thundering herd prevention: When a popular cache key expires, all concurrent requests that miss the cache simultaneously query the database. Prevent this with probabilistic early expiration: instead of expiring at exactly TTL, each request computes if (current_time + delta > expiry) where delta is drawn from a probability distribution — allowing one request to regenerate the cache before it actually expires. An alternative is the "mutex on miss" pattern: the first miss acquires a distributed lock (Redis SET NX with expiry), re-fetches from DB and populates cache, while subsequent misses wait for the lock and then re-check the cache. Cache warming: After a deployment or Redis restart, the cache is cold and all requests miss simultaneously. Implement a warm-up job that pre-populates the most frequently accessed keys from the database before routing production traffic to new instances.

Common failure modes

• Cache stampede (thundering herd): Popular key expires; hundreds of concurrent requests miss and hammer the database. Mitigate with mutex-on-miss or probabilistic early expiration.
• Stale cache serving incorrect data: An item is updated in the database but the cache TTL has not expired. Clients see old data. Use event-driven invalidation for data that must be fresh immediately after writes.
• Cache key collisions: Two different data types share the same key namespace; a product with ID 42 collides with a user with ID 42. Always namespace keys: product:42, user:42.
• Cache used as primary store: Application logic depends on data always being in cache without a fallback to the database. Redis restart causes data loss and application failure. Cache is volatile; the database is always the source of truth.
• High memory eviction rate: Cache is undersized, causing constant eviction of recently added entries. The cache hit rate collapses and the database receives nearly all traffic. Monitor eviction rate metrics and right-size the cache.

Decision checklist

• Cache hit ratio target has been defined (e.g., >90%) and the data access pattern supports it (some data has natural locality).
• A caching strategy (cache-aside, read-through, write-through, write-behind) has been chosen based on consistency and complexity requirements.
• TTL values are set based on the actual update frequency of each data type, not an arbitrary default.
• Thundering herd prevention is implemented for high-traffic cache keys.
• Cache is namespaced to prevent key collisions across data types and application versions.
• Fallback behavior on cache miss (and Redis unavailability) is implemented and tested.
• Cache hit ratio, eviction rate, and memory usage are monitored with appropriate alerts.

Example use cases

• Product catalog: An e-commerce site caches product detail pages in Redis with a 5-minute TTL. 95% of product page requests are served from cache, reducing database read load by 20× and dropping median response time from 120ms to 4ms.
• User session storage: A horizontally scaled API tier stores user session tokens in Redis with a 30-minute TTL. Any application instance can validate a session without a database lookup, enabling stateless horizontal scaling.
• External API response caching: A weather service API response is cached in Redis for 10 minutes. During peak traffic, thousands of requests per second are served from the cache while only one request per 10 minutes hits the external API, staying within rate limits.

Related patterns

• Cache Invalidation — the critical follow-on topic: how to keep cached data fresh when the source changes.
• CDN Architecture — extends caching to the network edge for static and semi-static content.
• Read Replicas — a complementary database-level read scaling strategy when caching hit rates are insufficient.
• Hot Partition Mitigation — caching is the primary fix for read hotspots in partitioned databases.

Further reading

• Redis — Common Caching Patterns and Recipes
• AWS — Caching Best Practices
• Martin Fowler — Two Hard Things (cache invalidation)