Cache stampede explained: when one expired key overloads a database
By Unlearnt
A cache stampede happens when many requests miss the same cached value and independently fetch or rebuild it at once. The cache normally shields the database from that work. When a popular entry expires, duplicate refreshes can overwhelm the system behind it.
Worked example: 50 requests, one missing result
Imagine a product page whose recommendation list takes 200 milliseconds to calculate. At noon its cached result expires. Fifty requests arrive before the first calculation finishes, and each starts the same database work. Assume one application process, successful reads and no other traffic; these are illustrative numbers, not a benchmark.
With request coalescing, the first request starts the calculation and registers that work under the cache key. The other 49 join that in-flight result instead of starting another calculation. Once it finishes, they can use its result and later requests can use the refreshed cache.
| Step | Independent refreshes | Coalesced refresh |
|---|---|---|
| Entry expires | No usable cached result | No usable cached result |
| 50 requests arrive before completion | 50 database calculations begin | 1 begins; 49 share the pending result |
| Refresh succeeds | Duplicate results refill the key | One result refills the key and serves the waiters |
Stampede, stale data and thundering herd
A stale-data bug asks whether a saved value is current. A stampede asks how much repeated work happens when the value is missing or being refreshed. Fresh data does not prevent overload, and limiting refreshes does not automatically make the result correct.
Thundering herd describes a wider pattern of many workers becoming active for the same event. A cache stampede is one example. Trace the miss and refresh sequence rather than assuming every database spike is a stampede.
One refresh per process is not one refresh for the fleet
A singleflight-style group coordinates calls sharing that group. Put one group inside each of three application processes and all three may start a refresh for the same key. A shared cache does not make their separate check-then-fetch sequences atomic.
Decide whether that remaining load is acceptable. A shared refresh coordinator can reduce duplication further, but now requests depend on its availability and latency. A worker that crashes or runs beyond a lock lease also needs a defined recovery path; a lock is not a complete failure-handling policy.
Choose what happens while the result is missing
There is no universal best response. An old recommendation list might be acceptable briefly; an authorisation decision may need a current authoritative check. Make that choice before treating stale responses as a performance fix.
| Approach | What it helps with | What still needs deciding |
|---|---|---|
| Coalesce requests for one key | Suppress simultaneous duplicate work | Coordination scope, wait time and refresh failure |
| Spread expiry times across keys | Avoid many different keys expiring together | One hot key can still get simultaneous misses |
| Serve stale data during refresh | Avoid making every reader wait | Maximum staleness and who owns refresh |
| Bound downstream work | Protect capacity during a cold cache or failure | Which requests wait, fail or receive a fallback |
Misconception: adding randomness to the TTL fixes every stampede
Expiry jitter spreads the expiry times of different entries. It does not stop fifty requests noticing that one popular entry has expired. Probabilistic early refresh is a different technique: it gives requests a chance to refresh before expiry. Neither term should stand in for checking how duplicate work is controlled.
Check the miss path, not just the hit rate
Test what happens with a cold cache and a slow or failed refresh. A high average hit rate can hide a short burst that exhausts downstream capacity.
- Count database refreshes per key while several requests arrive together.
- Repeat across multiple application instances, not just one process.
- Bound waiting and retries; ensure refresh failure does not release an unlimited wave of new attempts.
- Record which stale results are allowed and when callers must receive an error instead.
Check your understanding
Optional: apply the idea to a new scenario.
Three application servers share a cache. Each server combines simultaneous refresh requests only within its own process. A popular key expires and requests reach all three before any refresh completes. Why can the database still receive three refreshes, and what would you weigh before coordinating refreshes across servers?
Try explaining it to the tutor →Free to start, no account needed.