Prevent CDN Cache Stampedes with Origin Shield and Request Collapsing

My origin server died at 9:00 AM on a Monday because a 60-second cache TTL expired right as our morning newsletter went out. Over 1,400 concurrent users hit the same article URL within two seconds, every single edge server saw a cache miss at once, and our Postgres pool locked up instantly with 504 gateway timeouts.

We fixed this by setting up a multi-layered defense: request collapsing (proxy cache locks), stale-while-revalidate headers, and a dedicated origin shield tier. Here is how to configure these rules in Nginx, test them under simulated traffic spikes, and eliminate duplicate backend queries entirely.

What a Cache Stampede Actually Looks Like

A cache stampede (or thundering herd) happens when a high-traffic cache key expires while requests are still pouring in. Instead of one request refreshing the cache while others wait, every single incoming worker thread forwards the query straight to your origin.

If your CDN runs 45 global edge PoPs (Points of Presence) and each PoP gets 30 simultaneous hits for an expired page, your backend gets hammered with 1,350 identical database queries or Node rendering tasks at the exact same millisecond. CPU usage shoots to 100%, event loops freeze, and your origin starts spitting out 504 gateway timeout errors.

Stopping this requires fixing two specific bottlenecks:

• At the edge: Forcing concurrent incoming requests on any single PoP to share one upstream fetch (Request Collapsing).
• Across the network: Funneling cache misses across all global PoPs through one intermediate caching layer before anything touches your database (Origin Shielding).

Layer 1: Enable Request Collapsing (Proxy Cache Locking)

Request collapsing ensures that when 500 requests hit a proxy for an uncached asset, only the very first request goes upstream. The proxy parks the other 499 requests in memory until that first response comes back, then serves all of them from the fresh cache entry.

In Nginx, you handle this with the proxy_cache_lock directives.

Here is the exact proxy config I use in production:

# /etc/nginx/conf.d/caching-proxy.conf proxy_cache_path /var/cache/nginx/edge levels=1:2 keys_zone=edge_cache:100m max_size=10g inactive=24h use_temp_path=off; server { listen 80; server_name api.example.com; location / { proxy_pass http://origin_backend; proxy_cache edge_cache; proxy_cache_key $scheme$request_method$host$request_uri; proxy_cache_valid 200 301 302 10m; # Request Collapsing directives proxy_cache_lock on; proxy_cache_lock_timeout 5s; proxy_cache_lock_age 5s; # Headers to help debug edge behavior add_header X-Cache-Status $upstream_cache_status always; add_header X-Proxy-Host $hostname always; }
}

Here is what these directives actually do when traffic spikes:

• proxy_cache_lock on;: Tells Nginx that only one request can populate a new cache entry at a time.
• proxy_cache_lock_timeout 5s;: If the upstream request takes longer than 5 seconds, Nginx unblocks the waiting requests and lets them pass through to origin so clients do not hang forever.
• proxy_cache_lock_age 5s;: If the worker process building the cache crashes or stalls, another request takes over the upstream fetch after 5 seconds.

Layer 2: Async Background Refresh with stale-while-revalidate

Request collapsing prevents your database from melting, but that first user—and anyone queued behind the lock—still has to sit through a full origin round-trip. You can avoid that latency hit using the stale-while-revalidate and stale-if-error cache directives defined in RFC 5861.

With stale-while-revalidate, your CDN serves the stale cached asset instantly (5–15ms), while quietly firing a background subrequest to your origin to grab fresh data for future requests.

Set this directly in your application responses. Here is an Express / Node.js example:

// Express route returning stale-while-revalidate directives
app.get('/api/v1/posts', (req, res) => { const posts = fetchTrendingPostsFromDatabase(); // Fresh for 60s, stale allowed for 300s, stale on 5xx errors for 24h res.set({ 'Cache-Control': 'public, max-age=60, stale-while-revalidate=300, stale-if-error=86400', 'Content-Type': 'application/json' }); res.json({ data: posts });
});

If you run your own Nginx reverse proxy instead of relying solely on managed edge headers, tell Nginx to serve stale content while updating in the background:

# Add to your Nginx location block
proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
proxy_cache_background_update on;

The updating parameter tells Nginx to serve stale cache content if an upstream fetch for that specific key is already in flight. For edge-specific header nuances, check out our guide on CORS and Cache-Control headers for CDN assets.

Layer 3: Put an Origin Shield Between Edge PoPs and Origin

Request collapsing on an individual edge node only protects that single machine. If you use a global CDN like Fastly, Cloudflare, or CloudFront, you have dozens of PoPs worldwide. When an item expires everywhere at once, London, New York, Tokyo, and Frankfurt all miss their local caches simultaneously.

An Origin Shield is a designated caching proxy placed in the same cloud region as your backend servers. All edge PoPs route their cache misses through the Shield before hitting your actual origin application.

Here is the flow:

1. Edge PoPs: User requests hit regional edges (Singapore, Frankfurt, New York). Local request collapsing merges simultaneous hits.
2. Origin Shield: If a regional edge misses, it queries the Origin Shield. The Shield collapses misses coming in from all 40+ global PoPs into one request.
3. Origin Server: Your backend only handles a single request to regenerate the data, sends it to the Shield, and the Shield passes it down to the edge tier.

If you build your own shield proxy using Nginx or Varnish on a dedicated VPS, configure your upstream block like this:

# /etc/nginx/conf.d/origin-shield.conf
upstream production_app { server 10.0.1.50:8080; # Internal private IP of app origin keepalive 64;
} server { listen 8443 ssl http2; server_name shield.internal.example.com; ssl_certificate /etc/ssl/certs/shield.crt; ssl_certificate_key /etc/ssl/private/shield.key; location / { proxy_pass http://production_app; proxy_http_version 1.1; proxy_set_header Connection ""; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_cache edge_cache; proxy_cache_lock on; proxy_cache_lock_timeout 10s; proxy_cache_use_stale updating error timeout; proxy_cache_background_update on; }
}

Lock down your backend origin firewall so it only accepts connections from the shield’s private IP. This stops rogue edge requests from bypassing your shield and prevents 502 bad gateway and SSL handshake drops during routing changes.

Simulating and Testing Cache Stampedes with k6

Do not assume your proxy lock works until you test it under real concurrency while purging the cache. I use a k6 load testing script that runs 200 virtual users against a single endpoint with intentional cache clearing.

Here is the script I used to verify our configuration:

// stampede-test.js
import http from 'k6/http';
import { check, sleep } from 'k6'; export const options = { scenarios: { traffic_spike: { executor: 'constant-vus', vus: 150, duration: '30s', }, },
}; export default function () { const url = 'https://api.example.com/api/v1/posts'; const res = http.get(url, { headers: { 'Accept-Encoding': 'gzip', }, }); check(res, { 'status is 200': (r) => r.status === 200, 'cache header present': (r) => r.headers['X-Cache-Status'] !== undefined, }); sleep(0.1);
}

Run it directly from your terminal:

k6 run stampede-test.js

Here is what showed up in our origin logs during the benchmark:

• Before request collapsing: 150 virtual users triggered 150 concurrent SQL queries on cache expiration. Latency climbed to 2,840ms, and 14 requests failed with 504 timeouts.
• After request collapsing + stale-while-revalidate: Exactly 1 backend query hit the database on expiration. Average response times stayed flat at 14ms across all 150 virtual users.

The Gotcha: Vary Headers and Cache Key Splitting

The first time I deployed proxy_cache_lock, 4 to 8 parallel requests were still leaking through to our backend on every cache purge. The locks seemed broken.

The culprit was cache key fragmentation. Request collapsing only works if concurrent requests share the exact same internal cache key.

Three common mistakes split cache keys unexpectedly:

1. Vary: User-Agent: If your origin sends this header, the cache splits entries for Chrome, Safari, Firefox, and mobile engines. A stampede across different browsers triggers a separate backend fetch for each browser type.
2. Gzip vs. Brotli: A request with Accept-Encoding: gzip gets a different cache key than one with Accept-Encoding: br. Normalize encoding on your proxy.
3. Query Parameter Ordering: /api/posts?sort=desc&page=1 and /api/posts?page=1&sort=desc return the same payload, but CDNs treat them as completely different cache keys.

Fix this by stripping unneeded Vary headers and normalizing query strings before calculating the cache key:

# Ignore unnecessary origin vary headers that fragment the cache
proxy_ignore_headers Vary;
proxy_hide_header Vary; # Manually set a controlled Vary header to downstream clients
add_header Vary "Accept-Encoding" always;

For more details on how intermediate proxies evaluate variants, check the MDN Web Docs guide on HTTP Caching.

Frequently Asked Questions

What is the difference between Request Collapsing and Request Coalescing?

They are two names for the same thing. CDNs and reverse proxies use both terms to describe combining multiple concurrent requests for the same URI into a single backend fetch.

Does request collapsing add latency for users?

Only for users who arrive while an expired item is actively being fetched. They wait for that single upstream request (say, 80ms), which is far better than letting hundreds of concurrent queries choke your database and cause 5-second timeouts.

Can I use stale-while-revalidate for personalized user dashboards?

No. Never use shared CDN cache directives on authenticated endpoints that return private user data. Keep Cache-Control: private, no-store on any route using session cookies or Bearer tokens.

How long should my stale-while-revalidate window be?

A solid baseline is setting stale-while-revalidate to 3 to 5 times your max-age. If an asset is fresh for 60 seconds (max-age=60), allow stale responses for 300 seconds (stale-while-revalidate=300). That gives your workers plenty of time to fetch fresh data in the background before anyone sees a cold cache miss.

Next Steps for Origin Protection

Once your edge caches and origin shield are properly coalescing requests, your next step is tuning timeouts and connection pools between the CDN and your application layer.

See our guide on configuring origin shielding rules to prevent backend overloads to map out custom timeouts and keep your database stable during massive traffic spikes.