Changelog v3.16.1 (2026-06-01)

Version 3.16.1 (2026-06-01)

Full Changelog

Patch release that makes cache stampede protection actually fire for cached routine responses. The cache-options documentation has advertised stampede protection as a HybridCache feature, but the integration used IRoutineCache as a synchronous probe (Get / AddOrUpdate) that could not carry the SQL execution as the cache factory — so the protection never engaged. A burst of identical concurrent requests against a cold cache executed the underlying query N times, each taking a connection. In the worst case this exhausted Postgres' connection pool (remaining connection slots are reserved for roles with the SUPERUSER attribute), which combined with connection-retry backoff could pin the pool long enough to affect every app sharing the database.

What changed

IRoutineCache gains GetOrCreateAsync (additive)

A new method routes the cold-cache work through the cache so concurrent callers for the same key coalesce into a single execution:

csharp

ValueTask<object?> GetOrCreateAsync(
    RoutineEndpoint endpoint,
    string key,
    Func<CancellationToken, ValueTask<object?>> factory,
    TimeSpan? overrideExpiration = null,
    CancellationToken cancellationToken = default);

It ships as a default interface method (plain probe → factory → store, no coalescing), so any pre-existing custom IRoutineCache implementation compiles and behaves exactly as before — it simply gains no stampede protection until it overrides the method.

Note: although this is a new public API surface (conventionally a minor bump), it is shipped as a patch because it fixes an advertised-but-broken feature and is fully backward compatible via the default implementation.

Stampede protection per backend

• Memory (RoutineCache) and Redis (RedisCache) — coalesce concurrent factory invocations through an in-flight ConcurrentDictionary<string, Lazy<Task>>. A burst collapses to one execution; the rest await the in-flight result.
• HybridCache (HybridCacheWrapper) — delegates straight to HybridCache.GetOrCreateAsync, so Microsoft's built-in stampede protection now genuinely engages.

Middleware paths

• Scalar single-value and passthrough proxy responses (value-shaped) route through GetOrCreateAsync. The connection is opened inside the factory, so coalesced waiters never touch the database. The passthrough proxy case additionally coalesces identical upstream HTTP calls.
• Records / sets (the streaming path) use a per-key execution gate instead. This path streams rows to the client and disables caching mid-stream once a response exceeds MaxCacheableRows (default 1000), which does not fit the "compute one value, cache it, share it" factory model. The gate serializes concurrent requests for a key: the lead executes and (within the row limit) populates the cache, so the rest get a cache hit instead of re-executing. This caps concurrent DB executions per key at one in all cases — including over-limit responses, which serialize rather than run in parallel.

Effect

A burst of N identical requests against a cold cache now results in one database execution (within-limit) or a single serialized execution at a time (over-limit), instead of N concurrent executions. The worst-case fan-out from one event is bounded by the number of distinct cache keys (bounded by the schema), not by the number of clients.

Test coverage (read this honestly)

Automated coverage (NpgsqlRestTests/RoutineCacheTests/CacheStampedeTests.cs) runs against the in-memory backend with a live Postgres and asserts execution counts directly:

• 50 concurrent cold scalar requests → exactly 1 execution; warm-cache burst → 0 further executions; 4 distinct keys → exactly 4 executions (one per key).
• 50 concurrent cold set requests (within limit) → exactly 1 execution; over-limit set (1001 rows) → one execution per request, never cached, all responses correct.

The HybridCache path relies on Microsoft's own tested coalescing (Microsoft.Extensions.Caching.Hybrid) and the Redis path's coalescing is verified by inspection — neither is exercised by the test harness, which boots the core library with the default memory cache. Claims about those two backends are not backed by an automated test in this repo.

Known limitations

• Cross-process coalescing is out of scope. Coalescing is in-process per NpgsqlRest instance; multiple instances each execute once. (HybridCache's Redis layer still shares the cached value across instances.)
• Over-limit sets serialize, not coalesce. Responses above MaxCacheableRows (default 1000) are never cached, so the per-key gate makes concurrent requests for such an endpoint run one-at-a-time rather than sharing a result. This is deliberate: it caps both concurrent DB executions and peak memory (only one large set renders per key at a time) — but it does reduce throughput for a cached endpoint that returns more than MaxCacheableRows rows under load. Since such an endpoint is never actually cached, the right fix when this matters is to raise MaxCacheableRows so the result caches and coalesces, or to drop the cached annotation (restoring fully concurrent, uncached execution).
• The records/sets gate is held during the response stream. Because the gate wraps streaming to the client (not just the DB read), a slow or stalled lead client can delay other clients requesting the same key until it finishes or its request cancels. Waiters honor their own cancellation token, so a waiter that gives up is never stuck. The scalar and proxy paths are unaffected — their coalescing slot covers only the upstream call, not the client write.
• CommandCallbackAsync short-circuit under coalescing. If a user-supplied CommandCallbackAsync short-circuits the response on a cached scalar endpoint, coalesced waiters (not the lead) may observe an empty response. This affects only that specific hook on a cached endpoint.
• Cancellation. The shared factory runs on the lead caller's token; if the lead cancels mid-flight, waiters retry (re-probe the cache, or one becomes the new lead). A waiter may rarely observe cancellation if the lead cancels at the exact moment of coalescing. This is a deliberate safety choice — the factory uses the lead's live connection, so fully detaching the shared work risks using a disposed connection.