A stale account entry flushes unrelated route-cache entries
perfloop/nats-server · CACHE THRASH
https://perfloop.ai/t/oss/case_1bk24crq7f
Verdict
VERIFIED · settled 2026-07-20
What happened: Validated. The minimal stale-key deletion leaves unrelated B entries reusable after A’s generation refresh: the mixed routed-message benchmark reduced its primary median from 7,743 to 6,373 ns/op (17.693400490765854%, Mann–Whitney U p=0.00001082508822446903; 10 co-measured samples per arm), exceeding the 15% goal. Its supporting counters moved B L1 misses and B sublist matches from 8/op to 0/op, which reaches the cache-reuse ceiling for this fixture. The capacity-saturated distinct-cold-key guard held (829.55 to 823.3 ns/op; p=0.5288488601182102) while retaining the expected candidate-only prune transition. All three sealed checks passed, including the two-server account-removal/reload regression test now kept in server/routes_test.go. No further hot-path change is justified for this hypothesis: each unaffected B replay is already an L1 hit.
Hypothesis
`getAccAndResultFromCache` compares each hit against that entry's `pac.acc.sl.genid` at client.go:6333-46, but a mismatch then executes `clear(c.in.pacache)`, discarding every account/subject record. I ran `rg -n -C 2 'pa\.pacache|pacache\s*=' server --glob='*.go'`; it showed that non-account-scoped route parsing and gateway handling construct account-plus-subject cache keys, while `readCache` explicitly describes this map as an account-aware L1. I also inspected `Sublist.Insert`, which increments the individual sublist generation, and `Sublist.MatchBytes`, whose L1 fallback enters the shared sublist cache and, on its miss, tokenizes and traverses under locking. Thus a changed subscription set for account A can be detected by the next reused A key and erase still-valid B keys; later B messages must again take the account lookup path where applicable and call `MatchBytes`. The source establishes the broad invalidation shape, but not the mixed-account reuse rate or its CPU cost. The focused baseline check `go test ./server -run '^(TestRoute.*Cache|TestNoRaceRouteCache)$' -count=1` passed in 3.293s; it exercises existing route-cache behavior but does not measure mixed-account generation invalidation. A case session should prime several A and B keys on one route/gateway connection, mutate only A subscriptions, trigger the stale A lookup, then replay B keys while recording L1 hit/miss and reload counts by account, `Sublist.matches`, CPU/allocations, and routed-message latency. The expected signal is that B stays L1-hot after the A refresh with unchanged delivery correctness. The removed delta is the whole-map eviction plus the first later reload for each displaced B key; the trigger is a reused stale A entry, and the cadence is once at that stale lookup followed by each subsequently reused B key in normal per-message route processing.
Change to test: On a per-account-cache generation mismatch, invalidate and recompute only the stale key while reusing its perAccountCache record; retain entries whose own account/sublist generation is still valid, and perform capacity pruning only for a true new key. Preserve the current freshness behavior for the changed account and verify account-reload/import semantics before relying on per-entry invalidation.
Where it lives
perfloop/nats-server · server/client.go