Conservatively stop CRTC traversal on contended state

perfloop/dynamo · LOCK CONTENTION

https://perfloop.ai/t/oss/case_h9hh6gk7r0

Verdict

OPEN · opened 2026-09-07

Hypothesis

The current hot step blocks on `state.read()` while removals can hold `state.write()` across a supplied-hash scan. The local design already permits conservative undercount during concurrent shape changes, but explicitly forbids overcount and panics. A typed stop weakens the lookup result at a known boundary instead of dropping the read guard and combining an old edge/worker snapshot with a post-split child observation. This is a narrow, independently landable local change with low novelty and real quality risk; no telemetry yet shows that lock wait is a material cost.

Falsify the performance premise if a baseline profile at representative kept-up mixed load attributes no material request lookup time to `NodeState::read` waiting. Falsify the safety/quality premise if a deterministic held-write test at the first node or a descendant produces a panic, a score past the last confirmed prefix, an incorrect last-matched external hash or router-hint chain, or if a shadow exact/blocking comparison shows unacceptable overlap or selected-worker degradation for contention stops.

Architecture-independent. Limit this to the local CRTC primary selected by `router_event_threads > 1`; do not change Store/Remove semantics, scheduler policy, remote/distributed routing, or release a read guard before child lookup. The change must add no credit from the contended node, preserve the existing no-panic/no-overcount contract, and account for undercount rate and routing quality separately from latency. Do not infer benefit from raw completion latency alone; compare before/after Mooncake runs only after the held-write and shadow-quality checks pass.

This anchor crosses `PreprocessedRouting` links the preprocessor to a `ServiceBackend` backed by the KV-mode `RoutingHost`; this generated operator binding is not a direct static call edge., an index-invisible edge the agent declared; the static analysis could not confirm the edge, so confidence rests on the benchmark and verdict, not the code index.

Change to test: Make `find_match_step` return an explicit `ContendedStop` when `state.try_read()` cannot acquire node state, and make `walk_match_path` terminate at that boundary without changing the previously confirmed active workers, scores, matched depth, previous external hash, or router-hint chain.

Where it lives

perfloop/dynamo · lib/llm/src/http/service/openai.rs

Evidence

No usable result yet.

Timeline