Mutate pending streamed text instead of cloning it

perfloop/grok-build · AVOIDABLE COPY

https://perfloop.ai/t/oss/case_gz9tttwych

Verdict

VERIFIED · settled 2026-07-26

What happened: Validated for the sealed 128 × 1 KiB AgentMessageChunk merge stream: median elapsed time fell from 850,645 ns to 13,154.5 ns per stream (98.45% reduction; Mann–Whitney p=0.0000108). The co-emitted Dhat mechanism metrics fell from 24,970,240 to 261,120 allocated bytes and from 255 to 8 allocation blocks per stream. The focused persistence merge semantic check passed. This candidate intentionally retains the prior clone-and-rebuild path for AgentThoughtChunk, so the finding makes no performance claim for thought chunks or production traffic.

Hypothesis

Source inspection first confirmed that the assigned `write_update` forwards each persisted update to `append_update_commit_aware`. The focused source probe `rg -n -C 2 'maybe_merge_notification|pending_notification' crates/codegen/xai-grok-shell/src/session/persistence.rs` then showed that `run` calls this merge routine for every `PersistenceMsg::Update(Acp)`. In the mergeable text/thought branch, `pending.update.clone()` at line 1499 precedes `push_str` into the cloned text at line 1511, after which the original pending notification is discarded on the merge-success path. Because this is a clone of an enum containing the growing owned Text string and the clone is mutated, it should materialize the accumulated payload rather than merely copy a slice header. The removed delta is one accumulated-update deep copy per mergeable incoming chunk; its real cadence is every ACP stream update until a non-merge boundary or Flush. No runtime allocation or CPU profile was run, so the end-to-end impact remains a hypothesis. A case session should run a fixed-size streaming-chunk sweep (for example 1 KiB chunks at 10, 100, and 1,000 chunks) with allocation and CPU profiling around this routine. The confirming signal is allocation bytes and clone-frame CPU growing with accumulated response length before the change and falling after it, while JSONL/replay output remains byte- or event-equivalent.

Change to test: On the merge-success path, match the owned pending ACP notification mutably and append into its existing text buffer instead of cloning `pending.update`; preserve the original pending notification for non-merge/error paths and update only the current metadata/session fields required by existing semantics.

Where it lives

perfloop/grok-build · crates/codegen/xai-grok-shell/src/session/persistence.rs

Evidence

Timeline