Retained streaming metadata is overwritten by read-buffer reuse

perfloop-oss/mp4ff · Premature Release

https://perfloop.ai/t/oss/case_mwksvac3f5

Verdict

VALIDATED BY A DIFFERENT CHANGE · settled 2026-11-12

What happened: The check passes with this change.

Hypothesis

Streaming fragment callbacks can receive event-message bytes that have already changed during parsing. Decoded metadata keeps views into a read buffer that the parser reuses before the retained boxes are finished with it. A bounded check reproduces the wrong bytes; consumer prevalence and the memory cost of repairing ownership remain unmeasured. Confidence is medium for the broader repair scope.

ProcessFragments obtains temporary buffer views through ReadFullBox, decodes them with DecodeBoxSR, and calls ResetBuffer before retaining the resulting boxes. ReadFullBox explicitly limits its view to use before the next operation. DecodeEmsgSR retains MessageData through FixedSliceReader.ReadBytes, and DecodeUnknownSR retains its payload the same way. Those reads return slices of the input rather than independent storage. The subsequent box read can overwrite a still-live box's bytes. The explicit styp.Copy is a valid ownership control, but does not protect these other retained payloads. This is sequential reuse, not a proposed data race.

Property prop_2zrpga4dc0 found a reduced counterexample during its ten-second-budget native Go fuzz search. The exact returned corpus b2d27d2ffd9248bc and a normalized replay are stored. The deterministic checkProperty rerun at revision 702346275b141f91c229e5a759822d70d883d55c exited 1 with `go test ./mp4 -run '^FuzzPerfloopStreamPayloadLifetime$' -count=1 -timeout=60s -v`. One emsg v0 message byte, 0x30, preceding the first moof arrives at the first ready callback as 0x61, while DecodeFile preserves 0x30. No sample access or buffer growth is needed. The fixture contains two fragments, but the assertion stops processing in the first callback; it does not establish outcomes for later callbacks or successful completion of that invocation.

The independent Case should first replay that failure, then establish the backing-lifetime property across consecutive event and unknown boxes, unknown moof children, sample reads, subsequent fragments, and trailing-box callbacks, with both buffer reuse and growth. Byte-for-byte agreement with the owning DecodeFile reference is the confirming signal. The source supports that broader ownership question, but the present run directly demonstrates only the stated emsg failure. Stored checks prop_hxzcesvzrh and prop_ptbdf3f9tj passed all 169 bounded pre-moof/trailing sequence pairs and all six callback-error cases. They guard intentional summary scope and error behavior during repair.

This differs from hyp_fd3w4y2q1r, which removes media buffering performed solely to skip unread data. Here the metadata is intentionally retained and must remain valid. Grouped sample reads in hyp_maesfvj381 and track-default selection in hyp_h5pxyp6ckc do not repair this ownership boundary. For local cost context only, CPU-profile receipt call_XdWnKj9HYw2efTkOe2r4ROty ran the existing BenchmarkStreamSamples/GetSamples workload; its ProcessFragments frame at stream.go:699 carried 0.69 seconds inclusive out of 2.09 seconds of sampled CPU. That whole-path observation neither isolates this defect nor measures repair overhead or production frequency.

The refreshed open-PR page and the implementation and discussion of https://github.com/Eyevinn/mp4ff/pull/657 concern AAC configuration parsing and descriptor construction, not streaming buffer ownership. No overlapping implementation was present in that returned open-work set.

Change to test: Give retained streaming metadata backing storage that remains valid for its active fragment holders, rather than recycling borrowed decode buffers underneath them. Cover pre-fragment, nested moof and trailing payloads while preserving lazy media access, callback ordering and errors, retention limits, and the pre-moof-only scope of Emsgs and Prfts.

Where it lives

perfloop-oss/mp4ff

Evidence

The check passes with this change: `For each of the six pinned c2pa-rs-signed livesim2 A48/V300 media segments, StreamFile.ProcessFragments delivers EmsgBox.MessageData differing from DecodeFile at the fragment callback before any sample access, while processing completes successfully.`

Verification: no defect found.

Timeline