Replace carry-chained 64-byte UTF-8 scans with boundary-summary block batches

perfloop/simdjson · DATA PARALLEL GAP

https://perfloop.ai/t/oss/case_pg5w1h5y4k

Verdict

VERIFIED · settled 2026-08-06 · pull request opened as simdjson/simdjson#2814

What happened: The paired measurements met the required improvement.

Hypothesis

The traced Ice Lake entry forwards directly into the char overload and then `generic_validate_utf8<utf8_checker>`; that leaf walks `buf_block_reader<64>` one block at a time, calls `check_next_input`, advances, and finishes with a padded remainder plus `check_eof`. In the source, `utf8_checker` carries both `prev_input_block` and `prev_incomplete` from each block to the next, while its actual byte rules use only `prev<1>`, `prev<2>`, and `prev<3>`; on Ice Lake `simd8x64` is one 64-byte register. A boundary-summary layout can therefore remove the long loop-carried validation stage across the entry-to-leaf operation: all block interiors become independent vector work, and only the constant-size windows straddling each 64-byte join and the final EOF state need reduction. The floor remains a compulsory read of every input byte and an O(block-count) boundary reduction, so this is not a claim that standalone UTF-8 validation ceases to be linear or that it will help once memory bandwidth dominates. A later case should falsify the move with a pinned-Ice-Lake end-to-end `validate_utf8` cycles-per-byte and throughput comparison on long ASCII, multilingual valid, and deliberately malformed boundary-spanning inputs, with memory-bandwidth counters and differential fuzzing against the fallback implementation; reject it if the new batching does not reduce the measured compute-side cost before the workload reaches its read-bandwidth floor or changes any validity result.

Change to test: Add an internal bulk-validation path selected by `icelake::implementation::validate_utf8` for sufficiently long inputs: the char overload forwards one pointer/length request to a generic block-batch validator, which validates the interior of each 64-byte chunk independently, retains only a compact three-byte prefix/suffix and error summary per chunk, then reduces adjacent boundary summaries and the single EOF summary. Mask the first three lanes of each independent chunk and validate the cross-chunk windows in the boundary reduction, because UTF-8 rules reach back at most three bytes; keep the existing stateful reader for short inputs and preserve the existing partial-tail/`check_eof` semantics rather than changing the already-published tail staging path.

Where it lives

perfloop/simdjson · src/icelake.cpp

Evidence

Ice Lake 4 KiB malformed UTF-8 at a 64-byte boundary · 10 sample pairs

metric baseline candidate paired median change confidence range required result
cpu_time_ns 50.65 51.78 +1.9% (+0.96) +0.8543 to +1.543 ≤ 5 PASSED

Ice Lake 64 KiB ASCII UTF-8 validation · 10 sample pairs

metric baseline candidate paired median change confidence range required result
cpu_time_ns 1376 1084 −21% (−288.5) −302.1 to −275 < 0 PASSED
input_bytes_per_second 47630924243 60448017181 +26.5% (+12607509323) +12040025524 to +13272916935 ≥ 0 PASSED

Ice Lake 1 MiB ASCII UTF-8 validation · 10 sample pairs

metric baseline candidate paired median change confidence range required result
cpu_time_ns 22664 17813 −22.1% (−5016) −5616 to −4324 < 0 PASSED
input_bytes_per_second 46287989151 59805907364 +30% (+13876306613) +11374346288 to +16298332159 ≥ 0 PASSED

Ice Lake 1 MiB multilingual UTF-8 validation · 10 sample pairs

metric baseline candidate paired median change confidence range required result
cpu_time_ns 68800 56998 −17.2% (−11844) −12787 to −10885 < 0 PASSED
input_bytes_per_second 15249021425 18398821145 +20.7% (+3163721229) +2941723656 to +3276965465 ≥ 0 PASSED

Ice Lake 64 KiB malformed UTF-8 at a 64-byte boundary · 10 sample pairs

metric baseline candidate paired median change confidence range required result
cpu_time_ns 1173 981.3 −16.2% (−190.4) −202.7 to −174.9 < 0 PASSED

Ice Lake 1 MiB malformed UTF-8 at a 64-byte boundary · 10 sample pairs

metric baseline candidate paired median change confidence range required result
cpu_time_ns 19502 17171 −11.8% (−2303) −3343 to −1654 < 0 PASSED

Checks: 1 of 1 passed. Verification: no defect found.

Timeline