Avoid per-batch JSON decode and re-encode

perfloop/parseable · REDUNDANT SERIALIZATION

https://perfloop.ai/t/oss/case_ar4fzktbk5

Verdict

VERIFIED · settled 2026-08-20

What happened: The paired measurements met the required improvement.

Hypothesis

A focused source check, `rg -n -C 2 'record_batches_to_json\\(' src`, and the traced bodies showed the full consumer chain: `create_batch_processor` runs once for each `Ok(batch)`, constructs a one-batch `QueryResponse`, calls `to_json`, then emits `Bytes::from(format!("{response}\\n"))`. In `to_json`, `record_batches_to_json` uses `arrow_json::ArrayWriter` to write the RecordBatch into a Vec<u8> and immediately calls `serde_json::from_reader` on that same buffer; `to_json` then builds a Value array. On this streaming callsite `fields` is empty and `with_fields` is false, so the parsed tree has no observed fields-wrapper consumer before the final formatter; any direct path must still prove equivalent null and newline behavior. The removed delta is one JSON decode and one JSON re-encode, plus the intermediate map/Value allocations, per emitted batch; retain the necessary Arrow JSON encoding and own the output rather than caching it across requests. Trigger: streamed SQL responses with nonempty RecordBatches. Real cadence: once per successful batch result processed by the response closure, so work scales with both batch count and JSON payload size. No CPU or allocation profile was run, so the cost reduction remains a structural prediction. A case session should add byte/semantic-equivalence coverage for representative rows, nulls, and the terminal newline, then benchmark/profile `handlers.query` streaming representative batch sizes; confirmation is reduced serde JSON parse/serialize CPU and allocations together with lower end-to-end response latency.

Change to test: Add an owned direct RecordBatch-to-NDJSON-bytes path for the streaming caller, preserving the trailing newline and null semantics while bypassing the intermediate Vec<Map<String, Value>> and Value tree.

Where it lives

perfloop/parseable · src/handlers/http/query.rs

Evidence

8-batch default streaming SQL query handler (20,000 nullable rows/batch) · 10 sample pairs

metric baseline candidate paired median change confidence range required result
ns_per_batch 29664875 2743239 −91.2% (−27041228) −30676225 to −23539848 < −1483244 PASSED
allocation_calls_per_batch 180783 746.8 −99.6% (−180037) −180038 to −180035 < −9039 PASSED

Checks: 1 of 1 passed. Verification: no defect found.

Timeline