Defer Tag objects on known-field decoding

perfloop/protobuf-py · ALLOCATION HOT LOOP

https://perfloop.ai/t/oss/case_zgd8z5mtrd

Verdict

VERIFIED · settled 2026-08-17

What happened: The paired measurements met the required improvement.

Hypothesis

`read_message` calls `reader.tag()` once per wire field before looking up `tag.raw` in `_fields_by_tag`. `BinaryReader.tag` reads the varint and constructs `Tag(key)`; `Tag.__init__` derives the field number and `WireType` and writes three frozen slot attributes. For ordinary known scalar, enum, and length-delimited message fields in a non-group message, those derived values are not needed before decoder selection; repeated-field handling can derive the wire type only on its branch. This makes one short-lived Tag representation the removable delta at a real cadence of one per decoded wire field. A focused local timing check over seven samples of 250,000 one-byte tags, with GC disabled, measured a 0.400443s median for `reader.tag()` versus 0.099157s for direct `reader.varint(5, 0x0F)` (+303.8% for the representation path). That isolated check does not establish the cost share of full `Message.from_binary` decoding. A case should benchmark `Message.from_binary` on representative schemas with many known fields, verify identical values and errors for malformed, unknown, repeated, and group inputs, and show that Tag construction calls fall for eligible fields alongside lower end-to-end CPU or latency.

Change to test: Read the raw tag varint and perform the descriptor lookup first in the normal non-group loop; derive field number and WireType only in the group, unknown-field, or list branches that require them, while retaining malformed-tag validation and wire compatibility behavior.

Where it lives

perfloop/protobuf-py · src/protobuf/_message.py

Evidence

Warm parse of 15 known scalar fields with Message.from_binary (Python backend) · 10 sample pairs

metric baseline candidate paired median change confidence range required result
ns/op 37863 26096 −30.7% (−11627) −12075 to −10585 < −1893 PASSED
ops/s 26411 38321 +44.9% (+11867) +10278 to +12407 > 1321 PASSED

Warm parse of enum, map, nested, and unknown fields with Message.from_binary (Python backend) · 10 sample pairs

metric baseline candidate paired median change confidence range required result
ns/op 50465 33396 −34.2% (−17240) −20414 to −15108 < −2523 PASSED
ops/s 19816 29944 +51.3% (+10174) +9282 to +11592 > 990.8 PASSED

Warm parse of 200 repeated scalar fields with Message.from_binary (Python backend) · 10 sample pairs

metric baseline candidate paired median change confidence range required result
ns/op 489482 352611 −28.4% (−139169) −145750 to −128267 < −24474 PASSED
ops/s 2043 2836 +39.9% (+815.3) +748.7 to +830.8 > 102.1 PASSED

Warm parse of 200 packed fixed64 fields with Message.from_binary (Python backend) · 10 sample pairs

metric baseline candidate paired median change confidence range required result
ns/op 589529 434188 −26.1% (−153683) −159913 to −143852 < −29476 PASSED
ops/s 1696 2303 +35.6% (+603.5) +554.8 to +619 > 84.81 PASSED

Warm parse of 200 repeated message fields with Message.from_binary (Python backend) · 10 sample pairs

metric baseline candidate paired median change confidence range required result
ns/op 1093813 777645 −27.6% (−301500) −341163 to −272209 < −54691 PASSED
ops/s 914.3 1286 +39% (+356.6) +306.2 to +401.9 > 45.71 PASSED

Warm parse of 200 unknown varint fields with Message.from_binary (Python backend) · 10 sample pairs

metric baseline candidate paired median change confidence range required result
ns/op 451383 350473 −22.5% (−101483) −127121 to −93772 < −22569 PASSED
ops/s 2215 2853 +29% (+642.1) +584.8 to +762.4 > 110.8 PASSED

Warm parse of 200 ignored unknown varint fields with Message.from_binary (Python backend) · 10 sample pairs

metric baseline candidate paired median change confidence range required result
ns/op 331802 252735 −23.4% (−77538) −93197 to −55976 < −16590 PASSED
ops/s 3014 3957 +30.6% (+922.3) +627.9 to +1089 > 150.7 PASSED

Warm parse of an unknown group with 200 inner fields (Python backend) · 10 sample pairs

metric baseline candidate paired median change confidence range required result
ns/op 298378 215170 −28% (−83567) −97689 to −78970 < −14919 PASSED
ops/s 3352 4647 +38.4% (+1288) +1230 to +1461 > 167.6 PASSED

Warm parse of an ignored unknown group with 200 inner fields (Python backend) · 10 sample pairs

metric baseline candidate paired median change confidence range required result
ns/op 300389 218036 −27.4% (−82280) −112353 to −66362 < −15019 PASSED
ops/s 3329 4587 +38.4% (+1279) +978.8 to +1595 > 166.5 PASSED

Warm parse of a known group field with Message.from_binary (Python backend) · 10 sample pairs

metric baseline candidate paired median change confidence range required result
ns/op 7696 5328 −30% (−2305) −2667 to −2140 < −384.8 PASSED
ops/s 129931 187701 +43.3% (+56303) +49932 to +63036 > 6497 PASSED

Checks: 6 of 6 passed. Verification: no defect found.

Timeline