Redundant duplicate SHA256 hashing during fallback blob validation

perfloop/distribution · INEFFICIENT ALGORITHM

https://perfloop.ai/t/oss/case_6nr6g3vmet

Verdict

VERIFIED · settled 2026-08-17

What happened: The paired measurements met the required improvement.

Hypothesis

In registry/storage.blobWriter.validateBlob, when performing fallback full-hash validation (such as when resumable digests are disabled or on a chunked upload's final PUT), the code instantiates both a canonical digester and a verifier, then streams the entire blob through a TeeReader to both. When the client's requested algorithm is SHA256 (which matches digest.Canonical), both hashers compute SHA256 on the exact same bytes. This redundant hashing of large layers (hundreds of MBs to GBs) can be completely avoided by performing a single SHA256 hash when the algorithms are identical.

Change to test: Compare the client's requested digest algorithm with digest.Canonical. If they match, only stream and hash the blob once using the canonical digester and verify by directly comparing the digest strings. If canonical is already populated, only stream and hash once using the client's custom verifier without re-running the canonical digester.

Where it lives

perfloop/distribution · registry/handlers/blobupload.go

Evidence

128 MiB SHA-256 final PUT full-hash fallback after a 1 MiB chunk · 10 sample pairs

metric baseline candidate paired median change confidence range required result
ns/op 866955940 775516724 −10.8% (−93771017) −105747511 to −84456411 < −43347797 PASSED

128 MiB resumed SHA-512 final PUT after a 1 MiB chunk · 10 sample pairs

metric baseline candidate paired median change confidence range required result
ns/op 1038356739 943442102 −8.7% (−90151646) −128755616 to −85276487 < −51917837 PASSED

Checks: 4 of 4 passed. Verification: no defect found.

Timeline