Avoid duplicate checkpoints when closing the same upload writer twice
perfloop-oss/distribution · Excessive Round Trips
https://perfloop.ai/t/oss/case_557vnycy5y
Verdict
VERIFIED · settled 2026-11-11
What happened: The paired measurements met the required improvement.
Hypothesis
A resumed upload chunk saves an unchanged digest checkpoint during response construction and again during deferred cleanup. The extra storage-driver call may add chunk overhead, especially with a remote backend. End-to-end savings, production chunk sizes and request frequency remain unmeasured.
At f49f48736337e8fc9f7816a044819b0f7967d557, blobUploadResponse closes the upload before building continuation state, and blobUploadDispatcher also wraps the resumed handler in closeResources. An uncommitted, uncancelled blobWriter.Close invokes storeHashState before closing the driver writer. There is no completed-close guard. With no intervening write, the second call serializes the same digest and writes the same checkpoint path before the driver reports that it is already closed. Early handler exits need not take both Close calls; failed resume, committed and cancelled states do not have this two-checkpoint success-path shape.
The stored route check on prop_cjyj1ctcaa passed: `go test -count=1 -run=^$ -bench=^BenchmarkPerfloopPatchCheckpointBudget$ -benchtime=1x -benchmem ./registry/handlers` reported 2.000 checkpoint-writes/PATCH for filesystem and in-memory drivers at 0, 1, 4096 and 1048576 bytes (checkProperty call_YfFvwtsrqcemy6QKwE72lpaJ). It also checked continuation headers, another PATCH, Commit and read-back. These eight local samples establish driver-interface call counts, not remote request counts or a measured optimization gain.
The broader removal of cross-request close/resume work is already proposed by hyp_xc0czrd392's active-upload session cache. This proposal retains stateless, per-request checkpointing and removes only repeated work on one already-closed writer. It also differs from hyp_wzgesn443a, which removes the checkpoint during final Commit rather than an uncommitted PATCH. user:4 closed that Commit proposal and hyp_6z8h7v9dgk on 2026-11-06 for Tomás's GTM-scope reason; that lowers customer priority here but is not runtime evidence or a refutation of this mechanism.
Current upstream patches do not implement this change: https://github.com/distribution/distribution/pull/5063 only changes Close after Commit; https://github.com/distribution/distribution/pull/5064 ensures driver closure after checkpoint failure; https://github.com/distribution/distribution/pull/4869 removes a Cancel-side Close; and https://github.com/distribution/distribution/pull/1697 changes final Commit cleanup. Their behavior must remain compatible, without republishing their repairs. The open-PR pages were followed to their end. CONTRIBUTING.md directs code review to GitHub PRs, not a separate review archive. Recent merged-path history remains a gap: the checkout is shallow and the bounded history API reads returned HTTP 403.
The smallest useful Case check is a minimal completion-guard experiment against the same route benchmark. A reduction from two checkpoint calls to one after a successful first Close would confirm the removable work; unchanged counts or lost continuation would refute that approach. Measure allocations, I/O and elapsed time separately. Preserve close/resume cycles, signed continuation state, later PATCH, Commit/read-back, and checkpoint/driver-error behavior. Test and benchmark placement is provisional, and no production-only prevalence gate or minimum speedup is required.
Change to test: Avoid repeating checkpoint serialization and storage after an upload writer has completed a successful Close, while keeping newly resumed writers checkpointable and preserving first-close errors and failure retries. A local completion guard or equivalent single-close ownership is provisional; this does not require a cross-request session cache.
Where it lives
Evidence
Filesystem upload PATCH · 24 sample pairs
| metric | baseline | candidate | paired median change | confidence range | required | result |
|---|---|---|---|---|---|---|
ns/op |
12933586 |
7829206 |
−44.9% (−5809693) |
−8646071 to −1047637 |
gain > 0; gain > 0.05 × |baseline median|; arm medians separated |
PASSED |
B/op |
51375 |
39889 |
−22.4% (−11500) |
−12314 to −10499 |
loss ≤ 0.05 × |baseline median| |
PASSED |
allocs/op |
369.5 |
306 |
−17.1% (−63) |
−64 to −62 |
loss ≤ 0.05 × |baseline median| |
PASSED |
Checks: 6 of 6 passed. Verification: no defect found.