Result reads can report commit failure after heads are published

perfloop-oss/dolt · Uncatalogued correctness

https://perfloop.ai/t/oss/case_dhkjcxz9yp

Verdict

VERIFIED · settled 2026-11-13 · pull request opened as dolthub/dolt#12426

What happened: The check fails before this change and passes with it.

Hypothesis

A batch can report a read failure after the store has successfully published its heads. The API promises that failure leaves all heads unchanged, so this outcome can mislead a caller about whether its commit happened. A native fault check reproduces the mismatch; the frequency of the required cache miss and read failure is unknown.

database.CommitDatasets first calls db.update, which returns successfully after the root compare-and-swap is acknowledged. Only afterward, at lines 750-757, it reloads each new head through datasetFromMap to construct the returned snapshots. A read error there returns nil and the error without undoing the published root. The interface at go/store/datas/database.go:112-114 explicitly promises unchanged heads on failure. hooksDatabase.CommitDatasets and DoltDB.CommitHeadUpdates return immediately on this error, so the source also skips their normal post-publication hooks and listener path. That downstream consequence needs an entry-level regression check; it was not measured by the storage-only check.

The stored check on prop_nb73rdvxdc ran at revision 0d1e86b35a814050d1475214e7fdb76388adbe3b: go -C go test ./store/datas -run '^TestPerfloopBatchAtomicity$' -count=1 -timeout=90s -v. checkProperty run call_nuPVGsvkcOeYunnwWa1itOtp exited 1 from property assertions. All 85 stale-lock masks for zero through three branch/working-set pairs passed. It also completed 24 read-fault configurations: one or two branch updates, absent or existing heads, warm or purged value cache, and no fault, a fault enabled before the call, or a fault enabled after a successful root swap. The four cold-cache post-swap configurations returned an injected chunk read error after one acknowledged swap; a fresh, unwrapped database view observed the changed persisted heads. Cold existing-head pre-call fault controls returned errors with zero acknowledged swaps and unchanged roots. Warm post-swap controls succeeded because they did not need the failing chunk read.

The smallest returned replay creates one new branch with no initial commit. The chunk-store decorator delegates the real root swap and receives true,nil, then uses the existing public ValueStore.PurgeCaches seam and enables a Get error. This models a cold result read failing after publication; it does not simulate a crash, corrupt data, a competing writer, or a store returning an uncertain commit acknowledgment. Public cache purging and ChunkStore.Get errors are supported boundaries. Normal write caching makes this a conditional failure mode, not a claim about every invocation. Addressing it entails per-batch snapshot construction/state and review of retry and database binding, without a new public API. Its likelihood and production impact remain unmeasured.

This is independent of hyp_pptzaw039k's missing expected-head guard: the existing-head fault cases supply a matching nonzero expected head and have no concurrent writer. The broader contribution is to remove owner-generated fallible result materialization after acknowledged publication, rather than special-casing one injected error. The Case should first replay the retained input and verify persisted heads with a fresh reader, then establish consistent success/failure and hook delivery for supported update kinds, batch order, validation/build/precondition errors and optimistic retries. Unknown commit-acknowledgment outcomes are explicitly outside this finding. The stored native checker and its file layout are provisional proof choices, not extra frozen placement requirements.

The complete current open-PR listing showed no implementation at this storage owner. The inspected patch at https://github.com/dolthub/dolt/pull/12305 aborts transactions on a disappearing working set before publication; it does not address post-publication result reads. Local history is shallow, and the bounded GitHub path-history request returned HTTP 403, leaving recent merged history a named evidence gap. The prior session-value and root-rebuilding performance repairs concern different owners and changes; their customer pauses or negative measurements do not refute this failure-result contract.

Change to test: Prepare and validate the returned dataset snapshots before acknowledged publication, or retain sufficient built snapshots to avoid fallible result reads afterward. Preserve ordered results and post-commit hook delivery without attempting to roll back a root the store has already committed.

Where it lives

perfloop-oss/dolt · go/libraries/doltcore/doltdb/doltdb.go

Evidence

The check fails before this change and passes with it: `With acknowledged root publication, CommitDatasets returns successful ordered snapshots bound to the receiving database for commit, prebuilt commit, and working-set batches with absent or existing heads, even when the value cache is purged and subsequent chunk reads fail.`

The check fails before this change and passes with it: `For mixed branch and working-set batches in either order, DoltDB.CommitHeadUpdates returns the published heads, delivers commit hooks and the working-root listener, and registers hook replication waits before returning when only published-head reads fail after acknowledged publication.`

Checks: 5 of 5 passed. Verification: no defect found.

Timeline