Preserve prior shards and rename errors when publication fails

perfloop-oss/zoekt · Ignored Failure

https://perfloop.ai/t/oss/case_xcnjjp3z01

Verdict

VERIFIED · settled 2026-11-12 · merged as sourcegraph/zoekt#1227

What happened: The check fails before this change and passes with it.

Hypothesis

A replacement publication failure can retire the only usable prior index. For a repository in a compound shard, later successful cleanup can also turn that failure into a successful return. This is a supported filesystem-error path, but its deployment frequency is unknown.

Builder stages shards to avoid a mixed corpus. Finish's retirement comment at lines 756-758 places deletion after everything is renamed into place. Instead, the loop at lines 779-786 records a failed os.Rename and continues; the failed replacement remains in toDelete. Simple-shard cleanup removes that old file. Compound-shard cleanup tombstones the repository and unconditionally assigns SetTombstone's result to buildError, including nil.

checkProperty(prop_984sjn0yy3), at revision c11c1cb52bdc4fde9a31a68ca9df182b3770cdd7, exited 1 on TestPerfloopFinishPreservesPriorOnRenameFailure without timeout or build failure. The deterministic native filesystem fault removes exactly one completed staged source between Add and Finish, causing ENOENT. No replacement rename succeeds. With one prior simple shard, Finish returned an error but removed the old usable shard. With one two-repository compound shard and ShardMerging enabled, Finish tombstoned the target repository, left its neighbor alive, and returned nil. The test stores the replay event order; no additional replay files were generated.

The smallest next check is that two-storage-kind replay plus successful publication and valid compound retirement controls. The Case must establish error preservation and prior-data preservation on the supported publication failures. This witness does not require a crash-atomic multi-file transaction or rollback of replacements already published. Pre-publication temporary cleanup and delta-error repeat calls remain separate repairs.

Upstream was refreshed before submission. Actual hunks in https://github.com/sourcegraph/zoekt/pull/1050 leave this rename/retirement flow unchanged. https://github.com/sourcegraph/zoekt/pull/1089 changes the merge command's lifecycle and repositoryless-shard handling, not Builder.Finish publication; its merged predecessor https://github.com/sourcegraph/zoekt/pull/1088 only recovers empty shards during Explode. The checked https://github.com/sourcegraph/zoekt/pull/941 builder changes do not add a publication-failure policy. Targeted prior-repair searches found no equivalent repair.

Change to test: Keep still-usable prior shards live when replacement publication fails, and retain the publication failure through subsequent cleanup. Gate deletion or tombstoning on successful replacement publication without imposing crash-atomic rollback of successful renames.

Where it lives

perfloop-oss/zoekt · gitindex/index.go

Evidence

The check fails before this change and passes with it: `For full builds replacing a repository in simple or two-repository compound storage, a missing staged source for shard one or zero makes Finish retain that rename error on repeated calls, preserve un-replaced prior files without retirement, remove stale sidecars of published replacements, and keep an updated commit eligible for a smaller full retry that retires stale target shards while preserving the neighboring repository; successful publication retires only obsolete target data.`

The check fails before this change and passes with it: `For a delta Builder appending two data shards to two prior simple shards with unchanged branch names and index options and an updated commit, a missing lower-numbered staged source makes Finish retain its rename error on repeated calls, preserve the prior shard bytes, and leave their metadata updates unpublished.`

The check fails before this change and passes with it: `For a non-delta Builder with one prior simple shard, when a self-referential metadata sidecar makes IndexFilePaths fail before publication, Finish retains that enumeration error on repeated calls, preserves the prior shard bytes, removes every staged temporary file when removal succeeds, and clears finishedShards before returning.`

Checks: 3 of 3 passed. Verification: no defect found.

Timeline