Abort Git indexing after blob input fails

perfloop-oss/zoekt · Ignored Failure

https://perfloop.ai/t/oss/case_x7ke7zsgh4

Verdict

VERIFIED · settled 2026-11-14 · merged as sourcegraph/zoekt#1250

What happened: The check fails before this change and passes with it.

Hypothesis

A Git input error can replace a complete searchable repository with an incomplete prefix even though indexing reports failure. This affects both full and delta indexing. The behavior is reproduced with a two-file repository; real Git failure frequency and affected repository sizes remain unmeasured.

indexGitRepo creates the builder at line 640 and installs deferred Finish at line 650 as failure cleanup. Reader startup, cat-file ingestion, and go-git document creation can return before the explicit successful Finish at line 742. In particular, indexCatfileBlobs propagates Next and body-read errors after earlier documents may have been buffered or staged. Those input failures are not stored in the builder, so deferred Finish follows its normal publication and retirement path. Finish's documented failure cleanup and Builder's complete-shard staging distinguish this from an intentionally accepted partial-result API.

checkProperty(prop_8hejqyc1wm), at revision c11c1cb52bdc4fde9a31a68ca9df182b3770cdd7, exited 1 on TestPerfloopInputFailurePreservesIndex without timeout or build failure. Its four deterministic histories cross full/delta mode with shard limits 1/1024. A complete old index contains a.txt and b.txt. Using the fake-Git PATH seam already present in TestCatfileReader_ReportsProcessError, the next input delivers one valid new a.txt blob and fails before b.txt with exit status 17. IndexGitRepo returns false and that reader error, but searches subsequently return only new a.txt in all four histories. The two shard limits cover both already-staged and still-buffered prefixes. The test contains its replay fixture and script; the run generated no extra replay files.

The smallest next check is to replay those histories and contrast successful input, intentionally skipped/missing blobs, and stored worker errors. Case proof should audit input-error exits after builder creation, including startup and the go-git path, while preserving ordinary full and delta finalization. The observed witness is specifically a Next failure after one complete blob; no other failure boundary is claimed as already executed. Builder-internal validation-result retention and rename-failure retirement remain separate repairs with shared regression duties.

Upstream was refreshed before submission. Actual gitindex hunks in https://github.com/sourcegraph/zoekt/pull/1050 add admission and branch-set preparation but no abort transition around the deferred Finish. The checked https://github.com/sourcegraph/zoekt/pull/941 changes concern platform paths, not ingestion failure ownership. https://github.com/sourcegraph/zoekt/pull/1089 changes merge/explode artifact lifecycle rather than this producer-to-builder boundary. Targeted prior-repair searches found no equivalent abort repair; the existing FindAllShards I/O finding is unrelated.

Change to test: Separate abort cleanup from successful finalization on Git-input error exits so failed ingestion does not publish its input prefix or retire the prior complete corpus. Preserve explicit successful Finish and worker-error cleanup behavior.

Where it lives

perfloop-oss/zoekt · gitindex/index.go

Evidence

The check fails before this change and passes with it: `For full and delta Git indexing with an existing complete index and removable temporary files, reader-startup, next-header, blob-body, and go-git blob-creation failures before Finish preserve the old search results and index artifacts and discard buffered and staged output.`

Checks: 2 of 2 passed. Verification: no defect found.

Timeline