Avoid per-match metadata probes during file search

perfloop/codex · N+1

https://perfloop.ai/t/oss/case_s940sdypv5

Verdict

VERIFIED · settled 2026-07-24

What happened: Validated at the mechanism ceiling for the measured completed-session path: snapshot delivery improved from a 1,616,500 ns median to 1,426,500 ns (11.75%, p=0.0000108) while the deterministic metadata-probe evidence fell from 20 to 0 per 20-match snapshot. The focused 2,048-entry dense-plus-no-match initial-walk guard was inconclusive for a 0.84% slower median (451,150,000 ns to 454,950,000 ns; p=0.63), so it held as a non-regression bound rather than a claimed benefit. The source now explicitly defines FileSearchSession results as the one-walk snapshot, with a regression test for a post-walk file-to-directory replacement; no further metadata-probe reduction remains to pursue on this path.

Hypothesis

The worker iterates up to inner.limit matches on every changed nucleo.tick result and calls Path::new(full_path).is_dir() inside that per-match loop before building FileMatch. The TUI starts its session with the default FileSearchOptions limit of 20 and invokes update_query on every changed @ token. walker_worker already receives each discovered entry but injects only Arc<str> path data, so the later matcher has discarded the classification it needs. I ran `cargo test -p codex-file-search --lib session_emits_updates_when_query_changes`; it passed and confirms the existing sequential query-update behavior, but it does not measure metadata I/O. No syscall trace or CPU profile was collected, so the share of latency due to these probes remains hypothesized. A case session should complete a workspace walk with at least 20 dense matches, then repeatedly edit an @ query while tracing the matcher thread’s statx/newfstatat calls and timing changed-snapshot delivery. The baseline signal should show up to 20 metadata lookups per changed snapshot after the walk; the carried-type version should reduce that count to fallback-only while preserving directory results and symlink behavior.

Change to test: Carry an exact directory/file classification from walker_worker’s discovered entry through the Nucleo item data, and use that classification in matcher_worker instead of calling Path::is_dir for every output match. Preserve follow_links(true) semantics; use a metadata fallback only when the walker cannot provide an equivalent target classification.

Where it lives

perfloop/codex · codex-rs/tui/src/file_search.rs

Evidence

Timeline