[bvolpato]

Why

Literal --limit counted matching lines before grouping files, so repeated hits in one file could hide another file. Filename-only searches also built previews for matches that could never appear in the output. The ASCII trigram filter could exclude files whose Unicode lowercase form matched the query.

What changed

Rank and limit literal results by file, using exact matching-line counts and deduplicating overlapping workspaces. Bounded searches return up to three previews per file and report omitted matches through more_hits_in_file and text output. --no-limit returns every matching line, ranked by file.

If a selected file changes between counting and preview generation, reconsider candidates rather than returning an empty or stale result. Each retained file's count and previews come from the same content read.

For filename-only output, stream file contents and retain only the selected paths. When one candidate file is sufficient, stop after the first match and the binary-sniff prefix. Candidate filtering avoids ASCII trigrams that are unsafe under the existing Unicode lowercase rules. Queries without a safe trigram walk the workspace instead.

To reproduce the file-limit and Unicode bugs:

(
  tmp=$(mktemp -d)
  mkdir -p "$tmp/repo/.git"
  printf 'needle\nneedle\nneedle\n' > "$tmp/repo/a.txt"
  printf 'needle\nneedle\nneedle\nneedle\n' > "$tmp/repo/b.txt"
  printf 'Kelvin\n' > "$tmp/repo/unicode.txt"
  export IVYGREP_HOME="$tmp/home" IVYGREP_NO_AUTOSPAWN=1
  ig --add "$tmp/repo" --hash --no-watch
  ig --literal --no-watch --json -n 2 -C 0 needle "$tmp/repo"
  ig --literal --no-watch --file-name-only kelvin "$tmp/repo"
)

The first query returns both b.txt and a.txt. The second returns unicode.txt.

Validation

cargo fmt -- --check, cargo clippy --locked --all-targets -- -D warnings, cargo test --locked --test literal_cli --test cli_snapshot --test daemon_ipc --test mcp_e2e, cargo test --locked --no-default-features --test literal_cli, focused streaming, ranking, and file-change library tests, and ./test.sh --quick ran on macOS. A release binary from the initial revision completed scripts/e2e_all.sh --binary <release-binary>, including daemon equivalence and headless Web UI acceptance.

A new client stopped an incompatible version 9 daemon and returned results locally with autospawn disabled. A version 9 client searched through the new daemon without restarting it. Linux CI on the initial revision exercised the RLIMIT_DATA memory regression. The full ./test.sh suite and neural E2E were not rerun.

A temporary worktree combined the code changes from the three PR branches. It ran cargo test --locked --test skip_gitignore --test literal_cli, both E2E infrastructure unit-test modules, and scripts/e2e_all.sh --binary <debug-binary> with an overlong TMPDIR. The browser used one headless worker. Markdownlint checked the later architecture note.

The dependency audit fails on the unchanged main-branch lockfile because of existing rustls and zip advisories.

A daemon mutation soak on this head exceeded its anonymous RSS growth budget: 35.6 MiB against 32 MiB. An earlier run of the byte-identical binary and a rerun passed. The cause of the observed growth is not established. The soak sends hybrid search requests rather than literal file requests.

Evidence

On an Apple M5 Max with 64 GiB of memory, the filename-only reproduction used about 309 MiB peak RSS on the base release and 17 MiB on this release. Each measurement used the same indexed 700,000-byte file with 20,000 matching lines. The JSON outputs were identical across repeated runs and a follow-up measurement after the review fixes. Normal bounded output used about 17 MiB and reported 20,000 matches, three previews, and 19,997 omitted matches.

To reproduce the macOS memory measurement, set BASE_IG and PR_IG to release binaries built from the base and this branch:

(
  tmp=$(mktemp -d /tmp/ig-memory.XXXXXX)
  mkdir -p "$tmp/repo/.git"
  awk 'BEGIN { for (i=0; i<20000; i++) print "needle repeated event message here" }' > "$tmp/repo/repeated.txt"
  export IVYGREP_HOME="$tmp/home" IVYGREP_NO_AUTOSPAWN=1
  export RAYON_NUM_THREADS=4 TOKIO_WORKER_THREADS=2 RUST_LOG=off
  "$BASE_IG" --add "$tmp/repo" --hash --no-watch
  /usr/bin/time -lp "$BASE_IG" --literal --no-watch --file-name-only --json -n 1 -C 100 --hash needle "$tmp/repo" > "$tmp/base.json"
  /usr/bin/time -lp "$PR_IG" --literal --no-watch --file-name-only --json -n 1 -C 100 --hash needle "$tmp/repo" > "$tmp/pr.json"
  cmp "$tmp/base.json" "$tmp/pr.json"
)

On the same macOS 26.7.1 host, a warm-cache filename-only benchmark compared the base and this branch with ripgrep 15.2.0. The corpus was the sharkdp/bat archive at af1f53d9a977. It contained 909 source-tree files. ivygrep indexed 829 files and rg --files reported 850 under its filters. A UTF-16 file was excluded from both searches. The index was already built, query caching was disabled, four threads were used, and each tool ran 15 times in rotating order. Each query returned the same filename set from all tools.

Query	Base median / p95	PR median / p95	ripgrep median / p95
HighlightingAssets	12.75 / 19.01 ms	13.42 / 216.33 ms	23.65 / 24.94 ms
error	16.29 / 17.62 ms	16.36 / 18.13 ms	22.31 / 26.24 ms
key	15.89 / 20.19 ms	48.42 / 55.31 ms	23.69 / 28.73 ms
absent marker	14.36 / 15.62 ms	14.48 / 23.29 ms	22.68 / 27.58 ms

key falls back to a workspace walk to preserve Unicode matches. One outlier dominated the HighlightingAssets p95 for this branch. A later 50-invocation comparison for that query measured 12.93 ms median and 16.69 ms p95 for this branch, with a 19.35 ms maximum. This small warm benchmark does not measure index build time or cold-cache performance, and the tools have different file filters.

The file-change check recounts selected files before returning previews. On the same 700,000-byte file, bounded output with -n 1 -C 100 had a 24.69 ms median before the review fix and 26.15 ms after it, with identical JSON across 15 rotating invocations.

To repeat the comparison, extract the archive, initialize a Git directory, and index it with a release binary. Use an isolated IVYGREP_HOME, IVYGREP_NO_AUTOSPAWN=1, IVYGREP_DISABLE_QUERY_CACHE=1, RAYON_NUM_THREADS=4, and TOKIO_WORKER_THREADS=2. From the corpus directory, run the following commands for each query, normalize the paths, verify the filename sets, and rotate the tools across 15 warm invocations:

"$PR_IG" --literal --file-name-only --no-limit --no-watch -C 0 --exclude 'tests/syntax-tests/source/PowerShell/test.ps1' -- "$query" .
rg --threads 4 --hidden -g '!.git/**' -g '!tests/syntax-tests/source/PowerShell/test.ps1' --ignore-case --fixed-strings --files-with-matches -- "$query" .

Run the ivygrep command with BASE_IG as well. The absent marker was __ivygrep_absent_marker_9b65__.

Risk

The daemon protocol changes from version 9 to 10. Older version 8 and 9 clients retain legacy hit-limit semantics against the new daemon. A new client restarts an older incompatible daemon. No index format migration is required. Revert the change and restart the daemon to roll back.

Downsides

Bounded literal output can now omit previews that previously appeared in a single-file result. The exact matching-line count and omitted count remain visible. --no-limit restores all previews, but can use much more memory. A short query without a safe trigram can require a full file walk. An edit to a selected file during a search can cause another candidate scan. The search does not provide an atomic snapshot across the workspace. The matcher preserves existing lowercase and sigma handling, not full Unicode case folding such as ß matching ss.

Checklist

• Change is focused and follows existing project patterns.
• Tests cover changed behavior or regression, or testing is not applicable.
• ./test.sh --quick passed during iteration.
• ./test.sh passes, or Validation explains narrower proportional checks.
• Relevant CLI, daemon, MCP, browser, neural, or artifact E2E passes, or E2E is not applicable.
• User-facing docs and changelog are updated when needed.
• Performance and relevance claims include reproducible evidence.
• No private code, credentials, generated caches, or local paths are committed.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-17T23:12:21.690844Z	7fc16ff	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7fc16ffc38

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7fc16ffc38

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bvolpato]

Included in #454, merged as b6f4e75.

Reviewed head fd85b20da8b28d13dcf2f7a48c439c9a0cd3d013, including both resolved review fixes, is incorporated in the combined branch. The integration preserves the skip-gitignore guards from #455. I verified that the merged tree exactly matches the reviewed candidate; literal limits, ranking, edited-file consistency, Unicode, and CLI/daemon behavior passed combined native-platform validation. Closing as superseded.