Skip to content

[fix] Unblock v1.3.1 indexing, search, and security - #454

Merged
bvolpato merged 24 commits into
mainfrom
bvolpato/fix-index-replacement-v1.3.1
Oct 18, 2026
Merged

bvolpato merged 24 commits into
mainfrom
bvolpato/fix-index-replacement-v1.3.1

Conversation

@bvolpato

@bvolpato bvolpato commented Oct 17, 2026 •

Copy link
Copy Markdown
Owner

Why

Windows store replacement fails with metadata.sqlite3 still open and os error 32. Failed skip-gitignore indexing, literal file ranking and Unicode handling, and long macOS E2E socket paths also block the v1.3.1 preparation. Dependency audit additionally rejects the existing Rustls and Candle ZIP dependencies.

Related: #441. This consolidates the reviewed fixes from #455, #456, and #457.

What changed

  • Release cached readers before store replacement, hold exclusive workspace/base leases during watcher updates, and close bounded enhancement readers before inference or resource pauses. Preserve useful status while statistics are unavailable, including nullable MCP counts and matching generated Web assets.
  • Preserve failed skip-gitignore reindex errors and safe retries across CLI, daemon, and all-index paths. Keep those guards in literal file searches.
  • Fix literal file limits, ranking, counts, edited-file refill, and streaming Unicode matching. Protocol 10 retains the older hit-limit semantics for existing clients.
  • Use short aliases for long or multibyte temporary socket paths in standalone daemon and E2E procedures.
  • Upgrade Rustls and ZIP, using checksummed, versioned Candle core/NN forks with the minimal upstream ZIP 9 API repair and current Rust float-constant imports and unused Metal import cleanup. Keep the unchanged BERT implementation in private embedder modules, preserve the public embedder API and accelerator forwarding, and document source provenance and licenses. Prepare the existing manual crate-publication workflow for the new dependency order.
  • Preserve dependency-audit finding details and JSON artifacts on failure without changing the denied advisory classes or exit status.
  • Add pinned Rust and JavaScript/TypeScript CodeQL analysis. Refresh v1.3.1 documentation and evidence for the final source.

Validation

  • CARGO_BUILD_JOBS=4 RUST_TEST_THREADS=4 ./test.sh passed, including Rust regressions, formatting, ShellCheck, Clippy, Python checks, and deterministic frontend assets. RUSTFLAGS='-D warnings' cargo check --locked --all-targets passed, including maintained path dependencies.
  • Release ./scripts/e2e_all.sh --binary ... passed with a deliberately long TMPDIR, covering CLI procedures, daemon/local equivalence, and Chromium Web UI behavior.
  • ./scripts/e2e_neural_backend.sh --binary ... --model-profile general --expect-backend 'BERT embedding via Candle CPU' --expect-file src/lib.rs --check-worktree passed with the actual MiniLM model, semantic retrieval, and worktree inheritance/isolation.
  • Both maintained core versions passed their PTH/NPY/NPZ serialization coverage, including borrowed UTF-8 and owned CP437 filenames. Root all-feature dependency resolution uses the maintained Candle graph and ZIP 9.0.5.
  • cargo audit --deny yanked --deny unsound and pnpm -C web audit --prod found no vulnerabilities. Cargo retains the existing informational unmaintained-dependency warnings.
  • Audit-step execution passed clean, vulnerable, invalid-JSON, and missing-valid-report exit-status cases.
  • uv run scripts/run_current_head_benchmark.py --binary ... --require-neural, its evidence check, and uv run scripts/check_release_readiness.py --tag v1.3.1 passed. The benchmark and quality floors are unchanged.
  • Hosted CI passed native Linux x86/ARM, Windows neural/hash, and macOS Intel/ARM Accelerate/Metal tests and release E2E. Actual Candle Metal retrieval and worktree isolation passed. Rust/JavaScript CodeQL, benchmark guards, PR daemon/MCP soaks, relevance, coverage, and security checks passed. Full public-matrix execution and publication jobs are expected PR skips.

Evidence

The source-matched relevance report records source commit fdceb0b, the source-input fingerprint, and the exact release-binary SHA-256. All modes pass the unchanged quality floors: nDCG@10 0.8975, MRR 0.9257, and recall 0.96 or better, with verified neural execution. This uses the real final repository corpus; no comparative performance improvement is claimed. Public-corpus measurements retain their historical scope.

Risk

Busy-status consumers must handle omitted CLI statistics and nullable MCP counts/readiness. Watcher updates exclude concurrent workspace searches while indexing. The maintained Candle forks require upstream tracking and separate first publication before a future crates.io release; this PR does not publish crates, a release, or a tag. Local neural validation used Linux CPU and hosted Metal backend acceptance passed; CUDA execution remains unverified. CodeQL skips a full project build, so generated-output coverage is unverified.

The PR-ref CodeQL scan flags a pre-existing static typing-demo DOM pattern in docs/index.html. Its text comes from fixed markup with no external-input path. The code is unchanged from main, and the alert was not dismissed.

Checklist

  • Change is focused and follows existing project patterns.
  • Tests cover changed behavior or regression, or testing is not applicable.
  • ./test.sh --quick passed during iteration.
  • ./test.sh passes, or Validation explains narrower proportional checks.
  • Relevant CLI, daemon, MCP, browser, neural, or artifact E2E passes, or E2E is not applicable.
  • User-facing docs and changelog are updated when needed.
  • Performance and relevance claims include reproducible evidence.
  • No private code, credentials, generated caches, or local paths are committed.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-17T15:10:24.982870Z 03f406b PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-advanced-security

Copy link
Copy Markdown

You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool.

What Enabling Code Scanning Means:

  • The 'Security' tab will display more code scanning analysis results (e.g., for the default branch).
  • Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results.
  • You will be able to see the analysis results for the pull request's branch on this overview once the scans have completed and the checks have passed.

For more information about GitHub Code Scanning, check out the documentation.

@bvolpato bvolpato changed the title [fix] Close index replacement races and prepare v1.3.1 [fix] Unblock v1.3.1 indexing, search, and security Oct 18, 2026
@bvolpato

Copy link
Copy Markdown
Owner Author

[review-prs-verification]

Reviewed head 82d34dad535f8a6fd3d687c41857ee20f28adf41 against base 3e2a0421742b247c6b05fdedffc6a472da1e08f2.

Scope: reader/store replacement and watcher races, unavailable statistics across CLI/MCP/Web, the #455 skip-gitignore retries, #456 socket paths, #457 literal ranking/limits/Unicode, protocol compatibility, generated assets, and the checksummed Candle/Rustls/ZIP security repairs. Integration preserves both skip-gitignore literal fallback guards and the existing public embedding API.

Validation passed:

  • CARGO_BUILD_JOBS=4 RUST_TEST_THREADS=4 ./test.sh and RUSTFLAGS='-D warnings' cargo check --locked --all-targets.
  • Release CLI/daemon/Chromium E2E with a long TMPDIR; actual MiniLM BERT CPU retrieval/worktrees; both core versions' NPZ/PTH serialization paths.
  • Source-matched run_current_head_benchmark.py --require-neural, its --check, and check_release_readiness.py --tag v1.3.1. Quality floors are unchanged; rebuilt Linux artifact SHA-256 is 6836b50675da1b47bd3caa43d3a4200b209b1c9bf61636e367814f4a497bf980.
  • Final native Linux/ARM64, Windows neural/hash, macOS Intel/ARM Accelerate/Metal tests and release E2E, including actual Candle Metal retrieval/worktree isolation. Benchmark guards, paired generated corpus, PR daemon/MCP soaks, relevance, coverage, Rust/JS CodeQL, and all required security checks passed. The full public matrix and publication jobs are intentionally skipped on PRs.

Limits: CUDA execution is unverified. Existing informational unmaintained-dependency warnings remain. The PR-ref CodeQL alert concerns unchanged static typing-demo text with no external-input path; no alert was dismissed. Four verbatim upstream Rustdoc hard-break spaces remain. Future crate publication is separate from this merge.

No unresolved threads or required approvals remain. Ready to merge: yes.

@bvolpato
bvolpato merged commit b6f4e75 into main Oct 18, 2026
36 checks passed
@bvolpato
bvolpato deleted the bvolpato/fix-index-replacement-v1.3.1 branch October 18, 2026 02:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants