Skip to content

chore(bench): fixtures + analysis/transport benchmarks + baseline - #25

Merged
gandarfh merged 1 commit into
mainfrom
perf/bench-baseline
Jun 13, 2026
Merged

chore(bench): fixtures + analysis/transport benchmarks + baseline#25
gandarfh merged 1 commit into
mainfrom
perf/bench-baseline

Conversation

@gandarfh

Copy link
Copy Markdown
Member

Slice 2.5 foundation: measure before optimizing (the ADR-010 principle).

  • gen_fixtures.py: deterministic medium.md (466L/10 blocks) + large.md (5226L/50 blocks/~190 refs); refs point only at blocks above (real scope resolution).
  • bench_analysis.ml: in-process analysis cost (scan/diagnostics/semantic/completion/hover), no transport.
  • lsp_roundtrip.py: extended to run the fixtures E2E (didChange->diagnostics, semanticTokens/full) + a --check budget gate against the ADR-010 hard-fail thresholds.
  • BASELINE.md: numbers + the conclusions they drive.
  • CI: Linux-only budget gate in the ocaml job (hard-fail thresholds, not the noise-prone p95 targets).

Key baseline finding: the OCaml analysis is sub-millisecond on the large fixture (didChange->diagnostics 1.9ms p95). The one hot spot is semanticTokens/full on large at ~70ms p95 -- encoding/transport bound (of_blocks itself is 0.83ms), exactly what semantic tokens delta targets. The planned LSP memo/cache optimization would tune work already 100x under budget; the data redirects effort to semantic tokens delta and the frontend full-doc scanners (measured separately in httui-desktop).

@gandarfh
gandarfh merged commit a21b2ad into main Jun 13, 2026
9 checks passed
@gandarfh
gandarfh deleted the perf/bench-baseline branch June 13, 2026 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant