ci: execute all frontend and contract tests

[appergb]

User description

Closes #806

Summary

• add one deterministic npm test entry point that discovers every tracked src/**/*.test.ts, scripts/**/*.test.mjs, and scripts/check-*.mjs check
• invoke the pinned local tsx CLI through Node without npx, shell chaining, or runtime downloads
• build the frontend in pretest so the Tauri hotkey gate has a clean frontendDist
• run the aggregate command on macOS, Windows, and Linux CI after npm ci
• preserve the repository Windows lib-test limitation: the hotkey behavior runs on macOS/Linux and the same Rust test remains compile-checked on Windows

RED

• npm test failed because no aggregate runner existed
• after adding discovery, the runner failed on check-*.mjs, proving those checks would otherwise be omitted
• from an empty dist, the hotkey child failed at Tauri frontendDist, leading to the deterministic pretest build

GREEN / verification

• clean-dist CARGO_TARGET_DIR=/Volumes/mac/openless-target/issue-806 npm test (19 discovered checks; all passed)
• aggregate self-test injects child status 7 and asserts exact propagation plus fail-fast
• npm audit --audit-level=high (0 vulnerabilities)
• Node syntax checks for all changed .mjs files
• workflow YAML parse
• git diff --check
• exact-commit gitleaks scan

No product or UI behavior changes.

---

PR Type

Enhancement, Tests

---

Description

• Replace multiple CI steps with a single npm test command

• Add aggregate test runner that discovers all test files

• Add platform-specific skips for hotkey and speech tests

---

Diagram Walkthrough

flowchart LR
  A["npm test"] --> B["pretest: npm run build"]
  B --> C["test: node frontend-test-runner.mjs"]
  C --> D["discover test files"]
  D --> E["run .test.ts via tsx"]
  D --> F["run .test.mjs directly"]
  D --> G["run check-*.mjs directly"]
  E --> H["results"]
  F --> H
  G --> H

Loading

File Walkthrough

	Relevant files
Ci	ci.yml Consolidate CI test steps into one                                              .github/workflows/ci.yml Replaced separate contract test steps with a single npm test command. Removed individual check steps for macOS, Windows, PIN, and speech contracts. +3/-16
Configuration changes	package.json Add test scripts to package.json                                                  openless-all/app/package.json Added pretest script to build frontend before testing. Added test script to invoke the aggregate test runner. +2/-0
Enhancement	frontend-test-runner.mjs Aggregate test runner implementation                                          openless-all/app/scripts/frontend-test-runner.mjs New file that discovers and runs all test files (.test.ts, .test.mjs, check-*.mjs). Exports discoverTestFiles and runTestFiles for reuse. Supports CLI invocation for direct use. +87/-0
Tests	frontend-test-runner.test.mjs Self-test for aggregate runner                                                      openless-all/app/scripts/frontend-test-runner.test.mjs Self-test for the aggregate runner verifying discovery and failure propagation. Spies on spawn calls to validate correct command and arguments. +51/-0
Bug fix	check-hotkey-injection.mjs Skip hotkey runtime gate on Windows                                            openless-all/app/scripts/check-hotkey-injection.mjs Added early exit on Windows with explanatory comment. Maintains compile-only check on Windows via cargo test --no-run. +8/-0      macos-speech-usage-description-contract.test.mjs Skip speech contract on non-macOS                                                openless-all/app/scripts/macos-speech-usage-description-contract.test.mjs Added platform check to skip on non-macOS systems. +5/-0

---

[github-actions]

PR Reviewer Guide 🔍

(Review updated until commit c43b1d9)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 806 - Partially compliant Compliant requirements: One deterministic cross-platform command: npm test via scripts/frontend-test-runner.mjs. Added to CI after npm ci (the new step runs npm test). Avoids runtime downloads: uses local tsx dependency, no npx or shell chaining. No shell syntax that breaks Windows: uses Node spawn directly. Real-device smoke scripts are not included; platform-specific skips are handled inline within test files. Aggregated command fails fast on child failure (verified by included test frontend-test-runner.test.mjs). No product/UI changes. Non-compliant requirements: (none) Requires further human verification: (none)

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ No major issues detected

[appergb]

Independent senior re-review — logical verdict: REQUEST_CHANGES

Reviewed exact head 567f7ca70d5a62c3ab87dd8035032940d26b0804 against current beta 472b914985ec525bb082582fd7968a43c7d9ce41 (PR merge-base / recorded base 9991b325b8ea035ab167d486c5ba3c71688418a9). Submitted as COMMENTED because the authenticated account is also the PR author.

Strengths and confirmed behavior

• Independent tracked-file enumeration and discoverTestFiles() produce the same sorted set of 19 checks: all src/**/*.test.ts, all scripts/**/*.test.mjs, and all root scripts/check-*.mjs. The final default sort plus separator normalization gives stable slash-separated relative paths.
• TypeScript tests execute with the pinned local tsx@4.23.1 CLI through process.execPath; .mjs checks execute directly through the same Node binary. There is no npx, shell chaining, or runtime download path.
• Node 22 behavior is exercised locally (v22.17.1) and in CI; Windows used v22.23.1. The production runner's native absolute path resolution is correct on Windows.
• From a clean worktree with recursively initialized qwen-asr, clean npm ci, no dist, and a new Cargo target, npm test correctly ran pretest -> prebuild contract -> tsc/vite build -> 19 checks. The macOS/Linux hotkey gate compiled and ran hotkey_injection_gate_logs_pressed_and_cancels; the generated dist/index.html existed before Cargo consumed Tauri frontendDist.
• The Windows hotkey early-success path is consistent with the later cargo test --lib --no-run compile gate and the separate Rust-only backend suite. The aggregate workflow still includes the prior build, macOS capsule contract, Windows UI contract, preflight, PowerShell checks, Cargo gates, and version-sync gate.
• The injected self-test does verify exact helper status 7 propagation and fail-fast after two calls. More importantly, the exact-head Windows run supplies end-to-end evidence that a real child failure becomes the aggregate npm test exit status: the job stops immediately at that child and subsequent steps are skipped.
• Scope is limited to the test harness/workflow/package scripts and Windows hotkey gate; no product or UI behavior changed.

Critical

• None.

Important / required

1. The new self-test is not cross-platform and makes the required Windows gate fail.

   File: openless-all/app/scripts/frontend-test-runner.test.mjs:40-41

   The test passes appRoot: '/app' but hard-codes expected child paths as POSIX strings. On Windows, the production runner correctly resolves those paths to the current drive, so exact-head CI reports:

   • actual: D:\app\src\lib\passes.test.ts
   • expected: /app/src/lib/passes.test.ts

   The same defect also applies to the expected failing .mjs path on line 41. Because the self-test is itself discovered by the aggregate runner, Windows fails at Build frontend and run frontend/contract tests; cargo check, cargo test --lib --no-run, the Rust-only Windows suite, and version-sync verification are all skipped. This directly violates #806's cross-platform acceptance criterion and leaves only four of the five exact-head checks green.

   Build the expected paths with node:path (resolve/join) using the same platform-native semantics as the runner, then rerun the exact-head Windows job and confirm all later Windows gates execute successfully.

Minor

• None.

Independent verification

• Clean npm ci: 384 packages installed; 0 vulnerabilities.
• Empty-dist npm test: passed locally; 19 discovered checks, including the hotkey Cargo gate and all tracked frontend/contracts.
• npm audit --audit-level=high: 0 vulnerabilities.
• node --check for all three changed .mjs files: passed.
• ci.yml YAML parse, git diff --check, and tracked-vs-discovered set comparison: passed.
• Exact-commit gitleaks scan (9991b325...567f7ca7): no leaks.
• Current-beta merge-tree: clean (15a61140519e4e0ddac05b06d79a9345f63da146), with all five changed-file blobs identical to exact head.
• Exact-head checks: Android, macOS, Linux, and PR-Agent succeeded; Windows failed in frontend-test-runner.test.mjs as described above.
• Final drift check: PR head remains 567f7ca70d5a62c3ab87dd8035032940d26b0804; remote beta remains 472b914985ec525bb082582fd7968a43c7d9ce41. The PR is still Draft.

Ready-to-merge verdict

Not ready to merge. The aggregate runner design and coverage are otherwise sound, but its own platform-specific assertion currently breaks the mandatory Windows quality gate and prevents the downstream Windows checks from running.

[appergb]

Addressed independent review blocker at exact head 3c335783898ea8c303d6a0a9b66a990ce2e5a5fe:

• replaced POSIX-only /app/... expectations with node:path.resolve(...), matching the runner on Windows drive-letter paths
• rebased onto current beta e3e63cad040ff4bd5eefc439d9be065f2b4f8a43 and preserved the newly merged PIN security contract inside the aggregate npm test gate
• local aggregate run discovered 21 tests and passed all of them, including PIN persistence, hotkey injection, macOS/Windows contracts, and TypeScript suites
• Node syntax, workflow YAML parse, diff check, and gitleaks passed

Fresh exact-head CI and an independent re-review are required before merge.

[github-actions]

Persistent review updated to latest commit 3c33578

[appergb]

Logical verdict: APPROVE

Re-reviewed exact head 3c335783898ea8c303d6a0a9b66a990ce2e5a5fe against current beta e3e63cad040ff4bd5eefc439d9be065f2b4f8a43.

Findings:

• Critical: none.
• Important: none.
• Minor: none.

The previous Windows blocker is closed. The self-test now builds expected child paths with node:path.resolve, so the assertions follow native Windows path semantics instead of comparing D:\\app\\... against hard-coded POSIX paths. The exact-head Windows log confirms the aggregate step passed and reported all 21 tests.

Independent verification:

• Scope is limited to five CI/test-harness files; no product UI, runtime UI, or user-facing source paths change.
• Aggregate discovery exactly matches all 21 tracked src/**/*.test.ts, scripts/**/*.test.mjs, and scripts/check-*.mjs files, with no omissions or duplicates. This includes the merged pin-persistence-security-contract.test.mjs and githubOauth.test.ts coverage.
• A fresh isolated checkout with no pre-existing dist passed npm ci and npm test: pretest build, the real Rust hotkey injection gate, all 21 aggregate children, the PIN contract, and the runner self-test all completed successfully.
• The self-test still proves exact failure propagation and fail-fast behavior with synthetic statuses [0, 7, 0]: status 7 is returned and only the first two children are invoked. TypeScript tests use the repository-pinned tsx@4.23.1 CLI through process.execPath; no npx or runtime download path is introduced.
• Exact-head Windows CI logged 21 discovered tests, the expected Windows-only hotkey runtime skip, the aggregate self-test pass, and the PIN contract pass; it then also passed cargo check, Windows lib-test compilation, actual Rust-only backend tests, and version synchronization.
• Clean merge-tree generation produced b41bad44a5d17cf63183a62ad680d800902fdd5d, byte-equivalent to the exact head, with only the five reviewed paths.
• npm audit --audit-level=high reports 0 vulnerabilities; Node syntax checks, workflow YAML parsing, git diff --check, and gitleaks all pass.
• All five exact-head checks are green: PR Agent, Android, macOS, Windows, and Linux.

Given this evidence, no blocking or non-blocking correctness finding remains.

[appergb]

Rebased the aggregate test gate after #825 merged. Exact head is now 5eaa030d0c71d30c430e2e5293b110d64689b21e on beta eb099c4f411af546628408e30482d23b2fcc6bf8.

The workflow conflict was resolved by retaining the single aggregate npm test gate; the newly merged macOS speech contract remains in beta and is now automatically discovered by the runner. Local verification discovered and passed all 22 tests, including the C-locale Unicode speech plist suite, PIN persistence, hotkey injection, platform contracts, and TypeScript tests. Frontend build, npm audit 0, Node syntax, workflow YAML parse, diff check, and gitleaks passed.

Fresh exact-head five-platform CI and another independent exact-head review are required before merge.

[github-actions]

Persistent review updated to latest commit 5eaa030

[appergb]

Logical verdict: REQUEST_CHANGES (submitted as a COMMENTED review).

Re-reviewed exact head 5eaa030d0c71d30c430e2e5293b110d64689b21e against current beta eb099c4f411af546628408e30482d23b2fcc6bf8. The head is a direct child of that beta. git merge-tree --write-tree is conflict-free and returns 1eb88dc99ac52e5595d6f9a2a05f25c0ecd8df6a, exactly the head tree.

Important / required

1. [P1] Preserve the macOS-only contract semantics in the aggregate cross-platform step.

   discoverTestFiles() unconditionally includes every scripts/**/*.test.mjs file (frontend-test-runner.mjs:23-32), while the replacement workflow now runs npm test unconditionally on macOS, Windows, and Linux (ci.yml:155-156). Current beta added macos-speech-usage-description-contract.test.mjs; that contract invokes bash and the checker hard-codes /usr/bin/plutil. Before this PR, the workflow guarded this test with if: runner.os == 'macOS'; the aggregate replacement removes that platform condition without adding equivalent runner/test gating.

   Exact-head CI proves the failure on both non-macOS platforms:

   • Windows discovers all 22 tests, passes the runner self-test and native node:path.resolve assertions, then the macOS speech contract fails with status 127: /usr/bin/plutil: No such file or directory. The aggregate step exits 1.
   • Linux likewise discovers 22, reaches the same contract, receives status 127 for /usr/bin/plutil, and exits 1.
   • Because the runner correctly fail-fasts, all later aggregate children plus cargo check, the Windows compile/runtime Rust gates, and version-sync verification are skipped on those jobs.

   This is specifically a workflow-replacement regression, not a Windows path regression. Please keep discovery at the expected 22 tracked checks but preserve platform applicability—for example, make the macOS contract explicitly skip on non-Darwin systems or add a tested platform-aware runner mapping—then rerun exact-head Windows and Linux through every downstream gate.

Confirmed behavior

• Independent tracked-file enumeration exactly matches aggregate discovery: 22 tracked / 22 discovered, no missing files, extras, or duplicates. The list includes macos-speech-usage-description-contract.test.mjs.
• The prior Windows path bug is fixed: self-test expectations use node:path.resolve, and exact-head Windows passes frontend-test-runner.test.mjs before hitting the unrelated macOS-only contract.
• The self-test still proves fail-fast and exact child status propagation: injected statuses [0, 7, 0] return 7 and invoke only the first two children.
• TypeScript tests execute via process.execPath plus the pinned local tsx@4.23.1 CLI; .mjs tests execute through the same Node binary. No shell evaluation, npx, or runtime download path is introduced.
• A fresh independent worktree with recursively initialized submodules, no dist, no Cargo target, and clean npm ci passed local macOS npm test: pretest built dist, then all 22 children passed. The full Rust library suite also passed: 701 passed, 0 failed.
• Scope remains five CI/test-harness files. There are no UI, styling, component, or user-facing runtime changes.

Verification

• npm audit --audit-level=high: 0 vulnerabilities.
• cargo audit: exit 0 with 18 allowed warnings.
• Exact-commit gitleaks: 1 commit / ~5.05 KB scanned, no leaks.
• Changed .mjs syntax checks, workflow YAML parse, and git diff --check: passed.
• GitNexus compare: 5 files / 18 symbols, 0 affected product execution flows, low risk.
• Exact-head remote checks: Android cargo check, macOS checks, and PR-Agent pass; Windows and Linux fail at the macOS speech contract described above.

No other correctness or scope finding remains, but the two mandatory cross-platform jobs must be green before approval.

[appergb]

Remote CI on 5eaa030 correctly exposed an integration gap: the aggregate runner discovered the newly merged macOS speech plist contract on Linux/Windows, where /usr/bin/plutil is unavailable. Head c43b1d9 adds an explicit non-macOS skip inside that platform-specific contract and asserts the contract remains discoverable. Local verification: macOS contract pass, frontend runner self-test pass, aggregate npm test 22/22 pass, production build pass, npm audit 0, Node syntax/YAML/diff/gitleaks pass. A fresh exact-head independent review and all five required checks are pending.

[github-actions]

Persistent review updated to latest commit c43b1d9

[appergb]

Logical verdict: APPROVE (submitted as a COMMENTED review).

Fresh re-review of exact head c43b1d96e48650bac1583b8902cfc3c326d92cd2 against current beta eb099c4f411af546628408e30482d23b2fcc6bf8.

Findings:

• Critical: none.
• Important: none.
• Minor: none.

The previous exact-head blocker is closed without masking macOS coverage:

• Independent tracked-file enumeration exactly matches aggregate discovery: 22 tracked / 22 discovered, no missing files, extras, or duplicates. The list and the runner self-test explicitly include scripts/macos-speech-usage-description-contract.test.mjs.
• The contract now exits successfully before invoking bash//usr/bin/plutil only when process.platform !== 'darwin'. Exact-head Windows and Linux logs each show 22 discovered tests, the explicit non-macOS speech skip, and immediate continuation into the PIN contract and every later aggregate child.
• Exact-head macOS logs show the speech contract executing between the capsule and PIN contracts with no skip message. An independent local assertion on the macOS review host also ran the contract with { platform: "darwin", status: 0, stdout: "", stderr: "", skipped: false }, so the new guard does not bypass the real macOS validation.

Independent verification:

• The head is a direct child of current beta. git merge-tree --write-tree is conflict-free and returns 84cb849f75497255105e93d0015ac79360cb8d06, exactly the head tree.
• A fresh detached worktree started with no node_modules, no dist, and no Cargo target. After recursive submodule initialization and clean npm ci, npm test built the frontend first and then passed all 22 aggregate children, including the real hotkey Cargo gate and real macOS speech contract.
• The runner self-test proves fail-fast and exact child status propagation: injected statuses [0, 7, 0] return 7 and invoke only the first two children. Windows-native path expectations use node:path.resolve; exact-head Windows passes the self-test.
• TypeScript tests use process.execPath with the repository-pinned tsx@4.23.1 CLI. .mjs checks use the same Node executable; there is no shell evaluation, npx, or runtime download path.
• Full local Rust library suite: 701 passed, 0 failed. Exact-head macOS also passes 701; Linux passes its 687 cfg-specific tests; Windows compiles the full lib-test binary and passes all 130 Rust-only backend tests.
• Workflow replacement semantics are preserved: macOS, Linux, and Windows run the aggregate step, then their Cargo and version-sync gates. The Android job retains its separate build/updater/target-check path.
• Scope is limited to six CI/test-harness files. No product UI, component, style, or user-facing runtime source changed.
• Changed .mjs syntax checks, workflow YAML parse, and git diff --check: passed.
• npm audit --audit-level=high: 0 vulnerabilities. cargo audit: exit 0 with 18 allowed warnings. Exact-commit gitleaks: no leaks.
• GitNexus compare: 6 files / 19 symbols, 0 affected product execution flows, low risk.
• All five exact-head checks are green: Android cargo check, macOS checks, Windows checks, Linux checks, and PR-Agent.

Final drift check immediately before this review: head remains c43b1d96e48650bac1583b8902cfc3c326d92cd2; beta remains eb099c4f411af546628408e30482d23b2fcc6bf8. No correctness or scope finding remains.

[github-actions]

Persistent review updated to latest commit c43b1d9