fix(asr): Apple Speech 保留停顿分段，修复停顿前内容丢失

[Breadswim]

User description

摘要

Fixes #822。

Apple 的设备端识别会在自然停顿处把一次听写切成多个 utterance。每个带 speechRecognitionMetadata 的回调只表示一个独立话段；旧实现只取早到的 final，因而会丢掉停顿前内容。本 PR 跨话段累积文本，并等待 recognition task 真正完成后再收账。

修复内容

• metadata 边界是独立话段的权威证据：相邻话段即使完全相同或互为前缀（如 hello → hello、好的 → 好的我们继续）也分别保留，不再用跨 segment 前缀启发式吞掉复述或自我修正。
• 以 partial generation / commit state 区分当前话段 final 与同一 task 的全文回放；partial、final、错误抢救只提交一次。
• result 与 error 同次回调时先折叠 result，再按错误路径抢救，避免尾部文本被错误覆盖。
• 抽出纯 RecognitionLifecycle 状态策略，明确终止优先级：cancel > error salvage > completed+grace / final silent fallback > timeout > wait。completed 后迟到的 callback 会重新起算 250ms grace。
• 仅汉字、平假名、片假名及中日标点按无空格规则连接；俄语、阿语、韩语等非 CJK 脚本正常保留词间空格。
• 识别等待预算保持 max(60s, 音频时长 + 30s)，避免长录音被固定 60s 截断。

范围与兼容性

• 仅修改 openless-all/app/src-tauri/src/asr/local/apple_speech_provider.rs；无 UI、配置、依赖或其它 ASR provider 改动。
• 单话段 / 云端累计 partial 路径仍返回一次最终文本。
• 已同步 origin/beta d6ca07a6b244f58c38e9d29d73d6b3269b5a8499，采用普通 merge + non-force push。
• 当前 exact head：6f43edf8a894d3aafc6c1e01b4c4b807cd3e5b56。

测试与验证

严格按 TDD 执行：

• RED 1：新增相同/前缀独立话段与非 CJK 分词回归后，focused 结果 23 passed; 4 failed；失败值分别复现吞段与俄语粘连。
• GREEN 1：最小 accumulator / script-aware join 修复后，27 passed; 0 failed。
• RED 2：completion+grace、late callback、silent fallback、cancel/error/timeout ordering 六个状态机测试先因策略 todo! 失败；result+error 抢救测试另行 0 passed; 1 failed。
• GREEN 2 / 最终：cargo test --locked --manifest-path openless-all/app/src-tauri/Cargo.toml --lib apple_speech → 34 passed; 0 failed; 638 filtered out。

最终树额外验证：

• cargo test --locked --manifest-path openless-all/app/src-tauri/Cargo.toml --lib → 672 passed; 0 failed。
• cargo check --locked --manifest-path openless-all/app/src-tauri/Cargo.toml → success（仅仓库既有 warning）。
• changed-file rustfmt --edition 2021 openless-all/app/src-tauri/src/asr/local/apple_speech_provider.rs → success；未运行裸 cargo fmt。
• npm run build → Android IPC prebuild contract、TypeScript、Vite build 全部成功。
• npm audit --audit-level=high → found 0 vulnerabilities。
• cargo audit --file src-tauri/Cargo.lock → exit 0；无 denied advisories，报告 18 个仓库既有 allowed warnings（本 PR 不改依赖或 lockfile）。
• git diff --check origin/beta → success；最终 PR diff 仅 Apple Speech 单文件。
• GitNexus detect-changes → 1 file / 44 symbols / 0 affected processes / LOW risk；recognize_file 唯一生产上游调用者为 transcribe_pcm_blocking。
• GitHub CI run 29392368610 → success；Android cargo check、macOS checks、Linux checks、Windows checks 全部通过。

当前文件相对 beta 从 11 个测试函数增至 33 个，即本 PR 共新增 22 个 Apple Speech 单元测试。当前 CI 会在 macOS/Linux 实际执行 Rust lib tests，在 Windows 编译 lib tests 并执行 Rust-only backend tests；不再沿用“CI 不跑 cargo test”的旧描述。

真机证据

贡献者在原修复 head 上曾用 macOS 27.0 Beta / Apple Silicon / zh-CN 真机验证：

[apple-speech] utterance boundary: segment captured (13 chars)
[apple-speech] utterance boundary: segment captured (6 chars)
[apple-speech] recognition finished: 2 segment(s), 19 chars

输出为 来来来做个测试这是第一句话这是第二句话。本轮 review 修复未重新运行稳定版 macOS 真机 E2E；最终 head 的自动化状态/分段边界由上述单元与全量测试覆盖。

---

PR Type

Bug fix, Enhancement

---

Description

• Introduce SegmentAccumulator to retain pause-separated utterance segments

• Switch completion detection from first final to SFSpeechRecognitionTask completed state

• Add RecognitionLifecycle for cancel/error/completion decision priority

• Scale recognition wait budget with audio duration (min 60s)

---

Diagram Walkthrough

flowchart LR
  A["recognitionTaskWithRequest: resultHandler"] --> B["SegmentAccumulator.fold(text, utterance_ended, isFinal)"]
  B --> C{"Wait loop: check state, cancel, error, deadline"}
  C -- "task.state == completed + grace elapsed" --> D["salvage() -> joined text"]
  C -- "cancel_flag" --> E["cancel task, bail"]
  C -- "error present" --> F["salvage() -> return persisted segments"]
  C -- "timeout" --> G["bail timeout"]

Loading

File Walkthrough

Relevant files

Bug fix

apple_speech_provider.rs Multi-utterance accumulation and lifecycle-driven completion openless-all/app/src-tauri/src/asr/local/apple_speech_provider.rs Added SegmentAccumulator struct with fold/salvage logic for multi-utterance accumulation Replaced single-final detection with RecognitionLifecycle state machine (completed/grace/quiescence) Changed recognition loop to poll task.state and apply RECOGNITION_POLL sleep Implemented CJK-aware segment joining (bare join for Han+Japanese, space for others) Updated wait budget to max(60s, duration_ms + 30s); added unit tests (12+ cases) Extracted extract_callback() to handle simultaneous result+error +749/-65

---

[github-actions]

PR Reviewer Guide 🔍

(Review updated until commit e34fe8c)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis ✅ 822 - PR Code Verified Compliant requirements: 一次听写中包含≥1次自然停顿时，停顿前后的所有内容都出现在最终文本中，不丢段。 单话段（无停顿）场景的输出与修复前完全一致，不引入回归。 云端识别路径（partial 累计式）的输出不受影响。 识别中途出错时，已经累积的话段不被整段作废。 有覆盖多话段累积、逐话段 final、话段边界缺失等分支的单元测试。 Requires further human verification: 修复在稳定版 macOS（非 Beta）上的行为是否一致，需人工在稳定版上验证。 非中文 locale（如 en-US）下的多话段切分行为是否符合预期，需在对应语言下测试。

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ No major issues detected

[appergb]

Senior code review — logical verdict: REQUEST_CHANGES

Reviewed exact head feeecf44a4ec828f947a3eb9c852502fea12465d against beta at c645b16851727bb09d3a5274854bc1277da4bd9e. Submitted as COMMENTED per the current-account review workflow.

Strengths

• The change is tightly scoped to apple_speech_provider.rs; there are no React/TypeScript, CSS, configuration, dependency, or other-provider changes.
• Moving termination away from the first isFinal, accumulating boundary callbacks, salvaging a residual partial on errors, and retaining a completion grace window are directionally correct for issue #822.
• The reported Chinese pause case is represented by a focused accumulator test, and all five checks on this exact head are green. I independently ran cargo test --manifest-path openless-all/app/src-tauri/Cargo.toml --lib apple_speech: 23 passed, 0 failed.

Critical / required

1. SegmentAccumulator::push_segment can still delete legitimate spoken text. Lines 599–606 treat any adjacent segments where one text is a prefix of the other as a cumulative replay. But fold calls this same function for true utterance boundaries, where the PR says each text is a distinct spoken segment. For example, boundary segments 好的 followed by 好的我们继续 are collapsed to 好的我们继续 instead of 好的好的我们继续; hello followed by hello drops the second utterance entirely. Repetition and self-correction are normal dictation, so this reintroduces the exact data-loss class the PR is meant to fix.

   Please make dedup evidence-aware: preserve distinct boundary segments, and handle cumulative-final replay against the full accumulated transcript (or track an explicit cumulative/segmented mode) rather than inferring it from a prefix relationship with only the previous segment. Add regressions for equal repeated segments, a longer segment starting with the previous segment, and a shorter segment that is a prefix of the previous one.

Important

• joined uses “both boundary characters are non-ASCII” as a CJK test (lines 634–639). Supported locales include Korean, Russian, Arabic, Thai, Hindi, and accented Latin languages; their separate utterances generally need spacing, but this code concatenates them. Use an actual CJK/script-aware rule or preserve the recognizer-provided spacing semantics, and add at least one non-CJK non-ASCII regression.
• The new tests do not exercise the new termination/error state machine: no completed + grace, late callback, per-segment isFinal quiescence, timeout/cancel, or result/error ordering test. The PR says 12 new tests, but the diff adds 11; it also says CI does not run cargo test, while the current macOS/Linux workflow does run Rust unit tests. Please add a testable pure state transition helper (or equivalent seam) and cover the terminal/error boundaries before merge.

Minor

• None beyond correcting the verification text above.

Ready to merge verdict

Not ready to merge. The core pause-path architecture is promising and exact-head CI/focused tests are green, but the adjacent-prefix heuristic causes deterministic transcript loss and needs correction plus boundary regression coverage first.

[github-actions]

Persistent review updated to latest commit 6f43edf

[appergb]

Independent senior re-review — logical verdict: REQUEST_CHANGES

Reviewed exact head 6f43edf8a894d3aafc6c1e01b4c4b807cd3e5b56 against current beta 472b914985ec525bb082582fd7968a43c7d9ce41. Submitted as COMMENTED per the current-account review workflow.

Confirmed fixes and scope

• The old adjacent equal/prefix metadata-boundary loss is fixed: metadata-backed utterances are now committed without text heuristics, with equal, longer-prefix, and shorter-prefix regressions.
• The old non-ASCII spacing bug is fixed for the requested matrix: Russian, Arabic, and Korean receive spaces; Chinese and Japanese join without spaces around native punctuation.
• Callback state is private to each recognize_file invocation, so callbacks from one recognition task cannot mutate a later task's accumulator. The pure lifecycle helper now covers completed grace, a late callback restarting grace, final-only silence, cancel/error/timeout priority, and result+error salvage ordering.
• The PR remains scoped to apple_speech_provider.rs only (+734/-65): no UI, configuration, dependency, or other-provider changes.

Critical / required

1. Equal repeated per-segment final utterances are still deterministically collapsed.

   In SegmentAccumulator::fold (lines 638–652), a final without metadata is committed only when a new partial marked the generation active, there are no prior segments, or its text differs from the entire joined transcript. Therefore this callback sequence, which is the exact per-segment-final mode the implementation and tests claim to support, still loses speech:

   • fold("hello", false, true) commits the first utterance and leaves current_generation_active = false.
   • A second independent fold("hello", false, true) has a nonempty segment list and normalized(joined) == normalized(segment), so it is suppressed as a task-level replay.
   • salvage() returns "hello", not "hello hello".

   The coverage gap is precise: equal_boundary_segments_are_distinct_utterances uses metadata (true, false), while per_segment_finals_are_all_kept uses two different strings. Thus all 34 focused tests pass while the real repeated final-only case remains untested. This is still the core irreversible transcript-loss class from #822, not a cosmetic edge case.

   Please make replay suppression identity-aware (for example, explicit callback/task/generation evidence) rather than using equality with the accumulated transcript to decide whether a final belongs to a new utterance, and add an exact repeated per-segment-final regression. The fix also needs to keep the existing cumulative-full-final replay regression green.

Important

• None beyond the required data-loss fix above. The previous lifecycle/order and multilingual-spacing blockers are adequately addressed at this head.

Minor

• None.

Independent verification

• cargo test --locked --lib apple_speech --quiet: 34 passed.
• cargo test --locked --lib --quiet: 672 passed.
• cargo check --locked --quiet: passed (repository warnings only).
• npm run build: passed; npm audit --audit-level=high: 0 vulnerabilities.
• cargo audit: exit 0, with 18 repository-allowed unmaintained/unsound warnings and no failing vulnerability finding.
• gitleaks git --log-opts='d6ca07a6b244f58c38e9d29d73d6b3269b5a8499..6f43edf8a894d3aafc6c1e01b4c4b807cd3e5b56' --redact --no-banner: no leaks.
• git diff --check origin/beta...HEAD and rustfmt --check on the changed provider: passed. Repository-wide cargo fmt --all -- --check is already noisy in unrelated files, so it is not attributed to this PR.
• Current-beta merge-tree is clean; the merged tree keeps the exact head provider blob (3cbb4537fe8594724c096dbd3c566b070d75607b).
• All five exact-head checks are green: Android cargo check, macOS checks, Linux checks, Windows checks, and PR-Agent. The CI run reports head SHA 6f43edf8a894d3aafc6c1e01b4c4b807cd3e5b56; macOS/Linux ran Rust unit tests and Windows compiled/ran its Rust-only unit set.
• Final drift check after submission and current-beta refresh: refs/pull/824/head is still 6f43edf8a894d3aafc6c1e01b4c4b807cd3e5b56; beta is still 472b914985ec525bb082582fd7968a43c7d9ce41.

Ready-to-merge verdict

Not ready to merge. The old blockers are substantially fixed and the exact-head verification is otherwise strong, but the remaining equality-based final replay test still drops a legitimate repeated utterance in the advertised per-segment-final path.

[github-actions]

Persistent review updated to latest commit d37ace8

[github-actions]

Persistent review updated to latest commit 71a7776

[github-actions]

Persistent review updated to latest commit e34fe8c

[appergb]

Independent senior re-review — logical verdict: APPROVE

Reviewed exact head e34fe8c against authoritative live beta 4bdef5d. Submitted as COMMENTED because the authenticated reviewer is the PR author.

Prior blockers resolved

• Consecutive identical final-only callbacks are now preserved as distinct utterances. Replay suppression no longer relies on equality with the accumulated transcript; it is gated by a snapshot created only by an explicit metadata boundary. The exact final("hello") → final("hello") regression returns "hello hello".
• Task-level cumulative final replay remains deduplicated. The metadata-segmented regression followed by a whitespace-different full-final replay returns the accumulated transcript once.
• The earlier lifecycle/error concerns remain resolved: callback result is folded before its paired error is recorded; decision priority is cancel, error salvage, completed/final-silence finish, timeout, then wait; a callback after completed restarts the 250 ms grace window.
• Metadata-backed equal and prefix-related utterances remain distinct. Russian, Arabic, and Korean segments receive spaces, while Chinese/Japanese scripts and native punctuation join without inserted spaces.

Scope and current-beta integration

• Beta advanced during final verification through CI/test-runner-only PR #832. I refreshed and tested the new pair before submitting. Merge tree 94d0b8225ac25e1e586d0d3729f8d080b0b4937c is conflict-free; relative to current beta, the PR changes only apple_speech_provider.rs (+749/-65), and the merged provider blob is exactly the reviewed head blob.
• No UI, configuration, dependency, lockfile, or other provider change is introduced by this PR, so no visual/UI validation was required.

Independent verification

• Focused cargo test --locked --lib apple_speech on the current-beta merge: 35 passed, 0 failed (34 provider tests plus the matching current-beta coordinator route test).
• Full cargo test --locked --lib on the current-beta merge: 724 passed, 0 failed.
• New current-beta aggregate npm test: all 22 discovered frontend/contract tests passed, including its production-build preflight and hotkey-injection backend seam.
• cargo check --locked: passed with repository warnings only.
• npm run build: passed (Android IPC boundary contract, TypeScript, Vite); npm audit --audit-level=high: 0 vulnerabilities.
• cargo audit: exit 0, no denied advisory; 18 repository-allowed warnings.
• Changed-file rustfmt --check, git diff --check, and gitleaks over the exact branch-only range: passed.
• All five checks remain green on exact SHA e34fe8c: Android cargo check, macOS checks, Linux checks, Windows checks, and PR-Agent. macOS/Linux ran Rust lib tests; Windows compiled lib tests and ran its Rust-only set.

Verdict

Ready to merge. The deterministic repeated-final data-loss blocker is fixed without regressing cumulative replay deduplication, and no new blocking finding was identified.