Skip to content

feat(voice): a second, grammar-only decode makes commands reliable - #80

Merged
rockywuest merged 2 commits into
mainfrom
feat/command-grammar
Oct 26, 2026
Merged

rockywuest merged 2 commits into
mainfrom
feat/command-grammar

Conversation

@rockywuest

Copy link
Copy Markdown
Owner

Field report: voice commands are understood badly. The small Vosk model on the robot's cheap mic produces things like "legt dich hin" and "weder mit dem schwanz".

Change: every finished phrase is decoded a second time with a Vosk grammar that contains only command words and wake words (de/en/fr, chosen from the model name).

Problem with grammars: a grammar pushes every utterance into its vocabulary. Measured on the dog: "Nox, was siehst du gerade vor dir?" comes out as nox sitz dreh hallo geh.

Fix for that: word confidence. The forced words scored 0.47–0.86, real commands scored 1.00 throughout. So the grammar text is used only when every word reaches ≥ 0.95. Otherwise the full transcript goes on as before.

Validated on the dog with 10 German test phrases:

said full model sent
Komm her und wedel mit dem Schwanz komm her und weder mit dem schwanz komm her und wedel mit dem schwanz
Nox, was siehst du gerade vor dir? (correct) unchanged
Nox, wie spät ist es eigentlich? (correct) unchanged
Nox, leg dich hin nox leg dich hin nox lieg dich hin (same intent)
  • The extra decode costs 70–250 ms on the Pi 4.
  • VOICE_COMMAND_GRAMMAR=0 turns it off.
  • 8 new tests; 733 tests pass.

🤖 Generated with Claude Code

rockywuest and others added 2 commits October 26, 2026 22:23
The small Vosk model mishears short commands on the robot's mic ("weder mit
dem schwanz", "legt dich hin"). Each finished phrase is decoded again
against only the command vocabulary plus the wake words (70-250 ms on the
Pi 4, measured). A grammar forces everything into its words — a question came
out as "nox sitz dreh hallo geh" — but those forced words score 0.47-0.86
while real commands score 1.00, so the grammar text replaces the transcript
only when every word is >= 0.95. VOICE_COMMAND_GRAMMAR=0 turns it off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…er the echo filters

- A grammar does not emit words it doesn't know, so a confident reading could
  shrink "nox wie spät ist es" to "nox". It must now have as many words as
  the full transcript.
- The replacement happens only after the echo and self-echo checks, which
  test both readings, so the dog's own words can't become a command.
- The utterance buffer keeps at most 30 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rockywuest
rockywuest merged commit 03d7e93 into main Oct 26, 2026
4 checks passed
@rockywuest
rockywuest deleted the feat/command-grammar branch October 26, 2026 21:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant