Skip to content

feat(tts): add Typecast TTS extension#2202

Open
hmmhmmhm wants to merge 9 commits into
TEN-framework:mainfrom
hmmhmmhm:TASK-13879
Open

feat(tts): add Typecast TTS extension#2202
hmmhmmhm wants to merge 9 commits into
TEN-framework:mainfrom
hmmhmmhm:TASK-13879

Conversation

@hmmhmmhm

@hmmhmmhm hmmhmmhm commented Jul 6, 2026

Copy link
Copy Markdown

Summary

  • Add a Typecast TTS extension backed by the official typecast-python SDK.
  • Stream Typecast WAV output, strip the initial WAV header, and forward PCM16 mono audio to TEN at Typecast's fixed 32 kHz sample rate.
  • Expose the API endpoint as params.url, with https://api.typecast.ai as the production default.
  • Add runtime, configuration, vendor-error, cleanup, PCM conversion, and TTS guarder coverage.

Validation

  • Typecast TEN runtime tests: 9 passed
  • TTS guarder full suite: 15 passed, 2 skipped (HTTP TTS does not emit WebSocket connection status; subtitle alignment is opt-in)
  • GitHub Actions workflow reproduced locally with ghcr.io/ten-framework/ten_agent_build:0.7.14:
    • voice-assistant: check, build-agent-deps, lint, all extension tests, and Go server tests passed
    • voice-assistant-realtime: check, build-agent-deps, lint, all extension tests, and Go server tests passed
    • Black: 860 files unchanged
    • pylint: 10.00/10
  • python3 -m compileall, JSON validation, and git diff --check passed

TTS Guarder Snapshot

================== 15 passed, 2 skipped in 363.11s (0:06:03) ==================

Remaining Manual Check

  • TEN-Agent webpage audio-session verification requires valid Agora and STT credentials. The local workspace contains the Typecast credential but not those required credentials, so this check is still pending.

@hmmhmmhm
hmmhmmhm force-pushed the TASK-13879 branch 2 times, most recently from 36ec0c0 to 26ea9c3 Compare July 6, 2026 04:45
@hmmhmmhm hmmhmmhm changed the title [TASK-13879] Add Typecast TTS extension Add Typecast TTS extension Jul 6, 2026
@hmmhmmhm hmmhmmhm changed the title Add Typecast TTS extension feat: Add Typecast TTS extension Jul 6, 2026
@hmmhmmhm hmmhmmhm changed the title feat: Add Typecast TTS extension feat(tts): add Typecast TTS extension Jul 14, 2026
Comment thread ai_agents/agents/ten_packages/extension/typecast_tts_python/config.py Outdated
@wangyimin-agora

Copy link
Copy Markdown
  1. Please add more test cases. You can refer to other TTS extensions, for example, rime_ttp_tts. Moreover, please ensure all test cases pass in CI.
  2. Please run tts guarder for your new tts extension and ensure all cases can pass, then add a snapshot in the PR description or comment.
  3. Please verify your TTS extension on the TEN-Agent webpage, which provides AI communication.

@hmmhmmhm

Copy link
Copy Markdown
Author

Addressed the review items in 13a7779 and bf1e7dc:

  • Renamed the public endpoint setting from host to url.
  • Expanded the Typecast runtime suite to 9 cases covering config normalization/validation, empty input, vendor errors, WAV-to-PCM conversion, successful streaming, and SDK cleanup.
  • Added all TTS guarder property fixtures and fixed-rate handling.
  • TTS guarder result: 15 passed, 2 skipped in 363.11s.
  • Reproduced the CI workflow locally in the official build image for both voice-assistant and voice-assistant-realtime; Black, dependency build, pylint, all extension tests, and Go server tests passed.

The two guarder skips are expected for an HTTP TTS extension: WebSocket connection-status reporting and opt-in subtitle alignment. TEN-Agent webpage audio verification remains pending because the local workspace does not contain the required Agora/STT credentials.

# Conflicts:
#	ai_agents/agents/integration_tests/tts_guarder/tests/bin/start
@hmmhmmhm

Copy link
Copy Markdown
Author

Rebased the validation against the latest upstream state by merging origin/main in 2e5d539. The only conflict was the fixed-sample-rate guarder list; both the newly added EZ-AI provider and Typecast are retained. After resolving it, the Typecast guarder was rerun from a clean install and passed again: 15 passed, 2 skipped in 362.91s.

@YiminW

YiminW commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Hi, if TEN-Agent webpage audio verification cannot be finished in the coming few days, please provide pcm dump generated by TTS Guarder tests. I will check the audio file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants