Repository navigation
feat: chat attachments and speech-to-speech talk - #3
Merged
Merged
Conversation
Chat: - Drag-and-drop files onto the terminal to queue attachments, sent with the next message. Text files are inlined, images sent as data URLs. - Vision-capable models receive images; text-only models skip them (badge still shows what was dropped). Input bars turn amber on unsupported images. - Ctrl+U clears the pending queue. Talk: - New `padwan-talk` command / `talk` subcommand: speech-to-speech language tutor over OpenAI realtime (gpt-realtime), gated behind the `voice` extra (sounddevice + PortAudio). Deps: - padwan-llm[realtime]; local editable padwan-llm/piou sources while the content-parts/vision + paste-hook work lands upstream.
…ask cancellation, timed mic reads
The transport lets roughly one send through per read-poll tick, so per-chunk appends starved the stream and the commit fired with most audio still queued locally. Batch everything captured between sends, wait for the queue to drain before committing, and print the seconds of audio actually sent per turn.
Headless prompt loop over one persistent AgentSession (same store and options as chat:send). Async stdin so exit never blocks on a reader thread; Ctrl-C routes through task cancellation; Ctrl-D exits.
…KEY in the environment
bytearray slice+del memmoved the whole queued backlog on every 50 ms audio callback; BytesBuffer appends by reference and reads only copy at chunk boundaries.
…urce Bump the pin to >=0.8.0, drop the [tool.uv.sources] override, and migrate talk.py to the 0.8.0 realtime API: top-level RealtimeClient factory with session config on the constructor, connection opened via async with.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
chat:send: file and image attachments — drag-and-drop in the TUI, content parts built with the padwan-llm multimodal helpers; images unsupported by the model are skipped with a warning.talk: speech-to-speech voice chat on padwan-llm's Realtime client — hold-Space push-to-talk (release to send), batched mic audio flushed before commit, your transcript printed before the assistant's reply, generic assistant persona (--instructionsoverrides), clean Ctrl-C shutdown.just chatopens the chat TUI with.envloaded automatically;env.templatedocuments the expected keys.Depends on padwan-llm
feat/multimodal(polarsen-io/padwan-ai#24) — therealtimeextra and content-part helpers are not in a published release yet, hence the local editable source and draft status.