Skip to content

MAINT Simplify OpenAI Realtime receive state handling - #2988

Merged
Roman Lutz (romanlutz) merged 2 commits into
microsoft:mainfrom
ShivamSharma43:maint/2414-realtime-receive-state
Oct 9, 2026
Merged

Roman Lutz (romanlutz) merged 2 commits into
microsoft:mainfrom
ShivamSharma43:maint/2414-realtime-receive-state

Conversation

@ShivamSharma43

Copy link
Copy Markdown
Contributor

Description

Refactors RealtimeTarget.receive_events_async around an explicit per-turn state object, as proposed in #2414. Fixes #2414.

  • _RealtimeReceiveState (a private dataclass) holds the audio buffer, the transcripts, the response id the turn is bound to, the count of accepted events, and the soft-finish deadline. It encodes the existing rules:
    • Events for another response id are stale.
    • A response.done that arrives as the first event, with no audio, is left over from a prior turn's soft-finish.
    • The deadline is set once, by the first audio.done.
  • _apply_receive_event is the per-event step. It has the same shape as _OpenAIRealtimeDispatcher._route_event_async on the streaming side and handles stale filtering, delta accumulation, response.done, server errors and audio.done.
  • _next_receive_event_async is the wait step. It waits no longer than the fixed deadline and handles the stream ending, grace expiry, and a connection closing after audio has arrived.
  • receive_events_async still owns the receive lifecycle; its loop is now a few lines.

This is meant to preserve behavior. The fixed grace deadline already landed in #2420, so this PR keeps it (now as a rule enforced by the state object) rather than changing it. Barge-in isn't part of the atomic receive path; it lives in the streaming dispatcher, so that code is untouched. The only signature change is to the private _handle_response_done_event, which now takes a transcript count instead of the result object.

Roman Lutz (@romanlutz)

Tests and Documentation

  • All existing realtime tests pass unchanged, including the FIX bound OpenAI Realtime completion grace #2420 tests that pin the exact timeout sequence.
  • New tests cover these event sequences:
    • a normal turn
    • a repeated audio.done not moving the deadline
    • events for other responses being ignored
    • a late transcript within the grace period
    • real grace expiry
    • an error during grace
    • external cancellation during grace
    • stopping at the terminal response.done
  • The new receive-path tests also pass against the pre-refactor implementation.
  • On 5,000 randomly generated event sequences, the old and new receive_events_async gave the same results, exceptions, events consumed and timeouts.
  • pytest tests/unit/prompt_target passes (1604 passed, 120 skipped). ruff, ty and the other pre-commit hooks pass on the changed files.
  • No documentation changes. I didn't run JupyText because no notebooks or docs are affected.

This change was prepared with the help of Claude Code (an AI coding assistant).

Move the per-turn bookkeeping in RealtimeTarget.receive_events_async into a
typed _RealtimeReceiveState and split the loop body into a wait step
(fixed soft-finish deadline, stream end, connection close) and a per-event
step (stale-response filtering, delta accumulation, response.done, errors,
audio.done). receive_events_async still owns the receive lifecycle.

The grace deadline is set once by the first audio.done and is never
extended, as before; behavior is otherwise unchanged. Adds tests for the
event sequences: normal turn, late transcript within grace, grace expiry,
errors and cancellation during grace, stale responses, and stopping at the
terminal response.done.

Fixes microsoft#2414

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 4, 2026 19:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@romanlutz Roman Lutz (romanlutz) self-assigned this Oct 9, 2026
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@romanlutz
Roman Lutz (romanlutz) added this pull request to the merge queue Oct 9, 2026
Merged via the queue into microsoft:main with commit 1b89534 Oct 9, 2026
55 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MAINT Simplify OpenAI Realtime receive state handling

3 participants