Repository navigation
fix(chat): finalize streamed tool calls only when the stream completes - #629
Open
mattapperson wants to merge 6 commits into
Open
mattapperson wants to merge 6 commits into
mattapperson wants to merge 6 commits into
Conversation
A tool-call part was emitted as soon as the accumulated arguments parsed as JSON. An upstream early root close (valid JSON prefix followed by more fragments) therefore produced a truncated tool call followed by stray tool-input-delta parts, hiding the malformed input from the AI SDK's tool-call repair. Tool calls are now finalized in flush() with the full accumulated arguments; each is still emitted exactly once, including when the stream ends without a finish_reason. Related to #574 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… test Emit tool-input-start and the initial tool-input-delta as soon as a tool call's first chunk carries arguments, keeping only tool-input-end and tool-call deferred to flush. Add e2e/issues regression test for #574. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Handle tool-call deltas without an index: a new id (or function name) starts a new tool call, other deltas continue the most recent one. Previously these were stored at index -1 and never emitted. - Start a new tool call when a delta reuses an index with a different id. - Ignore an identical resend of already complete JSON arguments. - Emit tool-input-end for started tool calls dropped at flush (e.g. 'length'/'error' finish) so UI parts leave input-streaming. - Clarify the changeset on the tool-call / tool-input-end timing change.
mattapperson
force-pushed
the
claude/defer-streamed-tool-call-finalize
branch
from
November 4, 2026 14:39
c0284b9 to
958c45a
Compare
Treat an id-less indexless delta that repeats the current tool call's name as a continuation, and bind the first delta at an unseen index to the most recent tool call when that call started without an index and the delta does not identify a different call.
…oundaries A repeated function name without an id no longer continues the previous tool call once its arguments are already a complete JSON object/array.
A repeated function name without an id now starts a separate tool call only when the previous arguments are a complete JSON object/array and the delta begins a new, different one. Whitespace and identical resends continue the call. Cheap string checks gate the full parse so long same-name streams are not reparsed per fragment.
christineschen
approved these changes
Nov 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Related to #574
Root cause of #574 is server-side. OpenRouter's backend does not honor
eager_input_streaming: falseon tools; this package already forwards the field correctly (tools[i].eager_input_streaming), so the invalid streamed Claude tool arguments can't be fixed here. This PR fixes a provider-side bug that hid that failure.Bug: In
doStream, atool-callpart was emitted as soon as the accumulated arguments happened to parse as JSON. With an "early root close" from the upstream (a valid JSON prefix followed by more fragments), e.g. fragments'{"a":[1]}','</a>\n<b>','[2]}', the provider emitted a truncated-but-valid tool call with input{"a":[1]}and then kept emittingtool-input-deltaparts after it. The model's real (malformed) payload never reached the AI SDK's tool-call error handling /experimental_repairToolCall(#575).Fix: Tool calls are no longer finalized mid-stream. They are finalized in
flush()using the full accumulated arguments:tool-calls(including the existing missing/other->tool-callsinference from Kimi K2.5 returns undefined finishReason after tool calls, breaking agentic loops #420) orstop: every tool call is forwarded, invalid JSON as-is ('{}'only for empty arguments, as before).length,error,content-filter): only tool calls with parsable arguments are forwarded, matching previous behavior for truncated output.tool-input-start -> delta... -> end -> tool-calllifecycle, including single-chunk arguments and streams that end without afinish_reason.The early emission dates back to the original AI SDK v5 port (#77), inherited from the openai-compatible provider; nothing relies on mid-stream emission beyond event ordering. Because arguments are no longer parsed per delta, the
JsonCompletenessScanneroptimization from #586 is no longer needed and is removed (it was internal, not exported).Behavior change:
tool-input-start/tool-input-deltastill stream as arguments arrive (including single-chunk arguments), buttool-input-endandtool-callnow arrive at the end of the stream instead of as soon as the JSON closes, so a tool'sonInputAvailableand the UItool-input-availablestate fire at end of stream. With ai@7, tool execution timing is unchanged (tools already run after the model call ends). For whitespace trailing a complete object, theinputnow includes it (still valid JSON).Delta merging hardening (also in this PR):
index(optional in the schema since chore: make index optional #99) are now handled. Before, the first one landed attoolCalls[-1]and was never emitted. Now a delta with a newid(or, without an id, a function name) starts a new tool call and other deltas continue the most recent one. A repeated name with no id only splits off a separate call when the previous arguments are a complete JSON object/array and the delta starts a new, different one. The first delta at a new index binds to an indexless call it continues.indexwith a different non-emptyidstarts a new tool call instead of being concatenated onto the previous one under the wrong name. Provider indexes map to dense internal slots.{"a":1}sent twice) is ignored instead of producing{"a":1}{"a":1}. This only triggers when the accumulated arguments are a complete JSON container and the delta is byte-identical, so appending it could never produce valid JSON.length,error, ...) now get a closingtool-input-end(notool-call), so UI parts don't stay in input-streaming.Checklist
pnpm stylecheckandpnpm typecheckpnpm testand all tests passChangeset
pnpm changesetto create a changeset fileTests added (mocked SSE streams), in
src/chat/index.test.tspluse2e/issues/issue-574-streamed-tool-call-finalization.test.ts(which also checksexperimental_repairToolCallreceives the full arguments): early root close, multi-chunk args with braces in strings, single-chunk args, interleaved parallel tool calls, stream withoutfinish_reason, invalid args withstop, truncated args withlength(plustool-input-endfor the dropped call), no per-delta parsing, deltas withoutindex(single call, two calls, multi-chunk continuations), a different id reusing an index, repeated id at the same index, identical full-args resend, and a fragment equal to incomplete accumulated args (not dropped).