fix(schemas): add ExtraContent field to ChatStreamResponseChoiceDelta

[nghodkicisco]

PR Title

fix(schemas): add ExtraContent to ChatStreamResponseChoiceDelta

Problem Statement

When using Bifrost as a proxy layer between clients and OpenAI-compatible LLM providers (such as Google's Gemini via their OpenAI-compatible endpoint), streaming delta chunks that carry provider-specific metadata in an extra_content field are silently dropped during deserialization.

Google's Gemini models with extended thinking enabled return streaming chunks in this format:

// Thinking chunk
{"choices": [{"delta": {"content": "reasoning text...", "extra_content": {"google": {"thought": true}}, "role": "assistant"}}]}

// Final answer with thought signature
{"choices": [{"delta": {"content": "answer text", "extra_content": {"google": {"thought_signature": "CjIBjz1rX..."}}}}]}

// Tool call with thought signature
{"choices": [{"delta": {"tool_calls": [{"id": "call-123", "function": {...}, "extra_content": {"google": {"thought_signature": "..."}}}]}}]}

The extra_content field serves two purposes:

1. google.thought: true — marks chunks as internal reasoning/thinking (distinct from the final answer)
2. google.thought_signature — an encrypted opaque token that MUST be replayed on subsequent turns for Gemini to accept the conversation history

Without preserving extra_content on the delta, any proxy that routes through Bifrost's typed deserialization path:

• Loses thinking markers → clients cannot distinguish reasoning from final output
• Loses thought_signature → Gemini rejects subsequent turns with HTTP 400: "function call in content block N is missing a thought_signature"

This effectively breaks multi-turn agentic conversations with Gemini when extended thinking is enabled and traffic flows through a Bifrost-based gateway.

What This Solves

Adding ExtraContent json.RawMessage to ChatStreamResponseChoiceDelta ensures that:

1. Thinking content is visible — Clients see which chunks are reasoning vs final output
2. Multi-turn conversations work — thought_signature is preserved end-to-end, so Gemini accepts replayed conversation history on subsequent turns
3. Forward compatibility — Any future provider-specific metadata on streaming deltas flows through without requiring schema changes

Changes

• Add ExtraContent json.RawMessage field with json:"extra_content,omitempty" to ChatStreamResponseChoiceDelta
• No changes to UnmarshalJSON needed — the existing type Alias pattern automatically captures all struct fields including the new one
• Backward compatible: field is omitempty, only present when the upstream provider sends it
• Zero impact on providers that don't use extra_content

Note

ChatAssistantMessageToolCall already has an ExtraContent field (added previously) which handles extra_content on tool call objects. This PR completes the coverage by adding it to the delta itself, where thinking markers and the final-answer signature live.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: eb4b0cf4-8b08-4ea8-8904-f5de657d6fd0

📥 Commits

Reviewing files that changed from the base of the PR and between 654de46 and 89d577e.

📒 Files selected for processing (3)

• core/schemas/chatcompletions.go
• core/schemas/utils.go
• plugins/jsonparser/utils.go

🚧 Files skipped from review as they are similar to previous changes (2)

• core/schemas/utils.go
• core/schemas/chatcompletions.go

---

📝 Walkthrough

Summary by CodeRabbit

Release Notes

• New Features

  • Streaming chat responses now include an additional metadata field (extra_content) to preserve provider-specific information.

• Bug Fixes

  • Improved message cloning to avoid shared-data mutation, including deep-copying tool-call extra content and reasoning details.
  • Streaming response handling now preserves audio content during response copying.

Walkthrough

A new ExtraContent json.RawMessage field tagged json:"extra_content,omitempty" is added to ChatStreamResponseChoiceDelta in core/schemas/chatcompletions.go to carry provider-specific metadata in streaming deltas. The deep-copy logic in core/schemas/utils.go is updated to clone both tool-call ExtraContent bytes and the entire ReasoningDetails slice with its nested pointer fields. The JSON parser utility in plugins/jsonparser/utils.go is similarly updated to deep-copy streaming delta structs with the new ExtraContent field, ensuring that message copies do not share underlying data with originals.

Changes

Streaming Delta and Message Schema Extensions

Layer / File(s)	Summary
ChatStreamResponseChoiceDelta schema with ExtraContent core/schemas/chatcompletions.go	Adds ExtraContent json.RawMessage field (tagged extra_content,omitempty) to ChatStreamResponseChoiceDelta, extending the streamed chunk payload to include provider-specific metadata.
Core schemas deep-copy for ExtraContent and ReasoningDetails core/schemas/utils.go	Updates DeepCopyChatMessage to deep-copy tool-call ExtraContent bytes and clone ChatAssistantMessage.ReasoningDetails with all scalar and pointer fields, preventing shared-data mutations.
JSON parser deep-copy for streaming delta ExtraContent plugins/jsonparser/utils.go	Updates deepCopyChatStreamResponseChoiceDelta to explicitly initialize delta struct fields and conditionally copy ExtraContent and ReasoningDetails (with deep-copied pointer fields) from original delta into result when non-empty.

Sequence Diagram

sequenceDiagram
  participant ChatStreamDelta as ChatStreamResponseChoiceDelta
  participant CoreDeepCopy as DeepCopyChatMessage
  participant DeltaDeepCopy as deepCopyChatStreamResponseChoiceDelta
  
  ChatStreamDelta->>DeltaDeepCopy: original delta with ExtraContent
  DeltaDeepCopy->>DeltaDeepCopy: initialize struct fields
  DeltaDeepCopy->>DeltaDeepCopy: copy ExtraContent bytes to result
  DeltaDeepCopy->>DeltaDeepCopy: return delta without shared refs
  
  ChatStreamDelta->>CoreDeepCopy: message with tool calls and ReasoningDetails
  CoreDeepCopy->>CoreDeepCopy: deep copy tool call ExtraContent
  CoreDeepCopy->>CoreDeepCopy: deep copy ReasoningDetails slice and fields
  CoreDeepCopy->>CoreDeepCopy: return message without shared refs

Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

• maximhq/bifrost#4470: Both PRs preserve provider-specific extra_content; this PR's deep-copy logic in DeepCopyChatMessage and JSON parser directly handle ExtraContent added to ChatAssistantMessageToolCall in the related PR.

Suggested reviewers

• danpiths
• akshaydeo

Poem

🐇 A delta so lean, a field so small,
extra_content now answers the call.
Deep copies dance, no refs left to share,
Gemini thoughts hop through the air.
Reasoning details cloned with care — the bunny hops forward! 🌱

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the main schema change: adding ExtraContent field to ChatStreamResponseChoiceDelta. It is concise and directly reflects the core modification described in the PR.
Description check	✅ Passed	The description covers most required template sections including Summary, Changes, Type of change (Bug fix), and Affected areas (Core). However, 'How to test', 'Checklist' confirmations, and other template sections are missing or incomplete.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies"

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[greptile-apps]

Confidence Score: 4/5

The schema addition and jsonparser/utils deep-copy are correct, but the framework streaming accumulator will silently drop the new delta-level ExtraContent on every chunk, defeating the main purpose of the PR for most production deployments.

The deepCopyChatStreamDelta function in framework/streaming/chat.go is the copy site called for every streaming chat chunk in the accumulator path, yet it has no case for the new ExtraContent field. Gemini thought markers and thought_signature tokens will be nil in the copied delta even after this PR lands, meaning multi-turn agentic conversations with extended thinking still break through this path.

framework/streaming/chat.go — deepCopyChatStreamDelta needs a copy block for the new delta-level ExtraContent field.

Important Files Changed

Filename	Overview
core/schemas/chatcompletions.go	Adds ExtraContent json.RawMessage to ChatStreamResponseChoiceDelta; the existing type Alias UnmarshalJSON pattern captures it correctly with no other changes needed.
core/schemas/utils.go	Extends DeepCopyChatMessage to deep-copy toolCall.ExtraContent and the new ReasoningDetails slice on assistant messages; logic is correct and symmetric with existing copy patterns.
plugins/jsonparser/utils.go	Updates deepCopyChatStreamResponseChoiceDelta in the JSON-parser plugin to copy Audio, deep-copy ReasoningDetails, and copy the new delta-level ExtraContent; correct and complete for this code path.
framework/streaming/chat.go	deepCopyChatStreamDelta is missing the copy of the new delta-level ExtraContent field, so Gemini thought markers and thought_signature tokens are silently dropped on every chunk that flows through the framework streaming accumulator.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Gemini as Gemini (upstream)
    participant Parser as JSON Parser / Schema UnmarshalJSON
    participant Framework as framework/streaming deepCopyChatStreamDelta
    participant Plugin as plugins/jsonparser deepCopyChatStreamResponseChoiceDelta
    participant Client as Downstream Client

    Gemini->>Parser: SSE chunk with extra_content (thought / thought_signature)
    Parser->>Parser: Populate ChatStreamResponseChoiceDelta.ExtraContent ✅
    Parser->>Framework: Pass delta to accumulator
    Framework->>Framework: deepCopyChatStreamDelta — ExtraContent NOT copied ❌
    Framework->>Client: "chunk.Delta.ExtraContent == nil (thought_signature lost)"

    note over Plugin: jsonparser plugin path
    Parser->>Plugin: Pass delta to JSON parser plugin
    Plugin->>Plugin: ExtraContent deep-copied ✅
    Plugin->>Client: ExtraContent preserved

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Gemini as Gemini (upstream)
    participant Parser as JSON Parser / Schema UnmarshalJSON
    participant Framework as framework/streaming deepCopyChatStreamDelta
    participant Plugin as plugins/jsonparser deepCopyChatStreamResponseChoiceDelta
    participant Client as Downstream Client

    Gemini->>Parser: SSE chunk with extra_content (thought / thought_signature)
    Parser->>Parser: Populate ChatStreamResponseChoiceDelta.ExtraContent ✅
    Parser->>Framework: Pass delta to accumulator
    Framework->>Framework: deepCopyChatStreamDelta — ExtraContent NOT copied ❌
    Framework->>Client: "chunk.Delta.ExtraContent == nil (thought_signature lost)"

    note over Plugin: jsonparser plugin path
    Parser->>Plugin: Pass delta to JSON parser plugin
    Plugin->>Plugin: ExtraContent deep-copied ✅
    Plugin->>Client: ExtraContent preserved

Loading

_{Reviews (7): Last reviewed commit: "fix(jsonparser): deep copy ExtraContent ..." | Re-trigger Greptile}

[akshaydeo]

@nghodkicisco there a couple of deep copy flows we have - can you confirm you have covered all those ?

[nghodkicisco]

@nghodkicisco there a couple of deep copy flows we have - can you confirm you have covered all those ?

Thanks Akshay, reviewing and adding fixes.

[nghodkicisco]

@akshaydeo Added the gap fixes. Kindly review.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@plugins/jsonparser/utils.go`:
- Line 337: The ReasoningDetails field assignment in the delta copy operation
performs a shallow copy by directly assigning the slice reference, which is
unsafe because the slice elements contain pointer fields that can be mutated
across different accumulators/consumers. Replace the direct assignment of
ReasoningDetails with a deep copy by creating a new slice and iterating through
the original ReasoningDetails elements, copying each element (including any
nested pointer fields) to the new slice, then assign this new slice to the
copied delta's ReasoningDetails field to prevent mutation leaks.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: c899cc98-cf7b-4d25-9dd4-03930dcd9fb1

📥 Commits

Reviewing files that changed from the base of the PR and between 902a172 and eb6f7c4.

📒 Files selected for processing (1)

• plugins/jsonparser/utils.go

The merge-base changed after approval.

[nghodkicisco]

Hello @akshaydeo , notnsue where al these commits came from. Can you help me understand what all to keep in the PR, or would you need me to make any changes to get this fix in latest release? Thanks

[akshaydeo]

Rebase your branch with dev - that should fix this 🙇‍♂️

[nghodkicisco]

Rebase your branch with dev - that should fix this 🙇‍♂️

Thanks @akshaydeo , done, kindly help review the same and lemme know if anything else needed before merging. Thanks.