Skip to content

fix: moonshot reasoning_content on tool_calls #2022

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
steebchen wants to merge 3 commits into from
Closed

fix: moonshot reasoning_content on tool_calls #2022

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2a76fa8
fix: ensure moonshot reasoning_content on tool_calls
steebchen Apr 16, 2026
b44b9a0
refactor: drop redis for moonshot reasoning_content
steebchen Apr 16, 2026
9a80273
Merge remote-tracking branch 'origin/main' into fix-attachment-issue
steebchen Apr 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 6 additions & 20 deletions apps/gateway/src/chat/chat.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3755,34 +3755,20 @@ chat.openapi(completions, async (c) => {
}
}

// For Moonshot provider, enrich assistant messages with cached reasoning_content
// This is needed for multi-turn tool call conversations with thinking models
// Moonshot requires reasoning_content in assistant messages with tool_calls
// Moonshot's thinking models reject assistant tool_call messages that lack
// reasoning_content. If the client echoes `reasoning` (OpenAI-style) we map
// it across; otherwise fall back to an empty string so multi-turn tool
// conversations don't 400.
if (usedProvider === "moonshot") {
const { redisClient } = await import("@llmgateway/cache");
for (const message of messages) {
if (
message.role === "assistant" &&
message.tool_calls &&
Array.isArray(message.tool_calls) &&
message.tool_calls.length > 0 &&
!(message as any).reasoning_content // Only add if not already present
!(message as any).reasoning_content
Copy link

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The guard !(message as any).reasoning_content treats an empty string as “missing”. Since this code can set reasoning_content to "" intentionally, subsequent passes will still enter the block and may overwrite an explicit empty reasoning_content (or redo work unnecessarily). Use an explicit undefined/null check (e.g., message.reasoning_content == null or typeof ... === "undefined") to mean “not present”.

Suggested change
!(message as any).reasoning_content
(message as any).reasoning_content == null

Copilot uses AI. Check for mistakes.
) {
// Get reasoning_content from the first tool call (all tool calls share the same reasoning)
const firstToolCall = message.tool_calls[0];
if (firstToolCall?.id) {
try {
const cachedReasoningContent = await redisClient.get(
`reasoning_content:${firstToolCall.id}`,
);
if (cachedReasoningContent) {
// Add reasoning_content to the message for Moonshot
(message as any).reasoning_content = cachedReasoningContent;
}
} catch {
// Silently fail - reasoning_content caching is optional
}
}
(message as any).reasoning_content = (message as any).reasoning ?? "";
Comment on lines +3816 to +3818
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify whether BaseMessage (or adjacent message types) already define reasoning/reasoning_content
# so we can avoid local casting entirely.
rg -n -C3 --type=ts '\b(type|interface)\s+BaseMessage\b|reasoning_content|reasoning'

Repository: theopenco/llmgateway

Length of output: 50377

🏁 Script executed:

#!/bin/bash
# Verify the exact code at lines 3758-3771 in apps/gateway/src/chat/chat.ts
sed -n '3758,3771p' apps/gateway/src/chat/chat.ts | cat -n

Repository: theopenco/llmgateway

Length of output: 760

Remove as any from Moonshot message normalization.

Lines 3769 and 3771 use as any, which bypasses type checks and violates the TypeScript rule in this repo. Use a narrow local type instead.

💡 Proposed fix 
 if (usedProvider === "moonshot") {
+	type MoonshotAssistantMessage = BaseMessage & {
+		tool_calls?: unknown[];
+		reasoning?: string | null;
+		reasoning_content?: string;
+	};
 	for (const message of messages) {
+		const moonshotMessage = message as MoonshotAssistantMessage;
 		if (
-			message.role === "assistant" &&
-			message.tool_calls &&
-			Array.isArray(message.tool_calls) &&
-			message.tool_calls.length > 0 &&
-			!(message as any).reasoning_content
+			moonshotMessage.role === "assistant" &&
+			moonshotMessage.tool_calls &&
+			Array.isArray(moonshotMessage.tool_calls) &&
+			moonshotMessage.tool_calls.length > 0 &&
+			moonshotMessage.reasoning_content == null
 		) {
-			(message as any).reasoning_content = (message as any).reasoning ?? "";
+			moonshotMessage.reasoning_content = moonshotMessage.reasoning ?? "";
 		}
 	}
 }
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@apps/gateway/src/chat/chat.ts` around lines 3769 - 3771, The code is using
"as any" on the message object when normalizing Moonshot fields; define a narrow
local type (e.g., interface MoonshotMessage { reasoning?: string;
reasoning_content?: string }) and then cast the message to that type (message as
MoonshotMessage) into a local const (e.g., const msg = message as
MoonshotMessage), and replace uses of (message as any).reasoning and
reasoning_content with msg.reasoning and msg.reasoning_content so you keep
strict typing while performing the same normalization in the code paths that set
reasoning_content from reasoning.

}
Comment on lines 3812 to 3819
Copy link

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When message.reasoning is present, this copies it into reasoning_content but leaves the original reasoning field on the message. For Moonshot this means the same reasoning text can be sent twice in the upstream payload (once as reasoning, once as reasoning_content), increasing request size/token usage and risking context-limit issues. Consider deleting reasoning after copying (or only setting reasoning_content if reasoning exists and then removing it) so the upstream payload contains a single copy of the reasoning text.

Copilot uses AI. Check for mistakes.
}
}
Comment on lines +3805 to 3821
Copy link

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This Moonshot-specific message enrichment is central to preventing multi-turn tool-call 400s, but there doesn’t appear to be automated coverage asserting the behavior (mapping reasoning -> reasoning_content, and falling back to "" only when absent, without overwriting an existing reasoning_content). Adding a focused unit/integration test around this transformation would help prevent regressions, especially since Moonshot may not be exercised in the existing e2e matrix when provider keys aren’t configured.

Copilot uses AI. Check for mistakes.
Expand Down
25 changes: 0 additions & 25 deletions apps/gateway/src/chat/tools/parse-provider-response.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -803,31 +803,6 @@ export function parseProviderResponse(
}
}

// Cache reasoning_content for Moonshot thinking models when tool_calls are present
// This is needed for multi-turn tool call conversations because Moonshot requires
// reasoning_content to be included in assistant messages with tool_calls
if (
usedProvider === "moonshot" &&
reasoningContent &&
toolResults &&
Array.isArray(toolResults) &&
toolResults.length > 0
) {
for (const toolCall of toolResults) {
if (toolCall.id) {
redisClient
.setex(
`reasoning_content:${toolCall.id}`,
86400, // 1 day expiration
reasoningContent,
)
.catch((err) => {
logger.error("Failed to cache reasoning_content", { err });
});
}
}
}

// For non-reasoning models that return their answer in reasoning_content
// (e.g. CanopyWave Mimo), move reasoning to content so the response is visible.
if (!supportsReasoning && !content && reasoningContent) {
Expand Down
Loading