#1815 - Fix Gemini OpenAI-compatible guardrail IT sends gpt-5-mini instead of selected Gemini model - #1816
Open
simeshev wants to merge 7 commits into
Conversation
…stead of selected Gemini model Bind the resolved service model onto request-level OpenAI chat options so Spring AI 2.0 does not fall back to `gpt-5-mini` for Gemini, MiniMax, LM Studio, Docker, custom OpenAI-compatible, or OpenAI services. Add regression coverage for converter wrapping, factory-created services, and direct OpenAI-compatible autoconfiguration paths. Signed-off-by: Slava Imeshev <imeshev@yahoo.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes OpenAI-compatible providers sending requests with Spring AI/OpenAI’s default model (gpt-5-mini) instead of the Embabel-configured model by explicitly binding the resolved model id onto request-level OpenAiChatOptions.
Changes:
- Added
OptionsConverter<*>.withOpenAiModel(model)to force the service’s resolved model onto request-levelOpenAiChatOptions. - Applied model binding across OpenAI-compatible factory services and multiple autoconfiguration paths (OpenAI, OpenAI custom, Docker local models, MiniMax).
- Added regression tests to ensure model propagation and delegate option preservation across converters and wiring paths.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| embabel-agent-openai/src/test/kotlin/com/embabel/agent/openai/OpenAiModelBindingOptionsConverterTest.kt | New converter-focused regression coverage for model binding, validation, and preservation of delegate behavior. |
| embabel-agent-openai/src/test/kotlin/com/embabel/agent/openai/OpenAiCompatibleModelFactoryTest.kt | Verifies factory-created OpenAI-compatible services and prompt options carry the service model. |
| embabel-agent-openai/src/main/kotlin/com/embabel/agent/openai/OpenAiCompatibleModelFactory.kt | Wraps the provided options converter so request options are pinned to the resolved service model. |
| embabel-agent-openai/src/main/kotlin/com/embabel/agent/openai/converters.kt | Introduces withOpenAiModel wrapper for binding the model onto request-level OpenAiChatOptions. |
| embabel-agent-autoconfigure/models/embabel-agent-openai-custom-autoconfigure/src/test/java/com/embabel/agent/autoconfigure/models/openai/custom/AgentOpenAiCustomAutoConfigurationTest.java | Adds regression ensuring custom OpenAI-compatible services bind configured model on request options. |
| embabel-agent-autoconfigure/models/embabel-agent-openai-custom-autoconfigure/src/main/kotlin/com/embabel/agent/config/models/openai/custom/OpenAiCustomModelsConfig.kt | Ensures custom OpenAI model services bind their model id onto request options. |
| embabel-agent-autoconfigure/models/embabel-agent-openai-autoconfigure/src/test/java/com/embabel/agent/autoconfigure/models/openai/AgentOpenAiAutoConfigurationTest.java | New regression test ensuring standard OpenAI wiring binds configured model on request options. |
| embabel-agent-autoconfigure/models/embabel-agent-openai-autoconfigure/src/main/kotlin/com/embabel/agent/config/models/openai/OpenAiModelsConfig.kt | Applies model binding to standard OpenAI model service creation. |
| embabel-agent-autoconfigure/models/embabel-agent-minimax-autoconfigure/src/test/kotlin/com/embabel/agent/config/models/minimax/MiniMaxOptionsConverterTest.kt | Adds regression coverage for binding MiniMax’s configured model id on OpenAI-compatible options. |
| embabel-agent-autoconfigure/models/embabel-agent-dockermodels-autoconfigure/src/test/java/com/embabel/agent/autoconfigure/models/docker/AgentDockerModelsAutoConfigurationTest.java | Adds integration-style test ensuring discovered Docker models bind discovered id onto request options. |
| embabel-agent-autoconfigure/models/embabel-agent-dockermodels-autoconfigure/src/main/kotlin/com/embabel/agent/config/models/docker/DockerLocalModelsConfig.kt | Ensures Docker-discovered OpenAI-compatible models bind their id onto request-level options. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: Slava Imeshev <imeshev@yahoo.com>
…NAI_API_KEY
DockerLocalModelsConfig.dockerLlmOf built its OpenAiChatModel with only the
sync client:
OpenAiChatModel.builder()
.openAiClient(openAiClient) // sync client only
...
.build()
In Spring AI 2.0, OpenAiChatModel.Builder.build() also requires an async
client. When .openAiClientAsync(...) is not supplied, the builder falls back
to OpenAiSetup.setupAsyncClient(...), which constructs a fresh client and
demands a credential from the environment (OPENAI_API_KEY).
Docker local endpoints have no API key, so on any environment where
OPENAI_API_KEY is unset (including CI) that fallback threw:
IllegalStateException: At least one credential source must be specified:
credential (apiKey), workloadIdentity, or adminApiKey
The per-model catch in dockerLocalModelsInitializer swallowed the exception
("Failed to register Docker model ..."), so the model bean was never
registered. This was a latent production bug: any real Docker model would
silently fail to register whenever OPENAI_API_KEY was absent. The new
AgentDockerModelsAutoConfigurationTest regression test made the failure
visible, failing at context.getBean("dockerModel-docker-test-model") with
NoSuchBeanDefinitionException.
Root cause is a Spring AI 2.0 migration gap. The old OpenAiApi was replaced
by the openai-java SDK, which has separate sync/async clients.
OpenAiCompatibleModelFactory already handles this correctly by building both
clients from resolved credentials and wiring both into the chat model; the
Docker config was only updated to build the sync client and so inherited the
async-fallback trap.
Fix mirrors the factory: build an openAiClientAsync with the same "no-auth"
placeholder key and wire .openAiClientAsync(openAiClientAsync) into the
OpenAiChatModel builder, keeping model construction independent of ambient
environment credentials.
Signed-off-by: Slava Imeshev <imeshev@yahoo.com>
simeshev
requested review from
alexheifetz,
azanux,
igordayen,
jasperblues and
johnsonr
July 22, 2026 21:54
Contributor
|
Root cause: Spring AI 2.0 changed merge behavior — when per-request options are present, the model bean's default options no So, same in Anthropic. |
igordayen
reviewed
Jul 23, 2026
Contributor
There was a problem hiding this comment.
@simeshev - I'm testing a generic solution applicable to all LLM providers. once gets finalized, it supersedes this PR
Spring AI 2.0 stopped merging a ChatModel's configured options into a prompt that already carries per-request options: OpenAiChatModel and AnthropicChatModel buildRequestPrompt return the prompt unchanged when getOptions() != null. Embabel always supplies per-request options via its OptionsConverter, and provider option types coerce a null model to a baked-in default in their constructor (OpenAiChatOptions -> gpt-5-mini, AnthropicChatOptions -> claude-haiku-4-5). Net effect: every call ignored the selected model. This is the same root cause as #1815 (OpenAI/Gemini) and it also affects Anthropic and any OpenAI-compatible provider. Fix it once at the single chokepoint. SpringAiLlmService now binds the model configured on the underlying ChatModel (chatModel.getOptions().getModel()) onto the converted request options in both createMessageSender and createMessageStreamer: ``` internal fun bindModel(options: ChatOptions, model: String?): ChatOptions = if (model.isNullOrBlank()) options else options.mutate().model(model).build() ``` - Source is the bean's configured model, not the service name, so there is no name == wire-id assumption; it restores the pre-2.0 merge semantics. - mutate() dispatches to each provider's overridden builder, preserving the concrete option type (OpenAiChatOptions/AnthropicChatOptions) and all fields. - Null/blank guard leaves converter output untouched for models that expose no configured default (bare doubles, exotic providers). The per-site withOpenAiModel() wrappers (the #1815 fix) are now redundant but harmless: they bind model = modelId, then this layer re-binds the identical id read from the same bean, so the operation is idempotent. They can be retired in a follow-up; left in place here to keep the change focused. Tests (test-first): SpringAiLlmServiceModelBindingTest reproduces the bug (red: expected gpt-4.1 but was gpt-5-mini) then passes, plus direct bindModel unit tests; AnthropicModelBindingTest proves end-to-end that the selected Sonnet model reaches the wire via the unchanged AnthropicOptionsConverter. Updated three tests whose strict ChatModel mocks now need a getOptions() stub. Signed-off-by: Slava Imeshev <imeshev@yahoo.com>
…l binding SpringAiLlmService now binds the selected model onto request options for every provider (reading it from the ChatModel's configured options), so the OpenAI-compatible withOpenAiModel() wrapper is redundant: each call site was just re-binding, idempotently, the same model id the generic layer already reads from the same bean. Two mechanisms doing one job is worse than one. Production: - Delete the withOpenAiModel extension and pass the raw converter at all four call sites: OpenAiCompatibleModelFactory (Gemini/MiniMax/LM Studio/BYOK), OpenAiModelsConfig, OpenAiCustomModelsConfig, DockerLocalModelsConfig. Tests: - Delete OpenAiModelBindingOptionsConverterTest (covered the wrapper; its override/field-preservation coverage now lives in SpringAiLlmServiceModelBindingTest). - OpenAiCompatibleModelFactoryTest: replace the two converter-level binding tests with one asserting the load-bearing invariant — the factory bakes the model into ChatModel.getOptions(), which the generic binder reads. - Retarget the Docker/OpenAI/OpenAI-custom autoconfig regression tests to assert service.getChatModel().getOptions().getModel() instead of converter output, keeping config-site coverage at the correct layer. - Drop the wrapper test from MiniMaxOptionsConverterTest. No behavior change: binding moves from the converter to the shared call-time path. Model construction still bakes the selected model into every ChatModel bean, so the wire model is unchanged. Signed-off-by: Slava Imeshev <imeshev@yahoo.com>
Collaborator
Author
|
@igordayen - I have generalized the fix and removed OpenAI - specific converter. |
…ches the wire Unit tests with mocks can prove request options carry the right model, but not that the provider actually served it. These live ITs close that gap and guard both variants of #1815: - cross-endpoint (Gemini via OpenAI-compat): a wrong model id is rejected by the endpoint, so a successful call is itself the proof; - same-provider silent (OpenAI, Anthropic): both the selected and the default model are valid, so the only tell is the model the provider reports back. To assert the served model, surface it: LlmMessageResponse gains a `model` field, populated in SpringAiLlmMessageSender from the response metadata (also useful observability — which model actually answered). ITs (gated by *_API_KEY, aborted rather than failed when the key lacks model access, so CI-safe): - OpenAiServedModelBindingIT: served model is the selected gpt-5.4, not the gpt-5-mini OpenAiChatOptions default. - GeminiServedModelBindingIT: blocking + streaming calls reach the Gemini OpenAI-compatible endpoint with a gemini model (the reported #1815 case). - AnthropicServedModelBindingIT: served a sonnet, not the claude-haiku-4-5 default. All four pass against live OpenAI/Gemini/Anthropic APIs. Reading response metadata.model tripped strict ChatResponseMetadata mocks that only stubbed usage; made those mocks relaxed in the four affected test files so getModel() returns "" instead of throwing. Signed-off-by: Slava Imeshev <imeshev@yahoo.com>
…ches the wire Unit tests with mocks can prove request options carry the right model, but not that the provider actually served it. These live ITs close that gap and guard both variants of #1815: - cross-endpoint (Gemini via OpenAI-compat): a wrong model id is rejected by the endpoint, so a successful call is itself the proof; - same-provider silent (OpenAI, Anthropic): both the selected and the default model are valid, so the only tell is the model the provider reports back. To assert the served model, surface it: LlmMessageResponse gains a `model` field, populated in SpringAiLlmMessageSender from the response metadata (also useful observability — which model actually answered). ITs (gated by *_API_KEY, aborted rather than failed when the key lacks model access, so CI-safe): - OpenAiServedModelBindingIT: served model is the selected gpt-5.4, not the gpt-5-mini OpenAiChatOptions default. - GeminiServedModelBindingIT: blocking + streaming calls reach the Gemini OpenAI-compatible endpoint with a gemini model (the reported #1815 case). - AnthropicServedModelBindingIT: served a sonnet, not the claude-haiku-4-5 default. All four pass against live OpenAI/Gemini/Anthropic APIs. Reading response metadata.model tripped strict ChatResponseMetadata mocks that only stubbed usage; made those mocks relaxed in the four affected test files so getModel() returns "" instead of throwing. Signed-off-by: Slava Imeshev <imeshev@yahoo.com>
|
Contributor
|
The call chain for a normal doTransform call: ToolLoopLlmOperations.doTransform() The PR fixes SpringAiLlmService.createMessageSender — but that method is never reached because ChatClientLlmOperations // SpringAiLlmService — PR fixes this // ChatClientLlmOperations — overrides above, PR doesn't touch this Similarly for the two extra sites in ChatClientLlmOperations (lines 351 and 510) and both sites in The PR's approach works for providers that use SpringAiLlmService directly without overriding createMessageSender. |
igordayen
reviewed
Jul 23, 2026
Contributor
There was a problem hiding this comment.
@simeshev - may I suggest considering the PR I just finally released:
#1818
Upon extensive testing of:
- embabel-examples
- embabel-agent-experimental
- edge cases
100% Coverage on new code, Sonar clean.
Note - embabel-agent-experimental includes testing of LlmMessageSender providers directly.
Thank you for implementing a generalized approach. Appreciate if you could consider adding your newly added IT tests on top of #1818
Thanks
| val message: Message, | ||
| val textContent: String, | ||
| val usage: Usage? = null, | ||
| val model: String? = null, |
Contributor
There was a problem hiding this comment.
Is this enhancement not in scope for this PR?
| @@ -139,4 +139,3 @@ class UnfoldingToolInjectionStrategy : ToolInjectionStrategy { | |||
| val INSTANCE = UnfoldingToolInjectionStrategy() | |||
| } | |||
| message = embabelMessage, | ||
| textContent = assistantMessage.text ?: "", | ||
| usage = usage, | ||
| model = response.metadata?.model, |
| * [model] is null or blank, so models that expose no configured default (bare test doubles, | ||
| * exotic providers) keep the converter's output. | ||
| */ | ||
| internal fun bindModel(options: ChatOptions, model: String?): ChatOptions = |
Contributor
There was a problem hiding this comment.
If the model is null or blank, you'd just be calling .mutate().build(), which clones the options unchanged — same result as returning options directly.
| .joinToString(" | ") | ||
| .lowercase() | ||
| return listOf( | ||
| "not_found", |
Contributor
There was a problem hiding this comment.
How was it sourced? sufficiently representative?
| * | ||
| * Requires GEMINI_API_KEY. Skipped (aborted) if the key lacks access to the chosen model. | ||
| */ | ||
| class GeminiServedModelBindingIT { |
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes OpenAI-compatible services sending requests with Spring AI/OpenAI’s default
gpt-5-miniinstead of the configured Embabel model.Spring AI 2.0 does not merge
OpenAiChatModeldefault options into runtimeOpenAiChatOptionswhen request options are present. Embabel always supplies runtime options, so OpenAI-compatible providers need the resolved model id bound directly onto those request options.Changes
withOpenAiModel(model)forOptionsConverter<*>.Verification
mvn -fae clean verifypassed across all 73 modules.git diff --checkandgit diff --cached --checkpassed.