LM Studio: send completions to /v1, keep discovery on the raw address - #1885
Merged
Conversation
Every completion against an LM Studio model failed with '`choices` is not set' — a baffling message for what was really a 404. The two halves of this class disagreed about what baseUrl means: discovery appends /v1/models to it, while the OpenAI SDK appends /chat/completions (the factory's own docs say to bake the full path into the base URL). No single configured value satisfies both, so models listed perfectly and then nothing could be called. Normalised at each use: the SDK gets the /v1 root it expects, discovery keeps reading the configured address. Doing only the first broke discovery — it read the same inherited field and built /v1/api/v1/models, which registered nothing and left an appliance with a local default model unable to start at all. Both paths are now explicit about which form they want. Found with a logging proxy in front of LM Studio: the request went to POST /chat/completions and came back 'Unexpected endpoint or method'. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C7Etsae7WnXfshf5wjDELB
igordayen
reviewed
Aug 7, 2026
| * the OpenAI surface under `/v1`, so append it unless the operator has | ||
| * already done so. | ||
| */ | ||
| internal fun chatBaseUrl(baseUrl: String?): String { |
Contributor
There was a problem hiding this comment.
hardcoded URL, should be localhost?
Is it always local URL?
|
|
||
| @Test | ||
| fun `chat base url gains the v1 the SDK expects`() { | ||
| assertEquals("http://127.0.0.1:1234/v1", LmStudioModelsConfig.chatBaseUrl("http://127.0.0.1:1234")) |
Contributor
There was a problem hiding this comment.
127.... hardcoding in many places....
Contributor
Author
There was a problem hiding this comment.
Yes, please clean up. Thank you!
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every completion against an LM Studio model fails with
`choices` is not set— a baffling message for what is really a 404.What is wrong
The two halves of
LmStudioModelsConfigdisagree about whatbaseUrlmeans:loadModelsFromUrl) appends/v1/models, so it wants the plain server address.OpenAiCompatibleModelFactory, where the SDK appends/chat/completions— the factory's own KDoc says "bake the full path intobaseUrl" — so it wants the address to already end in/v1.No single configured value satisfies both. With the documented default (
http://127.0.0.1:1234) models list perfectly and then nothing can be called, which is the worst shape for a bug: the surface that proves the integration works is the one that works.What this does
Normalises at each use. The SDK gets the
/v1root it expects; discovery keeps reading the configured address. The configured value stays the plain server address an operator would naturally write, and an operator who has already written/v1is not given two.Doing only the first half is a trap I fell into: discovery reads the same inherited field, so normalising it there built
…/v1/api/v1/models, registered nothing, and left an appliance whose default model was local unable to start at all. Both paths are now explicit about which form they want.How it was found
A logging proxy in front of LM Studio, because the error message pointed nowhere useful:
Retries 40ms apart were the tell — not a model thinking, an instant rejection.
Verified
On a real machine: 17 LM Studio models discovered at boot, chat completions answered, and an agentic RAG loop driven end to end by a local model. 8 unit tests cover the normalisation, including the trailing-slash and already-has-
/v1cases.🤖 Generated with Claude Code
https://claude.ai/code/session_01C7Etsae7WnXfshf5wjDELB