fix(embeddings): one request per text for the Vertex single-content models - #4204
Merged
Conversation
…odels `GeminiEmbeddings.encode` sends each text as its own Content, which is what stops a multimodal model fusing a batch into one vector (#4001). On Vertex the SDK now refuses that outright for most of the family: `embed_content` routes every model whose name contains `gemini` — bar `gemini-embedding-001` — and every `maas` model to the single-content `embedContent` endpoint, and raises ValueError: The embedContent API for this model only supports one content at a time. client-side for anything longer. So an install configured with `gemini-embedding-2-preview` on Vertex fails every retain and every recall before a request leaves the process, and `test_api` has been red on the live test that covers it. Batching is the only free variable — each text still has to be its own Content to come back as its own vector — so these models get one text per request. `gemini-embedding-001`, the non-gemini Vertex models, and the whole Gemini API path keep the configured batch size. The predicate is mirrored from `google.genai._transformers`, not imported: it is private, and a copy that drifts fails loudly here (the SDK raises) rather than silently sending batches that never worked.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
test-api (1/3)is red onmainand on every open PR, on the live Vertex test added by #4001:It is not a test problem.
GeminiEmbeddings.encodesends each text as its ownContent— that is #4001's fix, and it is what stops a multimodal model fusing a whole batch into one vector. The SDK now refuses more than oneContentfor most of the Vertex family:embed_contentroutes every model whose name containsgemini, exceptgemini-embedding-001, and everymaasmodel to the single-contentembedContentendpoint, and raises client-side before anything reaches the network.So an install configured with
gemini-embedding-2-previewon Vertex fails every retain and every recall, not just the test.The fix
Batching is the only free variable — each text still has to be its own
Contentto come back as its own vector — so for those modelsencodeissues one request per text.gemini-embedding-001, the non-geminiVertex models (text-embedding-005, …) and the entire Gemini API (non-Vertex) path keep the configured batch size.The model predicate is mirrored from
google.genai._transformers.t_is_vertex_embed_content_modelrather than imported: it is private, and a copy that drifts fails loudly here — the SDK raises — instead of silently sending batches that never worked.Tests
Six parametrized cases counting the requests
encodeactually issues for three texts, against a mocked client:gemini-embedding-2-previewgoogle/gemini-embedding-2-previewtext-multilingual-maas-002gemini-embedding-001text-embedding-005gemini-embedding-2-previewThey assert the per-request
contentslengths, so a regression shows up as three texts back in one call rather than as an exception — the fix is the request shape. The live test from #4001 is unchanged and should go green in the CI jobs that carry GCP credentials.ruffandtyclean;pytest tests/test_gemini_embeddings.py— 46 passed, 1 skipped (the live test, no local credentials).