Repository navigation
Description
Describe the bug
When a Live connection drops and ADK reconnects using the session resumption
handle (Vertex AI, SessionResumptionConfig(transparent=True)), the server
sometimes rejects the new connection with close code 1008:
google.genai.errors.APIError: 1008 None. BidiGenerateContent session is already connected to an existing client
ADK reconnects immediately, before Vertex has released the previous socket.
_live_llm_flow.run_live_flow treats only 1000, 1006, 1011 as recoverable, so
the 1008 is re-raised and the whole run_live stream fails. In a phone-call
agent, that ends the call.
The condition is transient: the server releases the old session within about a
second. Other Gemini Live clients report the same error and fix it with backoff
plus a retry. adk-go has a matching change for 1008 in isResumable
(google/adk-go#1993).
To reproduce
- Vertex AI backend (
GOOGLE_GENAI_USE_VERTEXAI=1), a native-audio Live model,
RunConfig(session_resumption=types.SessionResumptionConfig(transparent=True), context_window_compression=...). - Run
runner.run_live(...)long enough for the connection to drop or rotate
(GoAway / network drop) while a resumption handle is held. - Intermittently, ADK's reconnect attempt fails at
llm.connect()with 1008 and
run_liveraises instead of retrying.
Traceback (trimmed)
File ".../google/adk/live/_live_llm_flow.py", line 595, in run_live_flow
async with llm.connect(llm_request) as llm_connection:
File ".../google/adk/models/google_llm.py", line 665, in connect
async with self._live_api_client.aio.live.connect(
File ".../google/genai/live.py", line 1198, in connect
raw_response = await ws.recv(decode=False)
websockets.exceptions.ConnectionClosedError: received 1008 (policy violation)
BidiGenerateContent session is already connected to an existing client
...
google.genai.errors.APIError: 1008 None. BidiGenerateContent session is already
connected to an existing client
The error is raised inside live.connect() while it waits for the setup
response, so this is the reconnect attempt. A first connection has no handle
and cannot collide.
Where
src/google/adk/live/_live_llm_flow.py (2.14.1; same logic on main):
except errors.APIError as e:
...
# Error code 1000, 1006 and 1011 indicates a recoverable connection drop.
if e.code in [1000, 1006, 1011]:
if invocation_context.live_session_resumption_handle and (not client_closed):
...
continue
There is also no delay between reconnect attempts (while True → continue).
Expected behavior
When a resumption handle is held, a 1008 "already connected" on reconnect is
retried with a short backoff (e.g. 0.5 s, 1 s, 2 s), within
DEFAULT_MAX_RECONNECT_ATTEMPTS, instead of failing the stream.
Suggested fix
- Add 1008 to the recoverable codes only for the reconnect path (a handle is
set), ideally matched on the "already connected" reason so other policy
violations still fail fast. - Add exponential backoff before each reconnect attempt (
asyncio.sleep), for all
recoverable codes. Reconnecting instantly is what loses the race. - Make sure the previous connection is fully closed before opening the new one.
Related (smaller)
- A connect-time
TimeoutError("timed out during opening handshake")orOSError
on a reconnect with a handle also escapes as "An unexpected error occurred in
live flow". With a handle held, it could get the same bounded retry. mainalready keeps the last non-empty resumption handle when an update arrives
with an emptynew_handle. In 2.14.x, an empty update overwrites the handle, so
a drop during generation or a tool call cannot be resumed. Could that fix ship in
the next release?
Environment
- google-adk 2.14.0 (live flow identical in 2.14.1)
- google-genai 2.34.0
- websockets 15.0.1
- Python 3.11, Linux (Docker), uvloop
- Backend: Vertex AI Live API,
transparent=Truesession resumption
Workaround
We catch APIError 1008, and connect-time TimeoutError/OSError raised from
the websocket, around runner.run_live ourselves. We then open a fresh
run_live on the same ADK session after a short delay, which reseeds the
history. That works, but it loses server-side resumption and duplicates logic
ADK already has.
Activity
MaiAditya commentedon Nov 19, 2026
Thanks for the detailed report and the adk-go pointer. If you're not planning to send the PR yourself, I'd like to take this, following your suggested fix:
run_live_flow, treat a 1008 whose message contains "already connected to an existing client" as recoverable only on the reconnect path (a resumption handle is held and the client hasn't closed), so other 1008 policy violations still fail fast.DEFAULT_MAX_RECONNECT_ATTEMPTS.I'd leave the connect-time
TimeoutError/OSErrorcase from "Related" for a follow-up unless you'd prefer it in the same change.ThamaluM commentedon Nov 19, 2026
Thanks, please go ahead. Two requests, since this is used for phone calls where any added delay is dead air:
attemptis reset today, so each rotation on a long call starts from zero delay.Small note on matching: the reason is only in the message (
APIError: 1008 None. BidiGenerateContent session is already connected to an existing client), so match onstr(e)/e.message.The
TimeoutError/OSErrorfollow-up is fine.surajksharma07 commentedon Nov 20, 2026
@ThamaluM Reproduced it with a mocked connection on 2.15.0 and main: 1008 still isn’t treated as recoverable, and there’s no delay between reconnects. The empty-handle fix you mentioned did make it into 2.15.0.
Until #8260 lands, a workaround is to wrap Gemini.connect() so it retries only the “already connected to an existing client” 1008 when a handle is set. Skip the delay on the first try, then back off 0.5s/1s/2s. Because the retry happens inside connect, the server-side resumption should be kept instead of replaying history.
One thing before anyone tries it: the snippet link in the draft, <link/gist to fix_8258.py>, is still a placeholder. Please paste the real gist URL or inline the code, otherwise there’s nothing for ThamaluM to run.
Could you test it on a few real calls with GoAway/drops and let us know if it holds up? @MaiAditya, if it works for them, please check #8260 against the same cases (zero first delay, backoff reset after a successful connect) before it goes for review.
MaiAditya commentedon Nov 20, 2026
Thanks for reproducing it. I checked #8260 against both cases:
_reconnect_delay(1)returns 0, so the first resume after a drop or GoAway is immediate. Retries after a failed attempt wait 0.5 s, 1 s, 2 s, then 4 s (the cap).test_reconnect_delay_is_immediate_then_doubles_up_to_the_cappins this.test_run_live_backoff_restarts_after_a_connection_delivers_an_eventto cover this.The CLA check is the remaining blocker on my side; I'm sorting that out now.
surajksharma07 commentedon Nov 20, 2026
Thanks @MaiAditya, went through 6d6c11f and traced the loop. The first resume is immediate. Failed retries wait 0.5/1/2/4 s and it raises after the 5th. The old connection is already exited before the wait. Resumption updates come through as events, so the counter resets on every GoAway rotation. A 1008 after a GoAway still lands in the APIError branch, so it gets the backoff even though the RESUME continue skips the max-attempts check (fine as the follow-up you noted). CLA is green now, so it’s just waiting on workflow approval and review.
@ThamaluM sorry, the link in my last comment was a placeholder. Here’s the snippet. Use model=ResilientGemini(model="") on the agent:
It only retries around the connect/setup step, so a 1008 during receive still goes through ADK’s normal handling. Let us know how it holds up on real calls.