[weiguangli-io]

Problem

In multi-threaded deployments (e.g. Vertex AI Agent Engine), each incoming
request runs in a fresh OS thread with its own asyncio event loop. When the
same Gemini instance is reused across requests, the previously
@cached_property api_client (a google.genai.Client) is bound to the
old, now-closed event loop via its underlying aiohttp / httpx session:

RuntimeError: Event loop is closed

Root cause: @cached_property stores the first Client created permanently on
the instance, regardless of which event loop is currently running.

Fixes #5538.

Solution

Replace @cached_property on api_client and _live_api_client with a
regular @property that stores a (loop, client) pair in __dict__. On each
access:

• If the running event loop matches the cached loop → return the cached client
  (no extra allocation on the hot path).
• Otherwise → create a fresh Client and cache it for the current loop.

This means each OS thread's event loop gets an isolated Client instance with
its own HTTP session, while calls within the same loop still share one client.

No public API is changed. Subclasses that override api_client with
@cached_property continue to work; the docstring hint to use @property
instead has been removed as the base class now handles this automatically.

Testing

• Added test_api_client_cached_per_event_loop: two threads each run their own
  event loop and share a single Gemini instance; asserts they receive distinct
  Client objects.
• Added test_api_client_cached_within_same_event_loop: asserts a single
  Client is reused for repeated calls within one event loop.
• Existing tests continue to pass.

[ypenn21]

nice im waiting on this change to be resolved.. thanks for submitting the PR

[kkj333]

Hi! The CI failure is caused by the existing tests using mock.patch.object(gemini_llm, "api_client"), which worked when api_client was a @cached_property (stored as an instance attribute) but breaks now that it's a @property.

The fix is to replace those mocks with mock.patch("google.genai.Client", ...) instead. There are about 20 occurrences in test_google_llm.py.

Happy to open a separate PR with the test fixes if that would help!

[rohityan]

Hi @weiguangli-io , Thank you for your contribution! We appreciate you taking the time to submit this pull request. Please fix the failing tests.

[GWeale]

the tests have been red since 30 april, so this can't be applied as it stands. closing on that, not on the fix.

#5538 stays open and api_client is still a cached_property on main, so nothing has overtaken you. the ci break was narrowed down here: the tests patch api_client as an instance attribute and need to patch google.genai.Client instead. the loop-keyed property was sound — send a fresh pull request off current main if you want it.