fix: prevent llm fallback retries after output

[chenghao-mou]

Outer retries could replay text or tool calls after a provider failure. Mark the error non-retryable when retry_on_chunk_sent=False; preserve pre-output and opt-in retries.

Addresses AGT-3492. Adopts @dtran26's diagnosis and reproducers from agents-js#2477, matching the outer-retry fix in agents-js#2480.

Validated: 35 fallback tests, make check, and 2,911 broader tests. Excluded baseline Google credential and missing livekit-server failures, both reproduced on main.

Initial prompt and agent context

Model: GPT-6

Fix the Python LLM fallback adapter’s outer retry bug in livekit/agents.

We investigated livekit/agents-js#2477 and confirmed that Python shares one part of the bug: when a provider emits text or tool calls, then raises a retryable APIError, retry_on_chunk_sent=False stops provider fallback but does not stop LLMStream’s outer retry loop. Generation restarts and appends duplicate output.

Verified Python revision: 4de6232. Fetch current main and check whether this still applies.

Relevant paths:

• livekit-agents/livekit/agents/llm/fallback_adapter.py: _run re-raises the original error after output without changing retryability.
• livekit-agents/livekit/agents/llm/llm.py: _main_task retries that error.
• AgentSession passes connection options with max_retry=3 by default. Direct adapter.chat() defaults to zero outer retries.

A controlled provider reproduces this without credentials or network calls:

1. First request emits “The answer.”, then raises APITimeoutError(retryable=True).
2. Next request succeeds with “The answer.”
3. With retry_on_chunk_sent=False and outer max_retry=3, output becomes “The answer.The answer.”
   Tool-call emissions duplicate through the same path.

Python already isolates errors by stream, accepts successful child retries, and cancels active children correctly. Those JS-specific failures do not need porting. Python’s Inference provider marks errors non-retryable after output, but custom providers can expose the adapter bug.

Please:

• Find or create a Linear ticket before creating the branch.
• Reproduce the bug, then make the smallest coherent fix.
• Cover text and tool-call replay, preserve pre-output retries and metadata-only behavior, and preserve explicit retry_on_chunk_sent=True.
• Run relevant tests and required checks.
• Leave unrelated working-tree changes untouched.

Local investigation:

• /tmp/agents-js-2477-investigation/python-parity.md
• /tmp/agents-js-2477-investigation/python_parity.py
  The harness asserts the observed bug; update expectations for regression tests.

Related Node fix: livekit/agents-js#2480
Original diagnosis and reproducer: livekit/agents-js#2477 by dtran26. Preserve contributor credit if that work shapes the fix.

Do not post comments, replies, or resolve threads on my behalf.

let's create a draft PR.

[devin-ai-integration]

Devin Review found 1 potential issue.