Skip to content

fix: time out stalled LLM streams - #2446

Open
askie wants to merge 1 commit into
MoonshotAI:mainfrom
askie:fix/stalled-llm-stream-timeout
Open

fix: time out stalled LLM streams#2446
askie wants to merge 1 commit into
MoonshotAI:mainfrom
askie:fix/stalled-llm-stream-timeout

Conversation

@askie

@askie askie commented Jul 31, 2026

Copy link
Copy Markdown

Related Issue

Related to #1798.

Supersedes #1799.

Problem

Provider request timeouts stop protecting the call after response headers arrive. If the streamed response then stops producing normalized message parts, generate() can remain blocked in iterator.next() indefinitely. Existing abort checks only run between parts, so they cannot recover a turn from this state.

What changed

  • Add a per-call streamIdleTimeoutMs driver option with a three-minute default in both current generation contracts.
  • Race every streamed iterator.next() against the idle deadline and the caller abort signal, covering the first part, subsequent parts, and stream completion.
  • Abort the provider transport through a merged signal and throw a retryable StreamIdleTimeoutError with provider, model, elapsed-time, and trace metadata.
  • Keep caller cancellation as the standard non-retryable AbortError.
  • Perform stream and iterator cleanup on a best-effort basis without awaiting provider cleanup promises that may also be stalled.
  • Strip the driver-only idle deadline before forwarding options to provider adapters.
  • Add contract tests for first-part and inter-part stalls, retry classification, transport abort, non-blocking cleanup, caller abort precedence, and option forwarding.
  • Export the new error from the @moonshot-ai/kosong root entry and add a CLI patch changeset.

No environment variable or new user-facing configuration surface is introduced, so the documentation audit found no user documentation update to make.

Checklist

  • I have read the CONTRIBUTING document.
  • I have linked a related issue, or explained the problem above.
  • I have added tests that prove my feature works.
  • Ran gen-changesets skill, or this PR needs no changeset.
  • Ran gen-docs skill, or this PR needs no doc update.

@changeset-bot

changeset-bot Bot commented Jul 31, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: ea96d22

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@moonshot-ai/kimi-code Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ea96d22866

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

stream.traceId ?? null,
),
);
}, idleTimeoutMs);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Bound the idle delay before passing it to setTimeout

When a caller supplies a finite streamIdleTimeoutMs above Node's maximum timer delay (2,147,483,647 ms), resolveStreamIdleTimeoutMs accepts it, but Node 24 emits TimeoutOverflowWarning and schedules this timer for 1 ms. A caller using a large duration to effectively relax the deadline will therefore have every stream fail almost immediately. Reject or clamp out-of-range values before scheduling; the mirrored agent-core-v2 implementation needs the same correction.

Useful? React with 👍 / 👎.

Pidbid added a commit to Pidbid/kkm that referenced this pull request Jul 31, 2026
Pidbid added a commit to Pidbid/kkm that referenced this pull request Jul 31, 2026
Pidbid added a commit to Pidbid/kkm that referenced this pull request Jul 31, 2026
Pidbid added a commit to Pidbid/kkm that referenced this pull request Jul 31, 2026
Pidbid added a commit to Pidbid/kkm that referenced this pull request Jul 31, 2026
Pidbid added a commit to Pidbid/kkm that referenced this pull request Jul 31, 2026
Pidbid added a commit to Pidbid/kkm that referenced this pull request Jul 31, 2026
Pidbid added a commit to Pidbid/kkm that referenced this pull request Jul 31, 2026
Pidbid added a commit to Pidbid/kkm that referenced this pull request Jul 31, 2026
Pidbid added a commit to Pidbid/kkm that referenced this pull request Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant