fix: time out stalled LLM streams - #2446
Conversation
🦋 Changeset detectedLatest commit: ea96d22 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ea96d22866
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| stream.traceId ?? null, | ||
| ), | ||
| ); | ||
| }, idleTimeoutMs); |
There was a problem hiding this comment.
Bound the idle delay before passing it to setTimeout
When a caller supplies a finite streamIdleTimeoutMs above Node's maximum timer delay (2,147,483,647 ms), resolveStreamIdleTimeoutMs accepts it, but Node 24 emits TimeoutOverflowWarning and schedules this timer for 1 ms. A caller using a large duration to effectively relax the deadline will therefore have every stream fail almost immediately. Reject or clamp out-of-range values before scheduling; the mirrored agent-core-v2 implementation needs the same correction.
Useful? React with 👍 / 👎.
…ract/generate.ts
…ract/provider.ts
…tract/generate.test.ts
Related Issue
Related to #1798.
Supersedes #1799.
Problem
Provider request timeouts stop protecting the call after response headers arrive. If the streamed response then stops producing normalized message parts,
generate()can remain blocked initerator.next()indefinitely. Existing abort checks only run between parts, so they cannot recover a turn from this state.What changed
streamIdleTimeoutMsdriver option with a three-minute default in both current generation contracts.iterator.next()against the idle deadline and the caller abort signal, covering the first part, subsequent parts, and stream completion.StreamIdleTimeoutErrorwith provider, model, elapsed-time, and trace metadata.AbortError.@moonshot-ai/kosongroot entry and add a CLI patch changeset.No environment variable or new user-facing configuration surface is introduced, so the documentation audit found no user documentation update to make.
Checklist
gen-changesetsskill, or this PR needs no changeset.gen-docsskill, or this PR needs no doc update.