Repository navigation
fix(ai-vercel-gateway): surface delta.reasoning as reasoning events on chat completions - #2152
pcbeingused333 wants to merge 2 commits into
Conversation
…n chat completions
AI Gateway streams thinking text on delta.reasoning when chat({ reasoning })
leaves the summary on. The chat adapter never overrode extractReasoning, so
the base dropped that text and no reasoning events were emitted.
Re-key aimock's delta.reasoning_content to delta.reasoning for the vercel-gateway chat provider and add it to the reasoning matrix, so the adapter's extractReasoning override runs end-to-end.
🦋 Changeset detectedLatest commit: 0e92bf9 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
No actionable comments were generated in the recent review. Walkthrough
Merge Risk:
|
|
Thanks for the PR, @pcbeingused333! 🙌 @jherr will take a look. Automated pre-review checks
Automated triage — a human review follows. |
The Vercel AI Gateway chat adapter sends
chat({ reasoning }), and with the summary on the gateway streams the thinking text ondelta.reasoning. The adapter never read that field.extractReasoningon the Chat Completions base returnsundefined, so the thinking was dropped and no reasoning events were emitted.🎯 Changes
packages/ai-vercel-gateway/src/adapters/text.tsoverridesextractReasoningand returns{ text }whendelta.reasoningis a non-empty string. One unit regression test, a sentence on the adapter doc, and a patch changeset.E2E:
vercel-gatewayjoins thereasoningset intesting/e2e/src/lib/feature-support.ts. aimock emitsdelta.reasoning_content, so the provider factory intesting/e2e/src/lib/providers.tspasses afetchthat re-keys it todelta.reasoning, the same wayopenai-compatible-legacyre-keys its frames. The frame-rewriting stream is now one shared helper (sseRewriteFetch) used by both.✅ Checklist
pnpm run test:pr, or these tests do not apply to this pull request. (The@tanstack/ai-vercel-gatewayvitest suite, thereasoningandchatE2E specs,tsc --noEmit,oxlint src --type-awareandoxfmt --checkwere run. The fulltest:prwas not.)docs/for this change, or this change is not user-facing.pnpm changeset), or this PR does not change a published package.🚀 Release Impact
Root cause
Issue. A Chat Completions stream with reasoning summary on produces no reasoning events. The answer text still arrives.
Cause. Vercel documents that reasoning content is streamed incrementally in
delta.reasoning. That field is outside the OpenAI Chat Completions chunk type, and the baseextractReasoningreturnsundefined. The request side already sends the gatewayreasoningobject, and addsexclude: trueonly when the summary is off.Fix. Override
extractReasoningand readdelta.reasoning. An empty string or a non-string is ignored. The Responses adapter is unchanged.Possible alternatives
Also accept
delta.reasoning_content, as the LLM Gateway and Cloudflare adapters do. Vercel's Chat Completions docs namedelta.reasoningonly, so this change reads that field and nothing else.Testing
In Docker
node:22, afterpnpm install --frozen-lockfile --filter @tanstack/ai-vercel-gateway...and building the package's workspace dependencies.With
packages/ai-vercel-gateway/src/adapters/text.tsfrommainand the new test:With this change,
npx vitest run --no-cacheinpackages/ai-vercel-gateway:tsc --noEmitandoxfmt --checkon the adapter, the test, the doc sentence and the changeset pass.oxlint src --type-awarereports no new errors.E2E, same image, after
pnpm install --frozen-lockfile --filter @tanstack/ai-e2e..., building the workspace dependencies andnpx playwright install --with-deps chromium, full provider matrix (CI=1).With the adapter source from
mainrebuilt,npx playwright test tests/reasoning.spec.ts:With this change rebuilt,
npx playwright test tests/reasoning.spec.ts tests/chat.spec.ts:tsc --noEmitintesting/e2eandoxfmt --checkon the two E2E files pass. The fullpnpm test:prand the rest of the E2E suite were not run. No call to the live gateway was made; the unit test mocks the SDK and the E2E run uses aimock.Manual check:
packages/ai-vercel-gateway/src/adapters/text.tsfrommain, then runnpx vitest run tests/text-adapter.test.tsinpackages/ai-vercel-gateway. Thedelta.reasoningtest fails.npx vitest runin that package. The suite passes.testing/e2e, runCI=1 npx playwright test tests/reasoning.spec.ts(full matrix;E2E_PROVIDERS=vercel-gatewaywithoutCInarrows it to this provider). Thevercel-gatewaycase shows the thinking block.Risk / rollback
Low risk: chunks without a non-empty string
delta.reasoningbehave as before. Revert this PR to drop the thinking text again.AI assistance was used to prepare this change and its tests.
Summary by CodeRabbit