Skip to content

fix(agent): do not execute tool calls from a max_output_tokens-truncated response - #125

Open
w0nche0l wants to merge 2 commits into
mainfrom
fix/skip-tool-calls-on-max-output-tokens-truncation
Open

w0nche0l wants to merge 2 commits into
mainfrom
fix/skip-tool-calls-on-max-output-tokens-truncation

Conversation

@w0nche0l

@w0nche0l w0nche0l commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Problem

When the provider stops a turn at max_output_tokens, the response can still carry a function_call item whose arguments are whatever prefix fit in the budget (production capture: a reasoning model spent 3,000 tokens thinking and was cut off at {"commands":). responseHasToolCalls and extractToolCallsFromResponse only look at output[].type, so the loop parsed the fragment, logged Failed to parse tool call arguments, fed the error back to the model, and made another request against the same exhausted budget. Each truncated turn cost one wasted round trip, and the run only ended when a stop condition fired.

Fix

New outputItemsWithoutTruncatedCalls(response): on a response with status === 'incomplete' and incompleteDetails.reason === 'max_output_tokens', drop every function_call whose item status is not completed (when the provider omits status, the arguments must parse). On any other response it is response.output unchanged. extractToolCallsFromResponse and responseHasToolCalls read through it, and model-result.ts uses it wherever a response's output is written to state or echoed into the next request, so a dropped call is never sent back without an output. Calls that completed before the cut-off execute normally, matching the Vercel AI SDK and OpenAI Agents SDK loops. With no surviving call the loop finalizes on the truncated turn and the caller sees incompleteDetails, the right signal to raise the budget.

Tests

tests/unit/max-output-tokens-truncation.test.ts:

  • extractors: fully truncated response yields no calls; mixed response yields only the completed call.
  • real callModel against a mocked betaResponsesSend: fully truncated turn finalizes after 1 request with nothing executed (without the fix: 5 requests before stepCountIs(3)); mixed turn executes the completed weather call once, never the shell call, and the follow-up request's input carries function_call/function_call_output for the weather call only.

Context: OpenRouterTeam/openrouter-web#43149 carries this as a patch on @openrouter/agent@0.10.0 until the next release.

…ted response

A response the provider stopped at max_output_tokens can still carry a
function_call item whose arguments are whatever prefix fit in the budget.
The loop parsed that fragment, failed, fed the parse error back to the
model, and issued another request against the same exhausted budget, so
every truncated turn cost one wasted round trip.

extractToolCallsFromResponse and responseHasToolCalls now yield no tool
calls when status is incomplete with reason max_output_tokens. The loop
finalizes on the truncated turn and the caller sees incompleteDetails.
…ore the cut-off

Review feedback: a parallel turn can complete one function_call and be cut
off at max_output_tokens during the next. Rejecting the whole response
discarded the completed call. Filter per item instead: on a truncated
response a function_call is executable iff its status is completed
(arguments must parse when the provider omits status). The dropped item is
also excluded from state and from the next request's input, so no call is
echoed without an output.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant