Skip to content

Commit 7fb8310

Browse files
ericallamclaude
andauthored
feat(chat): hand run() a streamText with the managed options already applied (#4884)
## Summary Every `run()` had to spread `chat.toStreamTextOptions()`, and leaving it out dropped six things with no error: the managed prompt and its cache control, the registry-resolved model, the prompt's sampling config, telemetry, the skill tools, and the `prepareStep` that delivers steering, compaction and injected context. Before: ```ts /trigger/chat.ts import { chat } from "@trigger.dev/sdk/ai"; import { streamText, stepCountIs } from "ai"; import { anthropic } from "@ai-sdk/anthropic"; export const myChat = chat.agent({ id: "my-chat", tools: { myTool }, run: async ({ messages, tools, signal }) => streamText({ ...chat.toStreamTextOptions({ registry, tools }), model: anthropic("claude-sonnet-4-5"), system: "You are a helpful assistant.", messages, abortSignal: signal, stopWhen: stepCountIs(15), }), }); ``` After: ```ts /trigger/chat.ts import { chat } from "@trigger.dev/sdk/ai"; import { stepCountIs } from "ai"; import { anthropic } from "@ai-sdk/anthropic"; export const myChat = chat.agent({ id: "my-chat", system: "You are a helpful assistant.", registry, tools: { myTool }, run: async ({ messages, tools, signal, streamText }) => streamText({ model: anthropic("claude-sonnet-4-5"), messages, tools, abortSignal: signal, stopWhen: stepCountIs(15), }), }); ``` `streamText` comes from `run`'s argument and shadows the one imported from `ai`, so the correct call is now the shorter one and the managed options cannot be lost by omission. `chat.toStreamTextOptions()` is unchanged and still supported, and is still the only option in a custom agent. ## What changes when your options collide with the managed ones Spread order decides the outcome today, and losing is silent: ```ts streamText({ ...chat.toStreamTextOptions(), tools: myTools }) // skill tools dropped streamText({ ...chat.toStreamTextOptions(), prepareStep: mine }) // steering, compaction and injection off ``` The managed `streamText` merges instead. `tools` are passed into the helper so skill tools survive, and a `prepareStep` you pass runs after the managed one rather than replacing it. Everything else you name is left alone and wins, telemetry included. `system` is the exception: it can be set on `chat.agent({ system })`, through `chat.prompt.set()`, or at the call site, but only in one of them. Two at once throws and names the one that already owns it. No shape merges two system values across every supported AI SDK version, since v5 rejects an array of blocks and a structured block carries the provider options that make prompt caching work. ## chat.headStart and chat.startHeadStart `buildStreamTextOptions` supplies `messages`, `stopWhen: stepCountIs(1)` and `abortSignal`. Step 1 belongs to the route handler and step 2 onward to the agent, so re-setting `stopWhen` after a spread hands over a stream that has already run past step 1. Before: ```ts import { streamText, stepCountIs } from "ai"; export const POST = chat.headStart({ agentId: "my-chat", run: async ({ chat: helper }) => streamText({ ...helper.toStreamTextOptions({ tools: headStartTools }), model: anthropic("claude-sonnet-4-6"), system: "You are a helpful assistant.", }), }); ``` After: ```ts export const POST = chat.headStart({ agentId: "my-chat", run: async ({ streamText }) => streamText({ model: anthropic("claude-sonnet-4-6"), system: "You are a helpful assistant.", tools: headStartTools, }), }); ``` Passing `messages`, `prompt`, `stopWhen` or `abortSignal` to that `streamText` is a type error, with a runtime throw behind it for JavaScript callers. `tools` is yours to pass. The old shape only warned in prose. ## Also in here - `chat.agent()` takes `system`, `registry`, `cacheControl` and `systemProviderOptions`, so a managed prompt's model and its cache breakpoint no longer have to be passed at the call site. - `ChatStreamText` is exported for typing a loop factored out of `run`. The signature is taken from the AI SDK's own declaration: ```ts import type { streamText as aiStreamTextSignature } from "ai"; type AiStreamTextFn = typeof aiStreamTextSignature; ``` The peer range spans `ai` v5, v6 and v7, whose options differ. `typeof` resolves to whichever version is installed, so generics and tool inference are the caller's own and a v8 option needs no change here. **Actions.** `onAction` no longer receives `streamText` or `tools`: an action is a state edit, and one that returns `chat.turn()` (added in #4816) is followed by `run()`, which already has both. The action docs on this branch describe that model. `chat.toStreamTextOptions()` now also applies `chat.agent`'s `system`, `registry`, `cacheControl` and `systemProviderOptions`, so the spread form is equivalent to the `streamText` handed to `run()`, as the docs say; previously an agent's system prompt was silently dropped on that path. Those options are published on every boot, including for a `hydrateMessages` agent, which skips the snapshot boot block where they were first set. ## Verification Typecheck and the full suite pass on both `ai@6.0.116` and `ai@7.0.66`. The option merge is a pure function so the merged object can be asserted directly, which is how `experimental_telemetry` being dropped was caught: most `streamText` options never reach the provider, so a test that observes the model cannot see them. Run end to end against a deployed agent with every `run` rewritten to the new form and no spread anywhere: steering, undo across a cold boot, and regenerate all still pass, a caller's own `prepareStep` runs while managed steering still fires inside the turn, and consecutive injections arrive one per turn. The handover-owned options are pinned by `@ts-expect-error` assertions in a typechecked test rather than only by the runtime throw. --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
1 parent bffc7cc commit 7fb8310

27 files changed

Lines changed: 1460 additions & 252 deletions
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
---
2+
"@trigger.dev/sdk": minor
3+
---
4+
5+
`run()` now receives a `streamText` with your agent's managed options already applied, so they cannot be lost by leaving out the spread:
6+
7+
```ts
8+
run: async ({ messages, signal, streamText }) =>
9+
streamText({ model, messages, abortSignal: signal });
10+
```
11+
12+
Spreading `chat.toStreamTextOptions()` still works and is equivalent. The difference is what happens when your options collide with the managed ones. Passing `tools` after the spread replaces the skill tools, and passing your own `prepareStep` replaces the managed one, which silently switches off steering, compaction and injected context. The managed `streamText` merges tools and composes `prepareStep` instead, so neither can be turned off by accident.
13+
14+
`system` can be set at the call site, on `chat.agent({ system })`, or through `chat.prompt.set()`, but only in one of them: setting it in two places throws, because no single shape merges two system values across every supported AI SDK version, and dropping one silently is the failure this seam exists to prevent. Injected instructions append to whichever one is in play.
15+
16+
`chat.agent()` also takes `registry`, `cacheControl` and `systemProviderOptions` now, so a managed prompt's model and its cache breakpoint no longer have to be passed at the call site. `chat.toStreamTextOptions()` applies them as well, so spreading it into the `streamText` imported from `ai` stays equivalent to the one `run()` receives.
17+
18+
`chat.headStart` and `chat.startHeadStart` hand their `run` the same thing, carrying the options the handover protocol depends on. There it matters more: re-setting `messages`, `prompt`, `stopWhen` or `abortSignal` after a spread breaks the handover rather than degrading a feature, and nothing caught it. On the managed one those four keys are a type error; `tools` is yours to pass.

docs/ai-chat/actions.mdx

Lines changed: 32 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -44,74 +44,76 @@ export const myChat = chat.agent({
4444
// returning void → side-effect-only, no model call
4545
},
4646

47-
run: async ({ messages, signal }) => {
47+
run: async ({ messages, signal, streamText }) => {
4848
return streamText({ model: anthropic("claude-sonnet-4-5"), messages, abortSignal: signal });
4949
},
5050
});
5151
```
5252

5353
**Lifecycle flow:** Wake → parse action against `actionSchema``hydrateMessages` (if set) → **`onAction`** → apply `chat.history` mutations → emit `trigger:turn-complete` → wait for next message.
5454

55-
## Returning a model response from an action
55+
When `onAction` returns `chat.turn()`, the flow continues instead of emitting `trigger:turn-complete`: the edit is snapshotted, then a turn runs on the edited history with `trigger: "action-turn"`, so `onTurnStart`, `run()`, `onBeforeTurnComplete` and `onTurnComplete` all fire and the answer is persisted like any turn's. See [Answering after an action](#answering-after-an-action).
5656

57-
`onAction` can return a `StreamTextResult`, `string`, or `UIMessage` to produce a response. All three are sent to the frontend and added to the conversation just like a normal turn's answer, but the rest of the turn machinery (`onTurnStart`, `onTurnComplete`, etc.) still does not fire. A returned `UIMessage` must have `role: "assistant"`; its text and `data-*` parts are delivered, and other part types are dropped.
57+
## Answering after an action
58+
59+
An action is a state edit. To answer after the edit, return `chat.turn()`: the edit is applied and snapshotted, then a turn runs on the edited history exactly as a message turn does. `onTurnStart`, `run()`, `onBeforeTurnComplete` and `onTurnComplete` fire, the turn counter advances, and the answer gets everything a turn has: the agent's system prompt and tools, steering, compaction, injected instructions and persistence.
5860

5961
```ts
60-
onAction: async ({ action, messages }) => {
61-
if (action.type === "regenerate") {
62-
chat.history.slice(0, -1); // drop the last assistant
63-
return streamText({
64-
model: anthropic("claude-sonnet-4-5"),
65-
messages,
66-
stopWhen: stepCountIs(15),
67-
});
62+
onAction: async ({ action }) => {
63+
switch (action.type) {
64+
case "undo":
65+
chat.history.slice(0, -2);
66+
return; // edit only, no turn
67+
68+
case "regenerate":
69+
chat.history.slice(0, -1);
70+
return chat.turn(); // answer the edited history
71+
72+
case "retry-formal":
73+
chat.history.slice(0, -1);
74+
chat.inject([{ role: "system", content: "Answer formally this time." }]);
75+
return chat.turn(); // with a one-shot instruction
6876
}
69-
// other actions return void → side-effect only
7077
}
7178
```
7279

73-
This is useful for actions that both mutate state and want a fresh model response (regenerate-from-here, retry-with-different-style).
80+
`run()` receives the edited history with no incoming user message, the same shape as a `regenerate-message` turn, and its `trigger` is `"action-turn"`, so a `run()` that returns early on `"action"` (the pre-May behaviour, when actions invoked `run()` directly) still answers. Returning anything other than `chat.turn()` or nothing is an error; a response can no longer be returned from `onAction` directly.
7481

7582
### Actions and persistence
7683

77-
An action is not a turn, so `onTurnComplete` never fires, and that is where an app that owns its own transcript normally writes. What that means depends on which persistence model you use.
84+
An action that returns nothing does not fire `onTurnComplete`, and that is where an app that owns its own transcript normally writes. What that means depends on which persistence model you use.
7885

79-
**Platform-managed** (no `hydrateMessages`): nothing to do. After an action that changed the conversation (a `chat.history` mutation, a response returned from `onAction`, or both), the runtime writes the snapshot, so the change survives the run ending.
86+
**Platform-managed** (no `hydrateMessages`): nothing to do. After an action that changed the conversation, the runtime writes the snapshot, so the edit survives the run ending. An action that returns `chat.turn()` is followed by a turn, which persists its answer the way every turn does.
8087

81-
**Your own store** (`hydrateMessages` registered): the runtime deliberately does not write, because your store is the source of truth. A history mutation and a returned response both live only in the running worker until you persist them, and a continuation rehydrates from your store, not from what the worker had in memory. `chat.pipeAndCapture` hands you the same assistant message the runtime would have captured:
88+
**Your own store** (`hydrateMessages` registered): the runtime deliberately does not write, because your store is the source of truth. A history edit lives only in the running worker until you persist it, and a continuation rehydrates from your store, not from what the worker had in memory. Mirror each edit in your store, not only additions: a regenerate is a delete *and* an insert. The answer that follows `chat.turn()` reaches your store through `onTurnComplete`, like any turn's answer.
8289

8390
```ts
84-
onAction: async ({ action, messages }) => {
91+
onAction: async ({ action, chatId }) => {
8592
if (action.type === "undo") {
8693
chat.history.slice(0, -2);
8794
await db.deleteLastExchange(chatId); // the rollback is yours to persist
8895
}
89-
9096
if (action.type === "regenerate") {
9197
chat.history.slice(0, -1);
92-
await db.deleteLastAssistant(chatId); // drop the answer being replaced
93-
const { message } = await chat.pipeAndCapture(
94-
streamText({ model: anthropic("claude-sonnet-4-5"), messages })
95-
);
96-
if (message) await db.saveMessage(message); // then store the new one
98+
await db.deleteLastAssistant(chatId); // the delete half
99+
return chat.turn(); // the insert half arrives in onTurnComplete
97100
}
98101
},
102+
onTurnComplete: async ({ chatId, newUIMessages }) => {
103+
await db.saveMessages(chatId, newUIMessages);
104+
},
99105
```
100106

101-
Mirror each mutation in your store, not only the additions. A `chat.history` mutation is invisible to your database, so a regenerate is a delete *and* an insert. Saving the new answer without removing the old one leaves both in the canonical transcript, and the next hydration returns the two of them. (An append-only or branching store is the exception: there you write a new version and resolve the head on read.)
102-
103-
Returning the stream instead of piping it yourself still works and still reaches the browser, but you have no message to store, so the next run does not know about it.
104-
105107
## Gating actions on HITL state
106108

107109
If you have a [human-in-the-loop](/ai-chat/patterns/human-in-the-loop) tool waiting on `addToolOutput`, you usually want to refuse competing actions like `regenerate` until the answer arrives. [`chat.history.getPendingToolCalls()`](/ai-chat/backend#chat-history) gives you exactly that signal:
108110

109111
```ts
110-
onAction: async ({ action, messages, signal }) => {
112+
onAction: async ({ action }) => {
111113
if (action.type === "regenerate") {
112114
if (chat.history.getPendingToolCalls().length > 0) return; // gated
113115
chat.history.slice(0, -1);
114-
return streamText({ model: anthropic("claude-sonnet-4-5"), messages, abortSignal: signal });
116+
return chat.turn();
115117
}
116118
},
117119
```
@@ -135,7 +137,7 @@ The action payload is validated against `actionSchema` on the backend; invalid a
135137
## See also
136138

137139
- [`chat.history`](/ai-chat/backend#chat-history): the imperative API actions use to mutate state
138-
- [Sending actions from the frontend](/ai-chat/frontend#sending-actions): `transport.sendAction` ergonomics
140+
- [Sending actions from the frontend](/ai-chat/frontend#sending-actions): sending actions through `useChat` so a turn that follows one renders like any turn
139141
- [`hydrateMessages`](/ai-chat/lifecycle-hooks#hydratemessages): fires before `onAction` when set
140142
- [Branching conversations](/ai-chat/patterns/branching-conversations): pairs action handlers with backend-controlled history
141143
- [Human-in-the-loop](/ai-chat/patterns/human-in-the-loop): gating fresh actions while a tool is waiting

docs/ai-chat/anatomy.mdx

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ Everything below maps onto one annotated agent:
1818

1919
```ts trigger/my-agent.ts
2020
import { chat } from "@trigger.dev/sdk/ai";
21-
import { streamText, stepCountIs } from "ai";
21+
import { stepCountIs } from "ai";
2222
import { anthropic } from "@ai-sdk/anthropic";
2323

2424
export const myAgent = chat.agent({
@@ -36,9 +36,9 @@ export const myAgent = chat.agent({
3636

3737
// The turn loop. Messages arrive accumulated; you stream back.
3838
// Options, levels, and alternatives — see Backend.
39-
run: async ({ messages, tools, signal }) =>
39+
run: async ({ messages, tools, signal, streamText }) =>
4040
streamText({
41-
...chat.toStreamTextOptions({ tools }),
41+
tools,
4242
model: anthropic("claude-sonnet-4-5"),
4343
messages,
4444
abortSignal: signal,

0 commit comments

Comments
 (0)