Skip to content

Commit e2737f6

Browse files
committed
fix(chat): consume injected instructions per turn, not per options build
Draining the lane on read handed the injection to whichever chat.toStreamTextOptions() call ran first and dropped it from the rest. A run() that builds options twice, a classifier pass and then the answer, sent the instruction to nobody if it passed the second one to streamText, with no error anywhere. Consumption is now keyed on the turn, so every build in the turn carries the same instructions and the turn after it carries none. A hand-rolled loop with no turn context still drains on read.
1 parent 86d67fa commit e2737f6

4 files changed

Lines changed: 89 additions & 6 deletions

File tree

.changeset/inject-system-to-instructions.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,4 +4,4 @@
44

55
`chat.inject()` with `role: "system"` now works. It previously put the system message into the conversation, which AI SDK 7 rejects for every provider: the next turn died with a generic "An error occurred." and persisted an empty assistant message, so the agent looked like it had stopped answering. System-role context is now appended to the model's instructions, which is also the only way to inject context the agent treats as trusted.
66

7-
Two things to know. Instructions are delivered by `chat.toStreamTextOptions()`, so a `run()` that calls `streamText` without spreading it does not receive a system-role injection. The conversational lane has no such requirement. And an injection applies to the next inference call only, rather than repeating on every turn that follows it.
7+
Two things to know. Instructions are delivered by `chat.toStreamTextOptions()`, so a `run()` that calls `streamText` without spreading it does not receive a system-role injection. The conversational lane has no such requirement. And an injection applies to the next turn only, rather than repeating on every turn that follows it. Every inference call in that turn sees it, so a `run()` that builds options more than once gets the same instructions each time.

docs/ai-chat/background-injection.mdx

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -216,9 +216,11 @@ rather than into the transcript.
216216
`messages` either way.
217217
</Warning>
218218

219-
- An injection applies to the next inference call only. The lane is drained once
220-
applied, so a block injected in `onTurnComplete` shapes the following turn and is
221-
not repeated on every turn after it.
219+
- An injection applies to the next turn only. A block injected in `onTurnComplete`
220+
shapes the following turn and is cleared after it, so it is not repeated on every
221+
turn from then on. Within that turn it is consumed once rather than once per read,
222+
so a `run()` that builds options more than once sees the same instructions in
223+
every build.
222224
- The injected text is merged into a single instruction rather than added as a
223225
second block, because AI SDK 5 rejects an array of system blocks while accepting
224226
one structured block. Merging changes the cached prefix, so a cached system prompt

packages/trigger-sdk/src/v3/ai.ts

Lines changed: 26 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2712,6 +2712,10 @@ const chatBackgroundQueueKey = locals.create<ModelMessage[]>("chat.backgroundQue
27122712
const chatInjectedInstructionsKey = locals.create<SystemModelMessage[]>(
27132713
"chat.injectedInstructions"
27142714
);
2715+
/** The turn that consumed the instructions lane, so a second read in the same turn still sees it. */
2716+
const chatInstructionsConsumedTurnKey = locals.create<number | undefined>(
2717+
"chat.injectedInstructionsConsumedTurn"
2718+
);
27152719

27162720
/**
27172721
* Run-scoped pipe counter. Stored in locals so concurrent runs in the
@@ -4732,8 +4736,28 @@ function toStreamTextOptions(options?: ToStreamTextOptionsOptions): Record<strin
47324736
*/
47334737
const injectedInstructions = locals.get(chatInjectedInstructionsKey);
47344738
if (injectedInstructions && injectedInstructions.length > 0) {
4735-
const injectedText = injectedInstructions
4736-
.splice(0)
4739+
/**
4740+
* Consumed once per turn, not once per read. A `run()` that builds options
4741+
* twice, a cheap classifier pass and then the answer, has to see the
4742+
* injection in both: draining on read hands it to whichever call ran first
4743+
* and drops it from the rest without saying so. Outside a turn there is no
4744+
* turn to scope that to, so the lane drains on read there instead.
4745+
*/
4746+
const currentTurn = locals.get(chatTurnContextKey)?.turn;
4747+
const consumedTurn = locals.get(chatInstructionsConsumedTurnKey);
4748+
4749+
let blocks: SystemModelMessage[];
4750+
if (currentTurn === undefined) {
4751+
blocks = injectedInstructions.splice(0);
4752+
} else if (consumedTurn !== undefined && consumedTurn !== currentTurn) {
4753+
injectedInstructions.length = 0;
4754+
blocks = [];
4755+
} else {
4756+
locals.set(chatInstructionsConsumedTurnKey, currentTurn);
4757+
blocks = injectedInstructions;
4758+
}
4759+
4760+
const injectedText = blocks
47374761
.map((block) => (typeof block.content === "string" ? block.content : ""))
47384762
.filter(Boolean)
47394763
.join("\n\n");

packages/trigger-sdk/test/inject-system-instructions.test.ts

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -232,4 +232,61 @@ describe("chat.inject with a system role", () => {
232232
await harness.close();
233233
}
234234
});
235+
236+
it("carries the injection into every options build in the turn, not only the first", async () => {
237+
/**
238+
* A `run()` that builds options twice, a classifier pass and then the
239+
* answer, has to see the injection in both. Consuming on read hands it to
240+
* whichever call ran first and drops it from the rest, silently.
241+
*/
242+
const model = new MockLanguageModelV3({
243+
doStream: async () => ({ stream: textStream("ok") }),
244+
});
245+
246+
const seen: { first: boolean; second: boolean }[] = [];
247+
let injectedOnce = false;
248+
249+
const agent = chat.agent({
250+
id: "inject-system-two-builds",
251+
onTurnComplete: async () => {
252+
if (injectedOnce) return;
253+
injectedOnce = true;
254+
chat.inject([{ role: "system", content: "SENTINEL-BOTH-BUILDS" }]);
255+
},
256+
run: async ({ messages, signal }) => {
257+
const first = chat.toStreamTextOptions();
258+
const second = chat.toStreamTextOptions();
259+
const has = (o: { system?: unknown }) =>
260+
JSON.stringify(o.system ?? null).includes("SENTINEL-BOTH-BUILDS");
261+
seen.push({ first: has(first), second: has(second) });
262+
return streamText({ ...second, model, messages, abortSignal: signal });
263+
},
264+
});
265+
266+
const harness = mockChatAgent(agent, { chatId: "inject-system-two-builds" });
267+
268+
try {
269+
await harness.sendMessage({ id: "u1", role: "user", parts: [{ type: "text", text: "one" }] });
270+
await new Promise((r) => setTimeout(r, 40));
271+
272+
await harness.sendMessage({ id: "u2", role: "user", parts: [{ type: "text", text: "two" }] });
273+
await new Promise((r) => setTimeout(r, 40));
274+
275+
await harness.sendMessage({
276+
id: "u3",
277+
role: "user",
278+
parts: [{ type: "text", text: "three" }],
279+
});
280+
await new Promise((r) => setTimeout(r, 40));
281+
282+
// Turn 1 predates the injection, turn 2 carries it in both builds, turn 3 is clear again.
283+
expect(seen).toEqual([
284+
{ first: false, second: false },
285+
{ first: true, second: true },
286+
{ first: false, second: false },
287+
]);
288+
} finally {
289+
await harness.close();
290+
}
291+
});
235292
});

0 commit comments

Comments
 (0)