Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
183 changes: 114 additions & 69 deletions continuations-explainer.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,9 +47,9 @@ tool to do so:
checkout(shipping_address, billing_address, card_details)
```

However, the implementation of this tool is actually spread across three
distinct pages (`/shipping`, `/billing`, `/checkout`). For the tool to complete,
the site needs to do a hard navigation across these three pages before finally
However, the implementation of this tool is actually spread across two
distinct pages (`/shipping` and `/billing`). For the tool to complete, the site
needs to do a hard navigation from `/shipping` to `/billing` before finally
submitting the checkout request to the server.

## Cross-frame tool invocation
Expand Down Expand Up @@ -80,7 +80,7 @@ tokens.

```js
partial dictionary ToolExecuteCallbackOptions {
required ModelContextToolInvocation invocation;
ModelContextToolInvocation invocation;
};

[Exposed=Window, SecureContext]
Expand All @@ -92,12 +92,22 @@ interface ModelContextToolInvocation {
```
Notes:

* `invocation` is passed to the execute callback of imperative tools registered
with `registerTool()`.
* `requestToken()` returns a DOMString to allow simple storage and transmission
of the token.
of the token. The token is a randomly generated, lowercase
[RFC 9562](https://www.rfc-editor.org/rfc/rfc9562) version 4 UUID, for
example `"6f0c8f9e-3b1a-4c2d-9e8f-0a1b2c3d4e5f"`.
* `requestToken()` returns a `Promise` to allow the token to be successfully
registered with agents and bound to the correct origin before returning.
* `requestToken()` rejects if the tool Promise associated with the invocation
has settled.
* A tool call can request at most one token. `requestToken()` rejects with an
`InvalidStateError` if:
* the document is not
[fully active](https://html.spec.whatwg.org/multipage/document-sequences.html#fully-active),
* the tool Promise associated with the invocation has settled, or the tool
call was canceled,
* a token was already requested for the tool call, or
* it is called from a callback passed to `resumeTool()`.

## Token Redemption

Expand All @@ -119,86 +129,104 @@ partial interface ModelContext {

* If the token is accepted for redemption, then the Promise resolves
successfully.
* The browser will call `callback` again shortly in the document that requested
it.
* The browser will then call `callback` shortly in the document that called
`resumeTool()`. The callback is invoked after the Promise returned by
`resumeTool()` has resolved.
* The `callback` will be passed the input object that was passed to the original
tool call and a new `ToolExecuteCallbackOptions.`
tool call and a new `ToolExecuteCallbackOptions`, with a new `signal` and
`invocation`.

**The following conditions apply on token redemption:**

* The token may only be redeemed once.
* Calls to resumeTool() while the initial tool is executing are allowed, but any
invocation of the callback will occur after the initial tool call is complete.
* Redemption fails if the document is not same-origin and part of the same
browsing context.
* If the original tool call is canceled by the caller (via AbortSignal for
web-platform callers, or internally by a built-in agent), then the token is
revoked.
* The browser may limit the lifetime of a token, i.e. it cannot be redeemed at
an arbitrary time later. The initial limit will be set by looking at agent
trajectory data, and will probably fall around 60 seconds.
* The token is implicitly canceled by the browser if the agent abandoned the
task, received different user instructions, was blocked from accessing the
site, etc.

If any of these conditions are not met, then `resumeTool()` rejects.

Note that `resumeTool()` is passed a `signal` via
`ToolExecuteCallbackOptions.signal` for the browser or agent to signal
cancellation of a resumed tool execution, the same way a `signal` was passed to
the original invocation of `executeTool()`. If that `signal` is triggered
during the execution of the `resumeTool()` callback, then its `Promise` will be
rejected.
* The token may only be redeemed once.
* The redeeming document must be same-origin with the document that requested
the token, and must be in the same tab. This may be the same document, a
different frame in the same page (for example, an `<iframe>`), or a document
loaded by a later navigation in the tab. Tokens can't be redeemed in other
tabs or windows (including popups).
* The redeeming document must be fully active, and must be allowed to use the
`"tools"` [permissions policy](https://www.w3.org/TR/permissions-policy/)
feature.
* The token must be redeemed within a limited time after it was requested.
Chromium currently allows 30 seconds. There is no limit on the number of
navigations between requesting and redeeming the token.
* `resumeTool()` may be called before the original tool call's Promise settles.
The callback does not wait for that Promise to settle.

If any of these conditions are not met, then `resumeTool()` rejects:

* with an `InvalidStateError` if the document is not fully active,
* with a `NotAllowedError` if the document isn't allowed to use tools, and
* with an `InvalidAccessError` if the token is malformed, unknown, expired,
already redeemed, or was requested by a different origin or in a different
tab.

## Cancellation

The caller may cancel the tool call after a token was requested (via
`AbortSignal` for web-platform callers, or internally by a built-in agent).

* If the token has not been redeemed, then it is **not** revoked. It can still
be redeemed until it expires, and the callback will run, but its result is
discarded. It is the developer's choice to redeem the token or not.

* If the token was redeemed and the callback is executing, then the `signal`
passed to the callback is aborted, and a `toolcancel` event is fired at
`document.modelContext` in the redeeming document.

## Agent Behaviors

When the tool requests a token, this is a signal to the agent that the tool has
not completed execution. Any output from the tool should be considered as
intermediate output that is not shown to the agent, and the browser should
expect final output to show to the agent when the tool invocation is completed.
not completed execution. The value returned by the original execute callback
(or its rejection) is ignored, and is not shown to the agent.

The browser will wait until all tokens are redeemed in the chain, all tool calls
have resolved, and show the model the final output of all tool calls. This
blocks the agent until the completion of a chain of continued tool calls.
The browser waits until the token is redeemed and the callback passed to
`resumeTool()` settles, and shows the agent the output of that callback as the
result of the tool call. This blocks the agent until the continued tool call
completes. The tool call fails if:

* the callback passed to `resumeTool()` rejects,
* the document running that callback is unloaded before it settles, or
* the token expires before it is redeemed.

# Examples

## Cross-document tool invocation

```js
// /billing.html
modelContext.registerTool({
// /shipping.html
document.modelContext.registerTool({
name: "checkout",
description: "Checks out the items in the cart.",
inputSchema: {
type: "object",
properties: {
billing_address: { type: "string" },
shipping_address: { type: "string" }
}, required: ["billing_address", "shipping_address"]
shipping_address: { type: "string" },
billing_address: { type: "string" }
}, required: ["shipping_address", "billing_address"]
},
execute: async (input, options) => {
// We'll need to continue on the next page for billing info.
const token = await options.invocation.requestToken();
// Handle billing_address (submit to server, etc.)
await submitBillingAddress(input.billing_address);
// Navigate to /shipping.html
window.location.href = `/shipping.html?token=${token}`;
// Handle shipping_address (submit to server, etc.)
await submitShippingAddress(input.shipping_address);
// Navigate to /billing.html
window.location.href = `/billing.html?token=${token}`;
}
)};
});
```

```js
// /shipping.html
// /billing.html
// Request resumption of the checkout tool.
modelContext.resumeTool(
document.modelContext.resumeTool(
new URLSearchParams(window.location.search).get('token'),
async (input, options) => {
// We'll need to continue again on the next page for confirmation.
const token = await options.invocation.requestToken();
// Handle shipping_address (submit to server, etc.)
await submitShippingAddress(input.shipping_address);
// Navigate to /confirmation.html
window.location.href = `/confirmation.html?token=${token}`;
// Handle billing_address (submit to server, etc.)
await submitBillingAddress(input.billing_address);
const order = await placeOrder();
// This is the result of the checkout tool call.
return `Order ${order.id} placed.`;
}
);
```
Expand All @@ -219,28 +247,37 @@ site structure for tools that cross document boundaries.

Tokens cannot be requested or redeemed on inactive documents or frames.

_Question:_ If a tool requests a token, and at a later time the tool call does
not complete (because the document becomes inactive or unloaded), then does the
token remain valid?
If the document that requested a token is unloaded before the original tool
call's Promise settles (for example, because the tool navigated it), then the
token remains valid. This is the main use case for continuations.

_Question:_ If a document requests a token then performs a cross-origin
navigation, does the token remain valid? If so, how many navigations are
allowed before the token expires?
If the document running the callback passed to `resumeTool()` is unloaded
before the callback settles, then the tool call fails.

If a document requests a token then performs a cross-origin navigation, the
token remains valid, and can be redeemed by a later document that is
same-origin with the document that requested it (for example, after returning
from a payment provider). There is no limit on the number of navigations, only
on the time before the token is redeemed.

Unregistering a tool after it has requested a token does not affect the
continuation.

# Security and Privacy Considerations

The token should be unguessable. Tokens should be created and tracked in a
trusted process to ensure access by permitted same-origin document(s) and
one-time redemption.
The token is unguessable: it is a randomly generated version 4 UUID. Tokens
are created and tracked in a trusted (browser) process, which enforces that
they are redeemed only once, only by same-origin documents in the same tab, and
only before they expire.

Its bearer will get access to the knowledge that a tool was called and the input
to that tool, which may contain private user information. For this reason, the
token is intended to be origin-bound. Leaking it to untrusted origins is not a
risk as it cannot be redeemed on them.

_Question:_ Do we allow tokens to be redeemed by a same-origin \<iframe\>
embedded in a cross-origin main frame? This depends on how we view tokens
related to storage partitioning.
Tokens may be redeemed by a same-origin `<iframe>` embedded in a cross-origin
main frame, provided the `<iframe>` is allowed to use tools. Redemption does
not check the top-level site.

# Accessibility Considerations

Expand All @@ -253,6 +290,14 @@ This proposal does not create new accessibility considerations.
- Document-to-worker tool calling (and vice versa)
- Document-to-server handoff

## Chained continuations

A callback passed to `resumeTool()` can't request another token, so a tool call
can only be continued once. Some tools span more than two documents, for
example a checkout flow across `/shipping`, `/billing` and `/confirmation`. A
future extension could allow the callback to request a new token from its
`invocation`, to continue the tool call again on a later document.

## Browsing context groups

Tools or agents may open additional browsing contexts (tabs, popup windows), and
Expand Down
Loading