Repository navigation
Signal tool stability with waitForToolsUntil() and ToolChangeEvent #373
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -190,6 +190,9 @@ A <dfn>model context</dfn> is a [=struct=] with the following [=struct/items=]: | |
| object's event loop, and because it is event-loop-local, it can get out of sync from the | ||
| [=traversable navigable=]'s more "global" map. | ||
|
|
||
| : <dfn>pending tool readiness promises</dfn> | ||
| :: a [=set=] of {{Promise}} objects, initially empty. | ||
|
|
||
| </dl> | ||
|
|
||
| A <dfn>tool definition</dfn> is a [=struct=] with the following [=struct/items=]: | ||
|
|
@@ -500,6 +503,64 @@ a [=list=] of [=origins=] |exposed origins|, and an [=origin=] |accessing origin | |
|
|
||
| </div> | ||
|
|
||
| <div algorithm> | ||
| To <dfn>update the tools-ready state</dfn> given a {{Document}} |document|, run these steps: | ||
|
|
||
| 1. [=Assert=]: these steps are running on |document|'s [=relevant agent=]'s [=agent/event loop=]. | ||
|
|
||
| 1. Let |context| be |document|'s [=Document/associated ModelContext|associated | ||
| <code>ModelContext</code>=]'s [=ModelContext/internal context=]. | ||
|
|
||
| 1. Set |document|'s [=Document/tools-ready=] to true if |context|'s [=model context/pending tool | ||
| readiness promises=] is [=set/is empty|empty=]; otherwise false. | ||
|
|
||
| </div> | ||
|
|
||
| <div algorithm> | ||
| To <dfn>wait for tools to be ready</dfn> given a [=top-level traversable=] |traversable|, returning | ||
| "<code>ready</code>" or "<code>abort</code>", run these steps: | ||
|
|
||
| 1. [=Assert=]: these steps are running [=in parallel=]. | ||
|
|
||
| 1. Let |activeDocument| be |traversable|'s [=navigable/active document=]. | ||
|
|
||
| 1. [=Assert=]: |activeDocument| is [=Document/fully active=]. | ||
|
|
||
| 1. Let |areAllReady| be an algorithm that returns a [=boolean=] by running these steps: | ||
|
|
||
| 1. If |activeDocument| is not [=Document/fully active=], then return true. | ||
|
|
||
| Note: If |activeDocument| becomes inactive while waiting (for example, because |traversable| | ||
| navigated or was destroyed), returning true here immediately unblocks the wait. | ||
|
|
||
| 1. [=list/For each=] [=navigable=] |navigable| of |activeDocument|'s [=Document/inclusive | ||
| descendant navigables=]: | ||
|
|
||
| 1. Let |document| be |navigable|'s [=navigable/active document=]. | ||
|
|
||
| 1. If |document|'s [=Document/tools-ready=] is false, then return false. | ||
|
|
||
| Note: Although [=Document/tools-ready=] is modified on |document|'s [=relevant agent=]'s | ||
| [=agent/event loop=] and read here [=in parallel=], implementations synchronize updates to | ||
| [=Document/tools-ready=] (for example, via inter-process communication to the browser | ||
| process) ordered with updates to [=model context/tool map=]. | ||
|
|
||
| 1. Return true. | ||
|
|
||
| 1. If the result of running |areAllReady| is false, then wait until either the result of running | ||
| |areAllReady| is true, or an [=implementation-defined=] timeout has elapsed since this step | ||
| began. | ||
|
|
||
| Note: The timeout is measured from when this wait begins. Calling | ||
| {{ModelContext/waitForToolsUntil()}} with additional promises while a wait is already in progress | ||
| does not reset or extend the timeout, and calling it after |areAllReady| returns true does not | ||
| delay an operation that has already been unblocked. | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Chrome's implementation skips some frames ... may be worth mentioning that implementations may adjust this algorithm when browser agents take observations.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This algorithm will be shared between browser-initiated observations and getTools() (in a follow-up PR). The section where this algorithm is called during browser agent observations notes that observation behavior is implementation-defined. See "The times at which a browser agent performs an observation are implementation-defined." |
||
|
|
||
| 1. Return "<code>ready</code>" if |activeDocument| is [=Document/fully active=]; otherwise | ||
| "<code>abort</code>". | ||
|
|
||
| </div> | ||
|
|
||
| <div algorithm> | ||
| The <dfn>tool execute steps</dfn>, given a [=string=] |toolName|, a {{Document}} |targetDocument|, a | ||
| [=string=] |inputArguments|, an algorithm |completionSteps|, and a [=unique internal value=] |uuid|, | ||
|
|
@@ -704,6 +765,13 @@ Upon creation of the {{Document}} object, its [=Document/associated ModelContext | |
| <code>ModelContext</code>=] must be set to a [=new=] {{ModelContext}} object created in the | ||
| {{Document}}'s [=relevant realm=]. | ||
|
|
||
| Each {{Document}} object has a <dfn for=Document>tools-ready</dfn> [=boolean=], initially true. | ||
|
|
||
| Issue(#328): [=Document/tools-ready=] should be false when the {{Document}} is created, unless the | ||
| {{Document}} is not [=allowed to use=] the "{{tools}}" feature. This gives authors an opportunity to | ||
| use {{ModelContext/waitForToolsUntil()}} until a deterministic {{Document}} lifecycle point. The | ||
| exact lifecycle point still needs to be resolved. | ||
|
|
||
| <hr> | ||
|
|
||
| <xmp class="idl"> | ||
|
|
@@ -727,7 +795,10 @@ The {{ModelContext}} interface provides methods for web applications to register | |
| <xmp class="idl"> | ||
| [Exposed=Window, SecureContext] | ||
| interface ModelContext : EventTarget { | ||
| readonly attribute boolean toolsReady; | ||
|
|
||
| Promise<undefined> registerTool(ModelContextTool tool, optional ModelContextRegisterToolOptions options = {}); | ||
| undefined waitForToolsUntil(Promise<any> promise); | ||
| Promise<sequence<RegisteredTool>> getTools(optional ModelContextGetToolOptions options = {}); | ||
| Promise<DOMString> executeTool(RegisteredTool tool, optional object inputObject, optional ModelContextExecuteToolOptions options = {}); | ||
|
|
||
|
|
@@ -742,6 +813,11 @@ is a [=model context=] [=struct=] created alongside the {{ModelContext}}. | |
|
|
||
|
|
||
| <dl class="domintro"> | ||
| <dt><code><var ignore>document</var>.{{Document/modelContext}}.{{ModelContext/toolsReady}}</code></dt> | ||
| <dd> | ||
| <p>Returns true if the document's tools are ready to be read.</p> | ||
| </dd> | ||
|
|
||
| <dt><code><var ignore>document</var>.{{Document/modelContext}}.{{ModelContext/registerTool(tool, options)}}</code></dt> | ||
| <dd> | ||
| <p>Registers a tool that [=agents=] can invoke. Returns a rejected promise if a tool with the | ||
|
|
@@ -750,6 +826,14 @@ is a [=model context=] [=struct=] created alongside the {{ModelContext}}. | |
| or {{ModelContextTool/outputSchema}} is invalid.</p> | ||
| </dd> | ||
|
|
||
| <dt><code><var ignore>document</var>.{{Document/modelContext}}.{{ModelContext/waitForToolsUntil(promise)}}</code></dt> | ||
| <dd> | ||
| <p>Signals that the document is asynchronously registering or updating its tools, and that tool | ||
| consumers should wait until <var ignore>promise</var> settles (or a timeout elapses) before | ||
| reading the document's tools. Multiple promises can be registered; the document's tools are | ||
| considered ready once all registered promises have settled.</p> | ||
| </dd> | ||
|
|
||
| <dt><code><var ignore>document</var>.{{Document/modelContext}}.{{ModelContext/getTools(options)}}</code></dt> | ||
| <dd> | ||
| <p>Returns a promise that resolves to a list of registered tools from this document and its | ||
|
|
@@ -784,6 +868,33 @@ is a [=model context=] [=struct=] created alongside the {{ModelContext}}. | |
| ``` | ||
| </div> | ||
|
|
||
| <div class=example id=waiting-for-tools> | ||
| The following delays tool discovery until asynchronous tool registration completes: | ||
|
|
||
| ```js | ||
| const toolsRegistered = (async () => { | ||
| const config = await fetch('/api/tools-config').then(r => r.json()); | ||
| for (const tool of config.tools) { | ||
| await document.modelContext.registerTool({ | ||
| name: tool.name, | ||
| description: tool.description, | ||
| execute: async (input) => { /* ... */ } | ||
| }); | ||
| } | ||
| })(); | ||
|
|
||
| document.modelContext.waitForToolsUntil(toolsRegistered); | ||
| ``` | ||
| </div> | ||
|
|
||
|
|
||
| <div algorithm> | ||
| The <dfn attribute for=ModelContext>toolsReady</dfn> getter steps are: | ||
|
|
||
| 1. Return [=this=]'s [=relevant global object=]'s [=associated Document|associated | ||
| <code>Document</code>=]'s [=Document/tools-ready=]. | ||
|
|
||
| </div> | ||
|
|
||
| <div algorithm> | ||
| The <dfn method for=ModelContext>registerTool(<var>tool</var>, <var>options</var>)</dfn> method steps are: | ||
|
|
@@ -916,15 +1027,61 @@ The <dfn method for=ModelContext>registerTool(<var>tool</var>, <var>options</var | |
|
|
||
| 1. Run the following steps [=in parallel=]: | ||
|
|
||
| 1. [=Notify documents of a tool change=] given |tool owner| and |exposed origins|. | ||
| 1. [=Notify documents of a tool change=] given |tool owner| and |exposed origins|. | ||
|
|
||
| 1. [=Queue a global task=] on the [=webmcp task source=] given |global| to [=resolve=] |promise| | ||
| with undefined. | ||
| 1. [=Queue a global task=] on the [=webmcp task source=] given |global| to [=resolve=] |promise| | ||
| with undefined. | ||
|
|
||
| 1. Return |promise|. | ||
|
|
||
| </div> | ||
|
|
||
| <div algorithm> | ||
| The <dfn method for=ModelContext>waitForToolsUntil(<var>promise</var>)</dfn> method steps are: | ||
|
|
||
| 1. Let |global| be [=this=]'s [=relevant global object=]. | ||
|
|
||
| 1. Let |document| be |global|'s [=associated Document|associated <code>Document</code>=]. | ||
|
|
||
| 1. If |document| is not [=Document/fully active=], then [=exception/throw=] an "{{InvalidStateError}}" | ||
| {{DOMException}}. | ||
|
|
||
| 1. If |document| is not [=allowed to use=] the "{{tools}}" feature, then [=exception/throw=] a | ||
| "{{NotAllowedError}}" {{DOMException}}. | ||
|
|
||
| 1. If |promise|.\[[Promise]].\[[PromiseState]] is not `pending`, then: | ||
|
|
||
| 1. If |promise|.\[[Promise]].\[[PromiseState]] is `rejected`, then [=mark as handled|mark=] | ||
| |promise| as handled. | ||
|
|
||
| Note: When |promise| is `pending`, [=promise/react|reacting=] to it in the steps below marks | ||
| it as handled. If it is already `rejected` when passed in, returning early skips | ||
| [=promise/react=], so we explicitly mark it as handled here to prevent an | ||
| `unhandledrejection` event from firing. | ||
|
|
||
| 1. Return. | ||
|
|
||
| 1. Let |context| be [=this=]'s [=ModelContext/internal context=]. | ||
|
|
||
| 1. [=set/Append=] |promise| to |context|'s [=model context/pending tool readiness promises=]. | ||
|
|
||
| 1. [=Update the tools-ready state=] given |document|. | ||
|
Comment on lines
+1066
to
+1068
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Web IDL wraps every Chromium returns early when the promise has already settled, so it fires nothing. The Suggest coalescing: have update the tools-ready state queue a task that compares the current state with the last state we notified about, and notify only when they differ. This fits with your PR comment about coalescing the register/unregister notifications.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Updated the spec to match the WPTs and ignore already settled promises, nice catch! Deferred the ToolChangeEvent dispatch changes to a follow-up PR, will take care of toolchange event batching there. Also, my comment was about an existing issue with no batching for toolchange in the register/unregister case tracked in #374. |
||
|
|
||
| 1. Let |settleSteps| be the following steps: | ||
|
|
||
| 1. [=set/Remove=] |promise| from |context|'s [=model context/pending tool readiness | ||
| promises=]. | ||
|
|
||
| 1. [=Update the tools-ready state=] given |document|. | ||
|
|
||
| 1. [=promise/React=] to |promise|: | ||
|
|
||
| - If |promise| was fulfilled, then run |settleSteps|. | ||
|
|
||
| - If |promise| was rejected, then run |settleSteps|. | ||
|
|
||
| </div> | ||
|
|
||
| <div algorithm> | ||
| The <dfn method for=ModelContext>getTools(<var>options</var>)</dfn> method steps are: | ||
|
|
||
|
|
@@ -1579,6 +1736,13 @@ steps: | |
|
|
||
| 1. [=Assert=]: |traversable|'s [=navigable/active document=] is [=Document/fully active=]. | ||
|
|
||
| 1. If the result of [=waiting for tools to be ready=] given |traversable| is | ||
| "<code>abort</code>", then return. | ||
|
|
||
| Note: This happens if |traversable| navigated to a new {{Document}} or was destroyed while | ||
| waiting. In the navigation case, implementations will typically [=perform an observation=] | ||
| again for the new [=navigable/active document=]. | ||
|
|
||
| 1. Let |observation| be a new [=observation=]. | ||
|
|
||
| 1. Let |flat descendants| be the [=Document/inclusive descendant navigables=] of |traversable|'s | ||
|
|
@@ -1633,6 +1797,11 @@ observation=] given any [=top-level browsing context=] in the [=user agent=] [=b | |
| group set=], at any time, although implementations typically reserve this operation for when the | ||
| user is interacting with a [=browser agent=] while web content is in view. | ||
|
|
||
| Note: Calling {{ModelContext/waitForToolsUntil()}} only delays [=observations=] that the [=user | ||
| agent=] chooses to perform while a {{Document}}'s [=Document/tools-ready=] is false. Toggling | ||
| [=Document/tools-ready=] does not by itself cause or guarantee that the [=user agent=] will | ||
| [=perform an observation=]. | ||
|
|
||
|
|
||
| <h2 id="security-privacy">Security and Privacy Considerations</h2> | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we filter this to the frames that have the
toolspermission and could potentially register tools?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
tools-ready for such documents should stay true. that seems better in case another call-site accesses that bit.
the waitForToolsUntil already throws if its used on a document that doesn't have this permission. i made a note that tools-ready's initialization should keep the permission policy in mind. for now it already starts off as true but that'll change once the document lifecycle question is resolved.