feat(voice): manage GPT-Live client delegation through async tools - #7258
feat(voice): manage GPT-Live client delegation through async tools#7258chasef07 wants to merge 1 commit into
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
There was a problem hiding this comment.
🟨 Backend exceptions leak into telemetry
When delegated work raises an exception containing customer data, ToolCallEnded.message publishes str(output) unmarked. Event consumers can persist sensitive exception text without redaction.
(Refers to this code)
Was this helpful? React with 👍 or 👎 to provide feedback.
| self._select_task = select_task or (lambda request: utils.shortuuid("task_")) | ||
| self._states: dict[str, _TaskState] = {} | ||
| self._requests: dict[str, DelegationContext] = {} | ||
| self._seen: set[tuple[str, str]] = set() |
There was a problem hiding this comment.
🔴 Delegation history grows quadratically
With the default selector, _states retains a full conversation snapshot for every completed delegation. Neither _states nor _seen removes completed entries. Long-running sessions can exhaust worker memory.
Learn more
The default selector creates a unique task key for every transport request. Each new _TaskState then imports the request's entire chat context, while _states retains every state for the session lifetime. The retained data therefore grows with the sum of all conversation snapshots rather than the conversation itself. _seen also retains every connection/request key, including keys from disconnected connections.
Example: After 1,000 independent delegations, the toolset keeps 1,000 task states. Later states each contain nearly the full 1,000-request conversation, producing roughly quadratic retained history.
Recommended fix: Remove completed one-request states when no execution can still reference them, and expire _seen entries when their connection ends. Preserve states only for task keys intentionally reused by select_task; this may require explicit task lifetime tracking or a bounded retention policy.
Was this helpful? React with 👍 or 👎 to provide feedback.
| call = llm.FunctionCall( | ||
| call_id=call_id, | ||
| name=name, | ||
| arguments=json.dumps(arguments), |
There was a problem hiding this comment.
🟡 Concurrent tasks collide on call IDs
When independent tasks reuse a backend call_id, execute_tool forwards both IDs into the shared executor. _ToolExecutor rejects the second call while the first remains registered. The second delegation fails instead of running its tool.
Learn more
A backend model's call ID correlates a call only within that model interaction or retained task history. ClientDelegation can run several independent task histories concurrently, but all child calls share one _ToolExecutor. That executor indexes running work globally by FunctionCall.call_id, so equal IDs from different tasks conflict even though their histories are independent.
Example: Task order and task weather both produce call_id="call_1" before either tool finishes. The order tool registers first. The weather tool then receives Task already running for call_id: call_1 and its delegation returns a failure.
Recommended fix: Give executor registrations a globally unique internal call ID, namespaced by task or delegation revision. Keep the backend's original call ID when constructing the model-facing retained FunctionCall and FunctionCallOutput, with an explicit mapping between model and executor IDs.
Was this helpful? React with 👍 or 👎 to provide feedback.
| extra={ | ||
| "delegation_id": request.id, | ||
| "task_id": task_id, | ||
| "revision": state.revision, | ||
| "connection_id": request.connection_id, | ||
| }, |
There was a problem hiding this comment.
🟨 Delegation metadata bypasses PII redaction
FunctionCall.extra stores request, task, and connection identifiers under unmarked keys. Lifecycle telemetry can export customer-derived identifiers without redaction.
Was this helpful? React with 👍 or 👎 to provide feedback.
| if result.fnc_call_out is not None: | ||
| self._state.history.insert(result.fnc_call_out) | ||
| child.session.emit( | ||
| "function_tools_executed", | ||
| FunctionToolsExecutedEvent( | ||
| function_calls=[call], function_call_outputs=[result.fnc_call_out] | ||
| ), | ||
| ) |
There was a problem hiding this comment.
🟨 Delegated tool outputs bypass redaction
FunctionToolsExecutedEvent publishes delegated tool results in ordinary output fields. Customer and business data can reach telemetry consumers without a PII marker.
Was this helpful? React with 👍 or 👎 to provide feedback.
1 participant
Summary
GPT-Live client delegation currently bypasses the framework's tool lifecycle: the example creates independent backend tasks, rebuilds their context, and manually sends results. A corrected request can therefore be followed by an obsolete answer.
Add the reusable
ClientDelegationSDK toolset, backed byAsyncToolsetand the existing_ToolExecutor. GPT-Live dispatches managed requests as internallk_agents_delegatecalls, preserving task registration, cancellation, draining, and lifecycle events. The example now consumes this SDK capability instead of owning a controller or a mandatory routing LLM.This follows the async-tool direction of draft #6602, reviewed at
197cbaa, and uses its builtin delegation tool name. The client-specific interface is proposed for review alongside that work; this does not copy its broaderAgentSession(delegation_llm=...)API. Adjacent fixes in #7230, #7234, #7229, #7239, #7231, and #7238 are outside this diff.Validation
test_gpt_live_client_delegation.py,test_tools.py,test_duplex_adapter.py, andtest_session_host.py.AgentSession, shared executor, and GPT-Live send/receive loops with an in-memory WebSocket: corrections, independent work, queued and streamed output races, cancellation-resistant results, tool outcomes, reconnect, handoff, shutdown, failures, and session isolation.git diff --checkpassed.make checkstops at 15 pre-existing formatting failures in README code blocks. Each was reproduced from unchangedorigin/main; the full-repository lint/type stages were not reached. These unrelated files are excluded from the PR.Limits