Repository navigation
feat(agent): count tool calls and failures per tool - #2
ivanmilevtues wants to merge 1 commit into
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
CodeBoarding reviewStatus: 2 changed components See the full change in CodeBoarding. graph LR
n_Coding_Agent_Application["Coding Agent Application"]
n_AI_Model_Providers_and_Runtime["AI Model Providers and Runtime"]
n_Durable_Execution_Engine["Durable Execution Engine"]
n_Protocol_and_Service_Mesh["Protocol and Service Mesh"]
n_Terminal_User_Interface_Framework["Terminal User Interface Framework"]
n_Build_and_Maintenance_Scripts["Build and Maintenance Scripts"]
n_Model_Context_Protocol_Client["Model Context Protocol Client"]
n_Codemode_Sandbox_Runtime["Codemode Sandbox Runtime"]
n_Telemetry_Instrumentation["Telemetry Instrumentation"]
n_Coding_Agent_Application -- "queries models and configures agent execution v…" --> n_AI_Model_Providers_and_Runtime
n_Coding_Agent_Application -- "renders terminal interface with" --> n_Terminal_User_Interface_Framework
n_Coding_Agent_Application -- "configures workflows and extensions in" --> n_Durable_Execution_Engine
n_Coding_Agent_Application -- "establishes connections and binds services via" --> n_Protocol_and_Service_Mesh
n_Coding_Agent_Application -- "authorizes and communicates through" --> n_Model_Context_Protocol_Client
n_Coding_Agent_Application -- "executes sandboxed code via" --> n_Codemode_Sandbox_Runtime
n_AI_Model_Providers_and_Runtime -- "prompts user login via" --> n_Coding_Agent_Application
n_Durable_Execution_Engine -- "invokes model runtime completions in" --> n_Coding_Agent_Application
n_Durable_Execution_Engine -- "queries model configurations from" --> n_AI_Model_Providers_and_Runtime
n_Durable_Execution_Engine -- "utilizes context and serialization utilities fr…" --> n_Protocol_and_Service_Mesh
n_Protocol_and_Service_Mesh -- "hosts and manages services defined in" --> n_Coding_Agent_Application
n_Protocol_and_Service_Mesh -- "replicates committed state from" --> n_Durable_Execution_Engine
n_Terminal_User_Interface_Framework -- "triggers user interface rendering in" --> n_Coding_Agent_Application
n_Build_and_Maintenance_Scripts -- "verifies and generates catalog definitions from" --> n_AI_Model_Providers_and_Runtime
n_Build_and_Maintenance_Scripts -- "invokes session and storage utilities in" --> n_Coding_Agent_Application
n_Build_and_Maintenance_Scripts -- "manages session documents via" --> n_Durable_Execution_Engine
n_Model_Context_Protocol_Client -- "persists OAuth credentials via" --> n_Coding_Agent_Application
n_Codemode_Sandbox_Runtime -- "calls back into tool execution in" --> n_Coding_Agent_Application
classDef added fill:#1f883d,stroke:#0b5d23,color:#ffffff;
classDef modified fill:#bf8700,stroke:#7d4e00,color:#ffffff;
classDef deleted fill:#cf222e,stroke:#82071e,color:#ffffff,stroke-dasharray:5 3;
class n_AI_Model_Providers_and_Runtime modified;
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d537fb8ae3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ? await failToolCallsFromTruncatedMessage(toolCalls, emit) | ||
| : await executeToolCalls(currentContext, message, config, signal, emit); | ||
| toolResults.push(...executedToolBatch.messages); | ||
| for (const result of executedToolBatch.messages) recordToolCall(result.toolName, result.isError); |
There was a problem hiding this comment.
When a tool invokes another tool through the exported runToolCall() pipeline, the nested call never reaches this block, so only the outer tool is counted. This occurs in packages/coding-agent/src/core/agent-session.ts, where nested tool calls delegate directly to runToolCall(); for example, a codemode call that invokes read twice records one codemode call and zero read calls or failures. Record outcomes in the common runToolCall/finalization path, while avoiding double-counting model-issued calls.
Useful? React with 👍 / 👎.
| errors: number; | ||
| } | ||
|
|
||
| const stats = new Map<string, ToolCallStat>(); |
There was a problem hiding this comment.
When multiple Agent instances run in the same Node process, this singleton map merges their calls even though agent state and sessionId are instance-specific. A session reading getToolCallStats() therefore reports tools and failures from unrelated sessions, and calling resetToolCallStats() for one session erases the other sessions' totals, including concurrent runs. Store the counters on the agent/run or key the read, record, and reset APIs by session.
Useful? React with 👍 / 👎.
This branch has not been deployed
1 participant
Test PR for the one-file CodeBoarding setup (webview earendil-works#210 / action earendil-works#143), on a repository that keeps its analysis in main. Adds a small tool-call-stats module that the agent loop records into, so the review has a new component and relations to report. Not meant to merge.