Skip to content

Python: Allow concurrent MCP calls with request-owned headers - #10147

Merged
Eduard van Valkenburg (eavanvalkenburg) merged 6 commits into
mainfrom
parallel-toolbox-alternatives
Nov 6, 2026
Merged

Eduard van Valkenburg (eavanvalkenburg) merged 6 commits into
mainfrom
parallel-toolbox-alternatives

Conversation

@eavanvalkenburg

@eavanvalkenburg Eduard van Valkenburg (eavanvalkenburg) commented Nov 6, 2026 •

Copy link
Copy Markdown
Member

Motivation & Context

Concurrent calls through one FoundryToolbox or an MCPStreamableHTTPTool with a header_provider currently serialize across the entire RPC, even when their effective headers are identical. Independent calls scheduled with asyncio.gather() therefore cannot overlap.

Scope headers to each outgoing HTTP request and allow operations with the same effective header identity to share a session concurrently. Preserve isolation when identity changes, including discovery, recovery, and connection cleanup.

Description & Review Guide

  • What are the major changes? Replace the mutable active-header fallback with copied, owner-tagged request snapshots. Track active network attempts rather than holding a lock across a complete logical call. Identity changes, discovery, reset, close, and recovery drain active operations before replacing a session. Retries release their attempt before recovery, then reconcile identity and rebuild argument filtering and metadata from the refreshed catalog. Concurrent failures share recovery, and close-generation checks prevent older recovery from reopening after shutdown. Resource reads enter admission before requiring a ready connection. Connection replacement restarts complete tool/prompt catalogs without retaining old pages or cursors; another replacement fails explicitly. Configured discovery refreshes tools when prompt recovery replaces their connection. The HTTP wrapper tags the central request() path and rejects unsupported untagged sending methods. Invocation depth preserves outer activity during nested operation scopes. The existing MCP lifecycle ADR records these invariants and the design tradeoffs.
  • What is the impact of these changes? Public signatures, initialization, dependencies, header precedence, and caller-owned connection ownership stay unchanged. Same-identity tools, prompts, Toolbox resource reads, and Agent run preparation can overlap. A different identity requires a framework-owned session transition; caller-owned connections are not recreated. HTTP bearer-token refresh remains request-local. request_timeout also bounds admission/reconciliation and recovery as separate budgets; None disables those budgets. These are not end-to-end deadlines and do not wrap the admitted RPC or lifecycle cleanup. Calls still share server-side MCP session state, so applications must order state-dependent operations themselves.
  • What do you want reviewers to focus on? Request ownership and redirect handling, same-identity overlap, identity-change isolation, nested callbacks, cancellation, shutdown during recovery, and catalog publication. Check that a queued exclusive transition blocks unrelated new operations without blocking callbacks needed by an active operation. Also review fresh retry metadata, bounded catalog restarts, and the distinct admission, RPC, recovery, and cleanup boundaries.

From python/, .venv/bin/python -m pytest packages/core/tests/core/test_mcp.py packages/core/tests/core/test_mcp_http_auth.py packages/core/tests/core/test_mcp_observability.py packages/core/tests/core/test_mcp_skills.py packages/foundry_hosting/tests/test_toolbox.py --import-mode=importlib -m 'not integration' --tb=short --disable-warnings passed: 1,526 passed, 3 skipped, 2 deselected. Existing deprecation and telemetry warnings remain. This is the relevant regression selection, not a full-workspace unit sweep.

From python/, uv run --no-sync poe test-typing -P core and uv run --no-sync poe test-typing -P foundry_hosting both passed, including all five configured test checkers for each package.

From the repository root, uvx --from ruff==0.17.3 ruff check python/packages/core/agent_framework/_mcp.py python/packages/core/tests/core/test_mcp.py python/packages/core/tests/core/test_mcp_http_auth.py python/packages/core/tests/core/mcp_http_helpers.py python/packages/foundry_hosting/agent_framework_foundry_hosting/_toolbox.py python/packages/foundry_hosting/tests/test_toolbox.py and uvx --from ruff==0.17.3 ruff format --check python/packages/core/agent_framework/_mcp.py python/packages/core/tests/core/test_mcp.py python/packages/core/tests/core/test_mcp_http_auth.py python/packages/core/tests/core/mcp_http_helpers.py python/packages/foundry_hosting/agent_framework_foundry_hosting/_toolbox.py python/packages/foundry_hosting/tests/test_toolbox.py both passed.

From python/, uvx --from pyright==1.1.414 pyright --pythonpath .venv/bin/python --project pyproject.toml packages/core/agent_framework/_mcp.py packages/foundry_hosting/agent_framework_foundry_hosting/_toolbox.py passed, with no errors or warnings.

Related Issue

No linked GitHub issue. Maintainer-directed implementation review; this is not presented as a trivial-change or repository-automation exception.

AI Assistance

  • No material AI assistance was used.
  • This is an AI-assisted contribution. I reviewed, understood, and verified all submitted content and accept responsibility for it.

AI assistance details: Copilot assisted with design, implementation, regression tests, documentation, and this PR description.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR links to an agreed issue with no competing open PR, or the Related Issue section documents a trivial-change or repository-automation exception.
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

Copilot AI balanced review requested due to automatic review settings November 6, 2026 12:35
@agent-framework-automation agent-framework-automation Bot added documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs python Usage: [Issues, PRs], Target: Python labels Nov 6, 2026
@github-code-quality

github-code-quality Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Code coverage overview

Languages: Python

Python / code-coverage/python

main c04fecd parallel-toolbox-alt... 67d1f41 +/-
92% 92% 0%
View coverage of the 3 most impacted files.
File main c04fecd parallel-toolbox-alt... 67d1f41 +/-
packages/core/a...ework/_tools.py 96% 96% 0%
packages/core/a...amework/_mcp.py 91% 92% +1%
packages/foundr...ing/_toolbox.py 97% 99% +2%

ℹ️ Calculations represent line coverage. Results may change as calculations are updated.


Updated November 06, 2026 14:51 UTC

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Critical lifecycle-barrier gaps can tear down sessions during active RPCs, and inherited header context can execute direct prompt calls under a stale identity.

2 open issues
What changed in this PR

This PR enables concurrent same-identity MCP HTTP calls in Python by replacing shared active headers with request-owned snapshots. It also coordinates session transitions and recovery with active operations.

Changes:

  • Adds request-scoped headers, operation tracking, and separate admission and recovery timeout budgets.
  • Updates Foundry Toolbox prompt handling, resource reads, and HTTP-client cleanup.
  • Adds concurrency and lifecycle regression tests and documents the new behavior.
File Description
skills/​agent-framework-builder-python/​references/​integrations.md Documents MCP concurrency, ownership, and timeout semantics.
python/​packages/​foundry_hosting/​tests/​test_toolbox.py Tests overlapping calls, token refresh, and shutdown coordination.
python/​packages/​foundry_hosting/​agent_framework_foundry_hosting/​_toolbox.py Updates prompt identity handling, resource operations, and cleanup ordering.
python/​packages/​core/​tests/​core/​test_mcp.py Adapts recovery tests and verifies request-header snapshots.
python/​packages/​core/​tests/​core/​test_mcp_observability.py Updates recovery setup for connection-loss logging tests.
python/​packages/​core/​tests/​core/​test_mcp_http_auth.py Adds concurrency, identity-transition, timeout, cancellation, and recovery coverage.
python/​packages/​core/​tests/​core/​mcp_http_helpers.py Adds a deterministic HTTP request barrier for tests.
python/​packages/​core/​README.md Explains concurrency guarantees and separate timeout budgets.
python/​packages/​core/​AGENTS.md Records header-scoping and lifecycle coordination guidance.
python/​packages/​core/​agent_framework/​_mcp.py Implements request snapshots, operation admission, shared recovery, and close-generation checks.

Review effort: Balanced

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAF Automated Review — Iteration 1

Result: Findings reported
Scope: full PR (1 commit(s)): 0693d64379ca
Model: gpt-5.6-sol

Overview

The change replaces mutable shared header state with owner-tagged request snapshots and adds strong lifecycle, cancellation, recovery, redirect, and identity-transition coverage. Direct tool, prompt, and Toolbox resource calls with the same effective identity can overlap while identity changes remain isolated. One residual admission path still serializes same-identity Agent.run() calls and can time out a concurrent run during a long MCP operation.

Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
1 verified finding remained after source verification (1 medium) across 1 file. Details are attached to the affected lines below.

Affected areas: python/packages/core/agent_framework/_mcp.py

@jpalvarezl

Copy link
Copy Markdown
Member

I reproduced two edge cases on ffaeb4f using a real framework-created MCP Client
and httpx.MockTransport. I didn't mock the framework's admission or recovery.

Resource reads during replacement initialization

Connect under CALL-A and retain a discovered skill. Start a tool call under
CALL-B, pause the replacement's initialize response, and call that skill's
get_content() under CALL-B before releasing initialization.

The read immediately raises RuntimeError: MCPTool is not connected. After
releasing initialization, the tool call succeeds and the same skill read returns
Caller: CALL-B. _read_resource() requires a connection before entering
_http_operation(), so it never reaches the admission barrier during this window.
Could we move that required lookup inside the managed operation scope?

Catalog pages surviving connection replacement

During an explicit load_tools(), session 1 returns old_head plus
nextCursor="old-cursor". I inject ClosedResourceError at the old Client's
list_tools(cursor="old-cursor") boundary. Recovery creates a real second Client.
Session 2's full catalog is new_head, new_tail; that old cursor returns only
new_tail.

The completed discovery exposes old_head, new_tail. A subsequent full
load_tools() correctly returns new_head, new_tail, without another connection
change. The request sequence during the fault is session 1 / no cursor, then
session 2 / "old-cursor": the old staged page survives the replacement.

Could we restart the complete catalog snapshot after replacing the connection,
rather than retrying the previous session's continuation cursor? Otherwise a
restart can publish mixed-session schemas/metadata, or fail if the new session
rejects the old cursor.

These are specific readiness/catalog-boundary cases; they don't argue against
the request-owned header approach.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAF Automated Review — Iteration 2

Result: No findings
Scope: 1 net-new commit(s): 83cf19af70ca
Model: gpt-5.6-sol

Overview

This incremental commit fixes the previously reported same-identity Agent preparation serialization while extending lifecycle tracking to framework-owned static/client-header connections and separating direct-prompt identity resolution from generated-prompt runtime context. Request snapshots, identity reconciliation, exclusive lifecycle barriers, finally-based operation release, and focused concurrency, prompt, and Foundry Toolbox regressions provide concrete guardrails. No Critical, High, or Medium defect introduced by this review range remains supported after reconciliation.

Reviewed the supplied incremental change set across correctness, security/reliability, architecture, and failure behavior.
No publishable findings remained after source verification for this scope.

@jpalvarezl Jose Alvarez (jpalvarezl) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, I like the request-owned direction. I have two robustness concerns and one documentation request:

  1. _MCPHeaderScopedClient tags stream, send, and delete, but __getattr__ silently delegates other request entry points to the underlying client. If a future MCP transport switches to request() or post(), the owner extension would not be added and the dynamic-header hook would skip the request without an obvious failure. Could we either wrap the central request path as well, or fail explicitly for untagged sending methods? A focused regression test through an alternate request method would also make this dependency clear.

  2. _MCPHTTPInvocation.active is a boolean, so if the same invocation ever re-enters _http_operation(), the inner finally will clear it while the outer operation is still running. The current call flow may prevent that today, but the state machine relies on that invariant. Could we use an active-depth counter, or add an assertion and focused test proving same-invocation re-entry cannot happen?

  3. Could we carry over the ADR from #10142, or add an equivalent, explaining why request-owned snapshots were selected over call-level same-identity leases? The concurrency and lifecycle contract is subtle, particularly around nested callbacks, identity transitions, and exclusive operations, and recording those invariants would help future maintainers.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAF Automated Review — Iteration 3

Result: No findings
Scope: 15 net-new commit(s): df7d02bdc99c, e9292752c1ea, 18cd5903b902, 2fa7b9ab21be, 0c479365fca9, 83c719065840, 4ffc820cd62d, 082217287cb4, 6e770b95c033, d0e191cf4d3d, 44f5495c0851, 9db7efb0841b, 80d0cbf5bc30, 28163f5e23d2, ffaeb4f0269d
Model: gpt-5.6-sol

Overview

The change replaces mutable shared-header state with request-owned snapshots and short-lived operation leases, allowing same-identity MCP calls to overlap while preserving exclusive identity transitions and lifecycle teardown. Owner tags, redirect stripping, generation-fenced recovery, caller-owned connection checks, and extensive concurrency/cancellation tests provide concrete guardrails. No publishable Critical, High, or Medium defect introduced by the reviewed range remained after reconciliation.

Reviewed the supplied incremental change set across correctness, security/reliability, architecture, and failure behavior.
No publishable findings remained after source verification for this scope.

@eavanvalkenburg

Copy link
Copy Markdown
Member Author

Addressed the robustness and ADR feedback and the resource/discovery repros in 9f5489d.

  • The HTTP wrapper now tags request() and explicitly rejects unwrapped sending methods instead of silently losing dynamic headers. A regression verifies the alternate request path and preserved caller extensions.
  • Invocation activity uses a depth counter. The re-entry regression verifies that exiting the inner scope leaves the outer operation active and still rejects nested close.
  • Resource reads require the connection only after entering admission. The retained-skill regression pauses replacement initialization and verifies that the read waits, then uses the replacement session and current call ID.
  • Tool and prompt discovery discard old staged pages and cursors after replacement and restart the complete snapshot once. Repeated replacement fails explicitly. Configured discovery also refreshes tools when prompt recovery replaces their session. Regressions cover accepting and rejecting servers, metadata, publication, and the restart bound.
  • Added the concurrency and identity-transition note to the existing MCP lifecycle ADR, including request-owned snapshots, nested callbacks, exclusive operations, ownership, and timeout boundaries.

The selected MCP/Toolbox regressions and both packages’ five configured test checkers pass. Strict source typing, lint, and formatting also pass. The PR description now reflects the current implementation and commands.

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 6, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 6, 2026
Merged via the queue into main with commit 25ecd67 Nov 6, 2026
54 checks passed
@eavanvalkenburg
Eduard van Valkenburg (eavanvalkenburg) deleted the parallel-toolbox-alternatives branch November 6, 2026 15:24
Eduard van Valkenburg (eavanvalkenburg) added a commit to eavanvalkenburg/agent-framework that referenced this pull request Nov 6, 2026
Release all 44 publishing workspace distributions with shipping changes since python-1.24.0, including the Python 3.10 removal. Use the approved minor bumps for Core/root, AG-UI, and Declarative, patch bumps for other changed stable integrations, and 261106 stamps for changed alpha/beta packages; no optional cohort-only bump. Update the Core standard-install cohort, apply strict API-based Core floors, and align exact hosting pins. Include the four Python Dependabot updates and the reviewed release batch manifest. Changelog includes the anticipated microsoft#10147 fix; the draft must wait for that PR to merge and be incorporated before release. Lab remains excluded from publication, with only its dependency lock updated.
Isham Mohamed (IshamMohamed) pushed a commit to IshamMohamed/agent-framework that referenced this pull request Nov 6, 2026
* Bump Python package versions for 1.25.0 release

Release all 44 publishing workspace distributions with shipping changes since python-1.24.0, including the Python 3.10 removal. Use the approved minor bumps for Core/root, AG-UI, and Declarative, patch bumps for other changed stable integrations, and 261106 stamps for changed alpha/beta packages; no optional cohort-only bump. Update the Core standard-install cohort, apply strict API-based Core floors, and align exact hosting pins. Include the four Python Dependabot updates and the reviewed release batch manifest. Changelog includes the anticipated microsoft#10147 fix; the draft must wait for that PR to merge and be incorporated before release. Lab remains excluded from publication, with only its dependency lock updated.

* Fix SQL Server cancellation test exception-info narrowing

* Retain deprecated declarative MCP cache parameter and correct release notes

Accept legacy positive cache_max_size values with a deprecation warning, without restoring initialized-session caching. Keep per-invocation security isolation and the approved 1.5.0 version, document the compatibility behavior and remaining lifecycle migration, and extend real-protocol isolation tests for explicit legacy values. Remove the changelog claim and reference for still-open PR microsoft#9617.

* Simplify deprecated MCP cache parameter handling and undo skill edits

This branch was successfully deployed

1 active deployment
github-app-auth — 67d1f415 Deployed Nov 6, 2026 by eavanvalkenburg via add_label #27296
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants