Python: Clean up dependency groups and compatibility

[eavanvalkenburg]

Motivation & Context

Python development dependency groups included test-only and feature-specific packages, while the maintenance tooling only understood a single dev group and could rewrite intentionally ranged dependencies as exact pins. This made dependency updates fragile, contributed to an OpenTelemetry resolution failure in the dependency maintenance workflow, and allowed the workspace override to hide the real Foundry Hosting and Mistral incompatibility reported in #7036.

Standalone PEP 723 samples also need to declare the specific Agent Framework distributions they import so they run without relying on the agent-framework meta-package. During the dependency refresh, newer OpenAI usage models and Azure SDK changes exposed compatibility and DevUI usage-mapping problems that also needed to be addressed.

Description & Review Guide

• What are the major changes? Reclassifies test-only and feature-specific dependencies into appropriate dependency groups; installs and validates all declared groups; updates dependency-bound and pin-refresh tooling to handle arbitrary groups while preserving ranges; refreshes the lockfile; adds minimal specific Agent Framework dependencies to PEP 723 samples; supports Mistral SDK 1.x and 2.x so Foundry Hosting can coexist with Mistral without an unsafe OpenTelemetry override; and preserves cache-read, cache-write, and reasoning token details in DevUI ResponseUsage.
• What is the impact of these changes? Dependency maintenance no longer constrains optional test dependencies unnecessarily or rewrites non-exact requirements, the reported Foundry Hosting/Mistral combination resolves normally, standalone samples install only what they use, and DevUI reports accurate usage details. The Foundry package is capped below the incompatible azure-ai-projects 2.3 API change.
• What do you want reviewers to focus on? Please focus on the dependency-group classifications, exact-pin-only maintenance behavior, the Mistral 1.x/2.x compatibility boundary, and the resulting lockfile changes.

Related Issue

Fixes #7036

Contribution Checklist

• The code builds clean without any errors or warnings
• All unit tests pass, and I have added new tests where possible
• The PR follows the Contribution Guidelines
• This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
• This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and the title prefix in sync automatically.

[github-actions]

Python Test Coverage Report •

File	Stmts	Miss	Cover	Missing
packages/mistral/agent_framework_mistral
_embedding_client.py	72	2	97%	28, 34
TOTAL	44146	5267	88%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
8844	33 💤	0 ❌	0 🔥	2m 31s ⏱️

[Copilot]

Pull request overview

This PR updates the Python workspace dependency-management model to treat multiple dependency groups (not just a single dev group) as first-class, improves dev pin refresh so it only touches exact pins, and adjusts package/sample dependencies to avoid masking real compatibility issues (notably Foundry Hosting + Mistral).

Changes:

• Generalize dependency-bound validation and optimizer tooling to install/validate all declared dependency-groups (plus dev extra/extras) instead of special-casing only dev.
• Update the dev pin refresh workflow to collect exact pins across development groups while preserving intentionally ranged requirements.
• Update PEP 723 sample metadata to depend on the specific Agent Framework distributions they import (instead of the agent-framework meta-package), and adjust package compatibility constraints/tests (Mistral + DevUI usage mapping).

Reviewed changes

Copilot reviewed 52 out of 53 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
python/scripts/dependencies/validate_dependency_bounds.py	Install all dependency groups when running per-package test/pyright validation.
python/scripts/dependencies/upgrade_dev_dependencies.py	Refresh exact development dependency pins without rewriting ranged requirements.
python/scripts/dependencies/_dependency_bounds_upper_impl.py	Generalize upper-bound probing and exact-pin refresh to arbitrary dependency groups.
python/scripts/dependencies/_dependency_bounds_lower_impl.py	Generalize lower-bound probing to arbitrary dependency groups.
python/pyproject.toml	Split root dev vs test groups; remove unsafe OTel override; update install/upgrade tasks to --all-groups.
.github/actions/python-setup/action.yml	CI setup now installs all groups via uv sync --all-groups.
python/DEV_SETUP.md	Developer setup instructions updated to use --all-groups and explain group intent.
python/.github/skills/python-package-management/SKILL.md	Guidance updated to reflect development groups vs test/feature groups.
python/packages/mistral/pyproject.toml	Broaden mistralai support range to include 1.x (while still <3).
python/packages/mistral/agent_framework_mistral/_embedding_client.py	Add Mistral SDK import fallback intended to support multiple mistralai layouts.
python/packages/mistral/tests/mistral/test_mistral_embedding_client.py	Add coverage asserting embeddings API supports output_dimension.
python/packages/foundry/pyproject.toml	Cap azure-ai-projects to <2.3.0 to avoid incompatible API change.
python/packages/lab/pyproject.toml	Move tau2 into a feature group and update shared dev tooling pin.
python/packages/hosting-responses/pyproject.toml	Rename dependency group dev → test for test-only dependencies.
python/packages/azurefunctions/pyproject.toml	Remove empty dependency-groups stanza.
python/packages/devui/agent_framework_devui/_mapper.py	Preserve cache-read/cache-write/reasoning token details in DevUI ResponseUsage.
python/packages/devui/tests/devui/test_mapper.py	Add tests for detailed usage aggregation and explicit zero-usage handling.
python/packages/core/tests/workflow/test_checkpoint_unrestricted_pickle.py	Update pickle-allowlist test for new InputTokensDetails shape.
python/samples/AGENTS.md	Document rule: PEP 723 scripts must list specific AF distributions (not meta-package).
python/samples/semantic-kernel-migration/processes/nested_process.py	Add explicit agent-framework-core dependency in PEP 723 metadata.
python/samples/semantic-kernel-migration/processes/fan_out_fan_in_process.py	Add explicit agent-framework-core dependency in PEP 723 metadata.
python/samples/semantic-kernel-migration/orchestrations/sequential.py	Add explicit agent-framework-openai + agent-framework-orchestrations deps.
python/samples/semantic-kernel-migration/orchestrations/magentic.py	Add explicit agent-framework-openai + agent-framework-orchestrations deps.
python/samples/semantic-kernel-migration/orchestrations/handoff.py	Add explicit agent-framework-openai + agent-framework-orchestrations deps.
python/samples/semantic-kernel-migration/orchestrations/group_chat.py	Add explicit agent-framework-openai + agent-framework-orchestrations deps.
python/samples/semantic-kernel-migration/orchestrations/concurrent_basic.py	Add explicit agent-framework-openai + agent-framework-orchestrations deps.
python/samples/semantic-kernel-migration/openai_responses/03_responses_agent_structured_output.py	Add explicit agent-framework-openai dependency in PEP 723 metadata.
python/samples/semantic-kernel-migration/openai_responses/02_responses_agent_with_tool.py	Add explicit agent-framework-openai dependency in PEP 723 metadata.
python/samples/semantic-kernel-migration/openai_responses/01_basic_responses_agent.py	Add explicit agent-framework-openai dependency in PEP 723 metadata.
python/samples/semantic-kernel-migration/copilot_studio/02_copilot_studio_streaming.py	Add explicit agent-framework-copilotstudio dependency in PEP 723 metadata.
python/samples/semantic-kernel-migration/copilot_studio/01_basic_copilot_studio_agent.py	Add explicit agent-framework-copilotstudio dependency in PEP 723 metadata.
python/samples/semantic-kernel-migration/chat_completion/03_chat_completion_thread_and_stream.py	Add explicit agent-framework-openai dependency in PEP 723 metadata.
python/samples/semantic-kernel-migration/chat_completion/02_chat_completion_with_tool.py	Add explicit agent-framework-openai dependency in PEP 723 metadata.
python/samples/semantic-kernel-migration/chat_completion/01_basic_chat_completion.py	Add explicit agent-framework-openai dependency in PEP 723 metadata.
python/samples/autogen-migration/single_agent/03_agent_thread_and_stream.py	Add explicit agent-framework-openai dependency in PEP 723 metadata.
python/samples/autogen-migration/single_agent/01_basic_agent.py	Add explicit agent-framework-openai dependency in PEP 723 metadata.
python/samples/05-end-to-end/m365-agent/m365_agent_demo/app.py	Switch PEP 723 metadata to use agent-framework-foundry.
python/samples/05-end-to-end/evaluation/self_reflection/self_reflection.py	Add explicit agent-framework-foundry dependency in PEP 723 metadata.
python/samples/05-end-to-end/evaluation/red_teaming/red_team_agent_sample.py	Add explicit agent-framework-foundry dependency in PEP 723 metadata.
python/samples/05-end-to-end/chatkit-integration/app.py	Add explicit agent-framework-chatkit + agent-framework-foundry deps.
python/samples/02-agents/providers/copilotstudio/copilotstudio_with_explicit_settings.py	Add explicit agent-framework-copilotstudio dependency in PEP 723 metadata.
python/samples/02-agents/observability/foundry_tracing.py	Add explicit agent-framework-foundry dependency in PEP 723 metadata.
python/samples/02-agents/middleware/atr_validation_middleware.py	Replace agent-framework meta-package with agent-framework-foundry.
python/samples/02-agents/harness/harness_research.py	Replace agent-framework meta-package with agent-framework-foundry.
python/samples/02-agents/harness/harness_data_processing.py	Replace agent-framework meta-package with agent-framework-foundry.
python/samples/02-agents/harness/build_your_own_claw/skills/valuation/scripts/valuation_metrics.py	Reformat JSON printing for readability (no functional change).
python/samples/02-agents/harness/build_your_own_claw/skills/risk-scoring/scripts/risk_score.py	Reformat JSON printing for readability (no functional change).
python/samples/02-agents/harness/build_your_own_claw/claw_step03_scaling_capabilities.py	Switch to agent-framework-foundry and adjust formatting/spacing in snippet blocks.
python/samples/02-agents/harness/build_your_own_claw/claw_step02_working_with_data.py	Switch to agent-framework-foundry in PEP 723 metadata.
python/samples/02-agents/harness/build_your_own_claw/claw_step01_meet_your_claw.py	Switch to agent-framework-foundry in PEP 723 metadata.
python/samples/02-agents/compaction/tiktoken_tokenizer.py	Add explicit agent-framework-core dependency in PEP 723 metadata.
python/samples/02-agents/auto_retry.py	Switch to agent-framework-foundry in PEP 723 metadata.

[github-actions]

Automated Code Review

Reviewers: 5 | Confidence: 56% | Result: All clear

Reviewed: Correctness, Security Reliability, Test Coverage, Failure Modes, Design Approach

---

Automated review by eavanvalkenburg's agents