Repository navigation
Python: [BREAKING] CodeAct: redact host-tool diagnostics by default - #9971
Conversation
GitHub Actions
Inactive
GitHub Actions
Inactive
agent-framework-automation
Bot
added
documentation
github-actions
Bot
changed the title
github-actions
Bot
changed the title
There was a problem hiding this comment.
🔵 Needs a closer look
Skipped Python and .NET Hyperlight native-backend tests leave redaction and failure signaling across the guest/FFI security boundary unverified.
No open issues 🎉
What changed in this PR
This PR introduces default host-tool diagnostic redaction across Python Monty and Hyperlight and .NET Hyperlight and LocalCodeAct. Applications can explicitly enable detailed errors for trusted channels while preserving failure signaling.
Changes:
- Add host-controlled error-detail options and redact diagnostics before generated code receives host-tool failures.
- Carry the policy through run snapshots, Python serialized state, and Hyperlight cache identities.
- Add regression coverage and document the breaking default behavior and trusted opt-in.
Native Hyperlight backend verification remains pending, as noted in the PR description.
| File | Description |
|---|---|
| skills/agent-framework-builder-python/references/integrations.md | Document Python diagnostic redaction and trusted opt-in. |
| skills/agent-framework-builder-dotnet/references/integrations.md | Document .NET diagnostic policy boundaries. |
| python/packages/monty/tests/monty/test_monty_codeact.py | Test option validation, snapshots, and redacted failures. |
| python/packages/monty/tests/monty/test_monty_codeact_integration.py | Cover caught and uncaught errors, concurrency, and cancellation. |
| python/packages/monty/tests/monty/conftest.py | Add shared failing and cancelled host-tool fixtures. |
| python/packages/monty/README.md | Explain redaction defaults and exception-handling migration. |
| python/packages/monty/AGENTS.md | Document the new option and propagation requirements. |
| python/packages/monty/agent_framework_monty/_provider.py | Expose and forward the diagnostic option. |
| python/packages/monty/agent_framework_monty/_monty_bridge.py | Redact host-tool exceptions before resuming generated code. |
| python/packages/monty/agent_framework_monty/_execute_code_tool.py | Validate, capture, and apply the diagnostic policy. |
| python/packages/hyperlight/tests/hyperlight/test_hyperlight_codeact.py | Cover redaction, cancellation, cache isolation, and backend behavior. |
| python/packages/hyperlight/tests/hyperlight/conftest.py | Add shared host-tool error fixtures. |
| python/packages/hyperlight/README.md | Explain redaction, opt-in, and cache isolation. |
| python/packages/hyperlight/agent_framework_hyperlight/_provider.py | Expose and forward the diagnostic option. |
| python/packages/hyperlight/agent_framework_hyperlight/_execute_code_tool.py | Apply redaction to callbacks and errors; isolate cached policies. |
| dotnet/tests/Microsoft.Agents.AI.LocalCodeAct.UnitTests/LocalExecuteCodeFunctionIntegrationTests.cs | Test caught and uncaught failures and construction-time policy capture. |
| dotnet/tests/Microsoft.Agents.AI.Hyperlight.UnitTests/ToolBridgeTests.cs | Test generic and detailed tool-error responses. |
| dotnet/tests/Microsoft.Agents.AI.Hyperlight.UnitTests/SandboxExecutorTests.cs | Verify policy snapshots and distinct cache fingerprints. |
| dotnet/tests/Microsoft.Agents.AI.Hyperlight.IntegrationTests/CodeActEndToEndTests.cs | Add backend coverage for diagnostic opt-in. |
| dotnet/src/Microsoft.Agents.AI.LocalCodeAct/README.md | Document default redaction and outer-error handling boundaries. |
| dotnet/src/Microsoft.Agents.AI.LocalCodeAct/LocalExecuteCodeFunction.cs | Capture the standalone function’s diagnostic policy. |
| dotnet/src/Microsoft.Agents.AI.LocalCodeAct/LocalCodeActProviderOptions.cs | Add the default-off detailed-error option. |
| dotnet/src/Microsoft.Agents.AI.LocalCodeAct/LocalCodeActProvider.cs | Capture the option and propagate it into run snapshots. |
| dotnet/src/Microsoft.Agents.AI.LocalCodeAct/Internal/ProcessBridge.cs | Redact exception details in failed tool responses. |
| dotnet/src/Microsoft.Agents.AI.LocalCodeAct/Internal/CodeExecutor.cs | Pass the snapshot policy into the process bridge. |
| dotnet/src/Microsoft.Agents.AI.Hyperlight/README.md | Document redaction and cached callback isolation. |
| dotnet/src/Microsoft.Agents.AI.Hyperlight/Internal/ToolBridge.cs | Apply diagnostic policy to host-tool error JSON. |
| dotnet/src/Microsoft.Agents.AI.Hyperlight/Internal/SandboxExecutor.cs | Capture and fingerprint the policy; redact execution exceptions. |
| dotnet/src/Microsoft.Agents.AI.Hyperlight/HyperlightCodeActProviderOptions.cs | Add the default-off detailed-error option. |
Review effort: Balanced
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
MAF Automated Review — Iteration 1
Result: Findings reported
Scope: full PR (1 commit(s)): 37e96174e8e3
Model: gpt-5.6-sol
Overview
The change consistently introduces a default-off diagnostic policy across the four provider and standalone surfaces, snapshots it per run, and separates Hyperlight cache identities so callbacks cannot cross policy boundaries. The new tests provide strong coverage for host-tool failures, caught versus uncaught behavior, cancellation, and policy propagation. Two Hyperlight lifecycle paths still let backend exception messages escape before the new policy is applied.
Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
2 verified findings remained after source verification (2 medium) across 2 files. Details are attached to the affected lines below.
Affected areas: dotnet/src/Microsoft.Agents.AI.Hyperlight/Internal/SandboxExecutor.cs, python/packages/hyperlight/agent_framework_hyperlight/_execute_code_tool.py
GitHub Actions
Inactive
Eduard van Valkenburg (eavanvalkenburg)
changed the title
GitHub Actions
Inactive
Eduard van Valkenburg (eavanvalkenburg)
removed
the
.NET
GitHub Actions
Inactive
GitHub Actions
Inactive
There was a problem hiding this comment.
MAF Automated Review — Iteration 2
Result: No findings
Scope: 1 net-new commit(s): 65b733a8de90
Model: gpt-5.6-sol
Overview
This incremental commit cleanly removes the earlier .NET implementation, documentation, and tests so the PR retains its stated Python-only scope and the .NET work can proceed in the companion PR. The affected .NET package paths now match the review base, and no residual Hyperlight or LocalCodeAct policy symbols remain. The supplied diff contains no Python or AgentMode hunks, so candidates in those areas cannot be anchored to this authoritative review range.
Reviewed the supplied incremental change set across correctness, security/reliability, architecture, and failure behavior.
No publishable findings remained after source verification for this scope.
Motivation & Context
Give Python Monty and Hyperlight a consistent, host-controlled policy for diagnostic details. Registered tool failures should remain failures without making backend exception text available to generated code unless the application explicitly opts in for a trusted channel.
This PR now contains only Python implementation, tests, documentation, and Python consumer guidance. The .NET changes are in the independent companion PR #9973.
Description & Review Guide
include_detailed_errorsoptions to the Python Monty and Hyperlight provider and standalone entry points. Redact host failures before guest execution resumes, carry the policy through run snapshots and serialized state, and isolate Hyperlight sandbox caches by policy. Update regression coverage and Python integration guidance.RuntimeErroror explicitly enable details for a trusted channel. Error-detail configuration remains separate from outer-client error handling and sensitive-data logging.The implementation files are byte-identical to the validated version before this language-only split. Targeted checks from that implementation:
python -m pytest python/packages/monty/tests/monty/test_monty_codeact.py python/packages/hyperlight/tests/hyperlight/test_hyperlight_codeact.py -m 'not integration' -p no:cacheprovider— 391 passed, 8 skipped.python -m pytest python/packages/monty/tests/monty/test_monty_codeact_integration.py -p no:cacheprovider— 54 passed with the real supported Monty runtime.Python Hyperlight backend checks skipped 4 cases because the supported backend environment was unavailable. These skips are not native verification.
Related Issue
N/A — maintainer-directed consistency update.
AI Assistance
AI assistance details: GitHub Copilot assisted with implementation, tests, documentation, this description, and the language-only PR split.
Contribution Checklist
breaking changelabel (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and the title prefix in sync automatically.