Repository navigation
[BREAKING] Python: fix(hyperlight): interrupt guest execution and enforce time limits - #10152
Vincent Biret (baywet) wants to merge 1 commit into
Conversation
…mits Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3d6807e7-70a6-4c43-8dc1-7bab20376cd4
GitHub Actions
Active
GitHub Actions
Active
agent-framework-automation
Bot
added
documentation
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
| start = time.monotonic() | ||
| execution.cancel() | ||
| with pytest.raises(asyncio.CancelledError): | ||
| await execution |
| break | ||
| try: | ||
| execution.result() | ||
| except asyncio.CancelledError: |
There was a problem hiding this comment.
🟡 Changes recommended
A cold invocation can incorrectly exhaust its budget while waiting behind another execution, and deterministic tests are missing for timeout and cancellation cleanup during initialization and warm-up.
2 open issues
What changed in this PR
Adds cancellation propagation and finite execution budgets to Python Hyperlight, aligning with the .NET work in #10066. Shipping still requires a compatible SDK release and dependency-floor update.
Changes:
- Adds a validated, configurable 30-second default budget, captured in run snapshots and serialized state.
- Interrupts guest execution on cancellation or timeout and waits for worker cleanup before reuse.
- Adds interruption and recovery tests and documents execution limits and migration requirements.
| File | Description |
|---|---|
skills/agent-framework-builder-python/references/integrations.md |
Documents execution budgets, cancellation, and SDK requirements. |
python/packages/hyperlight/tests/hyperlight/test_hyperlight_codeact.py |
Tests timeout validation and provider capture; updates sandbox fakes. |
python/packages/hyperlight/tests/hyperlight/test_execution_limits.py |
Adds real-guest cancellation, timeout, and recovery regressions. |
python/packages/hyperlight/tests/hyperlight/test_execution_control.py |
Tests timeout recovery, repeated cancellation, and queued cancellation. |
python/packages/hyperlight/tests/hyperlight/conftest.py |
Adds a controllable interruption fixture and updates lifecycle fakes. |
python/packages/hyperlight/README.md |
Explains limits, migration, SDK prerequisites, and native test setup. |
python/packages/hyperlight/agent_framework_hyperlight/_provider.py |
Exposes and forwards the execution timeout option. |
python/packages/hyperlight/agent_framework_hyperlight/_execute_code_tool.py |
Implements budget enforcement, interruption, cancellation cleanup, and setting capture. |
Review effort: Balanced
| control = _ExecutionControl(config.execution_timeout_seconds) | ||
| entry: _SandboxEntry | None = None | ||
| try: | ||
| entry = self._get_or_create_entry(config, control=control) |
| ) | ||
|
|
||
| def _build_sandbox() -> tuple[Any, Any]: | ||
| control.start() |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
MAF Automated Review — Iteration 1
Result: Findings reported
Scope: full PR (1 commit(s)): 858c047f5ade
Model: gpt-5.6-sol
Overview
The PR introduces per-invocation execution controls that preserve sandbox owner-thread confinement, wait through repeated cancellation, and restore cached sandboxes after interruption; focused tests cover cancellation, queued work, timeout, and reuse. Timeout validation, provider forwarding, and serialized state capture are coherent. The remaining risk is that the implementation unconditionally requires an SDK API absent from the declared and locked dependency, so standard installations fail before guest execution.
Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
1 verified finding remained after source verification (1 high) across 1 file. Details are attached to the affected lines below.
Affected areas: python/packages/hyperlight/agent_framework_hyperlight/_execute_code_tool.py
|
|
||
| def run(self, sandbox: Any, code: str) -> Any: | ||
| deadline = self.start() | ||
| handle = sandbox.interrupt_handle() |
There was a problem hiding this comment.
pyproject.toml still permits hyperlight-sandbox==0.8.0, and the lockfile resolves that version, but it does not provide interrupt_handle(). Every invocation from a standard install therefore fails at this unconditional call before guest execution. Please consume a compatible SDK release, raise the dependency floor, and regenerate the lockfile before shipping this change.
Motivation & Context
Cancelling an asynchronous Hyperlight invocation should stop its guest execution before the sandbox can be reused. Python guest execution also needs a finite default budget, consistent with the .NET work in #10066.
Description & Review Guide
execution_timeout_secondsoption to the tool and provider, defaulting to 30 seconds. Forward cancellation through an independently owned SDK interruption handle, wait for the native worker and interruption monitor to finish, and restore the warm snapshot before reuse. Capture the setting in provider run snapshots and serialized state. Add focused unit and real-guest regressions and update consumer guidance.asyncio.CancelledError. Runtime initialization and host tool callbacks cannot be interrupted and must enforce their own limits.Dependency prerequisite: This draft requires the Python
Sandbox.interrupt_handle()API from hyperlight-dev/hyperlight-sandbox#275. The declared released dependency does not yet provide it. A compatible SDK release and dependency-floor update are required before shipping; no local source paths are committed.Validation used temporary Linux x64 Azure KVM containers linked to SDK revision
530672881443ac26973688191a18f180dc373efd. The three original real-guest regressions failed for the intended behavior before implementation changes.Commands below ran from
python/packages/hyperlightwith that source-linked environment:python -m pytest tests/hyperlight/test_execution_limits.py tests/hyperlight/test_execution_control.py -v --tb=shortwithHYPERLIGHT_RUN_GUEST_TESTS=true: 8 passed, covering Python/Wasm and native JavaScript.python -m pytest tests -k "not pings_bing" --tb=shortwithHYPERLIGHT_RUN_GUEST_TESTS=true: 267 passed, 6 platform skips, 1 deselected, with the pre-existing warning described below.python -m pytest tests -k pings_bing --tb=shortwith networking enabled: 1 passed.python -m pytest -m 'not integration' --cov=agent_framework_hyperlight --cov-report=term-missing:skip-covered --cov-fail-under=85 tests --tb=short: 247 passed, 6 platform skips, 21 deselected; 88.31% coverage, above the 85% gate.ruff check --no-fix .andruff format --check .: passed with the repository-pinned Ruff.pyright --pythonpath /src/.venv/bin/python: passed, 0 errors for source.The configured test-typing profiles ran from
python:mypy --config-file pyproject.toml --explicit-package-bases --namespace-packages --python-executable /src/.venv/bin/python packages/hyperlight/tests: passed.pyrefly check -c pyrefly.toml packages/hyperlight/tests: passed.ty check packages/hyperlight/tests: passed.zuban mypy --config-file pyproject.toml packages/hyperlight/tests: passed.pyright -p pyrightconfig.tests.json --pythonpath /src/.venv/bin/python packages/hyperlight/tests: passed.The full package run reports a native
WasmSandboxcross-thread finalization warning. Repeating the existing host-error and owner-thread tests with unchanged production code frommainand the same SDK reproduces it:python -m pytest tests/hyperlight/test_hyperlight_codeact.py -k 'host_error_policy_with_real_sandbox or sandbox_run_result_is_finalized_on_owner_thread' --tb=short -W error::pytest.PytestUnraisableExceptionWarningproduced 4 passed, 1 failed because the warning was promoted to an error. This draft does not claim that existing warning is fixed.Cross-language checks used the same updated SDK and the unchanged #10066 source.
/srcwas the SDK checkout and/linked-sdk.targetsreplaced the .NET package reference with its local project:python -m unittest discover -s src/sdk/python/core/tests -p test_interruption.py -v: 4 passed, each exercising both backends.dotnet vstest /src/src/sdk/dotnet/core/Tests/HyperlightSandbox.Tests/bin/Debug/net8.0/HyperlightSandbox.Tests.dll --Logger:"console;verbosity=normal": 152 passed.dotnetdirectory,dotnet test --project tests/Microsoft.Agents.AI.Hyperlight.UnitTests -f net10.0 --no-build --no-restore -p:CustomAfterMicrosoftCommonTargets=/linked-sdk.targets -p:HyperlightGuestTests=true: 62 passed, 0 skipped.dotnet test --project tests/Microsoft.Agents.AI.Hyperlight.IntegrationTests -f net10.0 --no-build --no-restore -p:CustomAfterMicrosoftCommonTargets=/linked-sdk.targets -p:JsonSerializerIsReflectionEnabledByDefault=true: 5 passed, 0 skipped.Temporary Azure resources were deleted and their absence verified.
Related Issue
Related to #10066, which implements the .NET counterpart. This PR implements the Python integration rather than duplicating that work.
Depends on hyperlight-dev/hyperlight-sandbox#275. No separate Agent Framework issue is linked.
AI Assistance
AI assistance details: GitHub Copilot assisted with implementation, regression tests, documentation, validation, and this PR description.
Contribution Checklist
breaking changelabel (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.