Skip to content

Introduce Unrecoverable sandbox state - #1727

Open
ludfjig wants to merge 1 commit into
mainfrom
ll/snapshot-restore-transaction
Open

Introduce Unrecoverable sandbox state#1727
ludfjig wants to merge 1 commit into
mainfrom
ll/snapshot-restore-transaction

Conversation

@ludfjig

@ludfjig ludfjig commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Some failures can leave sandboxes in a state that cannot be restored (like poisoned but worse). This PR introduces such state as Unrecoverable. Unrecoverable sandboxes can no longer be used at all, and users need to create new one. I don't expect that this will be common at all, but it was an existing gap in our codebase that should be addressed.

This pr also tries to be a bit more transactional in certain operations, meaning it either fully completes successfully or errors cleanly without leaving sandbox in invalid state.

@ludfjig ludfjig added the kind/enhancement For PRs adding features, improving functionality, docs, tests, etc. label Aug 12, 2026
@ludfjig
ludfjig force-pushed the ll/snapshot-restore-transaction branch 6 times, most recently from eb0b4b3 to da5b23c Compare August 13, 2026 21:13
@ludfjig
ludfjig changed the base branch from main to ll/simplify_gdb August 13, 2026 21:17
@ludfjig
ludfjig force-pushed the ll/snapshot-restore-transaction branch from da5b23c to c180248 Compare August 13, 2026 21:38
@ludfjig
ludfjig force-pushed the ll/snapshot-restore-transaction branch from c180248 to b48dd56 Compare August 13, 2026 22:44
@ludfjig
ludfjig marked this pull request as ready for review August 13, 2026 22:54
Copilot AI lite review requested due to automatic review settings August 13, 2026 22:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the Hyperlight host sandbox lifecycle model by introducing an explicit unrecoverable state for MultiUseSandbox instances when certain restore failures (notably VM base mapping updates) leave the sandbox unsafe to continue using. It also updates the public API and tests to query sandbox lifecycle via a new status() method returning SandboxStatus.

Changes:

  • Add SandboxStatus (Ready | Poisoned | Unrecoverable) and expose MultiUseSandbox::status() (deprecating poisoned()).
  • Make restore() treat base mapping update failures as terminal by marking the sandbox Unrecoverable and rejecting future operations.
  • Add hypervisor test fault-injection helpers and expand restore-related tests to cover recoverable vs unrecoverable failure modes.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/hyperlight_host/tests/sandbox_host_tests.rs Update tests to use status().is_poisoned() instead of poisoned().
src/hyperlight_host/tests/integration_test.rs Update integration assertions to use SandboxStatus/status().
src/hyperlight_host/src/sandbox/snapshot/file_tests.rs Update snapshot tests to check poison via status().
src/hyperlight_host/src/sandbox/mod.rs Re-export SandboxStatus from the sandbox module.
src/hyperlight_host/src/sandbox/initialized_multi_use.rs Introduce SandboxStatus, implement status()/ensure_usable(), update poison/restore behavior, and add new restore failure-mode tests.
src/hyperlight_host/src/lib.rs Re-export SandboxStatus from the crate root.
src/hyperlight_host/src/hypervisor/hyperlight_vm/test_support.rs Add test-only VM fault injection and base mapping state helpers.
src/hyperlight_host/src/hypervisor/hyperlight_vm/mod.rs Adjust snapshot/scratch mapping update logic and expose test support module.
src/hyperlight_host/src/error.rs Add HyperlightError::UnrecoverableSandbox and classify it as non-poisoning.
CHANGELOG.md Document the new lifecycle API and unrecoverable behavior.
Suppressed comments (1)

src/hyperlight_host/src/hypervisor/hyperlight_vm/mod.rs:561

  • update_scratch_mapping has the same transactional issue as update_snapshot_mapping: it unmaps the old scratch region before mapping the new one, but only updates self.scratch_memory after the new map succeeds. If map_memory fails, scratch_memory still points at memory that has been unmapped in the VM, leaving the internal state inconsistent.
        if let Some(old_scratch) = self.scratch_memory.as_ref() {
            let old_base = hyperlight_common::layout::scratch_base_gpa(old_scratch.mem_size());
            let old_rgn = old_scratch.mapping_at(old_base, MemoryRegionType::Scratch);
            self.vm.unmap_memory((self.scratch_slot, &old_rgn))?;
        }
        unsafe { self.vm.map_memory((self.scratch_slot, &rgn))? };
        self.scratch_memory = Some(scratch);

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/hyperlight_host/src/hypervisor/hyperlight_vm/mod.rs
Base automatically changed from ll/simplify_gdb to main August 14, 2026 00:15
Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>
@ludfjig
ludfjig force-pushed the ll/snapshot-restore-transaction branch from b48dd56 to 47aa821 Compare August 14, 2026 00:15

@jsturtevant jsturtevant Aug 14, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be only avaliable when #[test]?

@jsturtevant

Copy link
Copy Markdown
Contributor

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/enhancement For PRs adding features, improving functionality, docs, tests, etc.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants