Skip to content

fix(collect): read files over exec_with_output when a sandbox has no streaming exec - #317

Closed
dmchoiboi wants to merge 1 commit into
mainfrom
michaelchoi/_claude/collect-artifacts-exec-fallback
Closed

dmchoiboi wants to merge 1 commit into
mainfrom
michaelchoi/_claude/collect-artifacts-exec-fallback

Conversation

@dmchoiboi

@dmchoiboi dmchoiboi commented Oct 23, 2026 •

Copy link
Copy Markdown

Why

collect_artifacts reads each file by streaming base64 < file through sandbox.exec(). Sandboxes that only implement the buffered exec_with_output, such as smol_vm and the ARP provider in current agentenvhub-sdk pins, fail with an AttributeError. This blocks tasks moving off the retired beta sandbox onto ARP (FDE-2526, P0).

What

collect_artifacts.py _collect_file:

Sandbox has Before After
exec (streaming) stream + incremental decode unchanged
only exec_with_output AttributeError: ... no attribute 'exec' one exec_with_output("bash", "-c", "base64 < f"), decoded in one piece

The size integrity check and the upload path are unchanged for both branches.

For reviewers

The streaming branch is the same code, indented one level. The fallback holds the whole base64 text in memory, which is fine at artifact sizes. A streaming exec for the ARP provider is in scaleapi/scaleapi#169759; this PR unblocks collection on the SDK versions already pinned.

Verification

  • New test_a_sandbox_without_streaming_exec_collects_over_buffered_exec (binary payload, 76-column wrapped base64). Before the change it fails with the same AttributeError users hit; after the change it passes.

  • tst/unit/task_step/test_collect_* and test_container_sandbox_routing: 107 passed. ruff clean.

  • Live on ARP staging (one 1-CPU VM, deleted afterwards), on a 3 MB random file:

    Check Result
    the ARP provider's streaming exec("bash","-c","base64 < f; echo to-stderr >&2; exit 3") streamed 990 chunks, decoded SHA-256 matches the source, stderr = to-stderr, wait() = 3
    collect_artifacts._collect_file via streaming exec uploaded bytes match the source
    collect_artifacts._collect_file via exec_with_output only (this PR) uploaded bytes match the source

🤖 Generated with Claude Code

@dmchoiboi
dmchoiboi requested a review from a team as a code owner October 23, 2026 14:48
…streaming exec

Sandboxes that only implement exec_with_output (smol_vm, and the ARP provider in agentenvhub-sdk <= 0.34.18) failed in _collect_file with AttributeError. FDE-2526.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dmchoiboi
dmchoiboi force-pushed the michaelchoi/_claude/collect-artifacts-exec-fallback branch from 3a058fa to 6a94df6 Compare October 23, 2026 14:50
await stderr_task
except Exception:
pass
if not hasattr(sandbox, "exec"):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 smol_vm collection still fails

SmolVmSandbox inherits Sandbox.exec, which raises NotImplementedError. That makes hasattr(sandbox, "exec") true, so collection still takes the streaming branch and fails instead of using exec_with_output.

Detect unsupported streaming execution, not just a missing attribute. Add a test using a sandbox that inherits the base exec method; the current fake has no such method and misses this case.

Knowledge Base Used: Provider integrations and sandbox deployment

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agent_env/task_step/task_steps/collect_artifacts.py
Line: 482

Comment:
**`smol_vm` collection still fails**

`SmolVmSandbox` inherits `Sandbox.exec`, which raises `NotImplementedError`. That makes `hasattr(sandbox, "exec")` true, so collection still takes the streaming branch and fails instead of using `exec_with_output`.

Detect unsupported streaming execution, not just a missing attribute. Add a test using a sandbox that inherits the base `exec` method; the current fake has no such method and misses this case.

**Knowledge Base Used:** [Provider integrations and sandbox deployment](https://app.greptile.com/scale-ai/-/custom-context/knowledge-base/scaleapi/agentenv-framework/-/docs/provider-integrations.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Cursor Fix in Claude Code Fix in Codex

@dmchoiboi dmchoiboi closed this Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant