[earakely-scale]

Why. agent-env run has --model but no way to pick the agent. Running a bundle's task with Pi or OpenHands instead of Claude Code means editing the task, and only task run --a2a-agent-id can override it, for stored tasks. Comparing harnesses on the same task should be one flag.

Change

• agent-env run <bundle> --agent <name> (and --dry-run) runs each task's solver as that agent. It resolves the name the same way as a task reference to an agent: an alias such as pi or openhands, an agent of this bundle by its folder name, an installed bundle's agent, or the store's by id.
  • A name that matches no agent is refused before anything runs, with the alias hint a2a-agent get gives (did you mean the alias 'pi'?).
  • A bundle whose tasks prompt no agent is refused too, since there is no solver to swap.
• Which steps it affects. The solver steps are the deploy_agent and install_agent steps for an agent a prompt_agent step prompts, minus the user sims those steps name (user_agent_name); plan.solver_steps picks them. User sims, judges and other agents keep the agents their tasks name.
• Tasks stay as written. The override is attached to each task as an extra reference, so the agent is pulled, written, image-checked and preflighted like any other. The task's config is untouched, so its version doesn't change.
• How the run applies it. Each task runs with user_overrides.step_params[<step>] = {a2a_agent_id, a2a_agent_version}, which deploy_agent already honors per step.
  • install_agent now reads it too. Otherwise it keeps its rule, so task run --a2a-agent-id installs as before. deploy_agent is unchanged.
  • A bundle's own agent or an installed bundle's runs at the version the run wrote. A store agent runs at the version the plan read, as bundles pin their other store references, so one put mid-run isn't deployed.
  • The task-write preflight checks the --agent agent at the steps it swaps, so an agent that can't be installed is refused before any sandbox starts.
• The dry run and the run print --agent pi: the solver deploys @local/agentenv-framework/agents/pi, in 1 of 1 task, plus the alias line.

Tests

• agent_override_test.py covers:
  • resolving an alias, the bundle's own agent and the store's;
  • a typo, and a bundle with no prompted agent;
  • solver step selection;
  • preflight checking the override, for a deploy and for an install, in both directions;
  • the dry-run line;
  • step params set on solver steps only, at the written version, or the store version the plan read;
  • the install step's override precedence;
  • the CLI.
• run_bundle_built_agents_test.py (slow tier, about 25 s): a task naming one built echo agent runs with --agent set to another. It checks which agent replied from the instance, then runs the task as written to confirm it stays v1.
• The plugin API check reports no break. The unit tier passes.

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge; no new actionable issues were found.

Summary

Adds agent-env run --agent to replace solver agents for one run without changing saved tasks.

• Bundle runs can swap the solver agent without changing task files.
• Process checks also count reads that lose a race as stopped.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A["run --agent NAME"] --> B["Resolve agent and solver steps"]
  B --> C["Check replacement agent"]
  C --> D["Write bundle entities"]
  D --> E["Set per-step agent ID and version"]
  E --> F["Run selected tasks"]
  F --> G["Saved task definitions stay unchanged"]

Loading

_{Reviews (6) · Last reviewed commit: "test: treat a process that exits mid-rea..." · Reviewed by Greptile}

[earakely-scale]

End to end on one m5.2xlarge (linux/amd64) admin VM.

• Setup: a clean venv with this branch's wheels, claude-haiku-5-5 via LiteLLM, each case in its own store. The user-sim and install cases ran at eec3448; the rest ran at 6169801 (eec3448 changes only preflight).
• How I checked which agent ran: the instance's deployed agent card (task get-instance) and the image Docker started.

Case	Result
run agents --task claude-code-smoke --agent pi, --sandbox local	dry run: prints --agent pi: the solver deploys @local/agentenv-framework/agents/pi, in 1 of 1 task; checks pi's published image run: passed (30 s) on pi, via step_params {"deploy": {"a2a_agent_id": "…/agents/pi", "a2a_agent_version": 1}}; Docker started agentenv-pi:src-5d316c3e… stored task: still names claude-code
Same task, --agent openhands, then without --agent	openhands: passed on OpenHands (75 s) without --agent: passed on Claude Code (18 s) task version: stays v1 across all three runs (v1, unchanged)
Solver prompted by a Claude Code user sim, --agent pi	passed; solver on pi, user sim still on Claude Code; only the solver step gets step_params. As written, both are Claude Code and it passes too.
A store agent: a2a-agent put --id my-pi --image-ref <pi's published ref>, then --agent my-pi	passed on pi, a2a_agent_version: null (the store's latest)
--agent pii	refused before anything runs: there is no agent 'pii' in this bundle, an installed bundle or the store; did you mean the alias 'pi'?
--agent pi --sandbox modal_vm	passed (16 s) on pi
install_agent naming no agent, into a Debian task container, --agent pi	dry run: lists the install as not preflighted, since it reads pi, which the run writes first run: installed pi into the container (deployed coder: the pi card, step_params on install-coder); all 6 steps completed check: scored 0, because pi read /workspace/notes.txt while the task's WORKDIR is /workspace/task_repo (see below)

pi works in /workspace once installed into a container, not in the container's working directory as Claude Code does: its start.sh runs from /opt/pi-a2a, and PiAgent defaults workspace to /workspace. A task naming pi in install_agent hits the same thing, so it isn't this PR's. I can fix it separately.

No containers or Modal sandboxes were left behind, and the results contain no secret text. The VM, security group and key pair are deleted.