feat(bundle)!: bundle init writes every env, agent and data artifact a task uses into the folder

[earakely-scale]

agent-env bundle init now writes every env, agent and data artifact a task uses into the folder, so the folder runs with no store. Before, it wrote the task and pinned each reference to the store.

What the folder holds

• envs/<name>/: each env's env.toml, beside its image's build context, pruned to what its Dockerfile reads.
• agents/<name>/: only with --include-scope user, since the agent a step names is the user's.
• artifacts/<name>/: files, file universes, environments, environment universes and CLIs.
• skills/<name>/: skills, in folders named by the skill's own name.
• A README with what a machine needs to run it ("Run it") and how to share it, and provenance.json with store ids and versions only, no bucket URLs.

Every reference to something written into the folder becomes its local name, with no pin.

How

• Each core type gets to_toml(out), the inverse of from_toml. A type never downloads anything itself; it asks the writer for another entity's local name, an image's toml value, or a file at a checked path.
• The get --output-dir writers for universes, CLIs, skills and file universes move onto their types, and the CLI calls them.
• Images:
  • A stored build context is downloaded once per stored object (grouped by ETag) and pruned into each image's folder.
  • An image whose context can't be written as a folder (one naming no Dockerfile and holding several, say) falls back to the registry ref it names, and the report says why.
  • Neither: the import fails, or with --include-scope user the image stays a pinned store ref, reported under Still needs the store.
• Versions:
  • Where the store holds the image's docker save tarball, the import reads the Python and global npm packages it holds, without Docker.
  • Python installs are pinned with a constraints.txt beside the Dockerfile, passed to each stage running pip install|wheel|download or uv pip as PIP_CONSTRAINT and UV_CONSTRAINT build args. Build args, not ENV, so the variables don't persist into the running container.
  • npm install -g gets exact versions.
  • A version is left unpinned, and the report says why, when:
    • a project lock file pins its packages already;
    • the build asks for a version the image ended past;
    • the package was installed from a path or URL;
    • pip can't read the version;
    • it is pip, setuptools or wheel.
  • The README says how to build an image unpinned.
• A hard link in a stored context is written as a file of its own holding what it links to, as tar -x extracts it.
• A step field naming an entity of a type it doesn't take is left out and reported. The store holds such a step as it is, though the step can't read the entity.
• An env named with no version takes the version the task pins elsewhere: such a step, as build_mcp_cli does, means the env an earlier step deployed, which a folder holds once. An artifact or agent named with no version keeps the latest, in a folder of its own, as the store run reads it.
• Files the OS or Python leaves in a stored artifact (__pycache__, .DS_Store) are left out and reported, as a bundle leaves them out of every folder it reads.
• bundle init prints the total download size first, and lists the entity folders it wrote.

agent-env run checks before it starts

• docker compose resolves when a gateway stack runs on this machine.
• docker buildx resolves when an image builds here. Docker 23+ fails docker build without it.
• A model endpoint is set when an agent or agent judge deploys, unless its step or the agent itself gives the variables.

Each refusal says how to install or set what's missing. VM and local builds now extract a stored context at extraction times (tar -xzmf): BuildKit reuses a file from a same-named folder whose path, size and time match, so a changed file of the same size could build as the old one.

Decisions

1. What a folder can't hold fails the import; with --include-scope user it stays in the store, pinned and reported. An image whose context can't be written but which names a registry is written as that ref instead, since a folder can hold a ref.
   • A type with no to_toml fails a default-scope import. That includes, for now, the sdk's plugin types, which get to_toml in a follow-up.
2. The contract test is a core-only round trip of every type through to_toml/from_toml.
3. No check after the build that the image matches the stored one.
4. The README's "Run it" section and the preflight checks above; bundle pack is left for later.
5. An image folder shared by several entities is copied into each.
6. Version pinning reads each stored image in full, one at a time. Its size is reported before the download starts.

Tests

• Unit: the tier passes (8,315). New: every core type round-trips through to_toml/from_toml; installed_packages on synthetic legacy and OCI docker save tarballs (whiteouts, opaque folders, linked layers); lock (multi-stage, uv pip, heredocs, lock files, versions asked for and ended past, ignore files, npm -g); hard links; the preflight checks.
• Integration (Linux, Docker): the CLI and Docker tiers pass, apart from failures main shares on that VM. New:
  • bundle_init_images_test: an image stored built at old versions imports, plans against an empty store, and rebuilds at the stored versions, though its Dockerfile names none.
  • vm_sandbox_build_test: two stored contexts built on one host as a VM builds them each hold their own file. It fails on main.
• Chaos (Linux), 182 trials:
  • SIGINT, SIGTERM and SIGKILL mid-import never leave a partial folder, and SIGINT exits 1 with no traceback. SIGKILL leaves hidden staging, as before.
  • Corrupt stored contexts fail by default, stay in the store with --include-scope user, and fall back to the ref when the image names a registry.
  • A corrupt stored tarball leaves versions unpinned and says so.
  • Concurrent imports into one folder: one wins. Into several: identical folders.
• Acceptance: real prod tasks imported with this branch, then run against empty stores on fresh Ubuntu and macOS (colima) machines, Haiku for every role. The agent was claude-code-cli, copied in from a --include-scope user import and named the default.

  • Imports, all succeeding:

    • CoopBench t03: 11 MCP servers in a multi env, whose shared stored context downloads once;
    • a multi env with two websites;
    • a context-built MCP server with its universe and knowledge base;
    • an hg4 build_mcp_cli task;
    • a Brookfield task with 12 CLIs;
    • 11 iOS skill artifacts;
    • a run_docker_container task whose Docker context is an artifact.

    The imports found the hard-link, wrong-type-field, unpinned-reference, __pycache__ and no-Dockerfile cases above.

  • Every folder dry-runs against an empty store on both machines.

  • Both machines:

    • the CLI task and the context-built MCP server task pass;
    • t03 runs all 18 steps, and the judges score Haiku 0, as before. Its stored warm-up scripts hard-code the gateway's port inside a VM, so the run used a local copy reading the env URL run_code gives.

  • The run_docker_container task builds its context from the folder and runs the container (Linux, as root, since the local provider's VM sandbox is this machine). It stops later, where run_code targets an agent container install_agent put inside that container: a local-provider issue, not this PR's.

  • The website and Brookfield folders stop before running. Their images are refs to a private registry, which agent-env reads only through its configured image store; the README says so.

  • The iOS skills task fails at add_skills: current core sends a stored skill as objects, which that agent's version doesn't take.

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge; no new blocking issue remains.

Summary

bundle init writes referenced entities and image build contexts into a runnable folder. The latest changes extract helpers and share constants without changing their behavior.

• Bundle folders now include the supported entities a task uses.
• Image folders now carry pruned build files and stored package versions.
• Local runs now check for Docker plugins and model settings they need.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Stored task or eval] --> B[Read referenced entities]
  B --> C[Write entity folders]
  B --> D[Download shared build contexts]
  D --> E[Prune files for each image]
  E --> F[Read saved package versions]
  F --> G[Pin compatible installs]
  C --> H[Write local references and reports]
  G --> H
  H --> I[Move completed bundle into place]

Loading

_{Reviews (6) · Last reviewed commit: "bundle init's guidance has one source ea..." · Reviewed by Greptile}