Skip to content

feat(bundle): pull the shipped agent's published image instead of building it - #293

Merged
earakely-scale merged 4 commits into
mainfrom
feat/pull-published-agent-image
Oct 22, 2026
Merged

earakely-scale merged 4 commits into
mainfrom
feat/pull-published-agent-image

Conversation

@earakely-scale

@earakely-scale earakely-scale commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Why. #288 publishes the Claude Code agent's image. This PR makes a run use it: a deploy of the default agent pulls the published image by its digest instead of building it, on every VM, from its folder. A remote agent deploy spent 51–58 s of its 56–63 s on that build. It also lets a provider that runs images by name run the agent, which it refused before because the image was only a build context.

Change

  • Lookup: published_images.published_agent_image(sources) finds the image published for a shipped agent whose build context has those sources. The digest comes from the record the wheel carries, or from the registry when there is no record.
  • Plan: Plan.published records the lookup once, for each agent image a bundle builds. Preflight, the ledger and the writer all read that one answer.
    • It matches by source digest, so a bundle holding an unchanged copy of the agent's folder gets the published image too. A changed folder doesn't.
  • Write: materialize registers such an image with put_ref: the published tag, its image_digest, and the staged build context beside it as the fallback. Nothing is built on this machine.
    • The plan keeps the source digest it found the publication for, and the write checks the staged context against it. If the folder changed after planning, the write refuses and asks for the run to be run again, so the image and its fallback context are never of different sources.
  • Ledger: the publication is one of the image's inputs. A version written before the image was published is written again as the published one (reason: published as <ref>), and its agent is rewritten with it; the next run reuses both. Unpublished images hash exactly as before, so no scheme bump.
  • Preflight checks a published image as the registry image it is. It isn't added to the images built here, and a provider that runs images by name takes it.
  • Local provider: prepare_image now pulls an image that has an image_digest, by that digest, onto this machine's Docker (host_images.pull). It builds the image from its context when the pull fails. It skips the pull only when Docker already holds the image as the one its digest names: pulled by that digest, or built here from its sources. A tag that two versions share can name another image. When the container then starts, the local sandbox doesn't pull the tag again over an image its provider prepared, or one held under a digest-pinned name. The VM providers and Modal already handle such images (feat(env): set up the infra envs from their published images, building only what isn't published #277).
  • Dry run says how the image would be written: <ref>, published for its sources, which each deploy pulls by its digest, or builds from its build context when it can't.
  • Docs: the agents README, CONTRIBUTING.md, AGENTS.md, and the provider guide's prepare_image row.

A bug fix in #277's bootstrap. _resolve_dockerfile looks a relative Dockerfile name up from the current folder first. Bootstrap passed bare names (Dockerfile), so a run started in a folder holding a Dockerfile of its own picked up the wrong file:

  • registering a published infra image failed: … is outside the build context;
  • building one stored an image with no Dockerfile recorded.

Bootstrap now passes each Dockerfile's whole path. That records the same dockerfile_path and the same source_digest.

Verified

  • The unit tier passes (8,956). New tests cover:
    • the lookup;
    • the plan, write and ledger transitions (published, then reused; unpublished, then published);
    • preflight on a provider that runs images by name, and on local;
    • the host pull: by digest, already held, falling back to a build, and no context;
    • the local provider's prepare_image;
    • the bootstrap fix, run from a folder holding a Dockerfile.
  • The shipped bundle's smoke task, dry-run with the registry stubbed to "published": on both local and modal, the Claude Code image is registered from ghcr.io/scaleapi/agentenv-claude-code:src-…, and nothing is built here.
  • Deploys against the real published image on local (amd64 and arm64), modal_vm and Modal containers all passed, as did the fallback to building when GHCR doesn't serve the digest. The timings are in the acceptance comment below; for example, a modal_vm run takes 21–22 s, against 86 s when it builds the image.

🤖 Generated with Claude Code

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; no new blocking issues remain.

Summary

Bundles now register shipped agents’ published images by digest, with their build contexts kept for fallback builds.

  • Matching agent sources use the published image instead of a bundle-built image.
  • Local Docker pulls published images by digest and reuses them for deploys.
  • Bootstrap uses each infra image’s full Dockerfile path.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Stage agent sources] --> B{Published image found?}
  B -->|Yes| C[Record publication in plan]
  C --> D[Check staged sources before write]
  D --> E[Store digest and fallback context]
  E --> F{Docker holds requested image?}
  F -->|Yes| I[Start prepared image]
  F -->|No| G[Pull by digest]
  G -->|Success| I
  G -->|Failure| H[Build fallback context]
  H --> I
  B -->|No| J[Store build context]
Loading

Reviews (4) · Last reviewed commit: "refactor(bundle): one staged context, th..." · Reviewed by Greptile

…lding it

An agent image a bundle builds from a shipped agent's sources (the Claude Code agent's folder, unchanged) is now
registered as the image agent-env publishes for those sources, by its digest, with the build context beside it.
Each deploy pulls it by the digest and builds it from the context only when it can't, so a run no longer builds the
default agent's image on every VM, and a provider that runs images by name can run it.

- The planner looks the publication up once (Plan.published), from the record the wheel carries or the registry, so
  preflight, the ledger and the writer all see the same answer.
- The ledger hashes the publication, so a version written before the image was published is written again as the
  published one ("published as <ref>"), and reused after that.
- Preflight checks a published image as a registry image: it isn't built on this machine first, and a provider that
  runs images by name takes it.
- The local provider's prepare_image pulls an image with a digest by it, onto this machine's Docker, building it from
  its context when the pull fails.
- Bootstrap names each infra Dockerfile by its whole path: a bare name was looked up from the current folder first,
  so a run started in a folder holding a Dockerfile registered or built the wrong one.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@earakely-scale
earakely-scale marked this pull request as ready for review October 21, 2026 22:03
@earakely-scale
earakely-scale requested a review from a team as a code owner October 21, 2026 22:03
Comment thread src/agent_env/providers/sandbox_providers/host_images.py
Comment thread src/agent_env/bundle/materialize.py Outdated
…anged since planning

- The local provider pulled a published image by its digest in prepare_image, then the container's start pulled its
  tag again, which fails with the registry unreachable though Docker holds the image. A local sandbox now skips
  pulling an image Docker holds under a name that never moves: one pinned to a digest, or a source tag agent-env
  publishes (published_images.is_source_tag).
- The plan keeps the source digest it found the publication for (Publication.sources), and the write checks the
  staged context against it, so an agent folder edited after planning isn't registered as the published image with
  a build context of other sources; the run asks to be run again instead.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@earakely-scale

Copy link
Copy Markdown
Collaborator Author

Acceptance on admin VMs, at 59be254.

  • Hosts: Linux amd64 (m5.2xlarge) and Graviton arm64 (m7g.2xlarge, standing in for a Mac).
  • Task: the shipped agents bundle's claude-code-smoke. It deploys Claude Code, has it run echo $((6 * 7)) through claude-haiku-5-5, and checks for 42.
  • Image: agentenv-claude-code:src-a9de08bf… (sha256:e20bc867…), published by the run of publish-pypi.yml started by hand on main. The tree under test has the protocol pinned as that run pinned it, so its tag matches.
  • Baseline: the same tree unpinned, whose image isn't published, so it's built as on main.
  • Result: every run passed.
published image baseline: built from its folder
modal_vm, run total 22.1 s, 20.9 s; the VM pulls in 4.5–5.3 s 86.4 s; a 73.0 s build in the VM
local, arm64, run total 18.8 s (a 6.4 s pull), then 6.5 s with it held 57.7 s, building on the host
local, amd64, run total 20.6 s (a 7.8 s pull), then 6.9 s with it held 31.9 s, warm build cache (no cold baseline)
Modal containers 44.1 s, run from the image by its digest not run
a digest GHCR doesn't serve local: pull failed, built from context, passed (70.3 s); modal_vm: same (72.3 s)

Notes

  • Dry run: it says ghcr.io/scaleapi/agentenv-claude-code:src-a9de08bf…, published for its sources, which each deploy pulls by its digest, or builds from its build context when it can't, and builds nothing here.
  • Local deploys pull once: they pull by the digest in prepare_image, and the container then starts with no second pull of the tag.
  • Cleanup: no Modal sandboxes are left in the test app, and no secret text appears in any result. The Modal and LiteLLM key files were removed from the VMs before teardown. The VMs, security group and key pair are deleted.

…d by its digest

The skip covered only agent-env's published source tags; an image registered with a digest under another tag was
still pulled again by name when its container started. host_images.pull now records each image it had Docker hold,
and a local sandbox skips any of those it still holds, so the provider no longer needs to know agent-env's registry.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Comment thread src/agent_env/providers/sandbox_providers/host_images.py Outdated
…ication, and digest-checked host pulls

- host_images.pull takes an image Docker holds under its name only when it holds it as the image the digest names
  (pulled by that digest, or built here from its sources), and otherwise pulls the digest and moves the tag: a tag
  two versions share can name another image. holds() is never true for an image with no sources.
- The images prepare_image pulled are the local provider's own record, which it hands each sandbox it creates
  (LocalSandbox.prepared), in place of a module-level set that tied the two through call order.
- StagedContext carries the staged copy's path, the Dockerfile's whole path and its identity (put_context's, built
  for each sandbox's platform), so the planner and both writers stage and hash a build context the same way.
- Publication.unwritten is the image as the run will store it, which preflight checks providers against, beside the
  fields the writer stores.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@earakely-scale
earakely-scale merged commit 8b3a813 into main Oct 22, 2026
14 of 15 checks passed
@earakely-scale
earakely-scale deleted the feat/pull-published-agent-image branch October 22, 2026 04:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant