feat(bundle): pull the shipped agent's published image instead of building it

AGENTS.md

@@ -106,7 +106,7 @@ and deserialized through a registry.
 | Package | Contents |
 |---|---|
 | `env/` | `Env` base class: a subclass declares `type`, implements `from_dict` and `async deploy(...) -> DeployedEnv`, and may override `reset` and `load_file_artifact_universe`. Built-ins in `env/envs/`: `MCPServerEnv` (`mcp_server`), `MultiEnv` (`multi`, MCP servers plus websites), `ServiceDBEnv` (`service_db`, PostgreSQL), `WebsiteEnv` (`website`), `GatewayEnv` (`gateway_server`). `env/gateway/gateway.py` is the env-side gateway server, container-only code. |
-| `artifact/` | Immutable versioned artifacts; every `put()` creates a new version. The document goes to the document store, binary payloads to the object store. Built-ins: `CliArtifact` (`cli`), `FileArtifact` (`file`), `FileArtifactUniverse` (`file_artifact_universe`), `DockerImageArtifact` (`docker_image`), `EnvironmentArtifact` (`environment`), `EnvironmentUniverseArtifact` (`environment_universe`), `SkillArtifact` (`skill`); `[artifacts] type_aliases` maps another stored spelling of a type onto one of them. A `DockerImageArtifact` holds a tar.gz of its image, or, written by `put_ref` (`--image-ref` on `env mcp-server put` and `a2a-agent put`), only a registry reference with its tag pinned to a digest, which sandboxes pull, or, written by `put_context`, only its build context, which a VM sandbox builds before running it (`VmSandbox.load_docker_images`) and Modal builds before creating its container (`ModalSandboxProvider.prepare_image`, outside a fallback chain's deadline); anything else that runs images by name (a container on the local provider, a lone server on any provider but Modal) refuses such an image. A build context (`utils/docker/context.py`) is what `docker build` would send, `.dockerignore` applied, written so equal files give equal bytes and an equal `source_digest`. |
+| `artifact/` | Immutable versioned artifacts; every `put()` creates a new version. The document goes to the document store, binary payloads to the object store. Built-ins: `CliArtifact` (`cli`), `FileArtifact` (`file`), `FileArtifactUniverse` (`file_artifact_universe`), `DockerImageArtifact` (`docker_image`), `EnvironmentArtifact` (`environment`), `EnvironmentUniverseArtifact` (`environment_universe`), `SkillArtifact` (`skill`); `[artifacts] type_aliases` maps another stored spelling of a type onto one of them. A `DockerImageArtifact` holds a tar.gz of its image, or, written by `put_ref` (`--image-ref` on `env mcp-server put` and `a2a-agent put`), only a registry reference with its tag pinned to a digest, which sandboxes pull, or a published image's tag with its `image_digest` and its build context, which sandboxes pull by the digest and build from the context when they can't (the infra images, and a bundle agent built from a shipped agent's sources, `Plan.published`), or, written by `put_context`, only its build context, which a VM sandbox builds before running it (`VmSandbox.load_docker_images`) and Modal builds before creating its container (`ModalSandboxProvider.prepare_image`, outside a fallback chain's deadline); anything else that runs images by name (a container on the local provider, a lone server on any provider but Modal) refuses such an image. A build context (`utils/docker/context.py`) is what `docker build` would send, `.dockerignore` applied, written so equal files give equal bytes and an equal `source_digest`. |
 | `task/`, `task_step/` | A `Task` holds its `TaskStep`s inline; `Task.run()` executes them as a DAG. `depends_on` (None means all prior steps) gates a step, independent steps run concurrently, `fail_task_on_error` makes a failure fatal or tolerated, `retry_config` rolls a failed span back through the step journal and re-dispatches it. Built-in steps live in `task_step/task_steps/` (`deploy_env`, `deploy_agent`, `prompt_agent`, the verifiers under `verifiers/`, and more). `task/episode/` runs a task with some of its agents played from outside, as a training loop does (`EpisodeSpec`). |
 | `store/` | Four store ABCs with local and cloud implementations: `DocumentStore` (SQLite, MongoDB, DynamoDB, Firestore with MongoDB compatibility), `ObjectStore` (filesystem, S3, Cloud Storage), `ImageStore` (local OCI registry, ECR), `SecretStore` (env vars or file, AWS Secrets Manager, Google Cloud Secret Manager). `VersionedEntityStore` implements the shared versioned get/put logic, `QueryBuilder` is the immutable chained query API, `store/base.py` holds the error types. `store/routing.py` is the `@local` namespace: while the CLI runs, an `@local/…` id and what an `@local` task's run records go to local stores beside the configured ones, and that run's writes to the configured stores are refused. A new backend must pass the conformance kits in `tst/store/`. |
 | `config/` | The `Config` singleton (`get_config`, `configure`, `reset_config`) in `config/runtime.py`, file discovery in `config/loader.py`, and `load_impl`, which resolves `module:Class` pointers. `agent_env.store` re-exports the config names for compatibility. |

CONTRIBUTING.md

@@ -98,7 +98,8 @@ Each of them pins its base images by digest, and Dependabot bumps those pins, so
 image: a bump changes the sources, and the next release builds it again.
 Installs pull them instead of building the infra images: a run on local stores registers a kind's images when GHCR has
 every one for the sources installed, by the digest the wheel records or, with no record, the one GHCR serves, and
-builds them otherwise; a sandbox that can't pull one builds it from its build context. The wheel and the release notes
+builds them otherwise; a sandbox that can't pull one builds it from its build context. A run that deploys the Claude
+Code agent registers its published image the same way, in place of its folder's build context. The wheel and the release notes
 record each image's digest. A failed image job keeps `agentenv-framework` off PyPI. Maintainers can publish the images
 alone by running the workflow by hand on `main`. GHCR keeps a new package private until a package admin makes it
 public, and the workflow warns about each one until then.

src/agent_env/agents/README.md

@@ -11,7 +11,9 @@ on every model call. The `LITELLM_*` variables stay in its environment too, so a
 endpoint. The image holds Claude Code, which is Anthropic's software under its
 [Commercial Terms](https://www.anthropic.com/legal/commercial-terms); a release also publishes the image, CLI included,
 at `ghcr.io/scaleapi/agentenv-claude-code:src-<hex>`, built from this folder with the protocol pinned as the wheel
-ships it. The CLI's version is the Dockerfile's
+ships it. A run that deploys the agent registers that image when it's published for the sources installed: each deploy
+pulls it by its digest and builds it from this folder only when it can't, and a provider that runs images by name can
+run it. The CLI's version is the Dockerfile's
 `CLAUDE_CODE_VERSION`. The image gives the scripts the model runs
 `node`, without npm or npx, and the Python packages `model-requirements.txt` pins.
 

src/agent_env/bundle/authoring.py

@@ -3,14 +3,18 @@
 from __future__ import annotations
 
 import os
+import shutil
 import stat
+import tempfile
 import unicodedata
-from collections.abc import Mapping, Sequence
+from collections.abc import Iterator, Mapping, Sequence
+from contextlib import contextmanager
 from dataclasses import dataclass
 from pathlib import Path
 from typing import Any, NoReturn, TypeVar, overload
 
 from agent_env.artifact.artifact import Artifact
+from agent_env.artifact.artifacts.docker_image import DockerImageArtifact
 from agent_env.entity.refs import EntityKind, parse_toml_ref
 from agent_env.env.env import Env
 from agent_env.env.envs._deployment import provider_refusal
@@ -255,6 +259,31 @@ def build_context_files(bundle: Bundle, entry: BundleEntry) -> dict[str, Path]:
     return dict(sorted(files.items()))
 
 
+@dataclass(frozen=True)
+class StagedContext:
+    """A copy of ``build_context_files`` for the image an entry's Dockerfile builds, so an image written from it was
+    made from what the ledger hashed."""
+
+    path: Path
+    dockerfile: str  # its whole path in the copy: a bare name is looked up from the current folder first
+
+    def identity(self, image_id: str) -> tuple[str, str]:
+        """The ``source_digest`` and ``image_name`` ``put_context`` gives ``image_id`` from this copy, built for each
+        sandbox's own platform, as a bundle writes its images."""
+        return DockerImageArtifact.context_identity(image_id, str(self.path), self.dockerfile, platform=None)
+
+
+@contextmanager
+def staged_build_context(bundle: Bundle, entry: BundleEntry, dockerfile: str) -> Iterator[StagedContext]:
+    """``entry``'s ``StagedContext`` for its ``dockerfile``, a path in its folder, removed on leaving."""
+    with tempfile.TemporaryDirectory(prefix="agent-env-build-") as staged:
+        context = Path(staged)
+        for key, path in build_context_files(bundle, entry).items():
+            (context / key).parent.mkdir(parents=True, exist_ok=True)
+            shutil.copy2(path, context / key)
+        yield StagedContext(context, str(context / dockerfile))
+
+
 def entry_file(bundle: Bundle, entry: BundleEntry) -> tuple[str, Path]:
     """The one file in an entry's folder, as ``(name, path)``. Raises BundleError otherwise."""
     key, path = next(iter(one_file(bundle, entry).items()))

src/agent_env/bundle/ledger.py

@@ -49,7 +49,9 @@
 @dataclass(frozen=True)
 class Digest:
     value: str
-    inputs: dict[str, Any]  # type, config, files {key: sha256}, needs and store_refs {"<kind> <id>": version}
+    # type, config, files {key: sha256}, needs and store_refs {"<kind> <id>": version}, and for an image published for
+    # its sources, published: "<ref>@<digest>"
+    inputs: dict[str, Any]
 
 
 @dataclass(frozen=True)
@@ -124,6 +126,8 @@ def digest(self, write: Write, needs: Mapping[tuple[str, str], int]) -> Digest |
             listing = build_context_files if built else folder_files
             for key, path in listing(self._plan.bundle_of(write.source.entry), write.source.entry).items():
                 inputs["files"][key] = _file_sha256(path) + (_mode(path) if built else "")
+        if (published := self._plan.published.get(write.id)) is not None:
+            inputs["published"] = published.pinned
         if write.kind not in (BundleKind.TASK, BundleKind.EVAL):
             # A document records the versions of what it references (an env its images, a universe its
             # environments), so one written anew means it must be written again. A task or eval names its
@@ -210,8 +214,9 @@ def _tracked(write: Write) -> bool:
     """Whether the ledger can list everything ``write`` is made from. Not yet for an env type with a ``from_toml`` of
     its own, which may read its folder in ways it can't see. An artifact, a skill included, is made from its toml,
     every file in its folder (``folder_files``) and what it references; an agent or env from its
-    toml and what it references. A built image is made from its build context, ``build_context_files``; what the
-    build fetches (its base image, packages) isn't an input. A referenced image is made from its ref's text, so a tag
+    toml and what it references. A built image is made from its build context, ``build_context_files``, and the
+    image published for it when there is one (``Plan.published``); what the build fetches (its base image, packages)
+    isn't an input. A referenced image is made from its ref's text, so a tag
     that moves in its registry isn't a change. An image folder's image is also made from its description."""
     if isinstance(write.source, BundleImage):
         return True
@@ -253,6 +258,9 @@ def _changes(row: Mapping[str, Any], digest: Digest) -> list[str]:
         reasons.append(f"type changed: {before['type']} → {after['type']}")
     if before.get("config") != after["config"]:
         reasons.append("config changed")
+    if before.get("published") != after.get("published"):
+        reasons.append(f"published as {after['published'].partition('@')[0]}" if after.get("published")
+                       else "not found published")
     for label, names in zip(("files added", "files removed", "files changed"), _diff(before["files"], after["files"])):
         if names:
             shown = ", ".join(names[:_LISTED]) + (f" and {len(names) - _LISTED} more" if len(names) > _LISTED else "")

src/agent_env/bundle/materialize.py

@@ -4,7 +4,9 @@
 that can't be written whole. It needs the CLI's namespace routing, which sends ``@local`` writes to the
 ``@local`` namespace's store. Holding the bundle's lock, it writes the entities in the plan's order, an image
 built from an entry's Dockerfile just before the entry, as its build context alone, which each deploy builds where
-it runs (a sandbox on this machine with this machine's Docker). Then it builds and preflights every task
+it runs (a sandbox on this machine with this machine's Docker). An agent's image built from a shipped agent's sources
+that agent-env publishes is registered as the published image instead, which each deploy pulls by its digest, with the
+build context beside it for a sandbox that can't pull it. Then it builds and preflights every task
 before writing any of them, and writes the evals last, since they name the tasks. Each write goes through the
 ledger, so one whose inputs haven't changed reuses the version the bundle last wrote. A reused task keeps
 whatever its steps took from config when it was first written, such as a rubrics verifier's default judge
@@ -15,12 +17,9 @@
 from __future__ import annotations
 
 import copy
-import shutil
-import tempfile
 from collections.abc import Callable, Iterator
 from contextlib import contextmanager, nullcontext
 from dataclasses import dataclass
-from pathlib import Path
 from typing import Any
 
 from agent_env.a2a_agent import A2AAgent
@@ -39,7 +38,7 @@
 from agent_env.utils.docker.build import DockerBuildError
 
 from ._fs import with_article
-from .authoring import AuthoringContext, build_context_files
+from .authoring import AuthoringContext, staged_build_context
 from .ledger import Ledger, locked_ids, materializing
 from .parse import BundleError, BundleKind
 from .plan import Plan, Write, artifact_writer, env_writer, unpinned_store_refs
@@ -86,7 +85,8 @@ def materialize(
     holds a lock this one needs, ``on_build`` before an image is built here, and ``on_write`` after each write,
     reused or not. An image the bundle builds whose id is in ``build_here``, as a deploy on this machine runs it
     (``preflight_run``), is also built with this machine's Docker, before it's written or, reused, when Docker doesn't
-    hold it, so a build that fails writes nothing; its version is its build context either way.
+    hold it, so a build that fails writes nothing; its version is its build context either way. One published for its
+    sources (``Plan.published``) is never built here: a deploy pulls it.
 
     ``dry_run`` checks and preflights what a run would, and writes nothing: no entity, ledger row or lock.
     Each write gets the version it would reuse or the store's next, which another run can take first. A step
@@ -143,44 +143,51 @@ def _write_entity(plan: Plan, write: Write, here: bool, on_build: Callable[[Writ
     if isinstance(write.source, ReferencedImage):
         return _write_referenced_image(plan, write)
     if isinstance(write.source, BuiltImage):
+        if write.id in plan.published:
+            return _write_published_image(plan, write)
         return _write_build_context(plan, write, here, on_build)
     return _WRITERS[write.kind](plan, write)
 
 
-@contextmanager
-def _staged(plan: Plan, image: BuiltImage) -> Iterator[Path]:
-    """A copy of the files the ledger hashes, ``build_context_files``, so an image the ledger reuses was made from
-    what it hashed."""
-    with tempfile.TemporaryDirectory(prefix="agent-env-build-") as staged:
-        context = Path(staged)
-        for key, path in build_context_files(plan.bundle_of(image.entry), image.entry).items():
-            (context / key).parent.mkdir(parents=True, exist_ok=True)
-            shutil.copy2(path, context / key)
-        yield context
-
-
 def _write_build_context(plan: Plan, write: Write, here: bool, on_build: Callable[[Write], None] | None) -> int:
     """Write the image an entry's Dockerfile describes as a docker_image artifact holding its build context alone,
     which each deploy builds where it runs, for its sandbox's platform; ``here``, built first with this machine's
     Docker, for the deploys here."""
     image = write.source
-    with _staged(plan, image) as context:
+    with staged_build_context(plan.bundle_of(image.entry), image.entry, image.dockerfile) as staged:
         if here:
-            sources, name = DockerImageArtifact.context_identity(write.id, str(context), str(context / image.dockerfile),
-                                                                 platform=None)
+            sources, name = staged.identity(write.id)
             try:
-                build_staged(context, image.dockerfile, name, sources, _before(on_build, write))
+                build_staged(staged.path, image.dockerfile, name, sources, _before(on_build, write))
             except DockerBuildError as e:
                 raise BundleError([f"{_path(plan, write)}: {_tail(str(e))}"]) from None
         try:
             return DockerImageArtifact.put_context(
-                write.id, description=_image_description(plan, write), context_path=str(context),
-                dockerfile_path=str(context / image.dockerfile), platform=None,
+                write.id, description=_image_description(plan, write), context_path=str(staged.path),
+                dockerfile_path=staged.dockerfile, platform=None,
             ).version
         except (ValueError, RuntimeError) as e:  # a context it can't write, or the object store
             raise BundleError([f"{_path(plan, write)}: {e}"]) from None
 
 
+def _write_published_image(plan: Plan, write: Write) -> int:
+    """Register the image published for the sources an entry's Dockerfile builds, by its digest, with its build context
+    beside it, which a sandbox that can't pull the image builds it from; nothing is built here. Refused when the
+    sources changed since the run planned it, so the image and its context are never of different sources."""
+    image, published = write.source, plan.published[write.id]
+    with staged_build_context(plan.bundle_of(image.entry), image.entry, image.dockerfile) as staged:
+        try:
+            if staged.identity(write.id)[0] != published.sources:
+                raise ValueError(f"its files changed after the run found {published.ref} published for them; "
+                                 "run it again")
+            return DockerImageArtifact.put_ref(
+                write.id, description=_image_description(plan, write), image_name=published.ref,
+                image_digest=published.digest, context_path=str(staged.path), dockerfile_path=staged.dockerfile,
+            ).version
+        except (ValueError, RuntimeError) as e:  # changed sources, a context it can't write, or the object store
+            raise BundleError([f"{_path(plan, write)}: {e}"]) from None
+
+
 def _build_reused(plan: Plan, write: Write, version: int, on_build: Callable[[Write], None] | None) -> None:
     """Build the image the bundle wrote earlier with this machine's Docker, when Docker doesn't hold it."""
     image = DockerImageArtifact.get(write.id, version)