diff --git a/AGENTS.md b/AGENTS.md index 6ba84592..db9a5064 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -106,7 +106,7 @@ and deserialized through a registry. | Package | Contents | |---|---| | `env/` | `Env` base class: a subclass declares `type`, implements `from_dict` and `async deploy(...) -> DeployedEnv`, and may override `reset` and `load_file_artifact_universe`. Built-ins in `env/envs/`: `MCPServerEnv` (`mcp_server`), `MultiEnv` (`multi`, MCP servers plus websites), `ServiceDBEnv` (`service_db`, PostgreSQL), `WebsiteEnv` (`website`), `GatewayEnv` (`gateway_server`). `env/gateway/gateway.py` is the env-side gateway server, container-only code. | -| `artifact/` | Immutable versioned artifacts; every `put()` creates a new version. The document goes to the document store, binary payloads to the object store. Built-ins: `CliArtifact` (`cli`), `FileArtifact` (`file`), `FileArtifactUniverse` (`file_artifact_universe`), `DockerImageArtifact` (`docker_image`), `EnvironmentArtifact` (`environment`), `EnvironmentUniverseArtifact` (`environment_universe`), `SkillArtifact` (`skill`); `[artifacts] type_aliases` maps another stored spelling of a type onto one of them. A `DockerImageArtifact` holds a tar.gz of its image, or, written by `put_ref` (`--image-ref` on `env mcp-server put` and `a2a-agent put`), only a registry reference with its tag pinned to a digest, which sandboxes pull, or, written by `put_context`, only its build context, which a VM sandbox builds before running it (`VmSandbox.load_docker_images`) and Modal builds before creating its container (`ModalSandboxProvider.prepare_image`, outside a fallback chain's deadline); anything else that runs images by name (a container on the local provider, a lone server on any provider but Modal) refuses such an image. A build context (`utils/docker/context.py`) is what `docker build` would send, `.dockerignore` applied, written so equal files give equal bytes and an equal `source_digest`. | +| `artifact/` | Immutable versioned artifacts; every `put()` creates a new version. The document goes to the document store, binary payloads to the object store. Built-ins: `CliArtifact` (`cli`), `FileArtifact` (`file`), `FileArtifactUniverse` (`file_artifact_universe`), `DockerImageArtifact` (`docker_image`), `EnvironmentArtifact` (`environment`), `EnvironmentUniverseArtifact` (`environment_universe`), `SkillArtifact` (`skill`); `[artifacts] type_aliases` maps another stored spelling of a type onto one of them. A `DockerImageArtifact` holds a tar.gz of its image, or, written by `put_ref` (`--image-ref` on `env mcp-server put` and `a2a-agent put`), only a registry reference with its tag pinned to a digest, which sandboxes pull, or a published image's tag with its `image_digest` and its build context, which sandboxes pull by the digest and build from the context when they can't (the infra images, and a bundle agent built from a shipped agent's sources, `Plan.published`), or, written by `put_context`, only its build context, which a VM sandbox builds before running it (`VmSandbox.load_docker_images`) and Modal builds before creating its container (`ModalSandboxProvider.prepare_image`, outside a fallback chain's deadline); anything else that runs images by name (a container on the local provider, a lone server on any provider but Modal) refuses such an image. A build context (`utils/docker/context.py`) is what `docker build` would send, `.dockerignore` applied, written so equal files give equal bytes and an equal `source_digest`. | | `task/`, `task_step/` | A `Task` holds its `TaskStep`s inline; `Task.run()` executes them as a DAG. `depends_on` (None means all prior steps) gates a step, independent steps run concurrently, `fail_task_on_error` makes a failure fatal or tolerated, `retry_config` rolls a failed span back through the step journal and re-dispatches it. Built-in steps live in `task_step/task_steps/` (`deploy_env`, `deploy_agent`, `prompt_agent`, the verifiers under `verifiers/`, and more). `task/episode/` runs a task with some of its agents played from outside, as a training loop does (`EpisodeSpec`). | | `store/` | Four store ABCs with local and cloud implementations: `DocumentStore` (SQLite, MongoDB, DynamoDB, Firestore with MongoDB compatibility), `ObjectStore` (filesystem, S3, Cloud Storage), `ImageStore` (local OCI registry, ECR), `SecretStore` (env vars or file, AWS Secrets Manager, Google Cloud Secret Manager). `VersionedEntityStore` implements the shared versioned get/put logic, `QueryBuilder` is the immutable chained query API, `store/base.py` holds the error types. `store/routing.py` is the `@local` namespace: while the CLI runs, an `@local/…` id and what an `@local` task's run records go to local stores beside the configured ones, and that run's writes to the configured stores are refused. A new backend must pass the conformance kits in `tst/store/`. | | `config/` | The `Config` singleton (`get_config`, `configure`, `reset_config`) in `config/runtime.py`, file discovery in `config/loader.py`, and `load_impl`, which resolves `module:Class` pointers. `agent_env.store` re-exports the config names for compatibility. | diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 51f0d7ce..9a802f62 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -98,7 +98,8 @@ Each of them pins its base images by digest, and Dependabot bumps those pins, so image: a bump changes the sources, and the next release builds it again. Installs pull them instead of building the infra images: a run on local stores registers a kind's images when GHCR has every one for the sources installed, by the digest the wheel records or, with no record, the one GHCR serves, and -builds them otherwise; a sandbox that can't pull one builds it from its build context. The wheel and the release notes +builds them otherwise; a sandbox that can't pull one builds it from its build context. A run that deploys the Claude +Code agent registers its published image the same way, in place of its folder's build context. The wheel and the release notes record each image's digest. A failed image job keeps `agentenv-framework` off PyPI. Maintainers can publish the images alone by running the workflow by hand on `main`. GHCR keeps a new package private until a package admin makes it public, and the workflow warns about each one until then. diff --git a/src/agent_env/agents/README.md b/src/agent_env/agents/README.md index 81411c80..4b88aa0c 100644 --- a/src/agent_env/agents/README.md +++ b/src/agent_env/agents/README.md @@ -11,7 +11,9 @@ on every model call. The `LITELLM_*` variables stay in its environment too, so a endpoint. The image holds Claude Code, which is Anthropic's software under its [Commercial Terms](https://www.anthropic.com/legal/commercial-terms); a release also publishes the image, CLI included, at `ghcr.io/scaleapi/agentenv-claude-code:src-`, built from this folder with the protocol pinned as the wheel -ships it. The CLI's version is the Dockerfile's +ships it. A run that deploys the agent registers that image when it's published for the sources installed: each deploy +pulls it by its digest and builds it from this folder only when it can't, and a provider that runs images by name can +run it. The CLI's version is the Dockerfile's `CLAUDE_CODE_VERSION`. The image gives the scripts the model runs `node`, without npm or npx, and the Python packages `model-requirements.txt` pins. diff --git a/src/agent_env/bundle/authoring.py b/src/agent_env/bundle/authoring.py index cf1db51f..ed99eaeb 100644 --- a/src/agent_env/bundle/authoring.py +++ b/src/agent_env/bundle/authoring.py @@ -3,14 +3,18 @@ from __future__ import annotations import os +import shutil import stat +import tempfile import unicodedata -from collections.abc import Mapping, Sequence +from collections.abc import Iterator, Mapping, Sequence +from contextlib import contextmanager from dataclasses import dataclass from pathlib import Path from typing import Any, NoReturn, TypeVar, overload from agent_env.artifact.artifact import Artifact +from agent_env.artifact.artifacts.docker_image import DockerImageArtifact from agent_env.entity.refs import EntityKind, parse_toml_ref from agent_env.env.env import Env from agent_env.env.envs._deployment import provider_refusal @@ -255,6 +259,31 @@ def build_context_files(bundle: Bundle, entry: BundleEntry) -> dict[str, Path]: return dict(sorted(files.items())) +@dataclass(frozen=True) +class StagedContext: + """A copy of ``build_context_files`` for the image an entry's Dockerfile builds, so an image written from it was + made from what the ledger hashed.""" + + path: Path + dockerfile: str # its whole path in the copy: a bare name is looked up from the current folder first + + def identity(self, image_id: str) -> tuple[str, str]: + """The ``source_digest`` and ``image_name`` ``put_context`` gives ``image_id`` from this copy, built for each + sandbox's own platform, as a bundle writes its images.""" + return DockerImageArtifact.context_identity(image_id, str(self.path), self.dockerfile, platform=None) + + +@contextmanager +def staged_build_context(bundle: Bundle, entry: BundleEntry, dockerfile: str) -> Iterator[StagedContext]: + """``entry``'s ``StagedContext`` for its ``dockerfile``, a path in its folder, removed on leaving.""" + with tempfile.TemporaryDirectory(prefix="agent-env-build-") as staged: + context = Path(staged) + for key, path in build_context_files(bundle, entry).items(): + (context / key).parent.mkdir(parents=True, exist_ok=True) + shutil.copy2(path, context / key) + yield StagedContext(context, str(context / dockerfile)) + + def entry_file(bundle: Bundle, entry: BundleEntry) -> tuple[str, Path]: """The one file in an entry's folder, as ``(name, path)``. Raises BundleError otherwise.""" key, path = next(iter(one_file(bundle, entry).items())) diff --git a/src/agent_env/bundle/ledger.py b/src/agent_env/bundle/ledger.py index 68cddacc..a7405dd7 100644 --- a/src/agent_env/bundle/ledger.py +++ b/src/agent_env/bundle/ledger.py @@ -49,7 +49,9 @@ @dataclass(frozen=True) class Digest: value: str - inputs: dict[str, Any] # type, config, files {key: sha256}, needs and store_refs {" ": version} + # type, config, files {key: sha256}, needs and store_refs {" ": version}, and for an image published for + # its sources, published: "@" + inputs: dict[str, Any] @dataclass(frozen=True) @@ -124,6 +126,8 @@ def digest(self, write: Write, needs: Mapping[tuple[str, str], int]) -> Digest | listing = build_context_files if built else folder_files for key, path in listing(self._plan.bundle_of(write.source.entry), write.source.entry).items(): inputs["files"][key] = _file_sha256(path) + (_mode(path) if built else "") + if (published := self._plan.published.get(write.id)) is not None: + inputs["published"] = published.pinned if write.kind not in (BundleKind.TASK, BundleKind.EVAL): # A document records the versions of what it references (an env its images, a universe its # environments), so one written anew means it must be written again. A task or eval names its @@ -210,8 +214,9 @@ def _tracked(write: Write) -> bool: """Whether the ledger can list everything ``write`` is made from. Not yet for an env type with a ``from_toml`` of its own, which may read its folder in ways it can't see. An artifact, a skill included, is made from its toml, every file in its folder (``folder_files``) and what it references; an agent or env from its - toml and what it references. A built image is made from its build context, ``build_context_files``; what the - build fetches (its base image, packages) isn't an input. A referenced image is made from its ref's text, so a tag + toml and what it references. A built image is made from its build context, ``build_context_files``, and the + image published for it when there is one (``Plan.published``); what the build fetches (its base image, packages) + isn't an input. A referenced image is made from its ref's text, so a tag that moves in its registry isn't a change. An image folder's image is also made from its description.""" if isinstance(write.source, BundleImage): return True @@ -253,6 +258,9 @@ def _changes(row: Mapping[str, Any], digest: Digest) -> list[str]: reasons.append(f"type changed: {before['type']} → {after['type']}") if before.get("config") != after["config"]: reasons.append("config changed") + if before.get("published") != after.get("published"): + reasons.append(f"published as {after['published'].partition('@')[0]}" if after.get("published") + else "not found published") for label, names in zip(("files added", "files removed", "files changed"), _diff(before["files"], after["files"])): if names: shown = ", ".join(names[:_LISTED]) + (f" and {len(names) - _LISTED} more" if len(names) > _LISTED else "") diff --git a/src/agent_env/bundle/materialize.py b/src/agent_env/bundle/materialize.py index 14a94cb6..4abb3dd5 100644 --- a/src/agent_env/bundle/materialize.py +++ b/src/agent_env/bundle/materialize.py @@ -4,7 +4,9 @@ that can't be written whole. It needs the CLI's namespace routing, which sends ``@local`` writes to the ``@local`` namespace's store. Holding the bundle's lock, it writes the entities in the plan's order, an image built from an entry's Dockerfile just before the entry, as its build context alone, which each deploy builds where -it runs (a sandbox on this machine with this machine's Docker). Then it builds and preflights every task +it runs (a sandbox on this machine with this machine's Docker). An agent's image built from a shipped agent's sources +that agent-env publishes is registered as the published image instead, which each deploy pulls by its digest, with the +build context beside it for a sandbox that can't pull it. Then it builds and preflights every task before writing any of them, and writes the evals last, since they name the tasks. Each write goes through the ledger, so one whose inputs haven't changed reuses the version the bundle last wrote. A reused task keeps whatever its steps took from config when it was first written, such as a rubrics verifier's default judge @@ -15,12 +17,9 @@ from __future__ import annotations import copy -import shutil -import tempfile from collections.abc import Callable, Iterator from contextlib import contextmanager, nullcontext from dataclasses import dataclass -from pathlib import Path from typing import Any from agent_env.a2a_agent import A2AAgent @@ -39,7 +38,7 @@ from agent_env.utils.docker.build import DockerBuildError from ._fs import with_article -from .authoring import AuthoringContext, build_context_files +from .authoring import AuthoringContext, staged_build_context from .ledger import Ledger, locked_ids, materializing from .parse import BundleError, BundleKind from .plan import Plan, Write, artifact_writer, env_writer, unpinned_store_refs @@ -86,7 +85,8 @@ def materialize( holds a lock this one needs, ``on_build`` before an image is built here, and ``on_write`` after each write, reused or not. An image the bundle builds whose id is in ``build_here``, as a deploy on this machine runs it (``preflight_run``), is also built with this machine's Docker, before it's written or, reused, when Docker doesn't - hold it, so a build that fails writes nothing; its version is its build context either way. + hold it, so a build that fails writes nothing; its version is its build context either way. One published for its + sources (``Plan.published``) is never built here: a deploy pulls it. ``dry_run`` checks and preflights what a run would, and writes nothing: no entity, ledger row or lock. Each write gets the version it would reuse or the store's next, which another run can take first. A step @@ -143,44 +143,51 @@ def _write_entity(plan: Plan, write: Write, here: bool, on_build: Callable[[Writ if isinstance(write.source, ReferencedImage): return _write_referenced_image(plan, write) if isinstance(write.source, BuiltImage): + if write.id in plan.published: + return _write_published_image(plan, write) return _write_build_context(plan, write, here, on_build) return _WRITERS[write.kind](plan, write) -@contextmanager -def _staged(plan: Plan, image: BuiltImage) -> Iterator[Path]: - """A copy of the files the ledger hashes, ``build_context_files``, so an image the ledger reuses was made from - what it hashed.""" - with tempfile.TemporaryDirectory(prefix="agent-env-build-") as staged: - context = Path(staged) - for key, path in build_context_files(plan.bundle_of(image.entry), image.entry).items(): - (context / key).parent.mkdir(parents=True, exist_ok=True) - shutil.copy2(path, context / key) - yield context - - def _write_build_context(plan: Plan, write: Write, here: bool, on_build: Callable[[Write], None] | None) -> int: """Write the image an entry's Dockerfile describes as a docker_image artifact holding its build context alone, which each deploy builds where it runs, for its sandbox's platform; ``here``, built first with this machine's Docker, for the deploys here.""" image = write.source - with _staged(plan, image) as context: + with staged_build_context(plan.bundle_of(image.entry), image.entry, image.dockerfile) as staged: if here: - sources, name = DockerImageArtifact.context_identity(write.id, str(context), str(context / image.dockerfile), - platform=None) + sources, name = staged.identity(write.id) try: - build_staged(context, image.dockerfile, name, sources, _before(on_build, write)) + build_staged(staged.path, image.dockerfile, name, sources, _before(on_build, write)) except DockerBuildError as e: raise BundleError([f"{_path(plan, write)}: {_tail(str(e))}"]) from None try: return DockerImageArtifact.put_context( - write.id, description=_image_description(plan, write), context_path=str(context), - dockerfile_path=str(context / image.dockerfile), platform=None, + write.id, description=_image_description(plan, write), context_path=str(staged.path), + dockerfile_path=staged.dockerfile, platform=None, ).version except (ValueError, RuntimeError) as e: # a context it can't write, or the object store raise BundleError([f"{_path(plan, write)}: {e}"]) from None +def _write_published_image(plan: Plan, write: Write) -> int: + """Register the image published for the sources an entry's Dockerfile builds, by its digest, with its build context + beside it, which a sandbox that can't pull the image builds it from; nothing is built here. Refused when the + sources changed since the run planned it, so the image and its context are never of different sources.""" + image, published = write.source, plan.published[write.id] + with staged_build_context(plan.bundle_of(image.entry), image.entry, image.dockerfile) as staged: + try: + if staged.identity(write.id)[0] != published.sources: + raise ValueError(f"its files changed after the run found {published.ref} published for them; " + "run it again") + return DockerImageArtifact.put_ref( + write.id, description=_image_description(plan, write), image_name=published.ref, + image_digest=published.digest, context_path=str(staged.path), dockerfile_path=staged.dockerfile, + ).version + except (ValueError, RuntimeError) as e: # changed sources, a context it can't write, or the object store + raise BundleError([f"{_path(plan, write)}: {e}"]) from None + + def _build_reused(plan: Plan, write: Write, version: int, on_build: Callable[[Write], None] | None) -> None: """Build the image the bundle wrote earlier with this machine's Docker, when Docker doesn't hold it.""" image = DockerImageArtifact.get(write.id, version) diff --git a/src/agent_env/bundle/plan.py b/src/agent_env/bundle/plan.py index 045d3363..ec61f51b 100644 --- a/src/agent_env/bundle/plan.py +++ b/src/agent_env/bundle/plan.py @@ -24,6 +24,7 @@ from agent_env.a2a_agent import A2AAgent from agent_env.artifact import Artifact, CliArtifact, EnvironmentArtifact, SeedsEnvironments from agent_env.artifact.artifact import FolderChild +from agent_env.artifact.artifacts.docker_image import DockerImageArtifact from agent_env.artifact.registry import canonical_type, get_artifact_registry from agent_env.config import get_config from agent_env.entity.refs import EntityKind, default_agent_refs @@ -32,6 +33,7 @@ from agent_env.env.envs.mcp_server import MCPServerEnv from agent_env.env.envs.multi_env import MultiEnv from agent_env.env.envs.website import WebsiteEnv +from agent_env.env.published_images import published_agent_image from agent_env.env.registry import get_env_registry from agent_env.env.store import ENVS_COLLECTION from agent_env.providers.env_providers.constants import ( @@ -50,7 +52,7 @@ from agent_env.task_step.task_steps.load_artifact import LoadArtifactTaskStep from ._fs import fold, relative, with_article -from .authoring import AuthoringContext, entry_files +from .authoring import AuthoringContext, entry_files, staged_build_context from .parse import NAMED_BY, Bundle, BundleEntry, BundleError, BundleKind, parse_bundle from .resolve import BuiltImage, BundleImage, Reference, ResolvedBundle, ResolvedEntry, build_step, resolve_bundle @@ -78,6 +80,26 @@ _NOTHING_TO_RUN = "this bundle has no tasks or evals to run" +@dataclass(frozen=True) +class Publication: + """The image published for the sources a bundle builds an agent's image from, which the run registers by its tag + and digest, with those sources as its build context.""" + + ref: str # its repository and source tag + digest: str + sources: str # the source digest of the build context it was found for + + @property + def pinned(self) -> str: + return f"{self.ref}@{self.digest}" + + def unwritten(self, image_id: str) -> DockerImageArtifact: + """The image as the run will store it as ``image_id``, but for its build context, which only the write uploads: + what a deploy's providers are checked against before it's written.""" + return DockerImageArtifact(id=image_id, description="", image_name=self.ref, image_digest=self.digest, + source_digest=self.sources) + + @dataclass(frozen=True) class Write: """An entity or built image a run writes.""" @@ -97,6 +119,9 @@ class Plan: store_refs: tuple[Reference, ...] # the store ids the writes reference, checked to exist store_latest: dict[tuple[EntityKind, str], int] # the version each unpinned store ref named when planned origins: Mapping[BundleEntry, Bundle] = field(default_factory=dict) # each entry from another bundle, with it + # Each agent image the bundle builds from a shipped agent's sources, by id: the image published for them, which the + # run registers in place of its build context alone. + published: Mapping[str, Publication] = field(default_factory=dict) def bundle_of(self, entry: BundleEntry) -> Bundle: """The bundle ``entry`` comes from: the installed one for an entry pulled in from one, else the run's.""" @@ -295,8 +320,9 @@ def plan(self, tasks: Sequence[str], evals: Sequence[str]) -> Plan: self._check_loaded_environments(closure, store_latest) if self.problems: raise BundleError(self.problems) - return Plan(self.resolved, self._ordered(closure), tuple(selected_tasks), tuple(selected_evals), store_refs, - store_latest, dict(self.origins)) + writes = self._ordered(closure) + return Plan(self.resolved, writes, tuple(selected_tasks), tuple(selected_evals), store_refs, store_latest, + dict(self.origins), self._published(writes)) def check(self) -> None: self._check_writers() @@ -793,6 +819,23 @@ def _ordered(self, closure: set[_Key]) -> tuple[Write, ...]: heapq.heappush(ready, (self.rank[dependent], dependent)) return tuple(writes) + def _published(self, writes: Sequence[Write]) -> dict[str, Publication]: + """The published image of each agent image in ``writes`` built from a shipped agent's sources, by id. Only an + agent's are looked up, as agent-env publishes no other kind of bundle image.""" + published = {} + for write in writes: + image = write.source + if not (isinstance(image, BuiltImage) and image.entry.kind is BundleKind.AGENT): + continue + with staged_build_context(self._bundle_of(image.entry), image.entry, image.dockerfile) as staged: + try: + sources, _ = staged.identity(write.id) + except ValueError: # a context its write refuses, which says why + continue + if (found := published_agent_image(sources)) is not None: + published[write.id] = Publication(*found, sources) + return published + def _bundle_of(self, entry: BundleEntry) -> Bundle: return self.origins.get(entry, self.resolved.bundle) diff --git a/src/agent_env/bundle/preflight.py b/src/agent_env/bundle/preflight.py index a09741d7..6147bdc5 100644 --- a/src/agent_env/bundle/preflight.py +++ b/src/agent_env/bundle/preflight.py @@ -134,10 +134,13 @@ def __init__(self, plan: Plan, sandbox: str | None): self.written = {write.id for write in plan.writes} self.agents = {write.id: write.source for write in plan.writes if write.kind is BundleKind.AGENT} self.envs = {write.id: write.source for write in plan.writes if write.kind is BundleKind.ENV} - self.built = {write.id: write.source for write in plan.writes if isinstance(write.source, BuiltImage)} - # Each image a folder names in a registry, as the store will hold it, so it's checked as a stored one is. + self.built = {write.id: write.source for write in plan.writes + if isinstance(write.source, BuiltImage) and write.id not in plan.published} + # Each image a folder names in a registry, or the bundle builds from sources whose image is published, as the + # store will hold it, so it's checked as a stored one is. self.referenced = {write.id: DockerImageArtifact(id=write.id, description="", image_name=write.source.ref) for write in plan.writes if isinstance(write.source, ReferencedImage)} + self.referenced.update({id: found.unwritten(id) for id, found in plan.published.items()}) # Each deploy of an image the bundle builds: (where, what, its provider's links, the images it accepts on each) self.uses: dict[str, list[tuple[str, str, list[SandboxProvider], Callable[[SandboxProvider], Accepts]]]] = {} diff --git a/src/agent_env/bundle/run.py b/src/agent_env/bundle/run.py index 80b79da3..ca7eb676 100644 --- a/src/agent_env/bundle/run.py +++ b/src/agent_env/bundle/run.py @@ -168,6 +168,9 @@ def images(self) -> tuple[str, ...]: f"{_label(plan, done.write)} v{done.version}: " + ("pulled from its registry, its tag pinned to a digest when it's written" if isinstance(done.write.source, ReferencedImage) + else f"{plan.published[done.write.id].ref}, published for its sources, which each deploy pulls by its " + "digest, or builds from its build context when it can't" + if done.write.id in plan.published else "a build context, built with docker on this machine" if done.write.id in self.built_here else "a build context, which each deploy builds where it runs, for its sandbox's platform") diff --git a/src/agent_env/env/bootstrap.py b/src/agent_env/env/bootstrap.py index d220dac2..2938e739 100644 --- a/src/agent_env/env/bootstrap.py +++ b/src/agent_env/env/bootstrap.py @@ -108,7 +108,7 @@ def _image(label: str, artifact_id: str, description: str, dockerfile: Path, ima say(f"Registering the published {label} image {ref}...") return DockerImageArtifact.put_ref(artifact_id, description=description, image_name=ref, image_digest=published[dockerfile], context_path=str(dockerfile.parent), - dockerfile_path=dockerfile.name) + dockerfile_path=str(dockerfile)) def _build_and_put(label: str, artifact_id: str, description: str, dockerfile: Path, image_name: str, *, @@ -119,7 +119,7 @@ def _build_and_put(label: str, artifact_id: str, description: str, dockerfile: P build_image(dockerfile, dockerfile.parent, image_name, platform=platform) say(f"Creating {label} DockerImageArtifact...") artifact = DockerImageArtifact.put(id=artifact_id, description=description, image_name=image_name, - build_context_path=str(dockerfile.parent), dockerfile_path=dockerfile.name) + build_context_path=str(dockerfile.parent), dockerfile_path=str(dockerfile)) say(f"Created {label} DockerImageArtifact: id={artifact.id} version={artifact.version}") return artifact diff --git a/src/agent_env/env/published_images.py b/src/agent_env/env/published_images.py index f7b64e9f..8028bf83 100644 --- a/src/agent_env/env/published_images.py +++ b/src/agent_env/env/published_images.py @@ -1,6 +1,7 @@ -"""The images agent-env publishes for the infra envs it ships, for linux/amd64 and linux/arm64: one per Dockerfile, built -from the Dockerfile's folder. Each is tagged by its build context's content, so an install holding the same sources -finds the image built from them; a release wheel also records the digest its release published under each tag.""" +"""The images agent-env publishes for the infra envs and the agents it ships, for linux/amd64 and linux/arm64: one per +Dockerfile, built from the Dockerfile's folder. Each is tagged by its build context's content, so an install holding the +same sources finds the image built from them; a release wheel also records the digest its release published under each +tag.""" from __future__ import annotations @@ -60,6 +61,18 @@ def published_digest(image: PublishedImage) -> str | None: return release_digests(RELEASE_FILE).get(ref) or _registry_digest(ref) +def published_agent_image(sources: str) -> tuple[str, str] | None: + """The ref and digest of the image published for a shipped agent whose installed build context has the source + digest ``sources`` (``BuildContext.source_digest(None)``), or None when no shipped agent has those sources or its + image isn't published.""" + tag = f"src-{sources.removeprefix('sha256:')}" + for image in AGENT_IMAGES: + if image.tag() == tag: + digest = published_digest(image) + return (image.ref(), digest) if digest else None + return None + + def _registry_digest(ref: str) -> str | None: if ref not in _SERVED: try: diff --git a/src/agent_env/providers/sandbox_providers/AGENTS.md b/src/agent_env/providers/sandbox_providers/AGENTS.md index 0564336d..62e0ef81 100644 --- a/src/agent_env/providers/sandbox_providers/AGENTS.md +++ b/src/agent_env/providers/sandbox_providers/AGENTS.md @@ -55,7 +55,7 @@ with `exposed_ports=[port]` and returns the VM, and the deploy then loads and st | `get_sandbox(sandbox_id) -> Sandbox` | yes | every step after the one that created the sandbox, and teardown | | `terminate_sandbox(sandbox_id)` | no | teardown. The default calls `get_sandbox` then `terminate()`, and treats `SandboxNotFoundError` as done; override when you can terminate without a live handle | | `create_container(...)` | inherited | containers that run an env or agent image. Override only when the platform boots images natively | -| `prepare_image(image, *, attribution)` | no | before a sandbox is asked for an image, outside any creation deadline; build a context-only image here | +| `prepare_image(image, *, attribution)` | no | before a sandbox is asked for an image, outside any creation deadline; build a context-only image here, and pull one with an `image_digest` by it, building it from its context when it can't be pulled | | `from_config(**config)` | no | receives `[sandbox.providers..config]` with `secret:` references resolved. Override to fail early with a `ConfigError` that names the missing key (`boxd` also rejects unknown keys) | | `close()` | no | release SDK clients | diff --git a/src/agent_env/providers/sandbox_providers/host_images.py b/src/agent_env/providers/sandbox_providers/host_images.py index 6e198e7b..76b8cf22 100644 --- a/src/agent_env/providers/sandbox_providers/host_images.py +++ b/src/agent_env/providers/sandbox_providers/host_images.py @@ -3,7 +3,8 @@ An image that is only its build context is built where it runs. Here that is this machine's Docker: built natively, under the image's name, labeled with the sources it was built from, and built again only when those change, so a later deploy, a parallel one or another run finds it. A VM on this machine starts with no build cache, so it loads a saved -copy of that build, kept once per sources. Neither is a store version: the image is its build context. +copy of that build, kept once per sources. Neither is a store version: the image is its build context. An image a +registry publishes is pulled here by its digest, and built from its build context only when it can't be. """ from __future__ import annotations @@ -12,6 +13,7 @@ import gzip import hashlib import io +import json import logging import platform as host import subprocess @@ -41,9 +43,17 @@ def held(image: DockerImageArtifact) -> bool: return holds(image.image_name, sources(image)) -def holds(image_name: str, image_sources: str) -> bool: - """Whether this machine's Docker holds ``image_name`` built from ``image_sources``.""" - return _built_from(image_name) == image_sources +def holds(image_name: str, image_sources: str | None) -> bool: + """Whether this machine's Docker holds ``image_name`` built from ``image_sources``; never when there are none, as + for an image with no build context, which nothing here built.""" + return image_sources is not None and _built_from(image_name) == image_sources + + +def holds_digest(image: DockerImageArtifact) -> bool: + """Whether this machine's Docker holds ``image``, one with an ``image_digest``, under its name as the image that + digest names: pulled by it, or built here from the image's sources. A tag can name another image than the one a + version records, so holding its name isn't enough.""" + return image.image_digest in _repo_digests(image.image_name) or held(image) def present(image_name: str) -> bool: @@ -102,6 +112,29 @@ async def build_here(image: DockerImageArtifact) -> None: await asyncio.to_thread(build, image) +def pull(image: DockerImageArtifact) -> None: + """Have this machine's Docker hold ``image``, one with an ``image_digest``, under its name, unless it already holds + it (``holds_digest``): pulled by its digest, or, when that fails, built from its build context. Raises RuntimeError + when it can't be pulled or built.""" + with holding_locks([f"image-pull:{image.pinned_name}"]): + if holds_digest(image): + return + pulled = subprocess.run(["docker", "pull", image.pinned_name], capture_output=True, text=True, errors="replace") + if pulled.returncode == 0: + subprocess.run(["docker", "tag", image.pinned_name, image.image_name], capture_output=True, check=True) + return + why = (pulled.stderr.strip().splitlines() or [f"docker pull exited {pulled.returncode}"])[-1] + if not image.buildable: + raise RuntimeError(f"Couldn't pull {image.pinned_name}: {why}") + logger.warning("Couldn't pull %s, so it's built from its build context: %s", image.pinned_name, why) + build(image) + + +async def pull_here(image: DockerImageArtifact) -> None: + """``pull`` off the event loop.""" + await asyncio.to_thread(pull, image) + + async def saved_here(image: DockerImageArtifact) -> str: """The object url of ``image`` built here and saved as a tar.gz, which a VM on this machine loads; saved once per sources.""" @@ -119,6 +152,19 @@ def _built_from(image_name: str) -> str | None: return (result.stdout.strip() or None) if result.returncode == 0 else None +def _repo_digests(image_name: str) -> set[str]: + """The digests this machine's image ``image_name`` was pulled by, none when it has none or there's no such image.""" + try: + result = subprocess.run(["docker", "image", "inspect", "--format", "{{json .RepoDigests}}", image_name], + capture_output=True, text=True, errors="replace") + except FileNotFoundError: # no docker on PATH, so no image here + return set() + try: + return {ref.rpartition("@")[2] for ref in json.loads(result.stdout) or []} if result.returncode == 0 else set() + except ValueError: + return set() + + def _saved_here(image: DockerImageArtifact) -> str: build(image) objects = get_config().get_object_store() diff --git a/src/agent_env/providers/sandbox_providers/local_sandbox.py b/src/agent_env/providers/sandbox_providers/local_sandbox.py index f43bc0ed..7bcd208d 100644 --- a/src/agent_env/providers/sandbox_providers/local_sandbox.py +++ b/src/agent_env/providers/sandbox_providers/local_sandbox.py @@ -38,7 +38,7 @@ from agent_env import config from agent_env.attribution import Attribution from agent_env.config.paths import state_root -from agent_env.providers.sandbox_providers.host_images import build_here, built_here, present +from agent_env.providers.sandbox_providers.host_images import build_here, built_here, present, pull_here from agent_env.providers.sandbox_providers.sandbox import SANDBOX_LABEL, NetworkPolicy, VmSandbox from agent_env.providers.sandbox_providers.sandbox_provider import ( SANDBOX_MODE_CONTAINER, @@ -186,6 +186,8 @@ class LocalSandbox(VmSandbox): ON_THIS_MACHINE = True platform = None # this machine's Docker runs its own platform, and emulates others where it's set up + # The images its provider's prepare_image had this machine's Docker hold, which its containers start from as they are. + prepared: frozenset[str] = frozenset() def url_from_sandbox(self, url: str) -> str: return LocalSandboxProvider.get_external_url(url) @@ -223,11 +225,17 @@ async def build_images(self, artifacts: list) -> None: await build_here(artifact) async def pull_images(self, image_names: list[str]) -> None: - """Pull each image but one this machine built from a build context, which no registry holds.""" - built = await asyncio.gather(*(asyncio.to_thread(built_here, name) for name in image_names)) - if pulled := [name for name, here in zip(image_names, built) if not here]: + """Pull each image but one this machine built from a build context, which no registry holds, and one its Docker + holds that its provider prepared (``prepared``) or under a name pinned to a digest, so a tag isn't pulled again + over the image a digest names.""" + held = await asyncio.gather(*(asyncio.to_thread(self._held, name) for name in image_names)) + if pulled := [name for name, here in zip(image_names, held) if not here]: await super().pull_images(pulled) + def _held(self, image_name: str) -> bool: + return built_here(image_name) or ((image_name in self.prepared or "@sha256:" in image_name) + and present(image_name)) + async def pull_published(self, artifacts: list) -> list: """Pull each published image but one this machine's Docker already holds under its name, so a run after the first needs no registry.""" @@ -579,6 +587,12 @@ class LocalSandboxProvider(SandboxProvider): SANDBOX_ACCEPTS = Accepts.NAME_OR_CONTEXT CONTAINER_ACCEPTS = Accepts.NAME_OR_CONTEXT + @property + def _prepared(self) -> set[str]: + """The images prepare_image had this machine's Docker hold, by name; made on first use, so a subclass with an + ``__init__`` of its own needn't call this one's.""" + return self.__dict__.setdefault("_prepared_images", set()) + def url_from_sandbox(self, url: str) -> str: return self.get_external_url(url) @@ -604,6 +618,7 @@ async def create_vm( sandbox = LocalSandbox(port_map=port_map) (sandbox.work_dir / _PORT_MAP_FILE).write_text(json.dumps(sandbox._port_map)) sandbox.network_policy = self.effective_network_policy(network_policy) + sandbox.prepared = frozenset(self._prepared) return sandbox async def create_container( @@ -670,9 +685,14 @@ async def _start_container(self, sandbox: VmSandbox, *, image_name: str, port: i await start_trusting(sandbox, sandbox.container_name, trust_dir) async def prepare_image(self, image: Any, *, attribution: Optional[Attribution] = None) -> None: - """Build a context-only image on this machine, whose containers the local provider starts by name.""" + """Have this machine's Docker hold the image the local provider starts containers from by name: one that is only + a build context built here, and one with a digest pulled by it, or built from its build context when it can't + be.""" if image.context_only: await build_here(image) + elif image.image_digest: + await pull_here(image) + self._prepared.add(image.image_name) async def create_sandbox( self, diff --git a/tst/unit/bundle/_support.py b/tst/unit/bundle/_support.py index 91714349..8aaf3a6f 100644 --- a/tst/unit/bundle/_support.py +++ b/tst/unit/bundle/_support.py @@ -1,6 +1,7 @@ """What the bundle tests share: laying out and planning a folder, the @local namespace's store, and a configured store that fails any test reaching it.""" +import shutil from pathlib import Path from agent_env.artifact.artifacts.docker_image import DockerImageArtifact @@ -8,6 +9,7 @@ from agent_env.bundle.plan import plan_bundle from agent_env.bundle.resolve import BuiltImage, resolve_bundle from agent_env.config import get_config +from agent_env.env.published_images import AGENT_IMAGES from agent_env.store.document_store import DocumentStore # The digest the registry fixture pins every tag to. @@ -30,6 +32,14 @@ def context_image(root, id, files=None, dockerfile="Dockerfile", folder=None): return DockerImageArtifact.put_context(id, description=id, context_path=str(context), dockerfile_path=dockerfile) +def shipped_agent(root, name="solver"): + """A copy of the Claude Code agent agent-env ships, as ``root``'s agent ``name``: the sources its image is published + for.""" + shutil.copytree(AGENT_IMAGES[0].dockerfile.parent, root / "agents" / name, + ignore=shutil.ignore_patterns("__pycache__")) + return root / "agents" / name + + def listing(folder): """The files under ``folder``, as POSIX paths relative to it, a link marked with a trailing ``@``.""" return sorted(path.relative_to(folder).as_posix() + ("@" if path.is_symlink() else "") diff --git a/tst/unit/bundle/conftest.py b/tst/unit/bundle/conftest.py index b1514c98..6ae819ec 100644 --- a/tst/unit/bundle/conftest.py +++ b/tst/unit/bundle/conftest.py @@ -1,17 +1,19 @@ """Fixtures the bundle tests share.""" +import shutil + import pytest from agent_env.artifact.artifacts import docker_image as docker_image_module -from agent_env.bundle import materialize as materialize_module +from agent_env.env.published_images import AGENT_IMAGES from agent_env.providers.sandbox_providers import host_images from tst.unit.bundle._support import DIGEST, listing @pytest.fixture def docker_on_path(monkeypatch): - monkeypatch.setattr(materialize_module.shutil, "which", lambda name: f"/usr/bin/{name}") + monkeypatch.setattr(shutil, "which", lambda name: f"/usr/bin/{name}") @pytest.fixture @@ -41,3 +43,11 @@ def pin(ref, auth): monkeypatch.setattr(docker_image_module, "pin_digest", pin) return asked + + +@pytest.fixture +def agents_published(monkeypatch): + """The registry serves each shipped agent's image, under DIGEST, and nothing else.""" + refs = {image.ref() for image in AGENT_IMAGES} + monkeypatch.setattr("agent_env.env.published_images._registry_digest", lambda ref: DIGEST if ref in refs else None) + return AGENT_IMAGES[0].ref() diff --git a/tst/unit/bundle/materialize_test.py b/tst/unit/bundle/materialize_test.py index 102c0414..886a3c3f 100644 --- a/tst/unit/bundle/materialize_test.py +++ b/tst/unit/bundle/materialize_test.py @@ -1,6 +1,7 @@ """Materializing a planned bundle: what gets written, what is reused, and what is refused before any write.""" import json +import shutil import subprocess import sys import textwrap @@ -36,7 +37,8 @@ from agent_env.task import Task from agent_env.task_step.task_step import TaskStep from agent_env.utils.docker.context import BuildContext -from tst.unit.bundle._support import DIGEST, RefusingStore, layout, local_builds, local_store, plan_of +from agent_env.env.published_images import AGENT_IMAGES +from tst.unit.bundle._support import DIGEST, RefusingStore, layout, local_builds, local_store, plan_of, shipped_agent ROOT = "@local/~/triage" LAYOUT = { @@ -464,6 +466,69 @@ def test_an_agent_folder_with_a_dockerfile_is_built_and_the_agent_written_over_i assert _summary(again)[image] == (1, True, ()) and _summary(again)[f"{ROOT}/solver"] == (1, True, ()) +def test_an_agent_image_from_a_shipped_agents_sources_is_registered_as_its_published_image_and_not_built( + bundle_dir, builds, agents_published): + shipped_agent(bundle_dir) + _steps(bundle_dir, [{"id": "agent", "type": "deploy_agent", "env_ids": [], "a2a_agent_id": "solver"}]) + image = f"{ROOT}/solver__agent_image" + plan = plan_of(bundle_dir) + + materialize(plan, build_here=local_builds(plan)) + + written = DockerImageArtifact.get(image, 1) + assert [(found.ref, found.digest) for found in plan.published.values()] == [(agents_published, DIGEST)] + assert (written.image_name, written.image_digest, written.pinned_name) == ( + agents_published, DIGEST, f"{agents_published}@{DIGEST}") + assert written.source_digest == f"sha256:{AGENT_IMAGES[0].tag().removeprefix('src-')}" + assert written.buildable and not written.context_only and written.tar_gz_object_url is None + assert builds == [] + assert A2AAgent.get(f"{ROOT}/solver").docker_image_artifact.image_digest == DIGEST + + +def test_an_agent_image_whose_sources_change_after_planning_isnt_registered_as_the_published_one( + bundle_dir, builds, agents_published): + """Its image and its build context would be of different sources.""" + folder = shipped_agent(bundle_dir) + _steps(bundle_dir, [{"id": "agent", "type": "deploy_agent", "env_ids": [], "a2a_agent_id": "solver"}]) + plan = plan_of(bundle_dir) + (folder / "kit.py").write_text("# changed after the run planned it\n") + + with pytest.raises(BundleError) as e: + materialize(plan, build_here=local_builds(plan)) + + assert list(e.value.problems) == [f"agents/solver: its files changed after the run found {agents_published} " + "published for them; run it again"] + with pytest.raises(NotFoundError): + DockerImageArtifact.get(f"{ROOT}/solver__agent_image") + + +def test_an_agent_image_from_sources_no_shipped_agent_has_is_its_build_context(bundle_dir, builds, agents_published): + (shipped_agent(bundle_dir) / "notes.md").write_text("mine\n") + _steps(bundle_dir, [{"id": "agent", "type": "deploy_agent", "env_ids": [], "a2a_agent_id": "solver"}]) + plan = plan_of(bundle_dir) + + _run(bundle_dir) + + assert plan.published == {} + assert DockerImageArtifact.get(f"{ROOT}/solver__agent_image", 1).context_only + + +def test_an_agent_image_written_before_its_image_was_published_is_written_again_as_the_published_one( + bundle_dir, builds, monkeypatch): + shipped_agent(bundle_dir) + _steps(bundle_dir, [{"id": "agent", "type": "deploy_agent", "env_ids": [], "a2a_agent_id": "solver"}]) + image, agent, ref = f"{ROOT}/solver__agent_image", f"{ROOT}/solver", AGENT_IMAGES[0].ref() + _run(bundle_dir) + monkeypatch.setattr("agent_env.env.published_images._registry_digest", lambda asked: DIGEST if asked == ref else None) + + published, again = _summary(_run(bundle_dir)), _summary(_run(bundle_dir)) + + assert published[image] == (2, False, (f"published as {ref}",)) and published[agent][:2] == (2, False) + assert again[image] == (2, True, ()) and again[agent] == (2, True, ()) + assert DockerImageArtifact.get(image, 2).image_digest == DIGEST + assert len(builds) == 1 # only the first run's, of its build context + + def test_an_image_is_built_from_the_files_the_ledger_hashes_so_a_reused_one_matches_them(bundle_dir, builds): layout(bundle_dir, { "agents/solver/Dockerfile": "FROM scratch\nCOPY . /agent\n", @@ -502,7 +567,7 @@ def test_a_changed_build_context_rebuilds_the_image_and_rewrites_its_agent(bundl def test_an_image_written_as_its_build_context_is_uploaded_with_no_build_and_no_docker(bundle_dir, builds, monkeypatch): - monkeypatch.setattr(materialize_module.shutil, "which", lambda name: None) + monkeypatch.setattr(shutil, "which", lambda name: None) layout(bundle_dir, {"agents/solver/Dockerfile": "FROM scratch\nCOPY run.sh /\n", "agents/solver/run.sh": "echo hi\n"}) _steps(bundle_dir, [{"id": "agent", "type": "deploy_agent", "env_ids": [], "a2a_agent_id": "solver"}]) image = f"{ROOT}/solver__agent_image" @@ -617,7 +682,7 @@ def test_an_image_the_ledger_reuses_needs_no_docker(bundle_dir, builds, monkeypa layout(bundle_dir, {"agents/solver/Dockerfile": "FROM scratch\n"}) _steps(bundle_dir, [{"id": "agent", "type": "deploy_agent", "env_ids": [], "a2a_agent_id": "solver"}]) _run(bundle_dir) - monkeypatch.setattr(materialize_module.shutil, "which", lambda name: None) + monkeypatch.setattr(shutil, "which", lambda name: None) assert _summary(_run(bundle_dir, dry_run))[f"{ROOT}/solver__agent_image"] == (1, True, ()) assert len(builds) == 1 diff --git a/tst/unit/bundle/preflight_test.py b/tst/unit/bundle/preflight_test.py index 2375b417..9053bd7a 100644 --- a/tst/unit/bundle/preflight_test.py +++ b/tst/unit/bundle/preflight_test.py @@ -37,7 +37,7 @@ from agent_env.task_step.task_steps.deploy_env import DeployEnvTaskStep from agent_env.task_step.task_steps.deploy_sandbox import DeploySandboxTaskStep from agent_env.task_step.task_steps.verifiers.rubrics_verifier import RubricsVerifierTaskStep -from tst.unit.bundle._support import layout, local_store +from tst.unit.bundle._support import layout, local_store, shipped_agent LOCAL = ("localhost:5000/local/img:v1", "file:///state/img-v1.tar.gz") REMOTE = ("123456789012.dkr.ecr.us-west-2.amazonaws.com/img:v1", "s3://bucket/img-v1.tar.gz") @@ -291,6 +291,22 @@ def test_an_agent_the_bundle_builds_is_refused_on_a_provider_that_creates_no_vm_ ] +@pytest.mark.parametrize("sandbox", ["containers", "local"]) +def test_an_agent_built_from_a_shipped_agents_published_sources_is_pulled_wherever_it_deploys(bundle_dir, monkeypatch, + agents_published, sandbox): + """Even on a provider that runs it by name, and on this machine, where it isn't built first.""" + registry = Config.sandbox_registry + monkeypatch.setattr(Config, "sandbox_registry", lambda self: {**registry(self), "containers": {"impl": _ContainersOnly}}) + shipped_agent(bundle_dir) + _task(bundle_dir, [AGENT]) + + dry = dry_run_bundle(bundle_dir, sandbox=sandbox) + + assert dry.built_here == frozenset() + assert dry.images() == (f"agents/solver (Dockerfile image) v1: {agents_published}, published for its sources, which each deploy " + "pulls by its digest, or builds from its build context when it can't",) + + def test_a_gateway_is_refused_on_a_provider_that_creates_no_vm_and_has_no_private_network(bundle_dir, monkeypatch): registry = Config.sandbox_registry monkeypatch.setattr(Config, "sandbox_registry", lambda self: {**registry(self), "containers": {"impl": _ContainersOnly}}) diff --git a/tst/unit/env/bootstrap_test.py b/tst/unit/env/bootstrap_test.py index 3bb93bcd..134b05c7 100644 --- a/tst/unit/env/bootstrap_test.py +++ b/tst/unit/env/bootstrap_test.py @@ -43,15 +43,14 @@ def write(env_id, *, platform, metadata=None, published=None): return calls -@pytest.mark.parametrize("kind, put, contexts", [ - (GATEWAY, bootstrap.put_gateway_env, [(bootstrap.GATEWAY_CONTEXT, "Dockerfile")]), - (SERVICE_DB, bootstrap.put_service_db_env, [(bootstrap.SERVICE_DB_DOCKERFILE.parent, name) - for name in ("Dockerfile", "Dockerfile.db-web", "Dockerfile.db-mcp")]), - (WEBSITE_BROWSER, bootstrap.put_website_browser_env, [(bootstrap.WEBSITE_BROWSER_CONTEXT, "Dockerfile")]), +@pytest.mark.parametrize("kind, put", [ + (GATEWAY, bootstrap.put_gateway_env), (SERVICE_DB, bootstrap.put_service_db_env), + (WEBSITE_BROWSER, bootstrap.put_website_browser_env), ], ids=[GATEWAY, SERVICE_DB, WEBSITE_BROWSER]) -def test_each_infra_image_keeps_the_build_context_it_is_built_from(local_stores, monkeypatch, kind, put, contexts): +def test_each_infra_image_keeps_the_build_context_it_is_built_from(local_stores, monkeypatch, kind, put): """So a VM or Modal running another platform than this machine builds it again from that; and those contexts are - what its recorded build inputs cover.""" + what its recorded build inputs cover. Each Dockerfile is named by its whole path: a bare name is looked up from + the current folder first.""" kept = [] def keep(id, *, description, image_name, build_context_path=None, dockerfile_path=None, platform=None): @@ -64,8 +63,7 @@ def keep(id, *, description, image_name, build_context_path=None, dockerfile_pat put("x", platform=None) - assert kept == contexts - assert [folder / name for folder, name in kept] == bootstrap._DOCKERFILES[kind] + assert kept == [(dockerfile.parent, str(dockerfile)) for dockerfile in bootstrap._DOCKERFILES[kind]] @pytest.fixture @@ -260,6 +258,19 @@ def test_a_published_image_is_registered_by_its_tag_and_digest_with_the_context_ assert bootstrap._built_from_stock(env, GATEWAY) +def test_a_published_image_is_registered_from_its_own_dockerfile_whatever_folder_the_run_is_in(local_stores, tmp_path, + monkeypatch): + """A run started from a folder holding a Dockerfile of its own registers the gateway's, with its sources.""" + (tmp_path / "Dockerfile").write_text("FROM scratch\n") + monkeypatch.chdir(tmp_path) + + image = bootstrap.put_gateway_env("default", platform=None, + published={bootstrap.GATEWAY_DOCKERFILE: _DIGEST}).docker_image_artifact + + tag = bootstrap._PUBLISHED[bootstrap.GATEWAY_DOCKERFILE].tag() + assert (image.dockerfile_path, image.source_digest) == ("Dockerfile", f"sha256:{tag.removeprefix('src-')}") + + def test_the_default_stores_are_local(local_stores): assert bootstrap.stores_are_local() diff --git a/tst/unit/env/published_images_test.py b/tst/unit/env/published_images_test.py index 3c9aa539..397efc60 100644 --- a/tst/unit/env/published_images_test.py +++ b/tst/unit/env/published_images_test.py @@ -1,5 +1,5 @@ -"""The images agent-env publishes for its infra envs: one per shipped Dockerfile, tagged by its sources, and the record -of their digests a release wheel carries.""" +"""The images agent-env publishes for its infra envs and its agents: one per shipped Dockerfile, tagged by its sources, +and the record of their digests a release wheel carries.""" import hashlib import json @@ -9,9 +9,11 @@ from agent_env.env import bootstrap, published_images from agent_env.env.published_images import ( + AGENT_IMAGES, REGISTRY, PublishedImage, _registry_digest, + published_agent_image, published_digest, release_digests, write_release_digests, @@ -119,3 +121,20 @@ def pin(ref, auth, *, timeout): assert [published_digest(image) for image in (served, missing, served, missing)] == [_DIGEST, None, _DIGEST, None] assert asked == [(served.ref(), None, 10), (missing.ref(), None, 10), (missing.ref(), None, 10)] + + +def test_a_shipped_agents_sources_find_its_published_image_and_other_sources_none(monkeypatch): + claude_code = AGENT_IMAGES[0] + sources = BuildContext.of(claude_code.dockerfile.parent, claude_code.dockerfile).source_digest(None) + monkeypatch.setattr(published_images, "_registry_digest", lambda ref: _DIGEST if ref == claude_code.ref() else None) + + assert published_agent_image(sources) == (claude_code.ref(), _DIGEST) + assert published_agent_image("sha256:" + "0" * 64) is None + + +def test_a_shipped_agents_sources_whose_image_isnt_published_find_none(monkeypatch): + claude_code = AGENT_IMAGES[0] + sources = BuildContext.of(claude_code.dockerfile.parent, claude_code.dockerfile).source_digest(None) + + assert published_agent_image(sources) is None + diff --git a/tst/unit/providers/sandbox_providers/host_images_test.py b/tst/unit/providers/sandbox_providers/host_images_test.py index 39f91348..eb82caed 100644 --- a/tst/unit/providers/sandbox_providers/host_images_test.py +++ b/tst/unit/providers/sandbox_providers/host_images_test.py @@ -1,8 +1,10 @@ """Images built from their build context on this machine: natively, once per sources, labeled with them, and saved once -for a VM here to load.""" +for a VM here to load; and published ones pulled here by their digest.""" import gzip import io +import logging +import subprocess import threading import time @@ -10,7 +12,8 @@ from agent_env.artifact.artifacts.docker_image import DockerImageArtifact from agent_env.providers.sandbox_providers import host_images -from agent_env.providers.sandbox_providers.local_sandbox import LocalSandbox +from agent_env.providers.sandbox_providers import local_sandbox +from agent_env.providers.sandbox_providers.local_sandbox import LocalSandbox, LocalSandboxProvider from agent_env.providers.sandbox_providers.sandbox import VmSandbox from agent_env.utils.docker.build import DockerBuildError @@ -132,6 +135,26 @@ async def pull(self, names): assert pulled == ["ghcr.io/acme/tool:1"] +@pytest.mark.asyncio +async def test_a_local_sandbox_doesnt_pull_a_tag_again_over_the_image_its_digest_named(monkeypatch, docker): + """One its provider prepared, or held under a name pinned to a digest; another held tag is pulled again, as is one + its provider prepared that Docker no longer holds.""" + pinned, gone = f"ghcr.io/acme/tool:1@{_PUBLISHED_DIGEST}", "ghcr.io/acme/gone:1" + held = {_PUBLISHED_REF, pinned, "ghcr.io/acme/tool:1"} + monkeypatch.setattr(local_sandbox, "present", lambda name: name in held) + sandbox = LocalSandbox() + sandbox.prepared = frozenset({_PUBLISHED_REF, gone}) + pulled = [] + + async def pull(self, names): + pulled.extend(names) + + monkeypatch.setattr(VmSandbox, "pull_images", pull) + await sandbox.pull_images([_PUBLISHED_REF, pinned, "ghcr.io/acme/tool:1", gone]) + + assert pulled == ["ghcr.io/acme/tool:1", gone] + + @pytest.mark.asyncio async def test_a_local_sandbox_builds_a_context_with_this_machines_docker(local_stores, context, docker): builds, _ = docker @@ -181,3 +204,127 @@ async def build_in_vm(self, artifacts): assert builds == [] and built_in_vm == [image.image_name] + +_PUBLISHED_REF = "ghcr.io/scaleapi/agentenv-claude-code:src-" + "b" * 64 +_PUBLISHED_DIGEST = "sha256:" + "a" * 64 +_PINNED = f"{_PUBLISHED_REF}@{_PUBLISHED_DIGEST}" + + +def _published(context): + return DockerImageArtifact.put_ref("solver__agent_image", description="d", image_name=_PUBLISHED_REF, + image_digest=_PUBLISHED_DIGEST, context_path=str(context), + dockerfile_path=str(context / "Dockerfile")) + + +def _docker_cli(monkeypatch, *, pulled_by=None, pull_fails=False): + """The docker commands ``pull`` runs, each recorded; ``pulled_by`` maps each image Docker holds to the digests it was + pulled by, and a pull exits 1 when ``pull_fails``.""" + ran = [] + + def run(argv, **kwargs): + ran.append(argv[1:]) + failed = pull_fails and argv[1] == "pull" + return subprocess.CompletedProcess(argv, int(failed), "", "Error response from daemon: denied" if failed else "") + + monkeypatch.setattr(host_images, "_repo_digests", lambda name: (pulled_by or {}).get(name, set())) + monkeypatch.setattr(host_images.subprocess, "run", run) + return ran + + +def test_a_published_image_is_pulled_by_its_digest_and_tagged_with_its_name(local_stores, context, docker, + monkeypatch): + ran = _docker_cli(monkeypatch) + + host_images.pull(_published(context)) + + assert ran == [["pull", _PINNED], ["tag", _PINNED, _PUBLISHED_REF]] + assert docker[0] == [] + + +def test_a_published_image_docker_holds_by_its_digest_isnt_pulled_again(local_stores, context, docker, monkeypatch): + ran = _docker_cli(monkeypatch, pulled_by={_PUBLISHED_REF: {_PUBLISHED_DIGEST}}) + + host_images.pull(_published(context)) + + assert ran == [] and docker[0] == [] + + +def test_a_published_image_built_here_from_its_sources_isnt_pulled(local_stores, context, docker, monkeypatch): + ran = _docker_cli(monkeypatch) + image = _published(context) + docker[1][_PUBLISHED_REF] = image.source_digest + + host_images.pull(image) + + assert ran == [] and docker[0] == [] + + +def test_a_tag_docker_holds_as_another_image_is_pulled_by_the_digest_and_moved_to_it(local_stores, context, docker, + monkeypatch): + """Two versions can share a tag that names other images over time: the one a version records is the one run.""" + ran = _docker_cli(monkeypatch, pulled_by={_PUBLISHED_REF: {"sha256:" + "d" * 64}}) + + host_images.pull(_published(context)) + + assert ran == [["pull", _PINNED], ["tag", _PINNED, _PUBLISHED_REF]] + + +@pytest.mark.parametrize("printed, code, digests", [ + ('["ghcr.io/acme/tool@sha256:aa","mirror.gcr.io/acme/tool@sha256:bb"]', 0, {"sha256:aa", "sha256:bb"}), + ("null", 0, set()), + ("", 1, set()), +], ids=["pulled", "built", "absent"]) +def test_the_digests_an_image_was_pulled_by_are_read_from_docker(monkeypatch, printed, code, digests): + monkeypatch.setattr(host_images.subprocess, "run", + lambda argv, **kwargs: subprocess.CompletedProcess(argv, code, printed, "")) + + assert host_images._repo_digests("ghcr.io/acme/tool:1") == digests + + +def test_a_published_image_that_cant_be_pulled_is_built_from_its_build_context(local_stores, context, docker, + monkeypatch, caplog): + ran = _docker_cli(monkeypatch, pull_fails=True) + + with caplog.at_level(logging.WARNING): + host_images.pull(_published(context)) + + assert ran == [["pull", _PINNED]] + assert [tag for _, _, tag, _, _ in docker[0]] == [_PUBLISHED_REF] + assert f"Couldn't pull {_PINNED}, so it's built from its build context: Error response from daemon: denied" in ( + caplog.text) + + +def test_a_published_image_with_no_build_context_that_cant_be_pulled_says_why(monkeypatch): + _docker_cli(monkeypatch, pull_fails=True) + image = DockerImageArtifact(id="x", version=1, description="d", image_name=_PUBLISHED_REF, + image_digest=_PUBLISHED_DIGEST) + + with pytest.raises(RuntimeError, match=f"Couldn't pull {_PINNED}: Error response from daemon: denied"): + host_images.pull(image) + + +@pytest.mark.asyncio +async def test_the_local_provider_pulls_a_published_image_and_builds_one_that_is_only_its_context(local_stores, context, + monkeypatch): + """It starts containers from an image by name, so this machine's Docker holds the image first.""" + pulled, built = [], [] + + async def pull_here(image): + pulled.append(image.image_name) + + async def build_here(image): + built.append(image.image_name) + + monkeypatch.setattr(local_sandbox, "pull_here", pull_here) + monkeypatch.setattr(local_sandbox, "build_here", build_here) + provider, context_only = LocalSandboxProvider(), _put(context) + + for image in (_published(context), context_only, + DockerImageArtifact(id="t", version=1, description="d", image_name="img:v1", + tar_gz_object_url="s3://bucket/img.tar.gz")): + await provider.prepare_image(image) + + assert (pulled, built) == ([_PUBLISHED_REF], [context_only.image_name]) + assert (await provider.create_vm()).prepared == {_PUBLISHED_REF} # its containers start from it as it is + assert LocalSandboxProvider()._prepared == set() +