[burtenshaw]

Summary

Follow-up to #1591.

harbor-semantic-v1 refused every Harbor task that declared environment.docker_image, which excluded tasks that ship only a prebuilt image, including some Terminal Bench tasks. The #1591 plan doc listed digest-pinned image support as the follow-up. This PR implements it.

  [environment]
- docker_image = "org/image:tag"                  # runtime.startup SKIP, as before
+ docker_image = "org/image@sha256:<64 hex>"      # now runs

• A task whose docker_image is pinned by digest runs, with or without environment/Dockerfile. Harbor prefers a declared image over a Dockerfile, so the do-nothing trial pulls the pin. The oracle, replay and canary trials run the image it retained, as before.
• runtime.startup.measured and harbor-evidence.json record the pin as source_image.
• A docker_image without a digest stays a named runtime.startup SKIP. A tag can move between validation and training, so a tag-only report would not bind to the image that later runs.
• Task-authored Compose files stay unsupported.

Evidence

• tests/test_validation: 1970 passed, 66 skipped (Python 3.12, Harbor 0.25.0). Two tests cover the change: an unpinned tag is refused before any trial, and a pinned image with no Dockerfile passes with the pin in the report.
• Real Docker run: openenv validate <task> --level semantic --policy v2 on a task with no Dockerfile and docker_image = "debian@sha256:734a72b3…". runtime.startup, runtime.reward_well_formed, runtime.episode_determinism, semantic.oracle_max and semantic.floor_gap all PASS, with verified cleanup.
• ruff format, ruff check and usort check are clean on the changed files.

Limits

• The validator does not resolve tags. Tasks that declare a tag need an operator overlay that pins the digest and records the original bytes in original_source_digest. The plan doc now states this as the remaining follow-up.
• The pulled image stays in the local Docker cache. The validator only removes resources it created, and it cannot tell whether the image was already present.
• examples: discover, validate and evaluate pinned Hub environments #1593's pipeline docs still say environment.docker_image is unsupported. They need a matching update when that PR rebases over this one.

Merge Danger

Door: two-way

Blast Radius: validation core, Harbor binding only. Dockerfile tasks take the same path as before.

🤖 Generated with Claude Code

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: Missing environment dir breaks pinned images
  • Compose now falls back to the existing task directory for image-only packages, with coverage for a fully absent environment directory.

Preview (e16cb8e030)

diff --git a/docs/plans/rfc008-validation-completion.md b/docs/plans/rfc008-validation-completion.md
--- a/docs/plans/rfc008-validation-completion.md
+++ b/docs/plans/rfc008-validation-completion.md
@@ -123,15 +123,18 @@
 profile and an operator-supplied `validation/openenv.json` contract. Fresh Docker
 trials run a do-nothing agent, the oracle, an oracle replay and declared canaries,
 all pinned to one immutable image. The binding requires Python 3.12 or newer and
-`harbor==0.25.0` alongside `openenv[harbor]`. It refuses task-declared `docker_image`
-values, so the NOP trial builds from the task's Dockerfile. The validator keeps
+`harbor==0.25.0` alongside `openenv[harbor]`. The NOP trial builds from the task's
+Dockerfile, or pulls a task-declared `docker_image` that is pinned by digest
+(`name@sha256:<digest>`). It refuses a `docker_image` without a digest, because a
+tag can move between validation and training. The validator keeps
 `openenv-validation:<run id>` only when the run has no FAIL or ERROR and was not
 interrupted. It records the tag in `runtime.startup.measured` as
 `retained_image_tag`. Otherwise it removes the tag.
 
-**Follow-up.** Tasks that ship only a prebuilt image cannot run under
-`harbor-semantic-v1`. Some Terminal Bench tasks are affected. Supporting them needs
-a reviewable Dockerfile path or digest-pinned image support.
+**Follow-up.** Tasks that declare a prebuilt image by tag still cannot run under
+`harbor-semantic-v1`. Some Terminal Bench tasks are affected. An operator can
+resolve the tag to its digest in a validation overlay and record the original
+bytes in `original_source_digest`. The validator does not resolve tags itself.
 
 **Remaining.** Decide which runtime ids apply to one-shot tasks. Containment,
 network policy, resource bounds and oracle containment could apply. Observation

diff --git a/docs/source/reference/cli.md b/docs/source/reference/cli.md
--- a/docs/source/reference/cli.md
+++ b/docs/source/reference/cli.md
@@ -115,10 +115,15 @@
 passing task still reports WARN. This validates task packages, not the outer
 `harbor_env` server.
 
-Supported tasks use Linux, a single step, one Dockerfile, a shared verifier and
-public networking. Unsupported shapes include multi-step or non-Linux tasks,
-task-declared `environment.docker_image`, missing Dockerfiles, task-authored Compose
-files and separate verifier environments. GPU/TPU or external MCP requirements,
+Supported tasks use Linux, a single step, a shared verifier and public networking.
+The image comes from one Dockerfile or from a digest-pinned
+`environment.docker_image`, such as `org/image@sha256:<64 hex digits>`. Harbor
+prefers a declared image over a Dockerfile, so the do-nothing trial pulls the pin
+and `runtime.startup` records it as `source_image`. The validator leaves the pulled
+image in the local Docker cache. Unsupported shapes include multi-step or non-Linux
+tasks, an `environment.docker_image` without a digest, tasks with neither a
+Dockerfile nor a pinned image, task-authored Compose files and separate verifier
+environments. GPU/TPU or external MCP requirements,
 credentials, non-public or differing phase networks, task environment variables,
 healthchecks, skills, verifier collection and artifact extensions are also
 unsupported. A supplied provider, including `--space`, is unsupported. Each case

diff --git a/rfcs/008-environment-auto-validation.md b/rfcs/008-environment-auto-validation.md
--- a/rfcs/008-environment-auto-validation.md
+++ b/rfcs/008-environment-auto-validation.md
@@ -748,10 +748,12 @@
 `openenv[harbor]`. A missing or mismatched Harbor installation is a validator
 environment error. It exits 3 and names `harbor==0.25.0`.
 
-The profile supports single-step Linux tasks with one Dockerfile, one main Docker
-service, a shared verifier and public networking. Unsupported shapes include
-multi-step or non-Linux tasks, task-declared `environment.docker_image`, missing
-Dockerfiles, task-authored Compose files and separate verifier environments.
+The profile supports single-step Linux tasks with one main Docker service, a shared
+verifier and public networking. The image comes from one Dockerfile or from a
+digest-pinned `environment.docker_image` (`name@sha256:<digest>`), which the report
+records as `source_image`. Unsupported shapes include multi-step or non-Linux tasks,
+an `environment.docker_image` without a digest, tasks with neither a Dockerfile nor
+a pinned image, task-authored Compose files and separate verifier environments.
 GPU/TPU or external MCP requirements, credentials, non-public or differing phase
 networks, task environment variables, healthchecks, skills, verifier collection and
 artifact extensions are also unsupported. A supplied provider, including `--space`,

diff --git a/src/openenv/validation/harbor.py b/src/openenv/validation/harbor.py
--- a/src/openenv/validation/harbor.py
+++ b/src/openenv/validation/harbor.py
@@ -1,7 +1,8 @@
 """Native Harbor execution: fresh Docker trials graded through RFC 008 check ids.
 
-No served-protocol evidence is synthesized. A do-nothing trial builds the image, and
-the oracle, its replay and any canaries run pinned to that image.
+No served-protocol evidence is synthesized. A do-nothing trial builds the image, or
+pulls the task's digest-pinned one, and the oracle, its replay and any canaries run
+pinned to that image.
 """
 
 import asyncio
@@ -41,6 +42,8 @@
 from .types import CheckStatus, Lane, Level
 
 _IMAGE_ID = re.compile(r"sha256:[0-9a-f]{64}\Z")
+# A tag can move between validation and training; a digest cannot.
+_PINNED_IMAGE = re.compile(r"[^@\s]+@sha256:[0-9a-f]{64}\Z")
 _RESOURCE_ID = re.compile(r"[a-zA-Z0-9][a-zA-Z0-9_.-]*\Z")
 _PROJECT = re.compile(
     r"openenv-validation-[a-f0-9]+-(nop|oracle|oracle-replay|canary-[1-8])__env"
@@ -280,19 +283,22 @@
         return "multi-step Harbor tasks"
     if env.os.value != "linux":
         return "non-Linux Harbor tasks"
-    if env.docker_image is not None:
-        return "task-declared environment.docker_image instead of a Dockerfile build"
+    if env.docker_image is not None and not _PINNED_IMAGE.fullmatch(env.docker_image):
+        return "task-declared environment.docker_image without a sha256 digest pin"
     if config.verifier.environment is not None or (
         config.verifier.environment_mode is not None
         and config.verifier.environment_mode.value != "shared"
     ):
         return "separate Harbor verifier environments"
-    if not (task.task_dir / "environment/Dockerfile").is_file() or any(
+    if (
+        env.docker_image is None
+        and not (task.task_dir / "environment/Dockerfile").is_file()
+    ) or any(
         path.name.lower()
         in {"docker-compose.yaml", "docker-compose.yml", "compose.yaml", "compose.yml"}
         for path in (task.task_dir / "environment").rglob("*")
     ):
-        return "tasks without one Dockerfile or with task-authored Compose"
+        return "tasks without one Dockerfile or pinned image, or with task-authored Compose"
     if contract.resources.gpus or env.gpus or env.tpu or env.mcp_servers:
         return "GPU/TPU or external MCP requirements"
     if contract.requires_credentials:
@@ -333,9 +339,13 @@
 ):
     # Harbor's Compose collector does not reap the client on outer cancellation.
     runtime = type(environment).runtime()
+    project_directory = environment.environment_dir
+    if not project_directory.is_dir():
+        project_directory = project_directory.parent
+    project_directory = project_directory.resolve()
     argv = [*runtime.compose, "--project-name", environment.session_id]
     if runtime.supports_compose_project_directory:
-        argv.extend(["--project-directory", str(environment.environment_dir.resolve())])
+        argv.extend(["--project-directory", str(project_directory)])
     for path in environment._docker_compose_paths:
         argv.extend(["-f", str(path.resolve())])
     spawn = asyncio.create_task(
@@ -343,7 +353,7 @@
             *argv,
             *command,
             env=environment._compose_env_vars(include_os_env=True),
-            cwd=str(environment.environment_dir.resolve()),
+            cwd=str(project_directory),
             stdin=asyncio.subprocess.PIPE
             if stdin_data is not None
             else asyncio.subprocess.DEVNULL,
@@ -851,7 +861,8 @@
     inspector = DockerInspector()
     try:
         attempted = False
-        problem = _unsupported(read_harbor_task(subject.root), contract)
+        task = read_harbor_task(subject.root)
+        problem = _unsupported(task, contract)
         if skip_build:
             problem = "image build skipped by --skip-build"
         if provider is not None:
@@ -978,6 +989,10 @@
                                 "task_effective_digest": digest_before,
                             },
                         )
+                        source_image = task.config.environment.docker_image
+                        if source_image is not None:
+                            record["source_image"] = source_image
+                            startup.measured["source_image"] = source_image
                         if malformed:
                             reward_result = _result(
                                 "runtime.reward_well_formed",

diff --git a/tests/test_validation/test_harbor_validation.py b/tests/test_validation/test_harbor_validation.py
--- a/tests/test_validation/test_harbor_validation.py
+++ b/tests/test_validation/test_harbor_validation.py
@@ -153,9 +153,9 @@
             )
             self.hooks = {}
             self._openenv_cancelled_compose = []
-            self._environment_build_timeout_sec = read_harbor_task(
-                config.task.path
-            ).config.environment.build_timeout_sec
+            environment = read_harbor_task(config.task.path).config.environment
+            self._environment_build_timeout_sec = environment.build_timeout_sec
+            self.docker_image = environment.docker_image
             state.excluded_copied.append(
                 (config.task.path / "environment/.worktrees").exists()
             )
@@ -1175,16 +1175,78 @@
     assert not boundary.trials
 
 
-def test_task_declared_image_is_refused_before_trials(task, boundary):
+def test_unpinned_task_image_is_refused_before_trials(task, boundary):
     with (task / "task.toml").open("a") as stream:
         stream.write('[environment]\ndocker_image="registry.example/image:latest"\n')
     report = run_validation(task)
     startup = result(report, "runtime.startup")
     assert startup.status is CheckStatus.SKIP
-    assert "docker_image" in startup.evidence[0]
+    assert "digest pin" in startup.evidence[0]
     assert not boundary.trials
 
 
+def test_digest_pinned_task_image_runs_without_environment_dir(
+    task, boundary, tmp_path
+):
+    pinned = "registry.example/image@sha256:" + "d" * 64
+    (task / "environment/Dockerfile").unlink()
+    (task / "environment").rmdir()
+    with (task / "task.toml").open("a") as stream:
+        stream.write(f'[environment]\ndocker_image="{pinned}"\n')
+    artifacts = tmp_path / "pinned"
+    report = run_validation(task, artifacts_dir=artifacts)
+    startup = result(report, "runtime.startup")
+    assert startup.status is CheckStatus.PASS
+    assert startup.measured["source_image"] == pinned
+    native = json.loads((artifacts / "harbor-evidence.json").read_text())
+    assert native["source_image"] == pinned
+    # The do-nothing trial pulls the pin; later trials run the image it retained.
+    images = [trial.docker_image for trial in boundary.trials]
+    assert images == [pinned, *[startup.measured["retained_image_tag"]] * 2]
+
+
+@pytest.mark.asyncio
+async def test_compose_uses_task_dir_when_environment_dir_is_absent(
+    tmp_path, monkeypatch
+):
+    task_dir = tmp_path / "task"
+    task_dir.mkdir()
+    spawned = {}
+
+    async def create_subprocess_exec(*argv, **kwargs):
+        spawned["argv"] = argv
+        spawned["kwargs"] = kwargs
+        return object()
+
+    class Environment:
+        environment_dir = task_dir / "environment"
+        session_id = "trial__env"
+        _docker_compose_paths = [tmp_path / "compose.yaml"]
+
+        @staticmethod
+        def runtime():
+            return SimpleNamespace(
+                compose=("docker", "compose"),
+                supports_compose_project_directory=True,
+            )
+
+        @staticmethod
+        def _compose_env_vars(*, include_os_env):
+            assert include_os_env
+            return {}
+
+        @staticmethod
+        async def _collect_buffered_output(process, **kwargs):
+            return SimpleNamespace(return_code=0)
+
+    monkeypatch.setattr(asyncio, "create_subprocess_exec", create_subprocess_exec)
+    await harbor._run_compose(Environment(), ["pull"], cancelled=[])
+
+    project_flag = spawned["argv"].index("--project-directory")
+    assert spawned["argv"][project_flag + 1] == str(task_dir.resolve())
+    assert spawned["kwargs"]["cwd"] == str(task_dir.resolve())
+
+
 def test_explicit_contract_and_valid_toml_required(task, boundary):
     (task / "validation/openenv.json").unlink()
     with pytest.raises(ManifestError):

_{You can send follow-ups to the cloud agent here.}

^{Reviewed by Cursor Bugbot for commit 14396cc. Configure here.}