Enforce per-task network allowlists through direct and proxy gateways

.env.example

@@ -63,13 +63,17 @@ OPENROUTER_API_KEY=
 JINA_API_KEY=
 
 # ---------------------------------------------------------------------------
-# 5. Container proxy (not supported by current restricted task policies)
+# 5. Runtime egress (see docs/network-policy.md)
 # ---------------------------------------------------------------------------
-# Leave these empty. A general proxy can bypass Harbor's destination allowlist.
+# General container proxies bypass allowlists; leave empty for restricted tasks.
 CONTAINER_PROXY=
 CONTAINER_NO_PROXY=
 
-# Optional: comma-separated reachable upstream IPv4 DNS servers for the Docker
-# isolation sidecar. Bypasses an unreliable host DNS stub without opening HTTP
-# egress; only DNS port 53 to these IPs is exempted in allowlist mode.
+# Optional direct-mode upstream IPv4 DNS servers, comma-separated.
 CONTAINER_DNS=
+# Optional image override; defaults to hanhainebula/search-swe-egress:1.0.0.
+# Pulled automatically if missing locally.
+EGRESS_IMAGE=
+# Optional direct/proxy JSON path. Requires CONTAINER_DNS, CONTAINER_PROXY and
+# EGRESS_IMAGE to be empty. Leave empty for default direct access.
+EGRESS_CONFIG=

README.md

@@ -106,6 +106,9 @@ For other runs, fill only the matching sections already present in `.env`:
   `OPENROUTER_API_KEY`, or `JINA_API_KEY` only for the tasks identified by the
   comments in `.env.example`.
 
+If a proxy is required, set `EGRESS_CONFIG` in `.env` following the
+[network guide](docs/network-policy.md); otherwise leave it empty.
+
 The [evaluation guide](docs/evaluation.md) documents credential isolation,
 custom endpoints, proxies, and the complete per-task matrix.
 
@@ -194,7 +197,7 @@ hardware matrix plus GPU, network-policy, and custom-provider options.
 | --- | --- |
 | [Quick start guide](docs/quickstart.md) | A first CPU evaluation, end to end |
 | [Evaluation guide](docs/evaluation.md) | Per-task credentials, coding agents, GPU, network policy, and custom providers |
-| [Network policy](docs/network-policy.md) | Harbor egress modes and exact per-task host allowlists |
+| [Network policy](docs/network-policy.md) | Per-task allowlists, default direct gateway, and optional proxy egress |
 | [Asset guide](docs/assets.md) | Downloading, verifying, and restoring fixed data and models |
 | [Benchmark design](docs/benchmark.md) | Evaluation, repository layout, and data provenance |
 | [Contributing guide](docs/contributing.md) | Task-authoring workflow, validation, and PR expectations |

README_zh.md

@@ -98,6 +98,8 @@ VERIFIER_OPENAI_API_KEY=YOUR_DEEPSEEK_KEY
 - 可选 submission API：只在 `.env.example` 注释所列任务确实使用时，填写
   `TASK_1_1_OPENROUTER_API_KEY`、`OPENROUTER_API_KEY` 或 `JINA_API_KEY`。
 
+需要代理时，按[网络配置](docs/network-policy.md)设置 `.env` 的 `EGRESS_CONFIG`，否则留空。
+
 凭证隔离、自定义服务地址、网络权限和完整的逐任务配置矩阵见
 [评测指南](docs/evaluation.md)。
 
@@ -182,7 +184,7 @@ gateway、订阅 OAuth、Bedrock、Vertex、ACP 和自定义 Claude settings。
 | --- | --- |
 | [快速开始指南](docs/quickstart.md) | 完整的一次 CPU 评测流程 |
 | [评测指南](docs/evaluation.md) | 各任务凭证、编码智能体、GPU、网络策略和自定义模型服务 |
-| [网络权限](docs/network-policy.md) | Harbor 网络模式与各任务精确的 host allowlist |
+| [网络权限](docs/network-policy.md) | 各任务的 host allowlist、默认直连网关与可选代理出口 |
 | [资源说明](docs/assets.md) | 固定数据与模型的下载、校验和恢复 |
 | [基准设计](docs/benchmark.md) | 评测方式、仓库结构和数据来源 |
 | [贡献指南](docs/contributing.md) | 任务创作流程、验证要求和 PR 说明 |

docs/evaluation.md

@@ -260,13 +260,16 @@ Compose overlays request the real GPU.
 Harbor 0.22.0's Docker backend rejects `gpus = 1` during its own preflight and
 does not translate that field into the Compose GPU request. The shared launcher
 therefore supplies `--override-gpus 0` automatically while leaving the truthful
-task metadata and Compose reservations intact. If invoking `harbor run --env
-docker` directly with that Harbor version, add the same override. Recheck this
-workaround when upgrading Harbor.
+task metadata and Compose reservations intact. Direct Harbor invocations using
+`--env scripts.harbor_environments:PhaseScopedDocker` need the same override
+with that Harbor version. Recheck this workaround when upgrading Harbor.
 
 ## Runtime network enforcement
 
-Every current task uses a restricted agent or verifier phase. Keep
+Every current task uses a restricted agent or verifier phase, so the launcher
+automatically selects the direct gateway and pulls its image if missing. For an
+upstream HTTP(S) proxy, configure `EGRESS_CONFIG` using the
+[network guide](network-policy.md); proxy mode rejects public phases. Keep
 `CONTAINER_PROXY` unset: a general proxy would let the proxy choose arbitrary
 destinations and would defeat Harbor's hostname policy, so the launcher rejects
 it. Configure image-pull and Docker build proxies separately at the Docker

docs/network-policy.md

@@ -1,10 +1,111 @@
 # Runtime network policy
 
-Search-SWE uses Harbor 0.22.0's native `network_mode` and `allowed_hosts`
-fields. `allowed_hosts` contains hostnames only—not URLs, ports, or paths—and is
-valid only with `network_mode = "allowlist"`. Docker enforcement uses Harbor's
-egress-control sidecar. If the Docker host cannot enforce the requested policy,
-Harbor rejects the run instead of silently granting public access.
+Search-SWE uses Harbor 0.22.0's `network_mode` and `allowed_hosts` fields.
+Allowlists contain exact lowercase hostnames, not URLs, ports or paths, and
+apply separately to each task and execution phase. Restricted tasks use an
+independent trusted gateway for each agent/verifier environment, including when
+an upstream proxy is configured. CPU/GPU task images remain unchanged.
+
+## Default: direct access
+
+No extra setup is required. The launcher uses
+`hanhainebula/search-swe-egress:1.0.0`, pulls it if missing locally, and reuses it
+on later runs. Leave `EGRESS_IMAGE` and `EGRESS_CONFIG` empty. `EGRESS_IMAGE` or
+`--egress-image` can select a matching local build or repository digest.
+Images are validated against this checkout before task containers start;
+pull failures or incompatible images stop startup. Downloads use the Docker
+daemon's network/proxy settings, not the task's proxy configuration.
+
+Direct mode uses Docker DNS through the trusted gateway. If needed, set
+`CONTAINER_DNS` (or `--container-dns`) to comma-separated reachable IPv4 DNS
+servers. Only allowed host queries are forwarded; task processes cannot query
+upstream or Docker DNS directly. Changing DNS does not repair TLS or API errors.
+
+## Networks requiring a proxy
+
+Use an HTTP(S) proxy with CONNECT support and an HTTPS DNS-over-HTTPS (DoH)
+endpoint accessible through it. Create a JSON file outside task mounts and
+build contexts, replacing the example address and hostname below:
+
+```json
+{
+  "version": 1,
+  "mode": "proxy",
+  "image": "hanhainebula/search-swe-egress:1.0.0",
+  "upstream": {"url": "http://192.0.2.10:8080"},
+  "dns": {"doh_url": "https://resolver.example/dns-query"}
+}
+```
+
+Point `.env` to the file, leaving other overrides empty:
+
+```dotenv
+EGRESS_CONFIG=/absolute/path/to/egress.json
+EGRESS_IMAGE=
+CONTAINER_DNS=
+CONTAINER_PROXY=
+```
+
+`--egress-config PATH` overrides `EGRESS_CONFIG`. Relative CLI paths resolve
+from the current directory; environment-variable paths resolve from the checkout.
+Explicit config requires `image` and cannot be combined with image, DNS or general
+container-proxy overrides. Do not add the proxy to task allowlists.
+
+The proxy must be reachable **from Docker**; host `127.0.0.1` is not the host
+inside a container. There is no fixed VPN product/port or automatic discovery.
+SOCKS-only and TUN-only endpoints are not supported by this configuration.
+For a hostname URL such as `https://proxy.example:8443`, also set
+`upstream.address` to its reachable IPv4 address. Certificates are validated
+against the URL hostname; custom CAs and client certificates are unsupported.
+Loopback, link-local and task-local endpoints are rejected. Proxy URLs cannot
+contain credentials, queries or fragments; DoH URLs cannot contain query parameters.
+
+For proxy authentication, set `upstream.auth_file` to a JSON file containing
+`{"username": "...", "password": "..."}`. Keep it outside task mounts/build
+contexts, owned by the launcher user, with mode `0600` and parent mode `0700`.
+Relative auth paths resolve from the config file; symlinks/hard links are rejected.
+Only the trusted gateway receives these credentials.
+
+Explicit direct config uses `"mode": "direct"`, no `upstream`, and
+`"dns": {"servers": ["192.0.2.53"]}` with a reachable resolver (an optional UDP
+port is accepted). An empty `dns` object selects Docker DNS. `version` and
+`image` are still required. The launcher's `--dry-run` checks policy/config
+compatibility without pulling images, reading proxy credentials or calling APIs.
+
+## Supported runtime
+
+- Native Linux amd64, local rootful Docker through a Unix socket, Harbor 0.22.0,
+  and IPv4. Docker Desktop, remote/rootless Docker and userns remapping are unsupported.
+- Direct mode supports public and restricted phases. **Proxy mode rejects any
+  public phase**.
+- Custom network topologies, extra capabilities/devices, host/control mounts,
+  external/shared volumes and kept containers are rejected. Named volumes must
+  be project-local plain Docker volumes.
+- Authorization checks HTTP destinations and TLS SNI, not encrypted paths,
+  bodies or Host headers/domain fronting at an approved origin.
+
+## Isolation and recovery
+
+The gateway starts in no-network mode, applies the phase baseline, then allows
+task services to start. Task services cannot modify the firewall or forge trusted
+traffic marks. Policy changes revoke old connections. Proxy, DNS or control
+failure closes egress; there is no unrestricted fallback. A 30-second host-renewed
+lease bounds revocation after loss of host control; trusted host clocks are required.
+An already accepted upstream API operation cannot be undone by revocation.
+
+Normal teardown removes each environment's owned resources and private settings,
+while preserving images and host datasets. For leftovers after a failed cleanup:
+
+```bash
+python scripts/egress_cleanup.py
+python scripts/egress_cleanup.py --directory /path/from/list
+python scripts/egress_cleanup.py --directory /path/from/list --remove
+```
+
+These list, preview and remove one verified inactive instance respectively.
+Recovery refuses active owners, foreign resources or a different Docker daemon.
+Host logs under `egress/<instance>.log` record policies and kernel counters;
+these counters are not HTTP request counts.
 
 ## Current task matrix
 
@@ -40,7 +141,8 @@ The repository launcher derives and supplies the coding-model host. When
 invoking Harbor directly on a non-public task, add the matching hostname:
 
 ```bash
-harbor run --path tasks/TASK_ID --env docker \
+PYTHONPATH="$PWD${PYTHONPATH:+:$PYTHONPATH}" harbor run --path tasks/TASK_ID \
+  --env scripts.harbor_environments:PhaseScopedDocker \
   --agent scripts.harbor_agents:PreinstalledCodex --model MODEL_ID \
   --ak version=0.147.0 \
   --allow-agent-host MODEL_API_HOST
@@ -50,7 +152,8 @@ For the supported official-Anthropic Claude Code mode, use the pinned wrapper,
 an environment reference rather than a literal secret, and the fixed host:
 
 ```bash
-harbor run --path tasks/TASK_ID --env docker \
+PYTHONPATH="$PWD${PYTHONPATH:+:$PYTHONPATH}" harbor run --path tasks/TASK_ID \
+  --env scripts.harbor_environments:PhaseScopedDocker \
   --agent scripts.harbor_agents:PreinstalledClaudeCode \
   --model ANTHROPIC_MODEL_ID \
   --ak version=2.1.273 \
@@ -63,6 +166,11 @@ Bedrock selectors. Reproduce that sanitization when bypassing it; custom
 Anthropic endpoints and alternate Claude authentication modes are not part of
 the supported configuration.
 
+These commands use the default direct gateway image and pull it if missing locally.
+To select another matching image, add `--environment-kwarg egress_image=YOUR_IMAGE`. Manually using
+`--env docker` selects installed Harbor and does not receive this checkout's
+capability removal or old-connection revocation fixes.
+
 Do not use `--allow-environment-host` for a coding-model endpoint: that changes
 the environment baseline rather than only the agent phase. Do not add package
 registries, source-code hosts, wildcard domains, or a general HTTP proxy to a
@@ -74,29 +182,20 @@ Image pulls and Docker build downloads happen before untrusted task execution
 and are configured at the Docker daemon/build layer; they are not task runtime
 egress permissions.
 
-## Optional upstream DNS for isolated Docker runs
-
-If Docker's embedded resolver intermittently times out forwarding to the host
-DNS stub (for example, `127.0.0.53`), configure reachable upstream IPv4 DNS
-servers in the local `.env`:
-
-```dotenv
-CONTAINER_DNS=198.18.254.30,198.18.254.31
-```
-
-These example addresses belong to the diagnosed host's network; do not assume
-they work elsewhere. `--container-dns IP,IP` overrides the environment setting.
-Leave it unset to retain Harbor's default DNS behavior.
-
-The launcher generates a Docker Compose overlay for the isolation sidecar,
-shared by the task processes. It retains Harbor's original policy helper and
-adds only TCP/UDP port 53 exceptions to the configured DNS IPs during nonempty
-allowlist policies. `deny-all` removes those exceptions; ordinary API traffic
-still uses the original hostname allowlist. This is not an HTTP proxy and does
-not enable `CONTAINER_PROXY`. It neither changes host DNS nor restarts Docker.
-
-Overlays and a snapshot of the installed Harbor helper are retained in
-`/tmp/searchswe-dns-*` for the lifetime of the run. Do not remove them while a
-run is active (including its separate verifier); they may be removed afterward.
-Changing DNS does not fix unrelated TLS, API rate-limit, or authentication
-failures. Verify permitted and denied destinations after upgrading Harbor.
+## Regression requirements
+
+The [build and regression guide](../environments/egress/README.md#verification)
+covers these enforcement requirements:
+
+- Untrusted services cannot forge the gateway's firewall exemption marks or
+  modify its rules; capabilities and namespace boundaries must enforce this.
+- Tightening a policy terminates previously authorized tunnels and pooled
+  connections, not merely rejects new connections.
+- Agent, verifier, baseline, public, and no-network transitions are tested;
+  phase allowlists must not be replaced with their union.
+- Destination resolution works through the intended upstream path, including
+  when local DNS returns NXDOMAIN, times out, or gives an incorrect address.
+- Failure of the proxy, resolver, or policy update fails closed, without a
+  direct/unfiltered fallback. TLS certificate validation remains enabled.
+- The upstream endpoint is operator-configured, with no machine-specific IP,
+  port, VPN product, or public DNS provider required by task packages.

docs/quickstart.md

@@ -62,6 +62,9 @@ configuration groups and only the verifier receives the `VERIFIER_*` values.
 their keys are optional and are not needed for an implementation that uses only
 the provided corpus and local runtime.
 
+If a proxy is required, set `EGRESS_CONFIG` in `.env` following the
+[network guide](network-policy.md); otherwise leave it empty.
+
 The local `.env` is ignored by Git. Do not commit or print credentials. On a
 multi-user Unix host, restrict it after adding credentials:
 

environments/egress/Dockerfile

@@ -0,0 +1,14 @@
+# Use scripts/build_egress_gateway.py; it stages and verifies pinned APKs.
+ARG COMPONENT_IMAGE=searchswe-egress-component:build-required
+FROM ${COMPONENT_IMAGE}
+ARG GATEWAY_SOURCE_SHA
+ARG RUNTIME_LOCK_SHA
+COPY packages/ /tmp/searchswe-apks/
+RUN apk add --no-cache --no-network --repositories-file /dev/null /tmp/searchswe-apks/*.apk && rm -rf /tmp/searchswe-apks
+COPY runtime.lock.json /usr/share/searchswe-egress/runtime.json
+COPY gateway.py /opt/searchswe/gateway.py
+COPY --chmod=755 entrypoint.sh /opt/egress-sidecar/entrypoint.sh
+COPY --chmod=755 network-policy /usr/local/bin/network-policy
+LABEL org.search-swe.egress.gateway="1"
+LABEL org.search-swe.egress.source-sha256=${GATEWAY_SOURCE_SHA}
+LABEL org.search-swe.egress.runtime-lock-sha256=${RUNTIME_LOCK_SHA}

environments/egress/Dockerfile.component

@@ -0,0 +1,19 @@
+# Build from the audited helper image, then publish only the merged filesystem.
+# This excludes replaced GOST and OpenSSL binaries from all final OCI layers.
+ARG BASE_IMAGE=searchswe-harbor-base:build-required
+FROM ${BASE_IMAGE} AS assembled
+COPY packages/ /tmp/searchswe-apks/
+RUN apk add --no-cache --no-network --repositories-file /dev/null /tmp/searchswe-apks/*.apk && rm -rf /tmp/searchswe-apks
+COPY --chmod=755 gost-searchswe /bin/gost
+COPY LICENSE.* /usr/share/licenses/searchswe-egress/
+COPY third-party/ /usr/share/licenses/searchswe-egress/third-party/
+COPY component-manifest.json /usr/share/searchswe-egress/component.json
+
+FROM scratch
+COPY --from=assembled / /
+ENV PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+WORKDIR /bin/
+ENTRYPOINT ["/bin/gost"]
+ARG PATCH_SHA
+LABEL org.search-swe.egress.component="gost-x-0.10.9-searchswe-1"
+LABEL org.search-swe.egress.patch-sha256=${PATCH_SHA}