[Tomer Glottmann (tomergee)]

Makes the "injected daemon" section of the NeMo Gym integration real: a runtime layer that brings the NeMo servers into any task image, started by the guest, reached through endpoint().

Why

NeMo components that talk to a service inside the sandbox, NeMo Skills' code-execution sandbox server (ns_tools, the math recipes) and NeMo Gym's own exec server (what the ECS Fargate provider injects), need that service in the task image. SWE-bench-style images do not have it, and the integration README so far described the layer mechanism (#70) and the port proxy (#92) with a hypothetical nemo-daemon. This PR ships the daemon.

What

integrations/nemo-gym/layer/: one OCI image built from scratch (the recipe of #93's OpenHands layer): python-build-standalone CPython 3.12.15 plus hash-pinned wheels installed with uv, and two entry points served with the layer's own interpreter, so the task image contributes only its kernel and filesystem:

server	sidecar command	protocol
NeMo Skills local sandbox server	/opt/nemo/python/bin/python3 -m nemo_runtime.sandbox_server --host 0.0.0.0 --port 6000	POST /execute (ipython, python, shell, lean4), IPython sessions by X-Session-ID, GET /health
NeMo Gym exec server	/opt/nemo/python/bin/python3 -m nemo_runtime.exec_server --host 0.0.0.0 --port 19542	POST /exec, POST /upload, GET /download, GET /health

Both are vendored verbatim under layer/third_party/ with their Apache-2.0 license and a provenance README (NeMo Skills bcf059a, NeMo Gym d4fd5e8); the wrappers only choose the bind address (Flask's default is 127.0.0.1, which the router cannot reach) and the port. For language: python the layer's interpreter is appended to PATH as the fallback, so an image's own Python wins when it has one and an image without Python still executes code. lock.sh regenerates the hash-pinned requirements for amd64 and arm64.

Docs and tooling: the README section is rewritten around the real layer (build, template command, the two servers, trust note, limits updated: the NeMo Skills server no longer needs a Python in the image, the layer needs glibc 2.17+); the mapping table gains endpoint(); hack/runtime-layer-demo.sh registers the template and calls the servers from inside the environment, through the header form and through the path form; RUNTIMES.md and the top-level README point at it. Two things the live runs taught, both now in the README: the base template must name a task image of its own (--rootfs-image), because the guest's image is Alpine-based and cannot run the layer's glibc interpreter, so a base without one never bakes its golden snapshot and shows ERROR (the derived templates still work; the oh-agent-server base on the test cluster shows the same); and a recipe must select the layered base through create.template, not a per-sandbox provider_options template, which selects the base itself and skips the task image.

Test: tests/test_e2e.py::test_runtime_layer_endpoint, gated by SUBSTRATE_E2E_RUNTIME_TEMPLATE, creates a sandbox on the layered template, resolves endpoint(), runs print(6 * 7) through the proxy, keeps an IPython session across two calls, and checks that an undeclared port is refused. No provider code changes.

Verification

• Hermetic: pytest integrations/nemo-gym/tests 45 passed; make verify-boilerplate verify-gofmt; ruff clean on the new Python; shellcheck clean on the scripts.
• Live on a GKE cluster (gVisor pool, Substrate ddb46651, ate-env-api with the port proxy), layer image 87 MB compressed:
  • hack/runtime-layer-demo.sh with python:3.12-slim and with debian:bookworm-slim (no Python inside): serving 4 to 5 s after create with readiness gated on both sidecars; /execute answered from inside, through the header form and through the path form; an IPython session kept x between calls; language: python printed the layer's 3.12.15 in the Debian image; the exec server answered /exec.
  • test_runtime_layer_endpoint against the same template and the Debian image: passed in 6.4 s, no sandbox left behind.
• The template was registered with the CLI of the pending Substrate bump branch, because the cluster's Substrate (ddb46651, substrate#2780) no longer takes sandboxConfig in a template and main's ate-env manifest template still emits it. The layer, the scripts and the test do not depend on that.

Limits

• Both servers run what they are sent and authenticate nobody; the sandbox is the trust boundary, as with NeMo Skills' own sandbox container and the ECS exec server. The README says so.
• NeMo Skills' LocalSandbox client builds http://host:port/execute itself and cannot add headers, so a recipe that uses it directly needs the path form as its base URL or a per-sandbox proxy; ns_tools today points at one shared sandbox server, not one per Gym sandbox.
• linux/amd64 was built and tested; the Dockerfile and lock cover arm64 too, untested.