fix(docker): close two driver:docker channels that leaked grading material to the agent

[CarlesUIPath]

Summary

Under driver: docker, an eval agent could read its own grading material as follows:

1. During execution, the agent has direct access to the grading criteria via /work/input/taks.yaml. In certain cases, the agent trace reasoning compares the expected criteria directly against their own solution.
2. The skills root mounts the whole repo, and agents can access it. This means an agent could have access to more details about their grading.

This PR closes both channels, adds a lint guard so the skills layout can't recur, and is verified end-to-end with a live adversarial agent. That is, a test which asks the agent to cheat by finding reference files.

The existing reference anti-cheat window #106 (mode-000 on the staged reference copy) did not cover either channel: it guards the reference copies, while the criteria sat unshielded at /work/input and the plugin mount exposed the originals.

The fixes

• Fix A — delete the staged task definition in-container. /work/input is now writable, and the in-container entry point deletes both task.yaml and context.json immediately after load_task (_scrub_staged_inputs), before any agent turn on all paths (normal + regrade). Deleting should be a safe op since graders do not read at task.yaml after agent finishes.
• Fix B — allowlist / default-deny mask of auto-mounted plugin trees. New isolation/eval_material.py::mask_dirs: the plugin root stays mounted :ro, but every child dir outside the keep-set (.claude-plugin + the manifest-declared skill dirs) is --tmpfs-masked. Everything that isn't the skill surface is hidden by default, so an unknown/new eval layout can never leak; tests/, references, node_modules/ are masked for free. This is stronger than a chmod window. Graders do not look into the mounted tasks, since these resources are present in task dirs from the previous anti-cheat PR feat(reference)!: directory-only references + anti-cheat permission window #106.
• Fix C — CE065 lint. Flags any task_id: YAML or reference.directory left readable under an in-repo plugin root (colocated inside a skill dir, or a loose file at the root that a tmpfs can't mask). Reuses mask_dirs + manifest_skill_dirs as the single source of truth, pinned by an import-identity test.

Verification

• make verify: 5758 passed, 4 skipped (green; 2 pre-existing env-driven failures only appear when a local .env leaks TASK_DIR/model — unrelated files, unchanged from main).
• Live adversarial run (Claude agent, real container, ordered to peek): came back empty on all channels — find …/tests… and find … reference → no output; cat /work/input/task.yaml and cat /work/input/context.json → No such file or directory; verdict.txt: DENIED; anti-cheat criterion PASS, task SUCCESS.
• Deterministic docker probes + argv/unit tests for mask_dirs, _resolve_mount_path, nested-plugin dedup, the skills:"." stand-down, and _scrub_staged_inputs.
• Reviewed by a 3-model pass (Opus/Fable/Sonnet) and a merge+peek verification review — no reachable Critical/High/Medium.

Known residuals (documented defense-in-depth)

• The reference window's pre-existing gap (root can chmod back; full containment needs a non-root uid) is unchanged — out of scope.
• A tmpfs masks a directory, not a single loose file; CE065 covers in-repo plugins but can't reach out-of-repo ones. skills:"." stands the mask down (fail-safe, now logged + CE065-flagged).

Files

src/: isolation/eval_material.py (new), isolation/docker_runner.py, cli/run_task_internal_command.py, agents/_skills.py (manifest_skill_dirs made public/SSOT), orchestrator.py, fs_permissions.py.
tests/: test_eval_material.py (new), test_run_task_internal.py (new), test_docker_runner_mounts.py, test_custom_lint.py (CE065).
Docs: docs/DOCKER_ISOLATION.md, .claude/architecture-notes.md, CLAUDE.md.

🤖 Generated with Claude Code

[github-actions]

Claude finished @CarlesUIPath's task in 1m 21s —— View job

---

Code Review in Progress

Todo List:

• Read review guidelines from .github/code_review.md
• Read CLAUDE.md for project conventions
• Get full diff with git diff origin/main...HEAD
• Review each changed file with full context
• Check cross-file consistency
• Perform design-level scrutiny
• Identify what's missing
• Format and post final review

[bai-uipath]

Fix A is good, Fix B breaks real plugins. Suggest landing Fix A on its own and reworking the rest.

• The mask hides the plugin, not just eval material. The keep-set is only .claude-plugin plus the skill dirs, so hooks/, commands/ and agents/ (directories Claude Code loads by default) get masked too. On the skills nightly's staged .plugin-root (UiPath/skills#3546) that drops the plugin's hooks and commands and hides tests/scripts/stage-preview-sdk-workspace.sh, which the nightly pre_run calls, so the eval grades a different plugin than the one that ships. Fix before merge: keep the full plugin surface (default component dirs plus every manifest-declared path), or replace the mask with a preflight that errors when a mounted plugin root contains task_id: YAMLs.

• CE065 checks nothing real. It only walks this repo's tasks/, where no task resolves to a plugin root, and never sees the suites where this layout actually lives. Fix: cut it, or fold it into the runtime preflight above so it covers every suite.

• Needs a rebase, and the comments won't pass main's new prose gate. Several new docstrings and comment blocks exceed the caps, and the same rationale is repeated across about six files. Fix: one section in .claude/notes/isolation.md plus a Rationale: pointer.

• Fix A removes one copy of the criteria, not all of them. The same YAML (and its siblings) is still in /work/task_dir, behind a chmod window that gemini-3.8-flash bypassed in the BPMN sweep. Not a blocker, but the comments saying the agent "cannot read its criteria" should be scoped to /work/input.

[bai-uipath]

Verdict: good to merge. It blocks the in-container leaks and leaves the web hole open. It breaks one skills task, which needs a small fix before the coder-eval pin bump.

Gemini reruns

Gemini 3.8 Flash on Antigravity, the paper's settings. Two SkillsBench tasks, run in docker.

build	runs	cheated	how
0.10.2 (paper)	4	3	read tests/ answers in the container
main	20	3	downloaded ground_truth.json from GitHub with Antigravity's web tools
PR #179	19	0	none

The gap is a small sample (p = 0.21). All 3 main cheats went through the web, which #179 doesn't touch, so given enough runs they will show up on #179 too.

What it blocks

• Closed: the staged task.yaml/context.json in /work/input (now deleted after load), and eval files sitting next to the skills in a plugin root (now masked). A probe task confirmed that 4 of the 8 routes open on main are closed.
• Still open: Antigravity's web tools and the network, which is the route Gemini actually uses. The root chmod reopen of /work/task_dir is also still open, though no run tried it.

Skills-suite check

Run against skills main with the nightly config: docker, whole repo as the plugin.

• Unchanged: skills load identically (28 UiPath skills, 6 agents) and template fixtures are still visible. 45 of 46 real tasks set up exactly as on main, and the activation rate is 42 vs 43 of 91.
• Lost: the plugin's SessionStart hooks (telemetry only) and /uipath:install-permissions.
• Breaks one task: skill-troubleshoot-smoke-manifest-commands. Its pre_run reads $SKILLS_REPO_PATH/tests/..., which is now masked.
  • Fix: a task-level extra_mounts for tests/tasks/uipath-troubleshoot, restating the ~/.uipath mount.
  • Timing: land it before bumping the pin, currently 0.12.7.

Status

• Merge: main merged into the PR (8f6a364a), CI 17/17 green, waiting on review.
• Next: close the web hole by passing CapabilitiesConfig(disabled_tools=[SEARCH_WEB, READ_URL_CONTENT]) in antigravity_agent.py. The SDK supports it, and it's about 30 lines.