[anth-volk]

Fixes #1294

Stack

main ← #1295 ← #1297 ← #1296 ← #1298 (GitHub stack #1328, linked on 2026-10-20 with gh stack link).

#1322 taken out of the stack (2026-10-20): CI runs only for pull requests into main on purpose, to limit the number of simultaneous runs on the organization's shared runners. #1322, which removed that filter, is closed, and the old stack #1323 was removed. 4ced66de0 on #1295 restores the four files #1322 changed to their versions on main, and each branch above merges the one below it: #1297 94a5bff93, #1296 b0406fd76, #1298 7cb857716. None of those merges had conflicts, and test_ci_integration_job.py passes on every branch (9 passed). No commit was rewritten. From now on only the bottom pull request, which targets main, gets CI; each one above gets CI once it becomes the bottom.

• Base: main since 2026-10-20. Until then it was ci-pull-requests-into-any-base (Run the Tests workflow for pull requests into any base branch #1322's branch). 4ced66de0 is a revert of an unwanted change rather than a bug fix, so it has no red step.
• Head: 4ced66de0, the restore commit on top of 6b6887ef6. Before it, since 8d159a8c5, the branch gained three merge commits from its old base, all without conflicts and none rewriting a commit:
  • a645a3de8: Run the Tests workflow for pull requests into any base branch #1322's first commit and main at 41209b3;
  • e8c164966: Run the Tests workflow for pull requests into any base branch #1322's two review fixes (a test and the agent guide);
  • 6b6887ef6: main at a90eec3cb (Prepare the d1386 Route A rerun: CT-ladder base plan, fresh lineage evidence, launch-day sequence #1289, Place UK private rent and BRMA at the assigned area; bind Scotland's rents at BRMA grain (#1262, #1090) #1305), through Run the Tests workflow for pull requests into any base branch #1322.
• microcosm-collector#9's copy is still current: 8d159a8c5, the commit it copied the projection from, is in this branch's history, and the nine copied modules in packages/microcosm-graph/src/microcosm/graph/ still have the blob IDs its provenance.py records. So no re-copy is needed.
• Tests after each merge: this PR's two changed test files (test_uk_orrery_publication.py, test_graph_orrery_projection_without_store.py) gave 33 passed.

Why main was merged in (2026-10-20): GitHub builds each stacked pull request's test merge on top of the test merge of the pull request below it, starting from main: refs/pull/1322/merge has main as its first parent, and each later one has the previous one's. Once main gained #1305, #1296's change to rowwise_staging.py conflicted with it, so GitHub could not build #1296's test merge. It reported #1296 as conflicting although its branch contained its base, and started no CI for #1296 or #1298. main (a90eec3cb) was merged into #1322's branch, and each branch into the one above, with merge commits; #1296 is now mergeable.

Merge order: #1295, #1297, #1296, #1298, each after the one below it, with merge commits (or gh stack merge). Every branch above #1295 has its parent merged in rather than rebased, so no commit was rewritten. Squash-merging a lower pull request would leave its original commits in the branches above, which would then need rebasing.

Summary

The Orrery artifact is built in two halves. The build machine keeps evidence capture and verification against the content store. Projection needs only the compiler schema and the execution overlay, which makes it the half a collector worker can take over later. This PR makes that possible without changing what builds publish:

• The projection imports without the build runtime. microcosm.graph.orrery now imports and projects a schema plus overlay in a process where numpy, pandas, microcosm-frame and microcosm.graph.store cannot be imported. Two things pulled them in before:

  • orrery_evidence imported evidence for one protocol string and a hash helper;
  • the package init imported the executor, store, kernels, codecs and evidence eagerly.

  microcosm.graph now loads those runtime exports on first access (PEP 562). __all__ and every public name are unchanged, and the declared dependencies are untouched (that's O2).

• The execution overlay has its own file format. microcosm.graph.execution_overlay.execution_overlay_bytes and load_execution_overlay write and read the overlay as canonical JSON, keeping its existing protocol, microcosm.graph.execution-evidence.v1.

• UK finalize writes graph.execution.json beside graph.orrery.json and registers it as a build output: in the dataset manifest, a refused build's evidence manifest, and a failed build's orrery-failure/ bundle. It is not added to the Orrery publication set (orrery-publication/, orrery.upload.json); that waits for O5.

Overlay file format

• Path: graph.execution.json in the build output directory, next to graph.schema.json and graph.orrery.json.
• Protocol: microcosm.graph.execution-evidence.v1, the overlay's own protocol field. No wrapper protocol was added.
• Encoding: canonical JSON (UTF-8, sorted keys, no whitespace), with no transport tags. integer_literal and float_literal are added by the projection only.
• Reader: a regular file within the schema's 32 MiB serialized limit, which collection and projection already apply to the overlay. It refuses duplicate keys, non-finite numbers, values outside the schema's JSON bounds and other protocols. Graph-binding and record checks stay in the projection.

To project from the files:

orrery_json_from_schema(
    json.loads(schema_bytes),
    execution=load_execution_overlay(path),
)

Commits

Local checks were limited to the test files this branch adds or modifies, at the user's request; the rest of the suite is left to CI. Commit 1 ran the complete engine-free category before that change arrived. Commit 2's full-category run was stopped at about 40%. Every commit also passed tools/ci_test_plan.py verify, tools/graph_acceptance_burndown.py --verify, ruff check ., ruff format --check on the changed files, and a byte-identity harness. The harness runs a deterministic local-only finalize_graph, with and without a dataset manifest, over a fixed two-phase toy state, and compares every file's digest with origin/main.

1. Import the Orrery projection without numpy, pandas or the content store (dd71e0e71)
   • Red: 2 failed, for the intended reason. The restricted interpreter's finder raised on numpy, imported from microcosm/graph/__init__.py through kernel.py. One guard test passed (all public exports still resolve).
   • Green: 3 passed.
   • Complete engine-free category: 18,567 passed, 149 skipped, 3 xfailed, 1 failed. The failure, test_us_release_head_to_head_scorer.py::test_spawned_workers_score_identically_to_sequential_cli, is environmental: the scorer refuses --worker-max-slices on the local Python 3.14.6, which its own version table lists as unsupported. It is unrelated to this change.
   • Harness: byte-identical.
2. Read and write the execution overlay as its own file (632c266cb)
   • Red: 11 failed, for the intended reason (AttributeError: execution_overlay_bytes and load_execution_overlay didn't exist yet). The 2 commit-1 tests passed.
   • Green: 13 passed.
   • Harness: byte-identical.
3. Write the UK execution overlay beside the Orrery artifact (8d159a8c5)
   • Red: 4 failed, for the intended reason. The overlay file was missing (FileNotFoundError), or absent from the manifest, the evidence manifest or the failure bundle (KeyError). One guard passed (no overlay without recorded execution).
   • Green: 33 passed: both branch-changed test files in full, including the 15 existing finalize tests.
   • Harness: every existing file is byte-identical: graph.orrery.json, graph.schema.json, the evidence files, orrery.upload.json, the receipt, and everything in orrery-publication/. The only differences are the new graph.execution.json and exactly one new output entry in each manifest (orrery/graph.execution.json in the dataset manifest, graph.execution.json in the evidence manifest).

The new UK round-trip test runs a synthetic full build. Projecting graph.schema.json with the overlay read back from graph.execution.json must give bytes identical to the graph.orrery.json that finalize wrote, and the loaded overlay must equal collect_execution_evidence re-run against the store.

Findings and risks

• Overlay size.
  • Synthetic UK build (4 phases, 4 summaries): graph.execution.json is 121,145 bytes, beside a 348,283-byte graph.orrery.json (about 35%).
  • f10 fixture: not measured. No f10 build output exists on the machine this was made on, and no real build was run. The overlay is embedded whole in the Orrery artifact, which the plan measures at 24.4 MB for f10. The declaration-only graph is about 6.9 MB (Orrery export fails on full UK local builds: execution evidence exceeds the graph metadata bound #1290). So the f10 overlay should be at most about 17.5 MB, and it can never exceed 32 MiB, because collect_execution_evidence applies the schema's serialized bound.
  • Staging cost: registered outputs are staged, so a staged bundle carries the overlay bytes twice (once inside graph.orrery.json).
• Orrery export fails on full UK local builds: execution evidence exceeds the graph metadata bound #1290 still applies. At full UK local scale, collect_execution_evidence fails on that 32 MiB bound, so those builds produce neither an Orrery artifact nor an overlay file, before and after this PR. The collector-side size policy (O4) will need to decide what the overlay carries at that scale.
• Disclosure. The overlay carries every target's achieved value and weight extremes, but it is already embedded in graph.orrery.json, which is registered and staged the same way. The new file therefore adds no new disclosure, and it stays out of the public publication set.
• Package init behaviour. import microcosm.graph no longer imports the runtime modules. Any code that reached a submodule as an attribute (microcosm.graph.store.X) without importing it would now fail; I found no such use in the workspace. from microcosm.graph import store and from microcosm.graph import ContentStore both still work, and a guard test checks every __all__ name. Errors from a broken runtime module now surface on first use instead of at package import.
• Overlap with B4 (Reconstruct declaration-only Orrery publications for UK builds before #1144 #1291). B4 is adding declaration-only reconstruction as a new module. This PR doesn't touch orrery.py, and changes only two import lines in orrery_evidence.py. If Reconstruct declaration-only Orrery publications for UK builds before #1144 #1291 adds package exports, each one needs to go either in the eager imports (standard-library-only modules) or in _RUNTIME_MODULE_EXPORTS in microcosm/graph/__init__.py.
• Not done here.
  • No CLI flag. python -m microcosm.graph.orrery has no --execution PATH option; O3 can add one if the worker wants a command rather than the API.
  • Reference links aren't rebased. The overlay's native-evidence links are relative to the build directory, so a projection written elsewhere would need rebasing (documented).
  • Overlay written only after a successful projection. If the projection itself fails (for example over the 64 MiB output cap), no overlay is written. O5 may want the reverse.

🤖 Generated with Claude Code