[anth-volk]

Fixes #1291

Stack

main ← #1295 ← #1297 ← #1296 ← #1298 (GitHub stack #1328, linked on 2026-10-20 with gh stack link).

#1322 taken out of the stack (2026-10-20): CI runs only for pull requests into main on purpose, to limit the number of simultaneous runs on the organization's shared runners. #1322, which removed that filter, is closed, and the old stack #1323 was removed. 4ced66de0 on #1295 restores the four files #1322 changed to their versions on main, and each branch above merges the one below it: #1297 94a5bff93, #1296 b0406fd76, #1298 7cb857716. None of those merges had conflicts, and test_ci_integration_job.py passes on every branch (9 passed). No commit was rewritten. From now on only the bottom pull request, which targets main, gets CI; each one above gets CI once it becomes the bottom.

• Base: orrery-projection-without-store (Project Orrery artifacts without the content store, and write the execution overlay as its own file #1295).
• Merges: Project Orrery artifacts without the content store, and write the execution overlay as its own file #1295's branch is merged in three times, without conflicts: de4606dba (docs/orrery-adapter.md and docs/agent-guide.md merged cleanly, so the expected doc conflict with Project Orrery artifacts without the content store, and write the execution overlay as its own file #1295 is resolved), 17553f299 (Run the Tests workflow for pull requests into any base branch #1322's review fixes) and f2af7292e (main at a90eec3cb).
• New commits: three review fixes; see "Review fixes" below.
• Head: f2af7292e. Tests: this PR's two changed test files gave 68 passed at bb908fed6 and again at f2af7292e.

Why main was merged in (2026-10-20): GitHub builds each stacked pull request's test merge on top of the test merge of the pull request below it, starting from main: refs/pull/1322/merge has main as its first parent, and each later one has the previous one's. Once main gained #1305, #1296's change to rowwise_staging.py conflicted with it, so GitHub could not build #1296's test merge. It reported #1296 as conflicting although its branch contained its base, and started no CI for #1296 or #1298. main (a90eec3cb) was merged into #1322's branch, and each branch into the one above, with merge commits; #1296 is now mergeable.

Merge order: #1295, #1297, #1296, #1298, each after the one below it, with merge commits (or gh stack merge). Every branch above #1295 has its parent merged in rather than rebased, so no commit was rewritten. Squash-merging a lower pull request would leave its original commits in the branches above, which would then need rebasing.

Summary

Builds from before #1144 have no Orrery artifact, and the existing exporter can't give them one. They recorded no run binding, so the evidence validator refuses them. UK dense and national builds from that era did write their graph declaration (graph.json) and a code pin. This PR turns such a build's output directory into a declaration-only Orrery publication, marked as reconstructed. It works offline and uploads nothing.

This is a defined subset of #1170, tracked as sub-issue #1291: the declaration-only fallback for the build formats that recorded a declaration. Binding recovered receipts, timings and validation-check results to individual operations stays under #1170.

uv run python -m microcosm.build.uk_runtime.orrery_reconstruction \
  --build path/to/past-build \
  --output path/to/reconstructed-publication \
  [--revision <full git commit>]

The command writes a new directory, outside the build directory, in the same layout as the preserved publication directory that finalize_graph writes:

File	Contents
graph.schema.json	The recorded declaration's compiler schema, with the marker
graph.orrery.json	The Orrery document
orrery.upload.json	The inventory, in the format the emitter recomputes before it queues a directory
orrery.reconstruction.json	The summary report, outside the inventory

The command also prints a short summary. The CLI is documented in docs/orrery-adapter.md ("Reconstructed declarations").

Commits

Local runs were limited to the test files this branch adds, at the user's request. CI runs the full suite on this PR.

1. Mark declaration-only graph schemas as reconstructed (aee565df7). Adds the country-neutral microcosm.graph.reconstruction (marker, validation, reconstructed_schema), tools/orrery-contract/verify-reconstruction.mjs, docs and a changelog fragment.
   • Red: 21 failed, all at the stub's NotImplementedError (no import or fixture errors).
   • Green: 21 passed (test_graph_reconstruction.py, including Orrery's public parser through the new script).
2. Reconstruct Orrery publications for UK builds before run bindings (82ae80d35). Adds microcosm.build.uk_runtime.orrery_reconstruction and its CLI, docs and a changelog fragment.
   • Red: 18 failed, all at the stub's NotImplementedError.
   • Green: 18 passed (test_uk_orrery_reconstruction.py).

Review fixes:

3. Check a stale completion digest against the digest list (96abc1ec8). Dense builds closed by Register one UK full-build graph with all-geography calibration #901's driver are no longer refused (details under "Recorded digests"). The fixture now replays the drivers' write sequence in three layouts checked against the history with git show: the Register one UK full-build graph with all-geography calibration #901 window, the refreshed dense layout, and the national completion record.
   • Red: 4 failed, 21 passed. Three were refused with "rowwise_candidate_manifest.json does not match the digest recorded in build.json" (staged, unstaged, and the CLI with --revision, exit 2). The national-layout test failed only on the new stale_recorded_digests report field: that layout already reconstructed. The guard tests passed before the fix.
   • Green: 46 passed (both added test files).
4. Refuse a recorded declaration that is not a graph (5d44bc7e4). reconstructed_schema turns the serializer's TypeError (a top-level array or string, or a nested value of the wrong type) into a ValueError. The CLI therefore refuses with exit 2 instead of a traceback and exit 1.
   • Red: 4 failed, 46 passed, all at the uncaught TypeError. In the CLI case it escaped main.
   • Green: 50 passed.
5. Refuse an output directory inside the build directory (fc3e2327e). The output is resolved, and each existing ancestor is compared with the build directory by os.path.samefile. An output equal to the build or beneath it, including through .. or a link, is refused before anything is written.
   • Red: 4 failed, 50 passed. Output equal to the build was refused only as "already exists". The nested, .. and link cases wrote into the build.
   • Green: 54 passed.
6. Record the code revision that performed the reconstruction (14445a65e). The marker gains reconstruction_revision next to source_revision, and the docs, changelog and this description now say which parts come from the build (details under "Declaration resolution").
   • Red: 9 failed, 54 passed, against a stub that accepted the new parameter. Six were in the marker module (five KeyError: 'reconstruction_revision', plus code_revision's NotImplementedError) and three in the UK command (no revision lookup, no report field, no summary line).
   • Green: 63 passed.

Checks on the final tree:

• Both added test files: 63 passed (31 + 32).
• Lint: ruff check and ruff format --check on the changed files pass, and git diff --check is clean.
• Test layout: unchanged by the review fixes. tools/ci_test_plan.py verify and tools/graph_acceptance_burndown.py --verify pass.

Other local checks:

• The CLI works through python -m on a synthetic build, and a second run into an existing output is refused with exit status 2.
• Orrery 0.6.1's own CLI exports the reconstructed document to offline HTML.
• Run from another directory, code_revision reports this checkout's HEAD and its uncommitted state.

Before local runs were narrowed to the added test files, I also started the complete engine-free category, and both runs were stopped early:

• The first stopped at test_us_release_head_to_head_scorer.py::test_spawned_workers_score_identically_to_sequential_cli. That tool deliberately refuses --worker-max-slices on Python 3.14.6, which the local environment runs (3.14.7 is required), so the failure is unrelated to this branch.
• The second, with that test deselected, was stopped at 81% with no failures.

Neither run counts as a result for this PR.

Review fixes (2026-10-20)

Codex's $review-code review of 14445a65e found two bugs, and a later review of 2b020992f found a third. Each fix is its own commit, with tests written to fail first.

5b4da04e0 Read the reconstructing code's revision without inherited git variables

• Bug: microcosm.graph.reconstruction.code_revision ran git with the whole process environment. An inherited GIT_DIR, GIT_WORK_TREE, GIT_INDEX_FILE or GIT_COMMON_DIR, exported by an outer git command, a hook or a shell, overrides git -C, so git read another checkout. The marker then recorded that checkout's commit as reconstruction_revision, as clean. A file only the right checkout tracks came back as an unknown revision.
• Fix: git runs without the variables the installed git lists as local to one repository (git rev-parse --local-env-vars), the same list microcosm.frame.adapters.axiom strips for its checkout check. If git can't list them, the revision is unknown. That adapter's private helper isn't imported, because the adapter is a heavy module and reconstruction stays light; the duplicated lines could later move to a shared module.
• Test: test_code_revision_ignores_git_variables_that_select_another_checkout sets up two checkouts and selects the other one through all four variables. It checks a file both checkouts track by the same name, and a file only the reconstructing checkout tracks.
• Red: 1 failed, for the intended reason: the other checkout's commit was returned. The existing code_revision test passed.
• Green: 2 passed. test_graph_reconstruction.py gave 32 passed.
• docs/orrery-adapter.md and the 1291-orrery-reconstruction-marker fragment now say this.

2b020992f Check and write the same resolved output path

• Bug: reconstruct_uk_publication checked the resolved --output against the build directory, but wrote to the path as given. build/missing/../../publication names a folder beside the build, so the check accepted it. mkdir(parents=True) then created build/missing inside the build, which the command must only read. With a read-only build directory that mkdir failed, so the build couldn't be reconstructed at all.
• Fix: the output is resolved once, and that path is used for the check and for every write. An output that already exists at the requested path, such as a dangling link, is still refused rather than followed.
• Test: test_output_leaving_a_missing_folder_inside_the_build_creates_nothing_there, for a writable and a read-only build (skipped under root, which ignores directory permissions). Every file and folder in the build must be unchanged, and the publication must appear beside the build.
• Red: 2 failed, for the intended reasons. With the writable build, a missing folder appeared inside it; with the read-only build, PermissionError was raised on creating build/missing. The 4 existing tests for outputs inside the build passed.
• Green: 2 passed, along with those 4 and the existing-output test. Both changed test files gave 66 passed.
• docs/orrery-adapter.md says the command writes to the resolved path.

bb908fed6 Print the resolved output path the reconstruction wrote

• Bug (minor): main passed the --output spelling to summary_text, but the publication is written at the resolved path, which that spelling need not reach. For --output <build>/missing/../../cli_pub, the printed "Wrote … to" and report: paths did not exist, because missing cannot be entered.
• Fix: main resolves the output before the write, as reconstruct_uk_publication does, and prints that path. It still passes the given spelling, so the checks on it run.
• Tests:
  • test_cli_prints_the_resolved_output_path_it_wrote: for <build>/missing/../../cli_publication, both printed paths are the resolved ones, the given spelling is never printed, and the report exists at the printed path.
  • test_output_named_by_a_dangling_link_leading_outside_the_build_is_refused: the given name exists as a symbolic link, but the path it resolves to, outside the build, does not. It is refused as already existing, and nothing is written where the link points.
• Red: 1 failed, 1 passed. The printed-path test failed for the intended reason: the printed path was …/build/missing/../../cli_publication. The dangling-link test passed, because 2b020992f already checks the given spelling as well as the resolved path; it is a coverage test. With that check removed temporarily, it failed with DID NOT RAISE.
• Green: 2 passed. Both changed test files gave 68 passed. ruff check and ruff format --check were clean on the changed files.
• docs/orrery-adapter.md says the output must not exist as given or once resolved, and that the command prints the resolved path.
• The git environment helper stays a copy. microcosm.graph.reconstruction._git_environment duplicates the helper in microcosm.frame.adapters.axiom. Sharing it isn't worth it here:
  • importing the adapter would load numpy, pandas and microcosm.frame's bundle code into a module that imports only the standard library and microcosm.graph;
  • a shared module would mean editing microcosm.frame.adapters.axiom in this PR, and the two helpers handle failure differently: the adapter raises ValueError with a message about its checkout, after a 120-second timeout, while reconstruction returns None so that the revision is recorded as unknown;
  • whether the reconstruction tool moves to the collector is still open, and a shared helper in Microcosm would then have to be split again.

Declaration resolution

Decision: use the declaration file the build itself recorded (graph.json), with the code revision the build recorded. The declaration is not rebuilt by running the UK graph builders at the current revision. Only its bytes come from the build, though:

• From the build: the schema's graph section and graph_sha256, which reproduce the recorded file byte for byte.
• From the reconstructing code: the schema's compiled, fields and input_bindings sections, and the Orrery projection. A reconstruction made at another revision can differ in those parts. The marker therefore records that revision as reconstruction_revision, apart from the build's source_revision.

Details:

• Why the file still parses: the declaration format (serialize.py, decl.py) is unchanged from the UK graph driver (4bf7af4, 2026-09-25) to main. Since then, validation has only tightened (non-blank descriptions, expansion claims) and the schema string bound has loosened. A recorded file therefore reproduces byte for byte, and reconstructed_schema requires that: the schema's graph_sha256 must equal the file's SHA-256. A declaration that today's validator rejects, or a file that is not a graph at all, is refused, not reinterpreted.
• Source revision: the revision comes from rowwise_candidate_manifest.json (git_commit and git_dirty, which must agree with identity.code). build.json and sha256sums.txt bind that file by digest, and the command checks it against them. --revision must match the recorded commit. When the build recorded none, --revision supplies the commit and the marker records it as caller_asserted. With neither, the command refuses.
• Reconstruction revision: code_revision(path) reads the commit and working-tree state from the git checkout that tracks the reconstructing module's own file, not from the current directory. It returns nulls when no checkout tracks that file, so an installed copy inside an unrelated repository is never attributed to that repository. The report warns when the revision is unknown, or when the working tree is dirty or could not be checked.
• Refusals: a build that already records Orrery evidence (graph.schema.json or execution.evidence.json) is refused, with a pointer to python -m microcosm.graph.orrery. A directory without graph.json is refused, and the refusal names the build records that were found. An output inside the build directory is refused.

Alternatives considered:

1. Re-derive from a checkout of the recorded revision: run that revision's UK graph builders after checking that HEAD matches the pin. This is the only route for staged bundles, which never carried graph.json before Add country-neutral Orrery execution evidence and UK contracts #1144. Rejected for now: it needs that revision's environment and the build's exact arguments (posture, size, geography, spine checkpoint), and any gap would present a declaration that never ran.
2. A stored schema: none exists before Add country-neutral Orrery execution evidence and UK contracts #1144.
3. operations.json: it's an inventory only, and can't be projected.

Recorded digests

A file whose bytes disagree with a digest the build recorded for it is refused, with one exception, verified against the history with git show:

• Register one UK full-build graph with all-geography calibration #901's dense driver (430187f, merged 2026-09-29) recorded the candidate manifest's digest in build.json with the note staging receipts are appended after publication. Its close step, _close_attempt, then appended staging_delivery and staged_dataset to the manifest and refreshed only sha256sums.txt.
• Build the UK national release role on the graph (PR-2 of the #901 line) #1057 (merged 2026-09-30) added _refresh_completion (cb0d390), which reissues build.json over the final bytes and drops the note. The national graph path arrived in the same PR, so national builds from main always carry the refreshed layout.

A build.json digest with that exact note that no longer matches is skipped, and the manifest must match sha256sums.txt instead. The marker records this as fallback_digest_matched. A dense build from that window run with --no-staging or --no-staged-dataset wrote no digest list, so nothing the build recorded checks its manifest. The marker records that as only_stale_digest_recorded. In both cases the report lists the skipped digest, its note and what the file was checked against, and adds a warning. Any other mismatch is still refused.

Marker format

The marker is the graph-schema producer extension microcosm.reconstruction. It is saved in graph.schema.json, so any later re-projection of the saved schema keeps it (O4 and B2). Orrery documents show it at metadata.microcosm.extensions["microcosm.reconstruction"], and validate_reconstruction(schema) reads it.

{
  "protocol": "microcosm.graph.reconstruction.v1",
  "reconstructed": true,
  "basis": "declaration_only",
  "source_revision": {"git_commit": "<full lowercase commit>", "git_dirty": false, "origin": "recorded"},
  "reconstruction_revision": {"git_commit": "<full lowercase commit, or null>", "git_dirty": false},
  "inputs": [
    {"name": "build.json", "role": "completion_record", "sha256": "…", "bytes": 512, "verification": "no_recorded_digest"},
    {"name": "graph.json", "role": "declaration", "sha256": "<= graph_sha256>", "bytes": 4096, "verification": "no_recorded_digest"},
    {"name": "rowwise_candidate_manifest.json", "role": "code_pin", "sha256": "…", "bytes": 2048, "verification": "recorded_digest_matched"},
    {"name": "sha256sums.txt", "role": "digest_list", "sha256": "…", "bytes": 160, "verification": "no_recorded_digest"}
  ],
  "scope": "Rebuilt after the build from its recorded graph declaration, the only part taken from the build: …"
}

• Field rules: in source_revision, origin is recorded or caller_asserted, and git_dirty may be null. In reconstruction_revision, both fields are null when no checkout tracks the reconstructing code, and git_dirty alone is null when git could not check the working tree. Inputs are plain file names, listed in name order, and exactly one is the declaration.
• verification: recorded_digest_matched, fallback_digest_matched, only_stale_digest_recorded or no_recorded_digest (see "Recorded digests").
• No execution claims: the document carries no activities, artifacts, statuses or metadata.execution, so no operation appears as executed, cached or as having passed or failed a check.
• Publication ID: the inventory's ID is reconstructed- followed by 32 hex characters derived from the published files' digests. Reconstructing the same build twice from the same code revision gives identical bytes and the same ID.
• Protocol version: it stays v1 with the new field, since the protocol hasn't been released.

Overlap with O1 (#1294) and #1295

• No shared files with O1: this branch only adds modules. orrery.py, orrery_evidence.py and microcosm/graph/__init__.py are untouched. reconstruction.py imports only decl, schema and serialize, which O1 keeps light and eager, so it stays importable without the store.
• Imports O1 may move: the UK module imports orrery_json_from_schema from microcosm.graph.orrery and INVENTORY_NAME from orrery_publication. If O1 moves either, the fix is an import path.
• Doc overlap with Project Orrery artifacts without the content store, and write the execution overlay as its own file #1295: resolved. Project Orrery artifacts without the content store, and write the execution overlay as its own file #1295's branch is merged into this one with no conflict; see "Stack" above.

Risks and limits

• graph.json is not digest-bound by the build: no record from before Add country-neutral Orrery execution evidence and UK contracts #1144 binds it. A file overwritten after the build can't be detected. The marker records no_recorded_digest, and the report warns.
• Some candidate manifests are unchecked: a Register one UK full-build graph with all-geography calibration #901-window dense build run with staging disabled has no current digest of its manifest, which carries the source revision. The marker records only_stale_digest_recorded, and the report warns.
• National declarations lack the final readback: before Add country-neutral Orrery execution evidence and UK contracts #1144, the national graph.json was written before uk.full.national.readback was appended. The reconstruction shows exactly the recorded declaration, so that operation is absent.
• No real builds tested: real build directories from before Add country-neutral Orrery execution evidence and UK contracts #1144 were not available offline. Tests use fixtures that replay that era's write sequence, and the maintained UK dense declaration.
• Interim ID scheme: the publication ID scheme is local and interim, pending the I0 contract's service-owned deterministic IDs.
• Disclosure: a declaration-only document carries no target values or weights, only declarations (the manifests of validation checks included), the same as the existing declaration export.

Follow-ups

Out of scope for this PR:

• Deduplicate this module's JSON readers (_read_input, _json_object) with Project Orrery artifacts without the content store, and write the execution overlay as its own file #1295's overlay reader.
• TypedDicts for the marker and report shapes.

🤖 Generated with Claude Code