[anth-volk]

Fixes #1293

Stack

main ← #1295 ← #1297 ← #1296 ← #1298 (GitHub stack #1328, linked on 2026-10-20 with gh stack link).

#1322 taken out of the stack (2026-10-20): CI runs only for pull requests into main on purpose, to limit the number of simultaneous runs on the organization's shared runners. #1322, which removed that filter, is closed, and the old stack #1323 was removed. 4ced66de0 on #1295 restores the four files #1322 changed to their versions on main, and each branch above merges the one below it: #1297 94a5bff93, #1296 b0406fd76, #1298 7cb857716. None of those merges had conflicts, and test_ci_integration_job.py passes on every branch (9 passed). No commit was rewritten. From now on only the bottom pull request, which targets main, gets CI; each one above gets CI once it becomes the bottom.

• Base: us-calibration-epoch-thinning (Thin US calibration epoch telemetry to the UK's rule #1296). It is at the top because it changes uv.lock and its pinned digest; none of the four pull requests below it changes the lock.
• Merges: eb13b6e4a merges Thin US calibration epoch telemetry to the UK's rule #1296's branch, 79c6b9ba4 and 15923a198 merge it again after review fixes below, and 531591bbb merges it after main (a90eec3cb) came up the stack. The first three had no conflicts; 531591bbb's is described under commit 4 below. Commit 5 regenerates the schema for main's model change.
• Head: 6df396734.
• Lock digest: APPROVED_UV_LOCK_SHA256 still equals the SHA-256 of uv.lock (751721b2…), checked at 15923a198 and 6df396734. main has not changed uv.lock since 41209b3, and uv lock --check passes.
• Tests after the last commit: this PR's seven changed test files gave 560 passed and 8 skipped.

Why main was merged in (2026-10-20): GitHub builds each stacked pull request's test merge on top of the test merge of the pull request below it, starting from main: refs/pull/1322/merge has main as its first parent, and each later one has the previous one's. Once main gained #1305, #1296's change to rowwise_staging.py conflicted with it, so GitHub could not build #1296's test merge. It reported #1296 as conflicting although its branch contained its base, and started no CI for #1296 or #1298. main (a90eec3cb) was merged into #1322's branch, and each branch into the one above, with merge commits; #1296 is now mergeable.

Merge order: #1295, #1297, #1296, #1298, each after the one below it, with merge commits (or gh stack merge). Every branch above #1295 has its parent merged in rather than rebased, so no commit was rewritten. Squash-merging a lower pull request would leave its original commits in the branches above, which would then need rebasing.

Summary

This PR publishes a Draft 2020-12 JSON Schema for calibration_diagnostics.json schema 8. It is generated from the pydantic models in microcosm-diagnostics, so the collector can check submitted diagnostics at intake without importing Microcosm. The diagnostics format doesn't change, and nothing is published outside the repository.

• Generator. microcosm.diagnostics.calibration_diagnostics_v8_json_schema() generates the schema from CalibrationDiagnosticsV8 in validation mode and adds $schema, $id, $comment, title and a description. tools/generate_calibration_diagnostics_json_schema.py writes the committed file, or checks it with --check.
• Committed schema. The file is committed at packages/microcosm-diagnostics/src/microcosm/diagnostics/schemas/calibration_diagnostics.v8.schema.json and ships in the microcosm-diagnostics wheel. load_calibration_diagnostics_v8_json_schema() reads it. The package README, the root README and the agent guide document the path and the regeneration command.
• Drift test. The committed file must be byte-identical to freshly generated output.
• Validation tests.
  • Real diagnostics written by existing engine-free US, UK and transport builders validate against the committed file.
  • Valid and representative invalid documents are classified the same way by the schema and the models.
  • Every rule the schema can't express has a document that the schema accepts and the models refuse.
• Dependency. jsonschema was already locked as a microcosm-build runtime dependency. It joins the microcosm-diagnostics dev group, so no new package enters the lock.

Commits

Local checks were limited to the test files this branch adds or modifies, at the user's request. The full suite, including the complete engine-free category and the country jobs, runs in CI on this PR.

On 2026-10-20 the branch was rebased onto main at 41209b3, after microcosm#1308 merged and made it conflict. Commits 1 and 2 are the original two, replayed; commit 3 is new. "Rebase onto main" below lists how each conflict was resolved.

1. 509e4da6e Generate a Draft 2020-12 JSON Schema for calibration diagnostics schema 8 (was bcf6d3dc9)
   • Red: 50 failed, for the intended reason. Each was AttributeError: module 'microcosm.diagnostics' has no attribute ..., naming the generator, gap register or renderer that didn't exist yet.
   • Green: 50 passed. The changed test files (test_json_schema.py, test_schema.py) gave 116 passed.
   • After the rebase: the same two files gave 117 passed. The extra test is main's test_uk_area_fit_still_parses_legacy_support_figures.
   • Checks:
     • ruff check . clean;
     • ruff format --check clean on the changed files;
     • tools/ci_test_plan.py verify ok;
     • uv lock --check ok.
   • Lock finding: Before the original commit, a full engine-free run saw the edited lock before it was re-pinned. 37 tests errored with "Primary-QRF worker identity found an unapproved uv.lock digest". After the re-pin, those 8 modules gave 168 passed and 1 failed: test_spawned_workers_score_identically_to_sequential_cli, where the tool's own version check refuses the local Python 3.14.6. That failure is environment-only.
2. b7cf8a256 Commit the schema-8 JSON Schema and validate US and UK diagnostics against it (was 8ad318481)
   • Red: 9 failed, for the intended reason.
     • The 6 producer tests ran their builders and then stopped on AttributeError for load_calibration_diagnostics_v8_json_schema.
     • The 2 path tests stopped on AttributeError for committed_calibration_diagnostics_v8_json_schema_path.
     • The tool test hit FileNotFoundError for the tool that didn't exist yet.
     • The 2 existing tests whose setup moved into helpers still passed.
   • Green: The 7 changed test files gave 520 passed and 8 skipped (the skips are existing engine-dependent skips). That includes the 9 new tests.
   • After the rebase, on its own: 531 passed, 8 skipped and 3 failed. All 3 failures come from main's UKAreaFit change, which commit 3 fixes: the byte-exact drift test, the generator tool's test, and test_final_gate_kernel_diagnostics_match_the_published_json_schema. The last of these validates the diagnostics written by UKFullGateKernel, which runs the UK build's full set of validation checks on the final weights.
   • Checks:
     • ruff check . clean;
     • ruff format --check clean on the changed files;
     • tools/ci_test_plan.py verify ok;
     • the microcosm-diagnostics wheel builds and passes check-wheel-contents==0.6.3, with the schema file inside.
3. c0549d21b Regenerate the schema-8 JSON Schema for UKAreaFit's withheld support figures (new)
   • Why: microcosm#1301 changed UKAreaFit on main. Released diagnostics withhold an area's support figures (nonzero_households, nonzero_source_households, effective_sample_size) and carry support_below_floor instead. The figure fields became optional, so earlier documents still parse. The committed schema still required the three figures and didn't declare the flag.
   • Change: the file is regenerated with tools/generate_calibration_diagnostics_json_schema.py. Only the UKAreaFit definition changes: support_below_floor is an optional boolean or null, and each figure is optional, null, or at least 0 as before.
   • New tests in test_json_schema.py check the published file the way consumers read it:
     • test_published_schema_accepts_uk_area_fits_with_the_floor_flag_or_legacy_support_figures, with four cases: the floor flag only, neither the flag nor the figures, legacy figures, and explicit nulls. The models must accept each one too.
     • test_published_schema_and_models_refuse_a_negative_legacy_support_figure.
   • Red, against the file before regeneration: 3 failed, for the intended reason. The stale schema required the three figures, refused null for them, and didn't declare support_below_floor. The 2 guards passed: legacy figures were accepted, and a negative figure was refused.
   • Green: 5 passed. The 7 changed test files gave 539 passed and 8 skipped, including the 3 that failed at commit 2.
   • Checks after committing:
     • ruff check . clean;
     • ruff format --check clean on the 14 changed Python files;
     • tools/ci_test_plan.py verify ok;
     • uv lock --check ok;
     • generate_calibration_diagnostics_json_schema.py --check reports the file current;
     • the wheel builds, contains microcosm/diagnostics/schemas/calibration_diagnostics.v8.schema.json, and passes check-wheel-contents==0.6.3.

Rebase onto main

These are mechanical conflict resolutions, so they have no red step; the tests above were re-run after each one.

• uv.lock: git merged it without conflict. Its only difference from main's lock is this PR's jsonschema entry in the microcosm-diagnostics dev group, and uv lock --check passes, so uv lock would write the same file.
• APPROVED_UV_LOCK_SHA256 in packages/microcosm-build/src/microcosm/build/us_runtime/worker_identity.py: re-pinned to 751721b231ab8cc6b3d10164f61ca429f5171661951494503fb0873e3f67b117, the SHA-256 of the new lock. main had e5d5818b…, and the original commit had 561e9ab1…. worker_identity._approved_uv_lock_sha256() accepts the new lock.
• packages/microcosm-diagnostics/tests/engine_free/shared/test_schema.py: commit 1 moves the schema-8 document builders to test_support/microcosm_diagnostics/payloads.py. main changed one of them: _uk_payload()'s weakest area now carries "support_below_floor": True in place of the three figures. That change is now in uk_schema_8_payload(). main's new test_uk_area_fit_still_parses_legacy_support_figures stays in test_schema.py and calls uk_schema_8_payload(). Apart from that change and this PR's docstrings, the moved builders equal main's.
• packages/microcosm-build/tests/engine_free/uk/test_uk_graph_terminal.py: commit 2 moves the body of test_final_gate_kernel_owns_complete_diagnostics_and_reuses_decoded_result into the helper _run_uk_full_gate_kernel (replaced by Place UK private rent and BRMA at the assigned area; bind Scotland's rents at BRMA grain (#1262, #1090) #1305's _run_final_gate_kernel in commit 4). main added seven tests and a helper just before it, and they are kept. main also gave that body an import of uk_terminal_only_code_hash and a "terminal_code" context entry; both are now in the helper.
• README.md, docs/agent-guide.md and test_uk_diagnostics.py: git merged them without conflict.

Merging main up the stack (2026-10-20)

15923a198 merged #1296 with #1322's and #1297's latest review fixes, without conflicts (no red step; the seven changed test files gave 547 passed and 8 skipped). Then:

4. 531591bbb Merge us-calibration-epoch-thinning, bringing in main at a90eec3cb
   • Conflict: test_uk_graph_terminal.py. This PR and Place UK private rent and BRMA at the assigned area; bind Scotland's rents at BRMA grain (#1262, #1090) #1305 each moved the body of test_final_gate_kernel_owns_complete_diagnostics_and_reuses_decoded_result into a helper. Place UK private rent and BRMA at the assigned area; bind Scotland's rents at BRMA grain (#1262, #1090) #1305's _run_final_gate_kernel also takes brma_row and solved, so it is kept, with this PR's docstring sentence on the forbidden diagnostics rebuild. _run_uk_full_gate_kernel is gone; this PR's schema test calls _run_final_gate_kernel, and Place UK private rent and BRMA at the assigned area; bind Scotland's rents at BRMA grain (#1262, #1090) #1305's two withheld-row tests follow it.
   • Red, at this commit: the seven changed test files gave 5 failed, 554 passed and 8 skipped, for the intended reason: Place UK private rent and BRMA at the assigned area; bind Scotland's rents at BRMA grain (#1262, #1090) #1305 also changed microcosm.diagnostics.schema, so the committed JSON Schema is stale. The failures are the byte-exact drift test and the generator tool's test, which found the file differs from the generated schema, and the two UK writer cases and the UKFullGateKernel case, which the stale schema refused because it does not accept brma as a geography level.
5. 6df396734 Regenerate the schema-8 JSON Schema for withheld model figures and BRMA
   • Why: Place UK private rent and BRMA at the assigned area; bind Scotland's rents at BRMA grain (#1262, #1090) #1305 added the brma level to UK geography pass rates and uk_diagnostics.model_figures_withheld (UKWithheldModelFigures) for rows whose model figures a UK build withholds.
   • Change: the file is regenerated with tools/generate_calibration_diagnostics_json_schema.py. Three definitions change: UKWithheldModelFigures is new, UKDiagnosticsV1 gains model_figures_withheld, and UKGeographyPassRate accepts brma.
   • New test case: test_final_gate_kernel_diagnostics_match_the_published_json_schema is parametrized to also run Place UK private rent and BRMA at the assigned area; bind Scotland's rents at BRMA grain (#1262, #1090) #1305's fixture with a withheld BRMA row, so a real producer covers both new parts.
   • Red, against the file before regeneration: both cases of that test failed.
   • Green: 2 passed. The seven changed test files gave 560 passed and 8 skipped.
   • Checks: ruff check and ruff format --check clean on the changed test file; generate_calibration_diagnostics_json_schema.py --check reports the file current; uv lock --check ok.

Producers now validated against the committed file, after strict JSON parsing:

• the US national release writer (_write_release_calibration_diagnostics);
• the ACS local-area calibrate stage;
• the UK writer, with and without skipped targets and consumption drift;
• UKFullGateKernel, which runs the UK build's full set of validation checks on the final weights, with local fit and a measured rotated holdout, and with a withheld BRMA row;
• the transport toy country's end-of-build diagnostics kernel (test_transport_terminal_kernels.py).

Schema path

packages/microcosm-diagnostics/src/microcosm/diagnostics/schemas/calibration_diagnostics.v8.schema.json. Its $id is calibration_diagnostics.v8.schema.json, relative, following spec_engine/schema/*.schema.json.

Expressiveness gaps

The schema's description lists each of these, from CALIBRATION_DIAGNOSTICS_V8_JSON_SCHEMA_GAPS. A test fails if a validator function appears in the schema-8 model tree without an entry. Another test fails if an entry has no document that the schema accepts and the models refuse.

Gap	Enforced by	Why the schema can't say it
Finite numbers at any depth, including inside options, metadata and build	DiagnosticsModel.reject_non_finite_numbers, CalibrationDiagnosticsV8.finite_optional_scalar	JSON has no NaN or Infinity, but Python's json accepts those tokens, and a JSON Schema number can't exclude what they decode to. Intake must parse strictly.
Strict integers	model config strict=True	JSON Schema counts 2.0 and 1e3 as integers. The strict models refuse them because Python decodes them as floats.
Non-blank text	_require_non_blank	Not generated by pydantic. Expressing it through annotations would make pydantic emit patternProperties for the dict-key uses, which would leave values under non-matching keys unconstrained, so it stays a documented gap.
Digest anchoring under Python validators	Field(pattern=...)	In ECMA-262, $ matches only at the end. Python's re also lets $ match before a final newline, so Python validators accept a digest followed by \n.
Hierarchy identity: category.provider_id == provider.id, unique dimension ids	CalibrationHierarchy.category_matches_provider	Cross-field equality or uniqueness by key.
hierarchy.target.id == target_name	TargetDiagnosticV8.hierarchy_target_matches_row	Cross-field equality.
Target-surface matrix shape	TargetSurface.reconcile_matrix_shape	Cross-field equality and arithmetic.
Top-level counts against targets, n_records, n_specs	CalibrationDiagnosticsV8.reconcile_counts	Array length compared with a field value.
UK block presence and top-level reconciliation	CalibrationDiagnosticsV8.reconcile_counts	Partly expressible with if/then, but pydantic doesn't generate it. The value reconciliation and the sums can't be expressed.
UK weight summary, zero-weight strata, geography pass rates, local fit, rotated holdout, consumption drift	the UK* model validators and UKDiagnosticsV1.reconcile_uk_diagnostics	Sums, ratios, conditional nulls and per-fold cross-references.

The schema is also stricter than the models' parser in one place. The models compare Literal values with Python equality, so their parser also accepts 1 for true and true for 1, for example in rotated_holdout.skipped. The schema requires the declared JSON value, which is what the canonical writer always emits. The description says so.

Risks

• Lock digest re-pin. uv.lock is pinned by digest in microcosm.build.us_runtime.worker_identity.APPROVED_UV_LOCK_SHA256, so this PR re-pins it, following 3623ca5 (UK: track the defects uk-data is fixing that microcosm still has #1095). The pin is now the digest of main's lock at 41209b3 plus this PR's jsonschema dev entry. Any other PR that changes the lock conflicts on both lines, and whichever merges second re-pins.
• pydantic upgrades. A pydantic upgrade can change the generated text even when the models don't change. The drift test then fails, and the fix is to regenerate.
• The models stay the authority. Passing the schema is necessary but not sufficient. The collector should still parse strictly, and the gaps above are where a document can pass the schema and still be invalid.
• Relative $id. I didn't invent a public URL; publication is out of scope. A consumer that registers the schema by $id gets the relative id.

Unverified

• The complete engine-free category and the US and UK engine jobs weren't run locally, at the user's request. They run in CI on c0549d21b.
• No published release's diagnostics were validated, because these tests don't touch Hugging Face. The real-diagnostics evidence comes from the engine-free test builders listed above.
• Only python-jsonschema 4.26.0 was exercised, for both the metaschema check and validation. An ECMA-262 validator such as AJV wasn't tried. AJV's default strict mode should accept the schema, because it uses only standard keywords plus title, description, $comment and default.
• Whether the collector wants a URL $id, or if/then rules for the UK block's presence, is left to K4.

🤖 Generated with Claude Code