Skip to content

Build the 117th->119th CD crosswalk from the plan registry (fixes North Carolina) - #1043

Merged
MaxGhenis merged 3 commits into
mainfrom
us-cd-crosswalk-from-registry
Sep 28, 2026
Merged

MaxGhenis merged 3 commits into
mainfrom
us-cd-crosswalk-from-registry

Conversation

@MaxGhenis

Copy link
Copy Markdown
Contributor

Stacked on #1041 (the CD plan registry). It needs #1041 merged first; GitHub retargets it to main when that branch is deleted.

The bug

The packaged 117th→119th congressional-district crosswalk (us_runtime/data/congressional_district_vintage_crosswalk.csv) read the 117th plan from the 2020 Block Assignment Files' CD layer. That layer carries the 116th plans. The docs said the 116th and 117th plans are identical, which is false for North Carolina.

  • Census reports North Carolina as the only state whose districts changed between the 116th and 117th Congress. Its 117th districts are the 2019 remedial plan (HB 1029, S.L. 2019-249).
  • On 2020 blocks, NC's 2016 and 2019 plans put only 55% of the state's population in the same district.
  • So SOI's 117th-plan NC district targets were translated to the 119th districts through the wrong plan. The ACS district-calibration session measured the effect independently: 37% of NC's population lands in a different 119th district.

The fix

tools/build_us_congressional_district_vintage_crosswalk.py now builds from the pinned registry (load_pinned_us_cd_plan_registry). It overlays the registry's 117th_congress and 119th_congress plans on the same 2020 blocks, weighted by the registry's block populations, using the unchanged build_cd_vintage_crosswalk_rows.

  • Only North Carolina's rows change (1,444 → 1,449 rows); every other state is identical. Coverage and conservation are unchanged: 436 → 436 districts, zero unmatched population.
  • NC's 117th source-district populations now match the General Assembly's official 2020 figures for the 2019 plan within 0.09%. The old rows missed by up to 7.6% (district 1), with 7 of 13 districts beyond 0.5%.
  • New pins. The crosswalk SHA-256 is a347303f… (78,202 bytes). It is re-pinned in us/spec/sources.yaml, tools/build_us_multispine_pool.py, test_support/…/us_multispine_pool_tool.py and two tests. The provenance now records the registry SHA-256 and its 117th known deviations instead of the raw Census file list; the registry's receipt carries every underlying source.
  • Docs. The crosswalk module docstring, loader docstring and CONGRESSIONAL_DISTRICT_VINTAGE_CROSSWALK.md now describe the registry-based method and drop the "identical plans" claim.

Invariants and tests

The crosswalk keeps its existing guarantees: weights sum to 1 per source district, state and national totals are conserved exactly, and 436 districts appear on each side. The packaged-crosswalk validator enforces them and still passes. New tests:

  • The crosswalk's registry SHA-256 equals the registry receipt's pinned SHA-256, and its known deviations are the registry's 117th deviations (NC only).
  • NC's 117th source-district totals are within 0.5% of the official NCGA 2020 figures. The previous CSV fails this.
  • Differential (opt-in, MICROCOSM_US_CD_PLAN_REGISTRY): rebuilding from the real registry reproduces the packaged CSV's pair populations exactly. It passed locally.

Locally, 1,674 tests pass (10 skipped) across every engine-free test file that references the crosswalk or the US spec sources.

Effect

This changes the NC district inputs of future builds that translate SOI 117th CD facts. It publishes nothing and does not touch any release in flight, since those run from pinned worktrees. The design report of 2026-09-27 also found that the flagship release currently drops all district-level targets.

axiom: n/a: geography data infrastructure, no policy rules

🤖 Generated with Claude Code

The packaged crosswalk read the 117th plan from the 2020 BAF CD layer,
which carries the 116th plans. Census reports North Carolina as the only
state whose districts changed between the 116th and 117th Congress, so
North Carolina's SOI districts were mapped through its 2016 plan; the
two plans put only 55% of the state's population in the same district.

The builder now overlays the pinned registry's 117th_congress and
119th_congress plans (registry population weights). Only North Carolina's
rows change (1,444 -> 1,449 rows); every other state is identical. NC's
117th source-district populations now match the General Assembly's
official 2020 figures within 0.09%; the old rows missed by up to 7.6%.

Repins the crosswalk (sha256 a347303f..., 78,202 bytes) in sources.yaml,
build_us_multispine_pool.py and tests, and corrects the docs that called
the 116th and 117th plans identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis changed the base branch from us-cd-plan-registry to main September 28, 2026 14:16
…y NPZ

Review nits on #1043:
- Add an unconditional test on the packaged CSV: no pair crosses a state
  line, per-state totals equal the build receipt's state_conservation (all
  51, nothing unmatched), the national total equals the registry's
  331,449,281, and five states match their 2020 Census counts.
- Correct the NC regression comment: main's old crosswalk misses 10 of 13
  districts beyond 0.5% (worst 7.55%, district 1), not 7 of 13. Recomputed
  against the test's own NCGA reference; the new crosswalk's worst is 0.09%.

The exact per-pair rebuild test also passes locally against the pinned
registry NPZ (sha256 6b16fc21...).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Retargeted to main after #1041 merged (8d18463). Independent review (subfleet 20260928-101703-pr1043-review): APPROVE, no blocker or should-fix. It independently parsed the CSV (1,449 rows, 436 source and target districts, 331,449,281 people; only NC changes) and checked NC's 2019 plan against the IRS and NCGA sources. Its two nits are addressed in dbf3d90: an unconditional national and per-state conservation test on the packaged CSV, and a corrected regression comment (10 of 13 districts beyond 0.5%, verified). The skipped exact-rebuild test passes locally against the pinned registry NPZ (6b16fc21...). This will merge after #1046 so Route A's relaunch keeps its pinned crosswalk.

🤖 Generated with Claude Code

engine-us CI failed test_us_coverage_is_exact_complete_and_honest because the
crosswalk's new sha256 and byte size in us/spec/sources.yaml (the only spec
change in this PR) move three attested identities:
- EXPECTED_HASHES["full_checkpoint"] b7a336af... -> 3db095ce...
- EXPECTED_HASHES["geography_assignment"] f49425ca... -> 44e7f5f6...
- the US spec_sha256 pinned in test_us_multispine_pool_tool.py
  1cf1f44d... -> ffbb93ed...

The new values were computed from build_inventory_coverage on this tree
(only these two inventory items failed). docs/evidence/spec-engine/
us-f0-coverage.json was regenerated with tools/spec_engine_coverage.py; its
--check passes with 41/41 inventory checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Merge audit: gh pr checks 1043 exits 0 on the latest run at head 67b39b2; mergeable MERGEABLE; no CHANGES_REQUESTED. Independent review: subfleet 20260928-101703-pr1043-review, APPROVE at d8d96d9. Commits since then:

  • dbf3d90 addresses that review's two nits: an unconditional conservation test, and a corrected NC comment verified at 10 of 13 districts, worst 7.55%.
  • 67b39b2 re-pins the spec-engine digests that the new crosswalk sha legitimately moves (full_checkpoint, geography_assignment, and the multispine test's spec_sha256). The digests were computed from build_inventory_coverage on this tree; docs/evidence/spec-engine/us-f0-coverage.json was regenerated by tools/spec_engine_coverage.py, and its --check passes with 41/41 inventory checks.

The exact per-pair rebuild test also passed locally against the pinned registry NPZ (6b16fc21…). Route A's in-flight release is pinned to 4b57d15 and keeps the previous crosswalk; this fix reaches the next build.

🤖 Generated with Claude Code

@MaxGhenis
MaxGhenis merged commit bda72cb into main Sep 28, 2026
11 checks passed
@MaxGhenis
MaxGhenis deleted the us-cd-crosswalk-from-registry branch September 28, 2026 19:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant