Skip to content

Certify the Microcosm UK pair on policyengine-uk 2.162.14 (3c0d11b6 national, 6528c865 local area) - #725

Open
juaristi22 wants to merge 2 commits into
mainfrom
uk-certify-microcosm-pair-2162
Open

juaristi22 wants to merge 2 commits into
mainfrom
uk-certify-microcosm-pair-2162

Conversation

@juaristi22

Copy link
Copy Markdown
Contributor

@MaxGhenis, this head is ready for the UK hub's certification run. Could the hub pick it up? María will be around later this morning to decide whether the Budget dashboard moves to the release this produces.

This certifies the Microcosm UK pair built from PolicyEngine/microcosm#1411 for policyengine-uk 2.162.14, the first PyPI release that carries PolicyEngine/policyengine-uk#2695.

The pair

National: microcosm-uk-2024-25-national-20261028T061416Z-3c0d11b6 (microcosm_uk_2024_25).

  • Built with policyengine-uk 2.162.14, so it certifies on its own build (built_with_model_package).
  • The core stays 3.33.3, with core_binding: bundle_pin as on main.
  • In Microcosm, its release-cut certification passed 19 of 19 gates.

Local area: microcosm-uk-2024-25-dense-20261027T154955Z-6528c865 (microcosm_uk_2024_25_dense, 150,000 households, dataset_role: non_default_local_area).

  • Constituency and local-authority regions route to it (regional_release_kind: local_area).
  • It was built from b71ce793, the head of Settle property allowance elections without the charge where it cannot change them policyengine-uk#2695, and records policyengine-uk 2.162.12. 2.162.14 releases that package code unchanged: git diff b71ce793 2.162.14 -- policyengine_uk ':!policyengine_uk/tests' shows nothing.
  • It is routed on Allow a UK local-area release on an accepted model mismatch #699's accepted model mismatch (compatibility_basis: accepted_model_mismatch), with the audited input surface bound in the regional record:
    • audit_regional_input_surface.py against the build model 2.162.12 from PyPI (core 3.33.3);
    • no dropped column, 89 never-modelled columns, and no defaulted input;
    • auditor: juaristi22.
  • The recorded limitation says that the audit's baseline is the PyPI 2.162.12 release, which lacks #2695.

One spine. The two cuts' spines hold identical household, benefit-unit and person tables. Both were built from the same inputs, at microcosm db50ae8dd and daaf54302; the latter only re-pins the engine.

Impact

Commits

  • fc6e2119: the pair, certified by one certify-data run, plus the lock, release lock, TROs, the certified test constants and the changelog fragment.
  • 67566056: the household calculator snapshots, regenerated on 2.162.14 (PE_UPDATE_SNAPSHOTS=1). The UK households and the model surface move with the model; the US snapshots are unchanged.

How it was made

uv sync --frozen --extra uk --extra dev --python 3.13
uv pip install policyengine-uk==2.162.14
python scripts/audit_regional_input_surface.py \
  --regional-manifest-uri "hf://dataset/policyengine/populace-uk-private@microcosm-uk-2024-25-dense-20261027T154955Z-6528c865/releases/microcosm-uk-2024-25-dense/release_manifest.json" \
  --artifact microcosm_uk_2024_25_dense=<the cut's H5> --model-version 2.162.14 \
  --audited-by juaristi22 --limitation "<the limitation above>" --output regional_input_surface.json
python scripts/bundle.py certify-data --country uk --data-producer populace \
  --manifest-uri "hf://dataset/policyengine/populace-uk-private@microcosm-uk-2024-25-national-20261028T061416Z-3c0d11b6/releases/microcosm-uk-2024-25-national/release_manifest.json" \
  --regional-manifest-uri "hf://dataset/policyengine/populace-uk-private@microcosm-uk-2024-25-dense-20261027T154955Z-6528c865/releases/microcosm-uk-2024-25-dense/release_manifest.json" \
  --model-version 2.162.14 \
  --accept-regional-model-mismatch "<the reason in the regional record>" \
  --regional-input-surface-evidence regional_input_surface.json
uvx --from uv==0.13.4 uv lock --no-config --no-sources --default-index https://pypi.org/simple --python 3.13
uv sync --frozen --extra uk --extra dev --python 3.13
python scripts/release_lock.py
python scripts/bundle.py generate --include-tros
PE_UPDATE_SNAPSHOTS=1 python -m pytest tests/test_household_calculator_snapshot.py -q
python scripts/bundle.py check
python scripts/bundle.py check --include-tros --strict-tros
python scripts/check_uk_uc_floor.py --dataset microcosm_uk_2024_25 --year 2025 --output uc_floor.json

uk_qualification.rewrite_certified_constants rewrote the certified constants in tests/test_models.py, tests/test_release_manifests.py and tests/test_uk_regions.py.

Evidence run on this head

  • UC floor passed, Great Britain 2025, within 10%. The receipt holds weighted national totals only:
    • benefit units at 0.926 of DWP's figure;
    • spend at 0.958 of the OBR's.
    • 191 s, 18 GB.
  • Both bundle.py check runs pass, the second with --include-tros --strict-tros.
  • Locally: 139 tests on release manifests, UK regions, models, household snapshots and the UC floor, and 267 on the bundle, TRO and certification paths. The full suite runs in CI.

Open for the hub

#722 asks:

  • Q7: how long the local-area audit takes (here, 50 seconds);
  • Q1 and Q3: which steps must precede the merge, and whether independent steps can run in parallel.

On #720's timing, the release would land well after the Budget statement.

🤖 Generated with Claude Code

juaristi22 and others added 2 commits October 28, 2026 08:27
…ational, 6528c865 local area)

The national microcosm-uk-2024-25-national-20261028T061416Z-3c0d11b6 was
built with policyengine-uk 2.162.14 and certifies on its own build. The
local-area cut microcosm-uk-2024-25-dense-20261027T154955Z-6528c865 was
built from the head of PolicyEngine/policyengine-uk#2695 (b71ce793) and
records 2.162.12; 2.162.14 releases that package code unchanged, so its
constituency and local-authority regions route on #699's accepted model
mismatch, with the input-surface audit (auditor juaristi22: no dropped
column, no defaulted input) bound in the regional record. The UK model pin
moves from 2.151.0 to 2.162.14; the lock, the release lock, the TROs and the
certified test constants move with it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…2.14

PE_UPDATE_SNAPSHOTS=1 on the certified bundle. The UK households and the
model surface move with the model; the US snapshots are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Contributor Author

@MaxGhenis, for the UK hub. Here's what has already run on this head (67566056), so the hub can skip it, as it did on #720.

Already run here:

  • certify-data. The national certifies on its own build (policyengine-uk 2.162.14). The local area routes on Allow a UK local-area release on an accepted model mismatch #699's accepted mismatch, with the audit_regional_input_surface.py evidence bound in the regional record: no dropped column, no defaulted input, auditor juaristi22.
  • The relock, the release lock and the TROs (bundle.py generate --include-tros).
  • The household snapshots (PE_UPDATE_SNAPSHOTS=1).
  • The strict check (bundle.py check --include-tros --strict-tros) and bundle.py check.
  • The UC floor: passed. Great Britain, 2025: benefit units at 0.926 of the benchmark and spend at 0.958. These are weighted national totals.
  • Tests: locally, 139 tests on release manifests, regions, models, snapshots and the UC floor, plus 267 on the bundle, TRO and certification paths. CI is running the full suite on this head (8 of 12 passed so far, none failing).

Still to run, from the hub's evidence package: the national hash check, the 2×2 attribution, Budget inputs (with the uk-autumn-budget-dashboard-2026#124 list), the #574 reform contract, reform responsiveness, the attribution report and the evidence index.

A timing note before the long steps. On this cut, policyengine-uk 2.162.14 is far slower than 2.151.0, on which #720 ran:

policyengine-uk Build a simulation Income tax
2.151.0 20 s 1.9 s
2.162.12 (without PolicyEngine/policyengine-uk#2695) 127 s 120 s
2.162.14 61 s 54 s

#2695 halves the cost. But elections the ceiling leaves open are still compared in two branches of the whole simulation. The reform contract and responsiveness each run many full simulations, so on this head they will take much longer than on #720.

Should those two run here as specified? Or will the coordinator name a smaller set that must come before the merge, with the rest after it (#722 Q1)?

🤖 Generated with Claude Code

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant