Test the UK nation filters against policyengine-uk's country formula

[MaxGhenis]

Fixes #569

What this adds

Since #565 (6.2.3), the UK nation regions filter the stored household region column, and countries/uk/regions.py lists which regions make up each nation. policyengine-uk keeps its own mapping in the country formula, which derives country from region. tests/test_uk_regions.py checks the registry against _REGION_COUNTRIES, a hard-coded table in the test. Nothing checks either list against policyengine-uk.

tests/test_uk_nation_model_agreement.py takes the expected nations from policyengine-uk itself. A situation-based policyengine_uk.Simulation calculates country for every value of its Region enum, UNKNOWN included, in about 0.3s.

• Same nations: the registry's nations are exactly policyengine-uk's (apart from UNKNOWN).
• Exhaustive over Region: each nation filter keeps exactly the regions policyengine-uk places in that nation, and UNKNOWN falls in none.
• Property (Hypothesis, 100 examples): region-only data with 1–3 people per household, person, benefit-unit and household rows in separate orders, and str or bytes regions.
  • Each nation filter keeps exactly the households policyengine-uk assigns to that nation, once each, with all their people and their benefit units.
  • A nation with no households raises No households found.
  • The four nations are disjoint and together keep every household whose region policyengine-uk places in a nation.
• End to end: Simulation(..., scoping_strategy=country/<nation>).run() runs through the UK model for all four nations. The input has region and no country, and includes an UNKNOWN household, which every nation filter drops before the model runs. The output's country is the nation. The existing tests apply the strategies directly.

Scoping code is unchanged; it stays as #565 made it, filtering stored region with no model calculation.

What the tests catch

Mutation check: 14 mutants, each run against a copy.

• policyengine-uk's side. All four mutants fail this module and pass main's tests/test_uk_regions.py, because main's tests compare the registry with a hard-coded table rather than with the model:
  • a new Region value that the formula defaults to England;
  • Wales folded into England;
  • a new nation;
  • UNKNOWN mapped to England.
• The registry's side: dropping LONDON from England, or adding UNKNOWN to it, fails this module, as it fails main's tests.
• The filter cascade. Each of these fails this module:
  • benefit units left unfiltered, filtered by household id, or dropped;
  • a person dropped, or people filtered by position;
  • household rows duplicated;
  • bytes regions not matched.

Dropping the weights passes this module. Main's tests catch it.

Certified-data check of #565

#565 noted that the certified artifact was unvalidated. On enhanced_frs_2024_25_year_2026.h5, with main at 07bae75 and policyengine-uk 2.102.3, each nation filter keeps exactly the households policyengine-uk's country assigns to that nation:

region	households	weighted households
country/england	37,961	26,354,698.47
country/scotland	5,405	2,729,247.77
country/wales	3,821	1,590,754.53
country/northern_ireland	5,659	783,847.56

The four sets are disjoint and cover all 52,846 households (31,458,548.32 weighted). Simulation.run() for country/scotland returns the same 5,405 households and leaves the caller's tables unchanged.

Not fixed here

Scoped UK runs give the region all £31.73bn of UK business rates and region-relative deciles and poverty lines (#567). On 6.2.3, scoped Scotland household net income is £103.98bn, against £139.15bn for the same households in the national run. This reaches API nation results once PolicyEngine/policyengine-sim-api#733 deploys 6.2.3.

History of this PR

It first fixed #558 (nation regions raising on data without country) by having policyengine-uk calculate country on the unscoped dataset before filtering. #565 fixed the same failure while this was in review, filtering stored region, and its repository guidance rules out a preliminary model calculation. So this PR now carries only the tests. The earlier commits (1f6d02f, b1e138d, f12e77f) remain in this PR's history.

Tests: the new module (10 tests, about 4s), test_uk_regions, test_scoping_strategy, test_region_group_strategy, test_filtering, test_entity_utils, test_entity_utils_characterization and test_region pass locally on Python 3.13.

Review: an independent Opus review agreed the change follows the repository guidance (the model runs only inside the test, never in scoping). It asked for a stronger property, UNKNOWN kept in the end-to-end input, and an accurate commit message. All three are done in da51cf1.

axiom: n/a: tests of region scoping in the analysis wrapper, no policy encoded

🤖 Generated with Claude Code