Scoped UK runs give the region UK-wide business rates and region-relative ranks #567

What happens

A UK simulation scoped by row filter (nation, constituency or local-authority regions, or any RowFilterStrategy/RegionGroupStrategy) runs policyengine-uk on the filtered households only. Several policyengine-uk formulas normalise over the households in the dataset. On a filtered dataset they normalise over the region instead of the UK, so the region's households receive UK-wide totals and region-relative ranks.

Measured on 2026-10-08/09 with policyengine-uk 2.102.3 on the certified enhanced_frs_2024_25_year_2026.h5 (baseline policy, 2026). The nation rows were re-run on main at 07bae75 (6.2.3, #565's region filters) and match the first measurement exactly. Each figure compares a scoped run with the same households in the national run:

£31.73bn is the UK business-rates total. The national run splits it £27.28bn / £2.37bn / £1.83bn / £0.25bn across the four nations; every scoped run puts all of it on the region.

Which variables (Scotland, 5,405 households)

State Pension, LHA rates, weights and the absolute poverty thresholds match. The reform-only incidence variables (corporate_incident_tax_revenue_change, consumer_incident_tax_revenue_change, employer_ni_response_*) are zero in the baseline, so this run does not test them.

Mechanism (policyengine-uk 2.102.3, read this session)

• shareholding = household corporate_sector_wealth / Σ(corporate_sector_wealth × household_weight) over the dataset. business_rates = shareholding × UK business-rates revenue (all four nations). corporate_sdlt = shareholding × UK corporate stamp-duty revenue. corporate_land_value divides by the same dataset total. On a filtered dataset the shares sum to 1 over the region.
• household_income_decile ranks equiv_hbai_household_net_income within the dataset, so scoped runs report within-region deciles.
• in_relative_poverty_bhc compares income with 60% of the dataset's person-weighted median, so scoped runs use the region's median, not the UK's.

Who is affected

• policyengine.py 6.2.3: since Filter UK country simulations by stored regions #565, nation-scoped runs no longer raise; they return the totals above. Constituency and local-authority regions built with include_constituencies / include_local_authorities produce them too.
• The simulation API: app-v2 → policyengine-api → simulation gateway → executor resolves UK nation regions through this registry. The executor still pins 6.2.1, where nation runs raise. Use PolicyEngine 6.2.3 for UK country region filtering policyengine-sim-api#733 (draft) moves it to 6.2.3, and Restore Scotland population test with PolicyEngine 6.2.3 policyengine-api#3877 restores the Scotland population test on it. Once those deploy, API nation results carry these totals unless this is fixed first.

Options

1. Run nationally, then filter outputs, for UK row-filter strategies. Scoped results equal the national run restricted to the region by construction, including under reforms. A UK national run takes seconds. Regional deciles become UK deciles, unlike US state runs, which keep filtering before the run.
2. Pass national values into the scoped run (for example shareholding from the unscoped baseline). This keeps within-region deciles. It cannot cover reform-dependent quantities such as the relative poverty median under a reform, or the reform-only incidence variables.

Option 1 changes what regional decile outputs mean, so it needs a decision.