UK country simulations currently filter raw household data on country, but the deployed UK dataset stores region and the country model derives country from that input. The filter runs before model calculations and raises ValueError: Variable 'country' not found in household data.
This caused the Scotland population live test failure in API deployment run 37838389100.
Required selection rules:
- Scotland:
region == "SCOTLAND".
- Wales:
region == "WALES".
- Northern Ireland:
region == "NORTHERN_IRELAND".
- England: explicit membership in its nine English ITL1
region values.
Do not require a raw country column. The public identifier country/scotland is an area selector, not a column reference. Exclude unknown regions, preserve entities/weights, and support text and byte encodings. Add regression coverage that filters data with region and no country column. Put these rules in repository AI-facing guidance as well as user documentation, so future changes preserve them.
Implementation: #565. Expected patch release: 6.2.3 after current 6.2.2, provided no intervening release takes that version.
Coordinated follow-ups: temporary API test removal PolicyEngine/policyengine-api#3875 (issue PolicyEngine/policyengine-api#3874), simulation-worker dependency/deployment update PolicyEngine/policyengine-sim-api#733 (issue PolicyEngine/policyengine-sim-api#732), and API dependency update/test restoration PolicyEngine/policyengine-api#3877 (issue PolicyEngine/policyengine-api#3876). Both downstream packages must use the actual release containing the fix. A merged package PR alone does not repair deployed workers.
UK country simulations currently filter raw household data on
country, but the deployed UK dataset storesregionand the country model derivescountryfrom that input. The filter runs before model calculations and raisesValueError: Variable 'country' not found in household data.This caused the Scotland population live test failure in API deployment run 37838389100.
Required selection rules:
region == "SCOTLAND".region == "WALES".region == "NORTHERN_IRELAND".regionvalues.Do not require a raw
countrycolumn. The public identifiercountry/scotlandis an area selector, not a column reference. Exclude unknown regions, preserve entities/weights, and support text and byte encodings. Add regression coverage that filters data withregionand nocountrycolumn. Put these rules in repository AI-facing guidance as well as user documentation, so future changes preserve them.Implementation: #565. Expected patch release: 6.2.3 after current 6.2.2, provided no intervening release takes that version.
Coordinated follow-ups: temporary API test removal PolicyEngine/policyengine-api#3875 (issue PolicyEngine/policyengine-api#3874), simulation-worker dependency/deployment update PolicyEngine/policyengine-sim-api#733 (issue PolicyEngine/policyengine-sim-api#732), and API dependency update/test restoration PolicyEngine/policyengine-api#3877 (issue PolicyEngine/policyengine-api#3876). Both downstream packages must use the actual release containing the fix. A merged package PR alone does not repair deployed workers.