Repository navigation
Conversation
- Rules and Data hub pages read the country and stop showing US-only
copy (federal/state/local, CPS/FRS/SCF) on the UK side; their links
keep ?country= and ?embed= like the sidebar does.
- The header's Model menu says Behavioural responses on the UK side.
- The overview's UK ingredient cards no longer say 'Over 55 federal,
state, and local programs', and the UK walkthrough no longer cites
the elasticity of taxable income.
- The walkthrough says its five households are illustrative, and its
table scrolls instead of clipping columns on narrow screens.
- Parameter source links fetch the YAML tree from the public prefix
(/{country}/model/param-tree-*.json); the root path 404s on
policyengine.org, so every link fell back to a directory.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
The UK explorers put HMRC and DWP parameters under 'Federal: IRS, HHS,
USDA, SSA, HUD' and council schemes under 'City and county programs'.
The UK metadata API serves no moduleName, so all 937 UK variables fell
into 'Household inputs' and had no source link.
- categoryUtils takes a country. UK levels are UK government (by
department), Devolved (Revenue Scotland, Social Security Scotland,
Scottish Government, Welsh Revenue Authority), Local authorities,
Reforms, Assumptions and settings, and Household inputs. US behaviour
is unchanged.
- fetch-metadata snapshots policyengine-{country}/variables at the same
pinned commit as the registry; the UK explorer maps each variable to
the one file named after it, so variables group by department and
link to their source.
- Property tests: no UK path lands in a US level or the reverse, and a
filled module is always a real file named after the variable.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UK coverage tracker showed one static sentence ('over 50 tax and
benefit programs') although the build already snapshots policyengine-
uk's programs.yaml (50 programmes). It now lists each programme from
that snapshot, grouped into UK government departments, devolved bodies
and local authorities, with the registry's status, coverage, verified
years and notes, and the same provenance line as the US tracker.
Source links are built only from paths the pinned snapshot trees
contain: 20 of the 48 registry-derived variable links and 21 test links
would 404 in policyengine-uk, so the UK view links the parameter folder
and the variable's own file, pinned to the snapshot commit.
If the UK registry cannot load, the page says so; the bundled fallback
registry is US-only and is no longer shown for the UK.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ease policyengine.org/uk/model/data/pipeline described an older build: FRS 2023-24 with ~20,000 households, WAS Round 7, LCFS 2021-22, SPI 2020-21, a 1.3x fuel uprating, PyTorch weight splits and two final weight matrices. policyengine.py 6.3.4 pins policyengine-uk-data 1.58.0 (enhanced_frs_2024_25), so the page now describes that release: - FRS 2024-25: 16,288 households, 34,966 people. - WAS Round 8, LCFS 2023-24, ETB 1977-2024, SPI 2022-23 imputations. - 10,000 zero-weight SPI copies and a capital gains copy of each household, plus 270 HMRC size-band donors: 52,846 households. - One Output Area per household in Great Britain (the publish run sets PE_UK_DATA_OA_CLONES=1); Northern Ireland's 5,660 households get none (policyengine-uk-data#580). - 650 x 52,846 constituency and 360 x 52,846 local authority weight matrices; the default file's weights are the constituency fit's column sums, scaled back to 2024. - policyengine.py serves constituencies and local authorities as row filters on the households' Output Area codes. scripts/extract_uk_release.py builds src/data/ukDataRelease.json from the release's own files (sha256-checked against the release manifest), its calibration logs, the target registry, and a run of the pinned model on the saved file (scripts/run_uk_target_check.py). The pipeline copy quotes that file, so a new pin is a re-extraction. The extractor refuses a release other than the pin, unlabelled target groups, unclaimed local metrics, repeated source households and a pinned check from another model version; a differential test compares the committed JSON with a fresh extraction. The stage rows scroll inside themselves on narrow screens, as in #49. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/data/calibration said calibration details were 'available for the US model only', and /data/validation told UK readers PolicyEngine checks its tax calculations against NBER's TAXSIM across federal and state income taxes. Both pages now describe the UK release policyengine.py 6.3.4 pins. Calibration targets: the 645 UK-wide targets in 66 families, with publisher, source link, forecast vintage, area (Great Britain where the registry scopes a target to it), 2025 value and fit at the last logged epoch; and the constituency and local authority target families with their fit, held-out metric and missing-source counts. Validation: policyengine-uk 2.126.12 runs the saved enhanced_frs_2024_25 for 2025-26 and uk-data 1.58.0's own target matrix scores it: 73.6% of the 644 non-zero targets within 10% (87.0% at the end of calibration with 2.122.2). The page shows the OBR March 2026 receipts and spending lines against the model, fit by family, the held-out HMRC employment counts, known issues (policyengine-uk-data#537, #570, #580, #581; policyengine-uk#1996), and every target on request. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UK behavioural responses page showed the CBO's US elasticity
presets, a CBO-based US budget-constraint chart, US spelling ('labor')
and a link to the US methodology post, and said PolicyEngine uses the
elasticity of taxable income. It now describes what policyengine-uk
2.126.12 (the version policyengine.py 6.3.4 pins) models:
- Static by default.
- Labour supply: income and substitution elasticities (default 0) on
employment income, each relative change capped at 100%, fed back
into the reform.
- The OBR-method module (Simulation.apply_dynamics): elasticities by
family type and youngest child from the OBR's note on the Autumn 2023
NICs cut, for the two eldest adults in each household, hours only,
reported alongside the reform.
- Capital gains: retention-rate elasticity (default 0), with an
optional 1.4 elasticity for BADR gains from the OBR's January 2025
costing.
A differential test checks the defaults and elasticities against the
policyengine-uk source when UK_MODEL_SOURCE is set.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The release scores two OBR lines against narrower variables: total National Insurance against employee NICs (policyengine-uk-data#537) and the 'Incapacity benefits' line against income-related ESA. The validation table no longer shows £50.5bn as PolicyEngine's total NICs; it compares OBR's total with the three classes' sum (£207.7bn, +3.8%), names the ESA comparison for what it is, and says each figure sums the variable the target registry assigns. The pipeline's calibration-table rows count fitted area targets instead of areas × measures. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pinned run rebuilds the targets with today's code, and eight Scottish council tax band targets come out at different values. The extractor now scores the pinned estimates against the values in the calibration log, and lists the revised targets. Within 5% moves from 51.9% to 52.2%; within 10% stays at 73.6%. The extractor also records what the 2026-10-14 fact-check found each family to be, from the release's code: - areas: OBR's DWP benefit lines cover Great Britain; domestic rates cover Northern Ireland; SDLT covers England and Northern Ireland; SLC targets and the tenure targets cover England; country-level families name their nation - labels: ESA, DLA and PIP, fuel duty, NI, tenure, VOA bands, regional age, UC family type and mortgage payments say what is scored - publishers: Scottish VOA rows come from gov.scot; Plan 2 and 5 borrowers from Explore Education Statistics; the mortgage total is set in the build - notes on how families are built, including the local Universal Credit targets, which loss.py assigns to areas by row order - the build's policyengine-core version, targets carried from an earlier year (165), and targets by area Fit shares keep six decimals, so 649/650 shows as 99.8%, not 99.9%. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… code - Call the 645 targets national and regional, by area, not UK-wide - Fit figures are from epoch 510, the last the build saves and logs - State the year fallback as the code has it, and how many targets use it - Note how each family is built or scored, under its label - Say what local fit does not show: policyengine.py's local analyses use Output Area households at national weights, and the UC targets are misassigned - Compare the three NI classes with their own OBR forecasts - Explain the scoring mismatches: DWP lines are GB but scored on UK households, DLA and PIP against PIP, council tax scope, fuel duty - Give the build's own reason for holding out the HMRC counts - Correct #582's text and the Northern Ireland issue, and list the confirmed issues awaiting a policyengine-uk-data number Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the 2026-10-14 fact-check, each point checked in the release's code: - wealth: no separate pension wealth (non-DB pensions sit in corporate wealth); HPI rescaling is an unweighted mean over owners, GB regions only - FRS: claim flags, not take-up decisions; council tax fills missing or negative reports by one adult or more; rental areas by bedroom need; property purchase is a 3.85% chance set in the build - spending: a fuel-car flag, with its NTS share attributed as the code does; energy at 24.67p and 5.74p per kWh, no standing charge - services: bus subsidy £3.55bn; ETB survey year 2023; NHS drops the cost table's 85+ rows, so spending at 80+ is understated - top incomes: 100,000 SPI records across the groups; second stage also imputes tax-free savings, then zeroes 14 benefit reports and restores 3; 28.0% covers the copies' descendants too - capital gains: the weight split it returns differs from the one it fits; the recipient's own income band; the spline extends past p95; donors let calibration fit HMRC's bands - student loans: no income test; Pension Credit in expectation, FRS reporters only - geography: Scottish Output Areas drawn with equal chances; four unitary authorities have no Output Areas - calibration: dropout averages log weights; priors halved; epoch 510; UC targets misassigned - final dataset: runtime uprating leaves a 2025 run 0.4% lighter than calibration; local analyses use national weights Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…le's feedback policyengine-uk 2.126.12 is not static by default: employment_income adds employer_ni_fixed_employer_cost_change (incidence 1) and the salary sacrifice cap's return-to-pay and 0.16% haircut (cap from 6 April 2029). The page now says so and lists the three parameters. apply_dynamics writes the response into the reform simulation, so later taxes and benefits include it; the page said the opposite. Also: - relative changes, with the formula's floors (£1 net income, 0.01 wage) - exclusions by employment status, as the code tests them - is_married's fallback to every couple - capital gains: measured marginal rate on two adults, the 0.001 floor, the parameter's own OBR citation, and the guard against both conventions The differential test now checks the three pay parameters and the employment_income components against the policyengine-uk source. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The UK overview no longer promises any reform's effect on every household; household data is FRS households plus the build's copies, weighted towards official statistics; behaviour is optional elasticities plus the default pay channels - The walkthrough's behavioural step treated a fixed per-child credit as a marginal rate cut. A lump sum has no substitution effect, so the step now applies an income elasticity (−0.05) to the credit's share of net income, for both countries. The UK steps say the households, schedule and weights are illustrative. - UK hubs: the coverage registry's status, not a claim of full coverage; weights calibrated towards, not matching, official statistics Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Clear all stale snapshots before resolving the source and record parameter tree provenance alongside variable tree provenance. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Build exact class-to-module maps from commit archives, including multiline declarations. Keep an offline file-tree fallback and show unknown modules as unclassified. Clarify UK government totals, household results, reform assumptions and dynamic settings while preserving US output. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Require repository and commit matches for each tree, and gate computation links on exact variable existence. Keep independent tree failures from hiding other verified links and cover invalid uppercase registry variables. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UK hub posted issues for the release faults the pages describe: - #468: local UC targets misassigned (already open) - #538: the 0.44% weight round trip (already open) - #582: ESA/incapacity, corrected - #583: DLA in the PIP target - #584: council tax scope - #585: the capital gains split - #586: NHS 85+ - #587: the OA crosswalk - #588: GB benefit lines scored on UK households The validation page's known issues, the calibration notes and the pipeline stages now cite them. Also drop "slightly" from the revised-target sentence: Scottish band H moves 8.7%. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 14, 2026
Describe commit-matched coverage links and unclassified variables, and give the archive fallback test a complete Console-compatible logger for type checking. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Accept the new parameter tree object and legacy arrays, with string-path validation. Pin exact US and UK source URLs and directory fallbacks in rendering regressions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the GPT-6.1 Sol review (14 findings, each checked against the release's code before changing anything): - Behavioural: the employer NI pay formula is meant to hold employer cost, assumes pay stays above the reform's threshold, and exempts pay below the baseline threshold; the 0.16% is a separate haircut; the capital gains retention rate is a ratio; the tax-rate convention uses a different driver - Validation: Universal Credit is the one DWP line scored on Great Britain - Calibration: notes for the land value constructions, the fixed independent-school count and the salary sacrifice headcounts; "Source" columns, the local one marked as the build's own citations - Pipeline: attributions that only the build's code makes are labelled as its own; a 90% chance, not 90% flagged; a nominal 100,000 SPI sample; borrower counts and take-up rate as the build stores them - Local age note: the bands across all areas add to 90% - US data hub copy restored exactly to master's - Walkthrough comment: the example models only the income effect Tests: - the behavioural differential now runs policyengine-uk's own elasticity functions on every gender, marital status and youngest-child age, and checks each displayed row (a swapped pair of values now fails); it also checks the pay-channel formulas - new source guards check the extractor's areas, notes and publishers against policyengine-uk-data's code (UK_DATA_SOURCE) - the pinned target check records the dataset's sha256, data version and year, and the extractor rejects any other file, package or year (re-run: all 645 rows identical to the first run) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Five commits from a Subfleet build lane (fact-check E, coverage tracker and metadata reconstruction): - computation-tree links only for variables in the pinned snapshot (programs.yaml's JSA, ESA and DLA name no class) - all three snapshots cleared before a refresh; the parameter tree records its source commit; links verify only against a matching commit - an exact class-to-module map from the pinned commit's archive puts CB_HITC, has_student_loan and the other unmatched classes in their departments; unknown modules show as unclassified, not household inputs - UK groups: totals across all levels of government, Reforms holding modelling assumptions some of them on by default, Household variables, Dynamic response settings Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
policyengine.py 6.5.0 through 6.9.0 pin policyengine-uk 2.130.1 and
policyengine-uk-data 1.58.1 (enhanced_frs_2024_25, sha256 056e80fd…). The
pages described 6.3.4's pin (2.126.12, 1.58.0).
- Re-extracted from the 1.58.1 release:
- all four files match the manifest;
- calibration logs from publish run 38519051399, epoch 510;
- the 1.58.1 registry;
- the target check re-run with policyengine-uk 2.130.1 and core 3.33.0.
- What moved: pinned fit within 10% is now 78.3% (was 73.6%) and within 5%
61.3% (was 52.2%); calibration fit is now 86.8% and 67.4% at epoch 510;
the SPI lineage share is 27.8%. No target value changes on rebuild. The
composition and weight factor are unchanged, so the runtime shortfall
stays 0.44%.
- Copy, from the 1.58.0 → 1.58.1 diff:
- salary sacrifice amounts are held to pay, the forest trains on
respondents with pay, and converted donors' contributions come out of
pay;
- the £2,000 headcount test uses contributions as simulated, no longer
deflated to 2023 prices.
- Copy, from policyengine.py 6.9.0's packaged area lists (650
constituencies, 361 local authorities): Northern Ireland's areas and the
four 2023 unitaries have no Output Area households, so analyses of them
stop with an error.
- policyengine-uk 2.126.12 → 2.130.1 changes no behavioural parameter,
formula or elasticity the page states. The page now takes policyengine.py's
version from the release, and a test fails whenever UK_MODEL_VERSION and
the pin differ.
- Review finding 5: bus trip rates are described as stored in the build,
attributed to the NTS by its code.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ble targets Upstream since 1.58.1: - policyengine-uk-data#468 (UC by row order) is fixed on main after this release: #589, merged 2026-10-19, after the 1.58.2 tag; - #537 (NI) changed on main after this release; - #570 (capital gains) addressed on main after this release; - policyengine-uk#1996 (VAT) addressed after 2.130.1. The known issues now say so; the pinned release still has each one. On 1.58.1, capital gains are £55.5bn against HMRC's £68.8bn and VAT £313bn against OBR's £180bn. New known issue #593: some non-zero targets have no household that contributes to them, so no weighting can meet them. The count comes from the release data: 22 at epoch 510 and 19 with the pinned model. Examples are the £1m+ income bands and ages 85 to 90. The calibration page states the count too; a test checks both counts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ed SPI targets From the GPT-6.1 Sol re-review at 841e4dc (REQUEST CHANGES): 1. Module map provenance (major) - The UK explorer's class-to-module map now comes from the policyengine-uk release the production API serves: the API version, its tag and the tag's commit. - It is written to public/variable-modules-uk.json with repo, version and commit. - fetchMetadata applies it only when the repository and version match the API response; otherwise variables stay unclassified. - Tests reject a stale version, the wrong repository, missing provenance or commit, an API without a version, and a list-shaped map. - The build now writes the 2.130.1 map: tag 6b33955 → commit 714b143, 1,163 classes. 2. Behavioural test labels (major) - The test reads each row's demographics from its displayed labels with a fixed vocabulary, checks them against row.covers, and requires every grid cell exactly once. - It checks that the rendered table shows each row's labels. - scripts/uk_pay_channel_cases.py runs the employer NI, haircut and returned-income formulas themselves. The test checks each behaviour the page states, including the threshold lifted past pay. - Relabelling a row, swapping age bands and overlapping coverage each now fail. 3. Extractor restatements (major) - restatement_mismatches() derives each figure, scope and publisher from policyengine-uk-data's code and checks the extractor's emitted notes, labels, areas and publishers against it. - Twelve mutations (a dropped or altered note, publisher or area) each fail it. 4. Filename-only links (minor) - A paths-only tree now verifies no variable links. The afip.py case, which declares armed_forces_independence_payment, is a test. Also verified on 1.58.1: - hmrc_spi.py's 2025 projections reuse the base-year targets' names, and registry.get_all_targets keeps the first of each name. So the 156 SPI band targets are fitted at their 2023-24 values; for example £174.0bn against a projected £183.2bn. Each SPI family now carries that note, guarded against the source. Reported to the UK hub. - The geography stage says how the build uprates to 2025: its model's load-time growth, with separate weight and fuel indices. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"Northern Ireland domestic rates receipts (Northern Ireland)" now reads "Northern Ireland domestic rates receipts". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The note on the six SPI income families now says the 2023-24 figures were fitted for 2025 in this release, and that policyengine-uk-data#608 fixed it after this release. #608 is batch 3, merged 2026-10-19; its commit a608ddba merges the ODS outturn and the projection into one target per band. 1.58.0, 1.58.1 and 1.58.2 have the bug. - UK_DATA_VERSION_DESCRIBED names the release the pipeline copy was checked against (1.58.1). A test fails when the pinned data version differs. UK_MODEL_VERSION does the same for the behavioural page. 1.59.1 changes the SPI and CGT base years, among other things, so a move to it must reread the copy, not just re-extract the counts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
|
Hi Max, #56 replaces this PR's UK pipeline and calibration files ( |
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Max (2026-10-14): "i also noticed problems on policyengine-model on the uk side". This PR audits every page at policyengine.org/uk/model and fixes what was wrong or stale.
Each page now describes what policyengine.py 6.9.0 pins for the UK:
enhanced_frs_2024_25, sha256056e80fd….The audit started against 6.3.4 (2.126.12, 1.58.0). On 2026-10-20 the pages were moved to the 6.9.0 pin and re-verified against all three diffs.
Every number traces to a release artifact or a real run, and every mechanism to code at those tags. Where only a code comment names a source, the page says the build attributes it.
Five independent fact-checks (Subfleet, one per area) checked the pages against the sources, and their corrections are applied after verification in code. Two GPT-6.1 Sol reviews followed, and their 19 findings are addressed. A third reviews the final head.
What was wrong, by page
programs.yaml, grouped into UK departments, devolved bodies and local authorities. Source, parameter and computation-tree links appear only when the snapshot from the registry's own commit proves the file and class./param-tree-*.jsonfetched from the site root, which 404s on policyengine.orgUS-visible changes, all deliberate:
PR #49's files are untouched, except
DataPipeline.tsx(see merge order).Release faults found and filed (policyengine-uk-data, by the UK hub)
The pages also cite:
Sources
scripts/extract_uk_release.pybuildssrc/data/ukDataRelease.jsonandsrc/data/ukTargetChecks.json, and the pages quote those files. The extractor:scripts/run_uk_target_check.pyruns policyengine-uk 2.130.1 on the released file for 2025, scored by uk-data 1.58.1's owncreate_target_matrix.scripts/uk_obr_elasticity_grid.pyandscripts/uk_pay_channel_cases.pyrun policyengine-uk's own behavioural functions on made-up people.Invariants (tested)
UK_RELEASE_ARTIFACTS).UK_DATA_SOURCE). Twelve mutations each fail it.UK_MODEL_VERSIONmust equal the pin.Verification
UK_MODEL_SOURCE=…/pe-uk-2.130.1 bun run test: 265 passed.bun run lint: clean.bun run build: passes.UK_RELEASE_ARTIFACTS=… UK_DATA_SOURCE=…/uk-data-1.58.1 pytest scripts/test_extract_uk_release.py: 30 passed.Reviews
All in
~/reviews/pe-model-uk-audit-2026-10-14/:factcheck/out/;reviews/pr51-review-427c1ec.mdandreviews/pr51-review-841e4dc.md, with the responses in the commits after each;reviews/pr51-review-e051638.md.Merge order
src/components/data/DataPipeline.tsx. Whichever lands second keeps both sides: Rewrite the US data pipeline and calibration pages for the Microcosm default release #49's US intro andhasBranches, and this PR'sukPipelineIntro,ukBranchLabelsand overflow rows.axiom: n/a: site copy and data extraction only; no policy logic changed
🤖 Generated with Claude Code