Repository navigation
Update corporate SDLT and business rates statistics and project them with OBR receipts - #2322
Conversation
…with OBR receipts Corporate SDLT (HMRC buyer-type tables) now runs to 2024-25 and business rates to 2025-26 (Wales 2024-25); both grow with OBR March 2026 receipts forecasts by property type / nation. NI business rates were total rates (domestic and non-domestic); they are now LPS non-domestic rates paid. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tions - test_corporate_receipts_statistics: outturn equals the published tables, growth equals ratios of the OBR's published receipts, projections track them, shares within each line are invariant (Hypothesis), and the microsimulation allocates each statistic once (and holds it at the latest outturn under no_economic_assumptions). - corporate_sdlt.yaml: expectations for the 2026-27 statistic (£5,982.55 per 1e-6 shareholding), derived from the HMRC and OBR cells. - programs.yaml: business rates verified to 2030. - fiscal_partition.yaml: data notes on the business rates head (includes NI, which the OBR records under other taxes) and the household SDLT head (calibrated to the OBR total; policyengine-uk-data#564). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s; address review - England central list (Main Non-Domestic Rating Accounts) and Wales central list (StatsWales), which councils' figures exclude; GB statistics now sum to within 1% of the OBR's Business rates line in 2024-25 and 2025-26. - England 2019-20 and 2020-21 from MHCLG's November 2025 revised data. - Document the Welsh Covid-year relief treatment, NI's 2020-21 start, the 2039 index horizon, the 2024 index anchor and the SDLT shares assumption. - Tests: central lists, NI = LPS total less domestic, GB coverage of the OBR line, flat after 2039, explicit no_economic_assumptions cutoff guard. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The aggregate impact report (#2335) adds gov_spending, household_benefits and spending_heads to fiscal_partition.yaml. The tax tests now check the three tax totals only, require that spending heads borrow no tax head id or variable and that no spending head names a tax total, and apply dropped_when to the tax totals alone. Checked against #2335's file with these tax sections merged in: this file's tests and #2335's partition and guard tests pass. The vat head records that 76 households have negative VAT on policyengine-uk-data 1.58.0 (consumption or the full-rate expenditure share is negative in the data; policyengine-uk-data issue 565). Refs #2159 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…R lines Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… rates statistics Comment-only (review round 2, findings 2 and 3). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
5f08c24 to
47eb1f7
Compare
#2235 merged main and fixed its partition tests before landing. One conflict, in the vat head's data note in fiscal_partition.yaml: keep #2235's wording, which reports only the household counts. This PR's own data notes on business_rates and sdlt_household are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#2235 landed on main as 0def167, so this PR now targets main. No conflicts. Main's changes since #2235's head touch no statistic, receipts growth series or index this PR reads: the yoy_growth edits are to consumer_price_index_ahc, mortgage_interest and social_rent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_weighted_incidence.py (new on main via #2235's merge) sets the four business rates series so corporate incidence is £150 times shareholding. This PR adds ENGLAND_CENTRAL_LIST and WALES_CENTRAL_LIST to the sum, so they kept their real values (about £2.21bn in 2025) and five cases failed with 2.2e7 against 1.5. Set both to zero with the others. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#2322 now carries #2235's final head and main (a45baf5). One conflict, in the economic assumptions README's EFO refresh checklist: main added step 6 (move PROJECTION_END_YEAR on) where this PR added step 6 (refresh the receipts series). Keep both: receipts refresh as step 6, projection horizon as step 7, then the importer example. The importer, its tests and the yoy_growth.yaml comments merge without conflict; main has not touched the importer or its tests since this PR branched. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Hand-off to the UK hub (build job Head:
Tests: these ran locally at the head, in the foreground with
Ruff is clean, and the trusted fix-evidence checker passes on the body. CI: at 09:50Z, when this job ended, CI at Impact. Recomputed from parameters at this head against main, the change in Open items
|
Aggregate impact on certified data: passBase Checks: 0 broken by this change, 1 failing on both sides, 0 fixed. Moves flagged for review: 210 (thresholds in Notes
Moves flagged for review
170 more flags are in the report artifact. Receipts by tax head
Spending by benefit and service
Caseloads
Government balance, each component counted once
Model totals
Household income totals
Mean household net income in each side's own deciles
Standard reform costings (change in gov_balance)
Standard reforms: change in each reform's target
Household net income by decileMean change in household net income per household, and as a share of the decile's base income; households stay in the base run's deciles.
No change in any year: population; poverty rates; people in poverty; inequality. Checks failing on both base and head (1)
Counted-once checkEach of 84 components was raised by £1 per unit in 2026-27, and every head of its totals was read household by household: its own head must rise by exactly £1, no other tax head may move, no other benefit may rise, and the model's own total must move by what its heads moved. A head that breaks a rule is measured again twice: with every other head held at its baseline value, and with every other head held at its value after the first £1 while the component rises again. It fails where either still shows the response. 0 failed; 0 were skipped. Provenance
Dataset: Receipts by tax head: head levels
Spending by benefit and service: head levels
Caseloads: head levels
Government balance, each component counted once: head levels
Model totals: head levels
Household income totals: head levels
Population: head levels
Poverty rates: head levels
People in poverty: head levels
Inequality: head levels
Mean household net income in each side's own deciles: head levels
Standard reform costings (change in gov_balance): head levels
Standard reforms: change in each reform's target: head levels
|
test_head_totals_agree_with_the_model_weighted_sums checked that the partition's heads plus declared double counts equal gov_tax within a relative 1e-9. The model sums gov_tax for each household in float32, policyengine-core's float type, while the measurement sums the heads in float64, so the two agree only to float32 rounding. With this PR's corporate SDLT and business rates statistics, the 2026 totals on policyengine-uk-data 1.58.1 differ by 1.75e-9 (948,404,896,828 against 948,404,895,169), and the check failed in CI (#2342's run 39915137960). Use report_spec.yaml's identity_relative_tolerance (1e-6). The spec documents it as the tolerance for accounting identities between weighted totals, set by float32 rounding; the measurement applies the same value to its per-household identity checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings the certified receipts identity's float32 tolerance from #2322, which fixes this branch's CI failure in run 39915137960 (Test / Python 3/5, test_head_totals_agree_with_the_model_weighted_sums). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Merged by the UK hub (Claude session 8e19f9d5) at head
|
Fixes #2320
Summary
corporate_sdlt,business_ratesandbaseline_business_ratesallocate receipt statistics to households byshareholding. Those statistics stopped in 2018-2021 and were held flat in every later year. This PR:In 2026-27:
gov_taxandhousehold_taxrise by £9.16bn, and household net income falls by the same amount. The rise is £4.63bn in 2025-26 and £17.14bn in 2030-31.Every figure comes from the 11 October real runs on policyengine-uk-data 1.58.0, shown below. Weighted
corporate_sdltandbusiness_ratesequal their statistics on any dataset, becauseshareholdingdivides each household's corporate wealth by the weighted total, so the national change is the same on Microcosm; only its distribution across households differs. Recomputed from the parameters at this head, it is unchanged in every year (see "Stack update").Known overlap. £2.85bn of the 2026-27 change is corporate SDLT, which overlaps household SDLT until policyengine-uk-data#564 lands; see "SDLT calibration overlap" below.
#2235 (corporate SDLT counted once) merged on 22 October, and this PR now targets
main. #2342, the EFO receipts importer, is stacked on it.Sources
Every value was read from the published file.
extract_values.pyregenerates them all with sheet and cell references. A separate blind extraction, which was not shown these values, read 666 cells from the same publications, and every value it covers matches. Where it found later revisions or extra series, they are adopted below. It did not cover the England central list. Its seven accounts were checked against the published PDFs: each account's prior-year column equals the previous account's figure, and the round-2 reviewer confirmed 2023-24 to 2025-26 on gov.uk.Checks.
What was wrong with the old values
business_ratescontained NI domestic rates, whichdomestic_ratesalready charges. LPS's total less domestic equals its non-domestic series in every year to within £1; a test enforces this.Projection
The new series sit in
gov.economic_assumptions.yoy_growth.obr:stamp_duty_land_tax.residentialandstamp_duty_land_tax.non_residential, from Table 3.7;business_rates.great_britain, from Table 3.8;business_rates.northern_ireland, from Table 3.2.Each value is the ratio of consecutive published levels, to six decimals:
average_earnings's long-run path, because the OBR forecasts receipts only to 2030-31.Each outturn statistic declares
uprating:, and core uprating scales its last outturn value by the index ratio:Why growth rates in
yoy_growthrather than levels elsewhere: the projection then behaves like the model's other OBR series. In particular,no_economic_assumptionsholds it at the latest outturn, which is tested.Caveats
2024-25 is outturn. 2025-26 is outturn except for Wales, which is projected. Later years are projected.
Data notes, recorded in the YAML
business_ratesallocates every list's receipts by shareholding. That includes ratepayers such as Network Rail (£10.5m) and Dwr Cymru (£17.0m) on the Welsh central list in 2024-25.programs.yamlnow verifies from 2020.Interactions checked
corporate_tax_incidence(=corporate_sdlt+business_rates).hbai_excluded_income(= −corporate_tax_incidence).baseline_business_ratesnow sums the six series;business_ratesshares its formula. All move exactly with the statistics: the microsimulation test checks weighted totals, and the real runs show identical changes. HBAI household net income doesn't change, so poverty doesn't either.data_notes:business_rates: GB £37.26bn against the OBR's £37.41bn in 2026-27. NI is £0.78bn against the OBR's £0.40bn NI line, which the OBR records under other taxes.sdlt_household: the calibration overlap, below.1.58.0,targets/sources/obr.pylocates the current-receipts sheet by title and maps the OBR's total "Stamp duty land tax" to householdstamp_duty_land_tax.corporate_sdltadds again.corporate_sdlt.yaml. Expectations move from £3,130 to £5,982.55 per 1e-6 of shareholding, derived in the file header from the HMRC and OBR cells. Margins are £0.50, because the model chains six-decimal growth into a five-decimal index and lands within £0.10.programs.yaml. Business rates is now verified 2020-2030, with notes on the method. It was 2019-2021, "Parameters need updating beyond 2021".Invariants (tests in
test_corporate_receipts_statistics.py)average_earningsgrowth through 2045.corporate_sdltequals the corporate SDLT statistic;business_ratesandbaseline_business_ratesequal the business rates statistic;corporate_tax_incidenceis their sum andhbai_excluded_incomeits negative.no_economic_assumptions, both statistics hold at the latest outturn. The test asserts the scenario's cutoff assumption explicitly.Tests run (11 October, before the stack update)
test_corporate_receipts_statistics.py: 141 passed.stamp_duty/andexcise_integrationYAML: 24 passed. Partition tests: 39 passed.test_sdlt_counted_once_properties.py,test_import_obr_forecasts.py,test_lagged_series.py,test_council_tax_uprating.py,test_threshold_indexation.pyandtest_fiscal_year_parameters.py: 180 passed.main, so it was dispatched manually.test_dataframe_dataset.py::test_an_exported_frame_loads_back_and_exports_itself. That test also fails onmain(run 38143054188, 7343f94); the UK hub is fixing it (Exported input DataFrame no longer round-trips: main CI fails test_an_exported_frame_loads_back_and_exports_itself #2333).National impact (real runs)
Setup.
enhanced_frs_2024_25.h5, sha2565ba399ea5ab9a6c7d4e9aaf6c623080beb34d9ad5f330f6244dbad72b8d960b6.Microsimulationruns at current law, 2 versions × 6 years. No scaling.corporate_sdlt, base → headbusiness_rates, base → headgov_tax,household_tax,pre_budget_change_household_tax,gov_balance,corporate_tax_incidencehousehold_net_income(BHC and AHC),hbai_excluded_incomehbai_household_net_income,stamp_duty_land_tax,expected_sdltChecks, every year.
corporate_sdltandbusiness_ratesequal their statistics to within £0.01m.Distribution. Head − base, household net income per household, weighted mean £, by decile:
Both statistics are allocated by
shareholding, so the change grows with corporate wealth.Not changed here (follow-ups)
import_obr_forecasts.pyrefresh these growth series from a new EFO. It is stacked on this PR.residential.household.first_time_buyers_reliefandtransactions_by_age(2020-21) are kept verbatim.Review
Round 1 (independent review, Opus 5.5): REQUEST_CHANGES, with 14 findings. The responses, finding by finding:
Round 2 (same reviewer): APPROVE. All four major findings are resolved.
Round 3 (pending): the UK hub's independent review at the final head.
Stack update (23 October 2026)
Merges (no history rewritten):
f3c2ab2fcmerges Count corporate SDLT once and let SDLT abolition remove all of it #2235's final head (8c1b54dd9). There was one conflict, in thevathead'sdata_noteinfiscal_partition.yaml; I kept Count corporate SDLT once and let SDLT abolition remove all of it #2235's wording, which gives household counts only. This PR's two data notes are unchanged.b11dc8c47merges main ata45baf5e4, which includes Count corporate SDLT once and let SDLT abolition remove all of it #2235 as0def167a1. There were no conflicts. Main's changes since Count corporate SDLT once and let SDLT abolition remove all of it #2235's head don't touch these statistics, the receipts growth series oraverage_earnings.Two test fixes.
376f10307:test_weighted_incidence.py(added to main with #2235) sets the four business rates series so that corporate incidence is exactly £150 × shareholding. This PR addsENGLAND_CENTRAL_LISTandWALES_CENTRAL_LISTto that sum, so on the merged tree they kept their real values (about £2.21bn in 2025) and 5 of the test's 19 cases failed (2.2e7 against 1.5). The test now sets both central lists to zero, and all 19 cases pass.f6ed43708:test_aggregate_impact_certified.py::test_head_totals_agree_with_the_model_weighted_sumschecked that the partition's heads plus declared double counts equalgov_taxwithin a relative 1e-9. The model sumsgov_taxfor each household in float32 (policyengine-core's float type), while the measurement sums the heads in float64, so they agree only to float32 rounding. With these statistics, the 2026 totals on 1.58.1 differ by 1.75e-9 (£948,404,896,828 against £948,404,895,169), and CI failed it (run 39915137960, on #2342's branch, which carries this PR). The check now usesreport_spec.yaml'sidentity_relative_tolerance(1e-6). The spec documents that value as the float32 tolerance for identities between weighted totals, and the measurement applies it to its per-household identity checks. This test needs the certified dataset, so CI runs it, not the local runs below.National change at this head. This is recomputed from the parameters at this head against main (
a45baf5e4). With a dataset,shareholdingdivides each household's corporate wealth by the weighted total, so weightedcorporate_sdltandbusiness_ratesequal their statistics on any dataset, Microcosm included. The change ingov_taxandhousehold_taxis therefore the change in the two statistics; only its split across households depends on the data.gov_tax,household_taxThese match the 11 October real runs to the £m in every year they share. They also match CI's Aggregate impact report at
376f10307(base mainc65bced64, policyengine-uk-data 1.58.1) in every year, which reports 0 checks broken and no change in any year to poverty rates, people in poverty or inequality. The one failing check, negative VAT for 76 households (policyengine-uk-data#565), fails on both sides.Tests at this head (local, foreground,
-n 2, host load 40–90; CI runs the full suite):test_corporate_receipts_statistics.py: 141 passed, including the 4 cases that build a Microsimulation on a synthetic dataset.test_weighted_incidence.py: 19 passed. Before the fix, 5 failed.test_fiscal_partition.py: 39 passed.test_sdlt_counted_once_properties.pyandaggregate_impact/test_aggregate_impact_measure.py: 45 passed.test_program_registry.py,test_parameter_metadata.py,test_parameter_descriptions.pyandtest_no_economic_assumptions.py: 121 passed.stamp_duty/andexcise_integration.yaml: 46 passed.ruff format --check .andruff checkon the touched Python both pass.Reproducer: policyengine_uk/tests/test_corporate_receipts_statistics.py::test_sdlt_receipts_match_hmrc_tables, policyengine_uk/tests/test_corporate_receipts_statistics.py::test_business_rates_match_published_outturn, policyengine_uk/tests/test_corporate_receipts_statistics.py::test_projection_tracks_obr_receipts, policyengine_uk/tests/policy/baseline/gov/hmrc/stamp_duty/corporate_sdlt.yaml
axiom: n/a: official receipts statistics and OBR forecasts used for corporate incidence, not law
🤖 Generated with Claude Code