Repository navigation
Conversation
Contributor
Author
|
Batch dependency with pe-uk #2241 (fixes policyengine-uk#2238) and uk-data #556. #2241 stops charging a household's other-property stock as an additional-dwelling purchase. If this PR's target (OBR SDLT less corporate SDLT) is built on a policyengine-uk carrying #2241, but without an imputation of additional-dwelling purchases, calibration will push main-home SDLT to absorb the missing higher-rates share. #556 adds that imputation, calibrated to HMRC's HRAD statistics; a draft PR is being built now. So #556 should land in the same d833 batch as this PR, or the target should be held until it does. Max's decision on adopting #556 is d1354. |
70 tasks
Contributor
Author
|
Heads-up for the batch: #592 (hash-verified OBR workbooks, header-read years) removes |
The OBR's SDLT line covers every buyer, but policyengine-uk adds SDLT from non-natural persons separately as corporate_sdlt, so targeting stamp_duty_land_tax at the whole line counted the corporate share twice. Subtract the installed policyengine-uk's corporate statistic for each of the active OBR manifest's years, warn when that statistic is held flat (policyengine-uk before #2322, first released in 2.152.0), and refuse full builds against such a version. A statistic above the OBR line raises SourceError, which the receipts parser does not swallow. Fixes #564 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
force-pushed
the
sdlt-target-excludes-corporate
branch
from
October 25, 2026 03:06
0544d69 to
092982e
Compare
The £23.29bn 2026-27 example came from policyengine-uk#2322's head before #2241, which no release matches. Use 1.58.0 under the locked 2.122.2 (households £16.32bn against the OBR's £15.41bn in 2025-26, corporate £3.13bn on top) and the 2025-26 target with 2.152.0 or later. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Calibration fit against the base build: pass
A drop of more than 2.0% fails. Notes
National targets in a worse error band on the head (9)
Flags are for review: builds of the same code move 2 to 12 national targets to a worse band (#598). Bands are of |estimate / target - 1|; no target's estimate or error is published. Income distribution and single ages against the base build: passHBAI FYE 2025 (DWP), period 2024: equivalised income in £ a week, rates and Gini in per cent. A value outside HBAI's 95% interval is marked (outside).
FlagsNone. These measures report and flag but do not fail yet: their same-code spread is unmeasured (#598). Aggregate impact of the data change: passBase dataset Checks: 0 broken by this change, 4 failing on both sides, 0 fixed. Moves flagged for review: 188 (thresholds in Notes
Moves flagged for review
148 more flags are in the report artifact. Receipts by tax head
Spending by benefit and service
Caseloads
Government balance, each component counted once
Model totals
Components the model counts twice
Household income totals
Population
Poverty rates
People in poverty
Inequality
Mean household net income in each side's own deciles
Standard reform costings (change in gov_balance)
Standard reforms: change in each reform's target
Household net income by decileThe datasets hold different households, so households are not matched; compare the mean income in each side's own deciles. Checks failing on both base and head (4)
Counted-once checkCounted-once check not run: not requested. Provenance
Base dataset: Head dataset: Receipts by tax head: head levels
Spending by benefit and service: head levels
Caseloads: head levels
Government balance, each component counted once: head levels
Model totals: head levels
Components the model counts twice: head levels
Household income totals: head levels
Population: head levels
Poverty rates: head levels
People in poverty: head levels
Inequality: head levels
Mean household net income in each side's own deciles: head levels
Standard reform costings (change in gov_balance): head levels
Standard reforms: change in each reform's target: head levels
|
Review r2 of #569: the variation check refused a valid one-year manifest and any projection that happened to be flat, and let a £1 variation pass. #2322 gives both corporate SDLT revenue parameters an uprating index, so check for that (obr.unprojected_corporate_sdlt) in the target warning and the full-build preflight. Also import create_datasets' progress panels before the preflight runs, so its error handler can report a refused build instead of raising UnboundLocalError, and relax Hypothesis's time limit on the two SDLT properties, which failed the too_slow health check after policyengine-uk's imports on a loaded host. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r3 of #569: the test assumed projected statistics vary and that a version uprates both or neither, which a flat projection or a partial one would break. Expect both uprated from 2.152.0 (#2322) and, before it, neither, with £3.13bn in every year. Also qualify "whatever the weights" in corporate_sdlt_receipts: shareholding needs a non-zero weighted total. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #564.
What
obr/sdltnow targetsstamp_duty_land_taxat the OBR's SDLT line less policyengine-uk's corporate SDLT statistic for the same year:gov.hmrc.stamp_duty.statistics.{residential,non_residential}.corporate.revenue, read from the installed policyengine-uk (targets/sources/obr.py:corporate_sdlt_receipts,household_sdlt), for every year of the active OBR manifest (active_manifest().years, 2024-25 to 2030-31). The target keeps its name, reference and vintage.TESTING=1) stops at its preflight (utils/build_environment.py) if the resolved policyengine-uk does not project the corporate statistic: if eitherstatistics.*.corporate.revenueparameter has noupratingindex (obr.unprojected_corporate_sdlt). #2322 gave both the OBR's SDLT receipts for their property type; earlier versions have none and carry the 2020-21 figures forward, £3.13bn in all. The check reads the index, not the values, so a one-year manifest or a projection that happens to be flat is not refused. Target building, and so PR CI'sTESTING=1build, logs a warning in that case instead, so this PR's CI runs on the currently locked policyengine-uk 2.122.2.create_datasets.mainnow imports its progress panels before the preflight runs. Before, a preflight failure reached theexcepthandler beforedisplay_error_panelwas imported, so the build stopped with anUnboundLocalErrorand the reason only in the traceback's context (found by the review; the defect predates this PR, but this PR's guard is a new way to reach it).tests/test_property_purchased.pycompares households'stamp_duty_land_taxwith the household-only target, notexpected_sdltagainst total receipts, and checks that weightedcorporate_sdltequals the statistic subtracted. The newtests/test_obr_sdlt_target.pychecks the target identity each year and the invariants below.tests/test_obr_sources.py's fixed-column differential test subtracts the statistic forobr/sdlt.tests/test_obr_receipts_sheet.py's synthetic workbook now carries £100bn a year so its SDLT row stays above the statistic.tests/test_build_environment.pychecks the preflight's issue and that a refused build stops with the preflight's own error.Rebased on main
e1acdf96(2026-10-24): main's hash-bound manifest (#592) replaced_FY_COL_TO_YEAR, so years now come from the manifest and values fromread_row, and the tests load the verified workbook withload_workbook("receipts").Why
The OBR's SDLT line (EFO March 2026, Table 3.8; it equals Table 3.7's "Total SDLT" every year) covers all buyers in England and Northern Ireland. policyengine-uk adds SDLT paid by companies and other non-natural persons separately as
corporate_sdlt: the statistic times each household'sshareholding, which divides its corporate-sector wealth by the weighted total, so while that total is not zero, weightedcorporate_sdltis the statistic whatever the weights.corporate_sdltis ingov_taxandhousehold_tax. Calibratingstamp_duty_land_taxto the whole line therefore counted the corporate share twice.Measured
£bn, PolicyEngine years (2025 = 2025-26). Targets come from the committed EFO workbook and the installed policyengine-uk (
check569.py). policyengine-uk 2.157.0 (released 2026-10-25, measured that day) carries #2322 and #2342; its statistic is identical, to the pound, to that of #2322's headcafd3f1d, measured before the rebase. 2.157.3 (released 2026-10-25 05:21Z) gives the same statistic and targets in every year (check569_2157_3.json).Microsimulation runs on the released 1.58.0
enhanced_frs_2024_25.h5(sha2565ba399ea…d960b6), which was calibrated to the old target, 2025-26 (measure_2157.json,components_*.json):stamp_duty_land_taxsdlt_liablefilter)additional_residential_property_purchased(weighted value)non_residential_property_purchased(weighted value)corporate_sdltHousehold SDLT more than halves between the two versions, largely because of pe-uk#2241 (in 2.134.1 and later), not this PR: #2241's own run on the same data moved it from £16.32bn to £7.62bn, and other model changes between the two versions account for the rest. #2241 made additional-dwelling and non-residential purchases inputs that are nil unless a dataset imputes them, where before they were the household's property stocks priced as purchases whenever
property_purchased. The batch relock this PR needs brings #2241 as well. Without #573's imputation (d1354, parked), only main-home purchases carry household SDLT, so the calibration can reach the new target only by upweighting main-home buyers: £7.38bn to £10.60bn, 1.44 times, against 2.09 times for the old whole-line target. Weightedcorporate_sdltmatches the statistic to within £600 under both versions (float32). The household SDLT a rebuilt dataset reaches is pending: this PR's CI runs a reducedTESTING=1build on 2.122.2, and the full build comes with the batch.Is the new target attainable? This is a feasibility check, not the build pipeline (
reweight_experiment.py). Starting from 1.58.0's weights, I reweighted against the national targets only, using the pipeline's symmetric relative-error loss for 512 Adam steps at lr 0.1. Each run went once with the old SDLT target (the OBR total, £15.41bn) and once with the new one, all else equal. The pipeline also calibrates the constituencies and local authorities jointly, so this shows what meeting the target costs nationally, not what the full build will reach.Under 2.157.0 there are 648 national targets. 19 of their columns have no signal on 1.58.0.
known_gaps.yamllists 18 of those, but notdwp/uc_payment_dist/COUPLE_NO_CHILDREN_annual_payment_22_800_to_24_000, so main's coverage check refuses the matrix (coverage_2157.json). It is therefore built withrequire_coverage=False; a zero column adds the same constant to both losses (reweight_experiment_v2.py,reweight_experiment_2157.json). Households' SDLT starts at £7.36bn on the matrix. Both runs reach their SDLT target: the old one at £15.39bn (0.999 times) and the new one at £10.60bn (1.000 times). Their final losses agree to five decimal places (0.036688 and 0.036686). Both get there by upweighting main-home buyers (property_purchased), from 1.03m weighted households to 1.46m for the old target and 1.30m for the new one. No other target's absolute relative error differs between the runs by more than 0.4 percentage points (the largest isobr/state_pension, −7.8% against −7.4%), and 94.0% of the other targets are within 10% in both. Before the rebase, the same check on #2322's head (which predates #2241, so it still priced additional-dwelling purchases) hit both targets within 0.02%, with final losses equal to six decimal places (reweight_experiment.json). #573's imputation would let the calibration meet the target with fewer extra buyers; this check does not measure that.Invariants
For every year of the OBR manifest (2024-25 to 2030-31), with baseline policyengine-uk parameters and a non-zero weighted total of corporate-sector wealth (production calibration passes no reform; under a parameter reform, SDLT abolition or an
applied_before_data_loadscenario the model's corporate SDLT can differ from the statistic subtracted here):SourceErrornaming the year instead of producing a negative target (Hypothesis).SourceErroris not aValueError, so the receipts parser's per-row handler does not swallow it and the build stops (test_statistic_above_obr_sdlt_stops_the_receipts_parse); the manifest also requiresobr/sdlt, so the target cannot be dropped silently.corporate_sdltformula adds: a one-household simulation withshareholding= 1 in every year (rel. 1e-6), and weightedcorporate_sdltover the built dataset in 2025.unprojected_corporate_sdltlists exactly the statistics without an uprating index, for each of the four combinations set on the loaded parameters; and on the installed policyengine-uk, the statistics are uprated exactly when their values vary over the manifest years (both under 2.157.0; neither under 2.122.2).With households on target and that weighted total non-zero, model SDLT (
stamp_duty_land_tax+corporate_sdlt) is therefore the OBR's figure. Mutation checks (mut569.sh;mutations_2122.log,mutations_2157.log): each of eight mutants failstest_obr_sdlt_target.py,test_obr_sources.pyortest_build_environment.pyunder both 2.122.2 and 2.157.0. The mutants drop the subtraction, drop the non-residential statistic, raiseValueErrorforSourceError, remove the negative check, treat every statistic as projected, treat none as projected, drop the warning, and move the error panel's import back inside thetry, after the preflight (the original bug, which raisesUnboundLocalError).Guard and dependency
policyengine-uk>=2.122.0), and no open PR relocks to a version with #2322 (the highest, Fill benunit_reported_capital from FRS benefit-unit capital, so UC uses it too #552, locks 2.126.8). The batch that carries this PR must relock to 2.152.0 or later before its full build. Until then,make datawithoutTESTING=1stops with: "The resolved policyengine-uk does not project gov.hmrc.stamp_duty.statistics.residential.corporate.revenue or gov.hmrc.stamp_duty.statistics.non_residential.corporate.revenue: it carries the last outturn forward. Full builds need policyengine-uk 2.152.0 or later (Update corporate SDLT and business rates statistics and project them with OBR receipts policyengine-uk#2322)…" Verified: the preflight lists that one issue under 2.122.2 and none under 2.157.0 or 2.157.3.known_gaps.yamlentries (UC payment-distribution bands) that now have signal, as well as the one unlisted zero-signal band above (coverage_2157.json). A rebuilt dataset may differ.Tests
test_build_environment.py, everytest_obr_*.py,test_target_registry.pyandtest_property_purchased.py: 178 passed, 10 skipped (the dataset tests, with no built dataset in the checkout), under both the locked policyengine-uk 2.122.2 and 2.157.0.ruff format --checkandruff checkpass.test_property_purchased.pyon 1.58.0: the household ratio is 1.33 with 2.122.2 and 0.70 with 2.157.0, both inside the full bound (0.5, 1.5) and theTESTINGone (0.25, 4).axiom: n/a: calibration target in the data build, no policy rule changed.
Evidence:
~/reviews/uk-data-sdlt-corporate-564/(check569.py, check569_2122.json, check569_latest.json, check569_2157_3.json, coverage.py, coverage_2157.json, mutations_2122.log, mutations_2157.log, mutant_panel_from.txt, mutant_panel_to.txt, measure_manifest.py, measure_2157.json, components.py, components_2122.json, components_2157.json, reweight_experiment_v2.py, reweight_experiment_2157.json, reweight_experiment_2157_full.json, mut569.sh, tests569_2122.log, tests569_latest.log, review-r2/review.md, review-r3/review.md, and the pre-rebase measure.py, measure_2122.json, measure_2322.json, reweight_experiment.py, reweight_experiment.json).🤖 Generated with Claude Code