Repository navigation
Drop ons/council_tax_band_d/{code} target emission (#483) - #484
Open
vahid-ahmadi wants to merge 4 commits into
Open
vahid-ahmadi wants to merge 4 commits into
vahid-ahmadi wants to merge 4 commits into
Conversation
juaristi22
approved these changes
Sep 18, 2026
juaristi22
left a comment
juaristi22
left a comment
Collaborator
There was a problem hiding this comment.
Approving. Dropping the emission is the right shape for the Microcosm register, and it answers the question on #483: no, fencing in place is not needed. The local_council_tax_band_d_rate concern keeps covering targets/sources/la_council_tax either way, and its fence assumed exactly this state. One correction on our side: the fence's origin line says uk-data "did not wire them into the additive LA loss matrix", which was wrong until this PR (the target existed and was silently dropped). We will update that evidence to cite #483 and this PR once it lands, and re-pin the inventory sha256s for la_council_tax.py and test_la_council_tax_targets.py in Microcosm's uk_data_target_inventory.json.
Sequencing: please merge this before any policyengine-uk pin bump that includes PolicyEngine/policyengine-uk#1855.
Nits, none blocking:
- The
uv.lockchange (the package's own version line, 1.56.16 to 1.57.3) is not part of this fix. Keep it only if main's lock is behindpyproject.toml, otherwise drop it from the diff. - The
== 350and== 2563pins will need touching on every CSV refresh. Fine as a deliberate tripwire; the refresh commit should say so. - The conflict with #482 is real (same docstring region, same test file); whichever lands second keeps both paragraphs.
🤖 Generated with Claude Code
This was referenced Sep 18, 2026
A Band D council tax amount is a rate (£ per Band D dwelling), not an
additive household control. The loss matrix is an additive objective, so
binding a ~£2,500 per-dwelling rate into it is category-wrong — neither
as a sum nor as a mean.
Microcosm adjudicated the same question independently: its
uk_data_target_parity.json concern `local_council_tax_band_d_rate` is
status `reviewed_exclusion`, classification `non_linear`, reason "Band D
currency amounts are per-rate, not additive household controls", to stay
excluded "until a rate-aware non-linear objective exists".
Removed: the per-LA Band D amount Target emission (350 targets, all of
which were silently discarded today because `council_tax_band_d_amount`
does not exist in policyengine-uk) and the now-dead `_year_for_band_d` /
`_ref_for_band_d` helpers and `_ENGLAND_REF` / `_SCOTLAND_REF` /
`_YEAR_BAND_D_WALES` / `_YEAR_BAND_D_SCOTLAND` constants.
Kept: `band_d_amount` in storage/la_council_tax.csv (byte-identical) and
loadable, the `voa/council_tax/{code}/{band}` band-count targets (2,563,
unchanged — counts are additive and Microcosm binds them as
ported_local_declared), and the `housing/council_tax_net/{code}` targets
(318, unchanged).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- The net council-tax target reads total_council_tax_net straight from the CSV; the taxbase x Band D derivation happens upstream at CSV build time. Reword the comment accordingly. - Rename changelog.d/483.md to 483.removed.md so towncrier picks up the fragment type. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both pins are deliberate: they must be updated on every la_council_tax.csv refresh. Addresses a review nit on #484. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
vahid-ahmadi
force-pushed
the
fix/483-drop-band-d-target
branch
from
October 8, 2026 11:39
84b9927 to
aac11ac
Compare
Collaborator
Review of #484 — Drop ons/council_tax_band_d/{code} target emission (#483)Base repository: PolicyEngine/policyengine-uk-data Source DocumentsCriticalNone. Should AddressNone. SuggestionsNone. Evidence GapsNo material gaps reported. Notes
Validation Summary34 passed, 14 skipped in focused snapshot-module tests; all retained target fields and CSV exact equality proved; see pr-484-focused-pytest.log and pr-484-target-parity-diagnostic.log Official MHCLG rate/taxbase/revenue definition checked; 1 original source, SHA-256 manifest helper passed; no PDFs. CI at aac11ac: Check changelog fragment=SUCCESS, Lint=SUCCESS, Validate bundle release manifest contract=SUCCESS, Test=IN_PROGRESS. Detailed code, policy, source and reproduction artifacts were retained locally. Timingelapsed seconds: 1115.90s; setup seconds: 21.54s Review SeverityAPPROVE. Open findings: 0 critical, 0 should address, 0 suggestions. |
juaristi22
added a commit
to PolicyEngine/microcosm
that referenced
this pull request
Oct 15, 2026
_local_contract_covers still keyed the local council-tax rows on the voa.council_tax_stock.by_area. prefix that #929 retired, so the 25 mhclg, welshgov and scotgov by-area rows covered only the LA loss module. They now cover uk-data's la_council_tax source as well, ahead of the re-pin that follows PolicyEngine/policyengine-uk-data#482 and PolicyEngine/policyengine-uk-data#484. Three stale texts are corrected while the register regenerates: the English region concern names the taxbase rows and the nation grain, and the two private-rent concerns describe the calendar-2025 PIPR series in the pinned feed, not a June 2026 snapshot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22
added a commit
to PolicyEngine/microcosm
that referenced
this pull request
Oct 15, 2026
_local_contract_covers still keyed the local council-tax rows on the voa.council_tax_stock.by_area. prefix that #929 retired, so the 25 mhclg, welshgov and scotgov by-area rows covered only the LA loss module. They now cover uk-data's la_council_tax source as well, ahead of the re-pin that follows PolicyEngine/policyengine-uk-data#482 and PolicyEngine/policyengine-uk-data#484. Three stale texts are corrected while the register regenerates: the English region concern names the taxbase rows and the nation grain, and the two private-rent concerns describe the calendar-2025 PIPR series in the pinned feed, not a June 2026 snapshot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Calibration fit against the base build: pass
A drop of more than 2.0% fails. Notes
National targets in a worse error band on the head (1)
Flags are for review: builds of the same code move 2 to 12 national targets to a worse band (#598). Bands are of |estimate / target - 1|; no target's estimate or error is published. Income distribution and single ages against the base build: passHBAI FYE 2025 (DWP), period 2024: equivalised income in £ a week, rates and Gini in per cent. A value outside HBAI's 95% interval is marked (outside).
FlagsNone. These measures report and flag but do not fail yet: their same-code spread is unmeasured (#598). Aggregate impact of the data change: passBase dataset Checks: 0 broken by this change, 4 failing on both sides, 0 fixed. Moves flagged for review: 0 (thresholds in Notes
Moves flagged for reviewNone. Receipts by tax head
Spending by benefit and service
Caseloads
Government balance, each component counted once
Model totals
Components the model counts twice
Household income totals
Population
Poverty rates
People in poverty
Inequality
Mean household net income in each side's own deciles
Standard reform costings (change in gov_balance)
Standard reforms: change in each reform's target
Household net income by decileThe datasets hold different households, so households are not matched; compare the mean income in each side's own deciles. Checks failing on both base and head (4)
Counted-once checkCounted-once check not run: not requested. Provenance
Base dataset: Head dataset: Receipts by tax head: head levels
Spending by benefit and service: head levels
Caseloads: head levels
Government balance, each component counted once: head levels
Model totals: head levels
Components the model counts twice: head levels
Household income totals: head levels
Population: head levels
Poverty rates: head levels
People in poverty: head levels
Inequality: head levels
Mean household net income in each side's own deciles: head levels
Standard reform costings (change in gov_balance): head levels
Standard reforms: change in each reform's target: head levels
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #483.
Decision and provenance
A Band D council tax amount is a rate (£ per Band D dwelling), not an additive household control. The loss matrix is an additive objective — every target is compared against a weighted sum over households. Binding a ~£2,500 per-dwelling rate into it is category-wrong: not as a sum (which scales with the number of households in the LA), and not as a mean (still a rate inside an additive objective).
This is not a fresh judgement call. Microcosm reached the same verdict independently.
uk_data_target_parity.jsonconcernlocal_council_tax_band_d_rateis:non_linearreviewed_exclusionSo the resolution is to drop the emission, not to add a mean-returning compute route.
Why this is latent-bug removal, not a behaviour change today
council_tax_band_d_amountdoes not exist in policyengine-uk, sobuild_loss_matrix.py::_compute_simple_gbpraisesKeyError, returnsNone, and the callercontinues — all 350 of these targets are silently discarded today. The defect activates the moment policyengine-uk gains that variable (PolicyEngine/policyengine-uk#1855 adds it):_compute_simple_gbpwould then sum it against a ~£2,500 level.Removed
Target(name=f"ons/council_tax_band_d/{code}", variable="council_tax_band_d_amount", ...)loop inget_targets(), replaced by a comment block explaining the rate-vs-additive reason, citing Microcosm'slocal_council_tax_band_d_ratereviewed_exclusion and this issue, so it does not get helpfully re-added._year_for_band_d,_ref_for_band_d, and the constants_ENGLAND_REF,_SCOTLAND_REF(the identically-named_SCOTLAND_REFinvoa_council_tax.pyis a separate symbol and is untouched),_YEAR_BAND_D_WALES,_YEAR_BAND_D_SCOTLAND.test_band_d_targets_use_gbp_unit(vacuous once nothing matches).Deliberately kept
band_d_amountinstorage/la_council_tax.csv— needed as data for liability computation.git diff origin/main -- policyengine_uk_data/storage/la_council_tax.csvis empty: the CSV is byte-identical to main. The loader is untouched, and a new test asserts the column is still present and populated for 350 LAs.voa/council_tax/{code}/{band}band-count targets — 2,563, unchanged. These are counts, counts are additive, and Microcosm has themported_local_declaredand actively bound. Not touched.housing/council_tax_net/{code}— 318, unchanged._YEAR_BAND_D_ENGLANDkept: still used, because the England net council-tax requirement derives from the 2026-27 Band D level.Target counts, before and after
ons/council_tax_band_d/*voa/council_tax/*housing/council_tax_net/*Tests
New/changed in
test_la_council_tax_targets.py:test_no_band_d_amount_target_is_emitted— asserts no emitted target name starts withons/council_tax_band_d/(or containscouncil_tax_band_dat all).test_band_d_amount_column_still_loadable— the CSV column survives.test_band_count_target_count_matches_csv— now also pins the count at 2,563 so a future change to the Band D surface cannot quietly disturb the band-count family.Full suite, repo's normal invocation:
ruff check .reports 61 errors on this branch. Measured per-file rather than assumed: bothla_council_tax.pyandtest_la_council_tax_targets.pyreport 0 errors onorigin/mainand 0 on this branch (viagit show origin/main:<f> | ruff check --stdin-filename <f> -), so all 61 are pre-existing and live in files this PR does not touch.Not done / follow-ups
la-council-tax-all-bands) is still open, so this PR is based on currentmainand does not depend on it. Theband_A_amount..band_H_amountcolumns it adds are also rates and must likewise not be wired as additive targets — the follow-up there is "no additive target", not "add eight more". Once Extend LA council tax to all eight bands (MHCLG Table 9, 2026-27) #482 lands, the same comment should be added next to those columns. Flagging for @juaristi22 since this sits on the Microcosm parity surface.band_A_amount..band_H_amountparagraph into the same docstring region where this PR inserts its "No target is emitted…" paragraph, and adds 79 lines to the same test file. Whichever lands second will conflict; the resolution is trivial (keep both paragraphs).== 350and== 2563pins will need updating on any CSV refresh (the Scottish band-count follow-up, VOA 2026 stock). That is deliberate — it makes a silent change to these families impossible — but say so if you would rather keep only the CSV-derived assertion.changelog.d/483.removed.md(towncrier needs the.<type>.segment).build_loss_matrix.py:_compute_columnnever had anons/council_tax_band_d/route, and with the emission gone none is needed.🤖 Generated with Claude Code