Repository navigation
Compute gross council tax from LA Band D amounts and statutory band ratios - #1855
Conversation
|
Flagging a design question for review that I think is the crux of this PR, plus a downstream hazard it creates. The £61.21bn is an artefact, and the cause may be architectural rather than a missing dataset fieldThis PR computes liability as The PR attributes this partly to That observation is correct, but I do not think the conclusion "the dataset needs to assign LAs" follows. Checking
So Question for review: should this be computed per-LA under the LA weighting context instead of via a household-level enum lookup? I would rather have that ruled on before the approach is built further, since the remaining deliverables (validation against administrative totals, distributional impacts) all depend on it. Worth noting the £61.21bn also includes no single-person discount, exemptions, premiums or CTR, all of which are declared gaps here — so some overstatement is expected regardless. But the single-LA artefact is the larger term and it is the one with a design question attached. Downstream hazard: filed as PolicyEngine/policyengine-uk-data#483Independent of the above — merging this PR makes a latent defect in That repo emits a target at It will not raise. It will just calibrate against a wrong comparison. Detail and options in #483; that needs resolving before the pin is bumped, not before this merges. Verified separatelyThe Scotland correction in this PR is right and worth not losing: Scotland raised the E–H multipliers to 473/360, 585/360, 705/360, 882/360 from April 2017 (SSI 2016/123), so the flat 9ths only hold for England. Wales Band I = 21/9 traces to LGFA 1992 s.5(1A) itself (proportions 6:7:8:9:11:13:15:18:21 for Wales from April 2005), not to separate Welsh legislation. Also relevant to the Wales lane, from the parallel Axiom review (TheAxiomFoundation/rulespec-uk#344): LGFA 1992 s.12 was omitted with effect 1 April 2026 by the Local Government Finance (Wales) Act 2024 (asc 6), ss.18(6) and 23(7) — and 2026-27 is the year this PR's Band D amounts are drawn from. Context and full deliverable scope: #1854. 🤖 Generated with Claude Code |
juaristi22
left a comment
There was a problem hiding this comment.
The arithmetic is right, and both new variables have formulas, so they are safe for the Microcosm adapter. Requesting changes on data coverage, not on the design.
On the design question. The answer differs by consumer. policyengine-uk-data reweights one national sample per authority, so a household-level enum can never vary by authority there. Microcosm assigns an authority to every household row (an ONS code from its geography ladder), so a household-level local_authority lookup is exactly the right shape for us. What is missing is on our side: nothing maps that code into the LocalAuthority enum yet, so Microcosm builds also show the Maidstone default. We will supply local_authority as an input from the dataset build. The same gap already silences the bespoke CTR schemes in council_tax_reduction/config.py (Merton, Kingston upon Thames, Newham and the others test local_authority ==) on every dataset, which raises the priority of populating the input.
Changes requested:
- Enum coverage. Add the six 2023 unitaries (North Northamptonshire, West Northamptonshire, Cumberland, Westmorland and Furness, North Yorkshire, Somerset) to
LocalAuthority. Microcosm's roster is the April 2023 local authority list, so those six are exactly the codes that would fall to the default, and adding them lets all 350 Band D rows load. - Vintages. The parameter carries only 2026-27 for England and Wales and 2025-26 for Scotland. Microcosm measures at 2025, so England and Wales would be charged next year's level. The MHCLG Table 10 workbook prints the previous-year column alongside the current one, so fiscal 2025 values are in the same file, and Scotland's 2026-27 average Band D by council is published in the Scottish Government council tax datasets. Please carry 2025 and 2026 for all three nations.
- References. The England
amounts.yamlhref is the statistics collection page. Cite the specific table (Table 10, local authority level data, 2026-27) and say which Band D basis it is (including all precepts, if it is the area figure). The Scotland href should name the specific dataset. - Scotland ratios before 2017.
scotland.yamlstarts every band at 2017-04-01, including A to D, which SSI 2016/123 did not change. Date A to D from 1993-04-01 so the parameter is defined before 2017.
Agreed on the hazard: policyengine-uk-data must merge PolicyEngine/policyengine-uk-data#484 before bumping its pin past this PR. Microcosm has no exposure; the Band D rate was never ported (it is a signed exclusion in our parity register).
🤖 Generated with Claude Code
|
Thanks — all four addressed in 9673a4f. Point by point. 1. Enum coverage. Added the six April 2023 unitaries to Four source names needed aliasing to the enum's existing spellings rather than new members: MHCLG's "Bristol", "Durham", "Herefordshire" and "Kingston upon Hull" map to 2. Vintages. Both 2025 and 2026 now carried for all three nations. One correction to the premise: Table 10 does print the previous-year column, but only for lines 8 and 9 (the billing-authority-only Band D). The figure this parameter actually uses is line 17, the area figure — and there is exactly one line-17 column per release, no prior-year twin. So rather than reconstruct it from line 8 plus 16(a)–(d), I took fiscal 2025 from the revised 2025-26 release. Cross-check: the 2025-26 release's line-8 current column equals the 2026-27 release's line-8 previous column row for row (Adur 365.05 in both), confirming the same authority ordering and basis. Independent validation that I hit the same series you already had: every England and Wales 2026 value, and every Scotland 2025 value, reproduces the figure already in the file to the penny (0 mismatches across 344 rows). Sources actually used:
No gaps: all 350 rows have both years. 3. References. The collection-page href is gone; each nation now cites its specific workbook, and both the parameter description and the variable documentation state the basis:
4. Scotland ratios before 2017. Verified first, and the citation turned out to be wrong: SSI 2016/123 is the Local Authority (Capital Finance and Accounting) (Scotland) Regulations, nothing to do with council tax. The substitution order is SSI 2016/368, article 2, which substitutes Relative to D = 360 that gives A–D = 240/360, 280/360, 320/360, 1 — identical to the original s.74(1) proportion One step past your literal ask, flagged deliberately: dating only A–D from 1993 would still have left E–H undefined before 2017. E–H now also take their original 11/9, 13/9, 15/9 and 18/9 values from 1993-04-01, with the 2017-04-01 step to 473/360, 585/360, 705/360, 882/360 retained. Happy to drop that half if you would rather the parameter stay silent pre-2017 for E–H. Tests. One expected break, updated rather than worked around: the Edinburgh Band H 2026 case moved from 3830.60 to 3983.82, because 2026 now has its own vintage (Band D 1626.05) instead of holding 2025-26 (1563.51) flat forwards. That is exactly the mis-vintaging you flagged, so the moved number is the fix landing, not a regression. Only cosmetic note: regenerating the file moved Not rebased and not force-pushed — #1864 is stacked here and its head is untouched. 🤖 Generated with Claude Code |
|
Reviewed commit Confirmed integration blocker. The additions at CUMBERLAND and WEST_NORTHAMPTONSHIRE collide with enum members already on main. A read-only three-way merge against main Separate integration decision. council_tax_band_d_amount.py:22 looks up the household's Repair and acceptance steps:
Suggested acceptance commands, not executed in this packaging pass: uv run --no-sync python -c "import policyengine_uk"
uv run --no-sync policyengine-core test policyengine_uk/tests/policy/baseline/gov/local_authorities/council_tax -c policyengine_ukValidation completed: captured CI-log inspection and independent merged-file/AST duplicate detection. Policy tests were not rerun on a repaired tree; the full rate table, current data pipeline and population totals remain unverified. Live posting check (2026-10-02T15:12:23.228164+00:00): same reviewed commit; GitHub reports |
Add council_tax_band_d_amount (LA Band D amount, from a new LA-keyed parameter following the domestic_rates/rates.yaml precedent) and simulated_council_tax (Band D amount x statutory band ratio). Band ratios are added per country: England and Wales use the section 5 proportions 6:7:8:9:11:13:15:18 relative to Band D = 9, Wales adding band I at 21/9 from April 2005; Scotland uses the raised E-H multipliers (473/360, 585/360, 705/360, 882/360) from April 2017. The existing council_tax input variable is left untouched so the two can be compared. council_tax_band_d_amount also fills a dangling reference from policyengine-uk-data's la_council_tax target source. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…levels Review follow-up on the four data-coverage items. Enum coverage: add the six April 2023 unitary authorities (Cumberland, North Northamptonshire, North Yorkshire, Somerset, West Northamptonshire, Westmorland and Furness) to LocalAuthority. All 350 Band D rows now load, up from 344. Vintages: carry fiscal 2025 and 2026 for England, Wales and Scotland. England comes from MHCLG Table 10 line 17 in both the 2026-27 and the revised 2025-26 releases; the 2025-26 release's current-year column reproduces the 2026-27 release's previous-year column, and every England and Wales 2026 value reproduces the figures already in the file. Wales comes from the Welsh Government council tax levels data tables, Table 1 "Overall average band D". Scotland comes from the Scottish Government Band D council tax 1996-97 to 2026-27 dataset, whose 2025-26 column reproduces the Scottish values already in the file. References: cite the specific workbooks rather than the statistics collection page, and state the basis. For England and Wales this is the area figure, so it includes the county, police, fire, combined authority and parish or community council precepts; for Scotland it excludes the separately billed water and waste water charges. The variable documentation and the stale "six authorities absent from the enum" comment are corrected to match. Scotland ratios: SSI 2016/368 (not 2016/123, which is the capital finance regulations and was cited in error) substitutes 240:280:320:360:473:585: 705:882 into section 74(1) for financial years from 1 April 2017. Bands A to D are unchanged from the original 6:7:8:9 proportion, so they are dated from 1993-04-01; bands E to H keep the 2017 step and gain their original 11/9, 13/9, 15/9 and 18/9 values from 1993-04-01, so no band is undefined before 2017. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…land, an abolished district and the missing-authority default; document the authority context Main already added the six unitaries (52f4281), so composing this branch with it defined CUMBERLAND and WEST_NORTHAMPTONSHIRE twice and the package failed to import (TypeError: 'CUMBERLAND' already defined). Every Band D row still resolves to an enum member (350 of 350). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
9673a4f to
283b0ae
Compare
|
Thanks, María. Addressed at
Locally: YAML suite 2,627 passed; pytest (excluding microsimulation) 3,113 passed; ruff clean. |
Review of #1855 — Compute gross council tax from LA Band D amounts and statutory band ratiosBase repository: PolicyEngine/policyengine-uk Source Documents
CriticalNone. Should AddressA1 — Use a configured Towncrier type for the changelog fragment (OPEN)SHOULD ADDRESS — repository requirement. SuggestionsNone. Evidence GapsNo material gaps reported. Notes
Validation Summary162 passed, 1 warning in 2.16s, exit 0; bounded elapsed 6.03s. Exact snapshot Python import with bytecode disabled; run_bounded.py --seconds 540 review-code-test-runner.py current-pr-1855-code-tests-config.json; passing output current-pr-1855-code-tests-required.log, initial collection output current-pr-1855-code-tests.log. Focused nation vector and final standalone/payable/net-income assertions: run_bounded.py --seconds 150 current-council-tax-code-diagnostic.py 1855, exit 0 in 4.34s; output current-pr-1855-code-diagnostic.log. Existing Python3.13 interpreter used; no installs. Exact captured diff and source originals read. current-pr-1855-review-sources.json; current-pr-1855-review-source-cache-check.json. Independent 700-value comparison: current-pr-1855-policy-band-d-comparison.json. Source acquisition timing in current-policy-source-acquisition.json; source extracts checked independently of PR body. No duplicate policy-calculation diagnostics. Canonical source-manifest validator exited 0. CI at 283b0ae: Install + smoke-import (py3.11)=SUCCESS, Install + smoke-import (py3.12)=SUCCESS, Install + smoke-import (py3.13)=SUCCESS, Install + smoke-import (py3.14)=SUCCESS, Lint=SUCCESS, Test=SUCCESS. Detailed code, policy, source and reproduction artifacts were retained locally. Timingelapsed seconds: 1151.70s; setup seconds: 271.60s Review SeverityCOMMENT. Open findings: 0 critical, 1 should address, 0 suggestions. |
…(review: typed fragment) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks, María. Renamed the fragment to 🤖 Generated with Claude Code |
|
@juaristi22 A1 is fixed in 8b18c47 (the fragment is now |
juaristi22
left a comment
There was a problem hiding this comment.
Re-reviewed 8b18c47. My A1 is resolved: the changelog fragment is now council-tax-computed-liability.added.md. This commit only renames the fragment; the model code and tests are unchanged from the prior full review. I am clearing my change request and approving this head. Please rerun the cancelled full Test check before merging.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22
left a comment
There was a problem hiding this comment.
Fresh review of 7de51f6a984dabeea828191fbb3f3933c405a0a8. The earlier approval body attests 8b18c4753, before the main merge, despite GitHub’s updated review pointer. All eight owned blobs match that ancestor; current integration and public-source checks were performed afresh.
C1 — Add the required Axiom status record. The description has no axiom: line. docs/engineering/skills/github-prs.md:113–123 and .github/CONTRIBUTING.md:51 require it for policy additions/parameter changes; this PR adds statutory band ratios, 700 authority amounts and a computed output. Please state the truthful permitted status with the supporting reference/reason (including the documented needed route if applicable). This is a missing required completion record, not a claim that the external encoding is absent or incorrect.
A2 — Refresh the stale description (nonblocking). The actual table covers 350 authorities in both fiscal 2025 and 2026, including the six unitary authorities the body still lists as missing. Reconcile the one-year/344-authority claims and the blanket dataset-assignment statement with the current variable documentation; label the older Enhanced FRS aggregate exercise with its dataset/vintage. No wrong calculation is established by this documentation issue. Prior A1 (changelog name) remains resolved.
Validation: 226 current-head checks passed, including final payable-tax/CTR/income compatibility, explicit overrides, country/authority lookups and fiscal-year/parameter machinery. All 700/700 public Band D amounts match freshly acquired official workbooks to the penny; statutory England/Wales/Scotland ratios and dates verified. Ten official sources accepted. The new output remains a standalone gross area benchmark; private population totals were not rerun. Formatting, lint and Towncrier checks pass. No model implementation or statutory-value defect found.
Review status: COMPLETE. Review severity: REQUEST_CHANGES. Confirmed critical count: 1 (the required status record).
|
@juaristi22 thanks. Both points are addressed in the PR description only. No code changed, and the head is still C1. Added A2. The description now says:
Checked locally at 🤖 Generated with Claude Code |
|
@juaristi22 on C1: the axiom line is now
TheAxiomFoundation/rulespec-uk#468 is the dispatch-ready 🤖 Generated with Claude Code |
juaristi22
left a comment
There was a problem hiding this comment.
Re-reviewed 7de51f6a984dabeea828191fbb3f3933c405a0a8.
C1 and A2 are resolved. The description now records the permitted queued status through rulespec-uk#468; the issue has the required parity label, source/module detail, companion cases and signed-ingest blocker. This records queued work; encoding and differential parity remain pending.
The refreshed description correctly identifies 350 authorities and 700 amounts across fiscal 2025/2026, the six existing unitary definitions, dataset lookup assumptions and the historical aggregate exercise. The code commit is unchanged, so I retained the previous 226 passing checks and official 700/700 amount corroboration after validating the cached evidence. Current-head hosted checks pass. I am clearing my change request and approving.
Retain the documented downstream UK-data calibration fix before updating that repository's model pin.
Review status: COMPLETE. No open critical findings.
The restatement property test drew any stated parameter and any route. Scenario.parameter_changes types its values as numbers and booleans, so a draw of the list-valued gov.dwp.universal_credit.means_test.capital.person_sources through that route failed Scenario validation before any check ran. The council tax parameters this branch adds shift the derandomised draws onto that case in the merge queue. List values now use the other routes, and an explicit example pins the list restatement through the reform dict. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@juaristi22 this PR left the merge queue three times, for two different reasons:
In 0fb7c52 I merged main again and fixed the test strategy. List values now only go through the routes that accept them, and an explicit example covers the list case through the reform dict. The model code is unchanged. 🤖 Generated with Claude Code |
Aggregate impact on certified data: passBase Checks: 0 broken by this change, 1 failing on both sides, 0 fixed. Moves flagged for review: 0 (thresholds in Notes
Moves flagged for reviewNone. Spending by benefit and service
Caseloads
Household net income by decileNo household's net income changes in any year. No change in any year: receipts by tax head; government balance, each component counted once; model totals; components the model counts twice; household income totals; population; poverty rates; people in poverty; inequality; mean household net income in each side's own deciles; standard reform costings (change in gov_balance); standard reforms: change in each reform's target. Checks failing on both base and head (1)
Counted-once checkEach of 84 components was raised by £1 per unit in 2026-27, and every head of its totals was read household by household: its own head must rise by exactly £1, no other tax head may move, no other benefit may rise, and the model's own total must move by what its heads moved. A head that breaks a rule is measured again twice: with every other head held at its baseline value, and with every other head held at its value after the first £1 while the component rises again. It fails where either still shows the response. 0 failed; 0 were skipped.
Provenance
Dataset: Receipts by tax head: head levels
Spending by benefit and service: head levels
Caseloads: head levels
Government balance, each component counted once: head levels
Model totals: head levels
Components the model counts twice: head levels
Household income totals: head levels
Population: head levels
Poverty rates: head levels
People in poverty: head levels
Inequality: head levels
Mean household net income in each side's own deciles: head levels
Standard reform costings (change in gov_balance): head levels
Standard reforms: change in each reform's target: head levels
|
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Merged by the UK hub (session local_8e19f9d5, Opus 5.5) on Max's explicit instruction to merge the team's ready PRs before the Budget (21 October, about 23:00Z: "go ahead and merge them. none of us has time to wait on each other"). One merge-queue enqueue, pinned to head
🤖 Generated with Claude Code |
Makes gross council tax a computed quantity (Nuffield deliverable: "Council tax by band and local authority; national model integration"). Context and full deliverable scope: #1854.
axiom: TheAxiomFoundation/rulespec-uk#468 queued (LGFA 1992 ss.5, 10, 36, 74, 93 and SSI 2016/368 art. 2 are not in the corpus; each returns 404)
Axiom status: rulespec-uk has no encoding of LGFA 1992 Part I charging yet. TheAxiomFoundation/rulespec-uk#468 is the dispatch-ready parity issue for this PR: ss.5, 10 and 36 for England and Wales (with band I), and ss.74 and 93 with SSI 2016/368 for Scotland, with the authority-set Band D amount kept as an input. The scope comes from TheAxiomFoundation/rulespec-uk#344, and the worklist entries for ss.5 and 10 are in TheAxiomFoundation/rulespec-uk#346 (open). Encoding is blocked until these sections are ingested into the corpus, which needs a signed ingest manifest.
What it adds
council_tax_band_d_amount(Household, YEAR, GBP) reads a new LA-keyed parameter,gov/local_authorities/council_tax/band_d/amounts.yaml, following the existinggov/local_authorities/domestic_rates/rates.yamlprecedent for LA-keyed lookups.LocalAuthorityon main; this PR uses main's single definitions and adds no enum members.england.yaml: 6:7:8:9:11:13:15:18 relative to Band D = 9 (LGFA 1992 s.5), from 1993-04-01.wales.yaml: the same plus band I at 21/9, from 1 April 2005 (LGFA 1992 s.5; Council Tax (Valuation Bands) (Wales) Order 2003, WSI 2003/3046).scotland.yaml: all bands from 1993-04-01 on the original s.74(1) proportion; E to H then step to 473/360, 585/360, 705/360 and 882/360 from 1 April 2017 (Council Tax (Substitution of Proportion) (Scotland) Order 2016, SSI 2016/368, article 2). A to D are unchanged by that order.simulated_council_tax= Band D amount x band ratio, zero for Northern Ireland.Authority context
The Band D amount is looked up from the household's
local_authorityinput, so the dataset must supply it. Without it the household takes the enum default (Maidstone).local_authorityas an input from its dataset build (per @juaristi22's review of 18 September). This household-level lookup is the right shape for it.local_authority; it reweights one national sample per authority instead. On the Enhanced FRS, every household therefore gets Maidstone's level.This is documented on
council_tax_band_d_amount.Design choice
A new variable,
simulated_council_tax;council_taxstays the dataset input, untouched. Reasons:council_taxfeedscouncil_tax_less_benefitand the whole Council Tax Reduction stack. Redefining it would move CTR results for every household in the same change that introduces the arithmetic.local_authoritythe computed value is a single-authority counterfactual, so switching now would degrade the model there.simulated_council_tax_reduction_benunitconvention for computed-vs-reported pairs.Switching
council_taxover is a follow-up, once the dataset supplies authorities.Earlier aggregate exercise (historical, not rerun)
This table was computed at the first commit (7ef0352), when the table carried 344 authorities and a single vintage (2026-27 England and Wales, 2025-26 Scotland). Dataset
enhanced_frs_2024_25, period 2026, every household on the Maidstone default. It has not been rerun on the current two-vintage table; the Scotland row in particular would differ, since 2026 now has its own Scottish vintage.simulated_council_taxcouncil_tax(input)Against the ~£40bn England administrative total in the Nuffield application, the England figure was about 53% too high, for two reasons: every household is charged Maidstone's Band D (£2,502.59 in 2026, above the England mean), and this is gross liability (no single-person discount, exemptions or CTR). It measures how far the missing pieces reach, not a validation of the arithmetic.
Downstream hazard in policyengine-uk-data
policyengine_uk_data/targets/sources/la_council_tax.py:182emits targets withvariable="council_tax_band_d_amount". Today the variable does not exist, so the target is silently skipped. Once this merges and policyengine-uk-data bumps its policyengine-uk pin, the target goes live, and_compute_simple_gbpsumscouncil_tax_band_d_amountacross households while the target is a single Band D level (~£2.5k). Tracked in PolicyEngine/policyengine-uk-data#483; PolicyEngine/policyengine-uk-data#484 must merge before the pin is bumped past this PR. Both are open. Microcosm has no exposure.Tests
22 YAML cases in
policyengine_uk/tests/policy/baseline/gov/local_authorities/council_tax/simulated_council_tax.yaml:local_authorityfalls back to Maidstone (£2,502.59).At head 7de51f6: the council tax directory gives 22 passed locally, and all CI checks pass (smoke import on py3.11 to 3.14, Lint, Documentation build, Test / YAML, Test / Python 1 to 4).
Not covered
local_authorityin the Enhanced FRS (see Authority context).council_taxitself is unchanged, so CTR results are unaffected by this PR.🤖 Generated with Claude Code