Skip to content

UK pre-Budget data: measure stored values at their period, draw Carer's Allowance claims, and the #1278 items (#1324, #1319, #1278) - #1329

Merged
juaristi22 merged 21 commits into
mainfrom
uk-prebudget-1278
Oct 21, 2026
Merged

juaristi22 merged 21 commits into
mainfrom
uk-prebudget-1278

Conversation

@juaristi22

@juaristi22 juaristi22 commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

The data fixes for the 22 October national and dense build: #1324's calibration-period fix, #1319's Carer's Allowance draw, and the items #1278 lists before the pre-Budget build. It is one PR with ordered commits and one pin pass at the end, so each commit reads on its own. It must be on main by 21 October 23:59.

Closes #1324. Part of #1319: the draw lands here; its stage-health gate, the overlap rule and a bound caseload target follow after the Budget. Part of #1320: ALLPAY2 is counted once here, and the spousal-maintenance split waits on PolicyEngine/policyengine-uk-data#627. Part of #1278 and #1095.

Summary

#1324. Stored value columns measured at their binding's period

  • The bug. A plain-column binding read its value column off the frame, which stores it at the survey period (2024), and compared it with a 2025 target. UK national calibration reads stored input columns at the frame period: 76 £ rows compare 2024 values with 2025 targets #1324 lists 76 £ rows that compared 2024 values with 2025 facts: SPI component amounts, statutory maternity pay, workplace pensions and others.
  • The fix. A binding's value variable and the terms of its value_expression are now measured at the binding's period: its measurement_period, else the calibration period.
    • A frame-stored value off the frame's period goes through the measure resolver to the engine at 2025, as every engine-computed variable already does.
    • Engine values for stored names go in under a reserved engine_period__ name. Filters, conditions and bands keep their stored reads, and the stored columns are never overwritten.
    • A declared period that is neither the calibration period nor the frame's period refuses.
    • The profit after loss relief reads the frame only at the frame's period.
  • Rows that declare the frame's period.
    • ONS road fuel binds its calendar-2024 figure on the frame's own 2024 petrol and diesel columns, under a signed matched_period hold. The row no longer depends on the engine's fuel projection.
    • The seven bus and rail outturns held without an index declare the frame's period. This keeps their measurement and makes it explicit.
    • The England fare-receipt rows and the Northern Ireland bus receipts carry 2025 facts, so they go to the engine at 2025.
  • England bus fares. The England fare-receipt row now ends 9.6% above its target (10.1% with the rail row bound), against 2.3% below on the control's stored 2024 read. The cause is in policyengine-uk's pricing, not in the data:
  • The check. tools/check_uk_measurement_periods.py rebuilds each plain-column row's route from a finished run. It writes a receipt of periods and routes, and names any stored value read at a period the frame does not hold. It warns and does not fail, unless --strict is given. On the 19 October national build it names the rows UK national calibration reads stored input columns at the frame period: 76 £ rows compare 2024 values with 2025 targets #1324 lists.

#1278. Register items

Spine riders

  • State Pension at 52 weeks. policyengine-uk reads state_pension_reported back at 52 weeks, so 365.25/7 put every full-rate pensioner 0.34% above the full rate. This is an SP-only exception to UK: track the defects uk-data is fixing that microcosm still has #1095's weekly amounts.
  • Paying households' zero council-tax bills. A household that pays council tax, reports no reduction and carries a zero derived bill now takes its cell mean, as a missing bill does. The bill level waits until after the Budget.
  • ALLPAY2 is counted once, as miscellaneous income. A new test refuses any FRS field that feeds two person income columns.

#1319. Carer's Allowance claims for carers over the earnings limit

  • would_claim_carers_allowance followed reported receipt. A carer over the earnings limit cannot report the allowance, so a reform that widens it reached no new claimant. The April 2025 rise of the limit from £151 to £196 a week brought about 61,000 more people into eligibility (Independent Review of Carer's Allowance Overpayments, July 2025, chapter 7).
  • A new ungated spine stage, carers_allowance_take_up, keeps reported receipt as the anchor. It opens the flag at one rate (0.6, sensitivity 0.45 to 0.75) to carers outside Scotland whom only the earnings test keeps from the allowance. The draw is keyed on the person's identity.
  • Scotland stays closed, since Carer Support Payment reads the same flag and policyengine-uk does not yet apply its earnings limit. In law Carer Support Payment has an earnings limit, as the allowance does (Carer Support Payment: apply its earnings limit policyengine-uk#2548).
  • The receipt is aggregate-only. The stage has no health gate, and the flag stays out of the terminal take-up signal gate.

Hygiene

  • The SPI band receipt and 23 exclusion reasons are restated in aggregate terms, with the same meaning, decisions and windows. A disclosure-control item went to the author separately.
  • The uk.full.problem and holdout kernels review the local binding adjudications on the build's review date, not the wall clock.

Pin pass

  • Two commits at the end. The first regenerates the national target references and membership: 1,319 to 1,322, with measurement_period mirrored onto road fuel and the seven held bus and rail rows. The second regenerates the national compile-parity receipt, the release input coverage manifest and the charter-H2 spine fixture; the local receipt was regenerated and does not move.
  • The feed stays abf0921, and gates.json does not move.

Verification

For review

  • Register entries signed juaristi22 / 2026-10-20: the road-fuel matched_period hold (expires 2026-12-11), the plan-2 renewals (2026-11-26) and the rail entry (2026-12-11).
  • The council-tax docstring and changelog fragment now quote section 1.3 of the FRS 2024-25 methodology. It does not say whether CTPAID is asked or imputed.

Not in this PR (after the Budget)

🤖 Generated with Claude Code

juaristi22 and others added 16 commits October 20, 2026 13:43
A plain-column binding's value variable is read at its measurement period:
the binding's measurement_period, else the calibration period. When the
frame stores the variable at its own period (2024), the measure resolver's
probes report it missing, so the provider computes it at 2025, as it does
every engine-computed variable. Engine values for stored names are injected
under a reserved engine_period__ name, so filters, conditions and bands keep
their stored reads and the stored columns are never overwritten. A declared
period other than the calibration or the frame's period refuses. The profit
after loss relief reads the frame only at the frame's period.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eriod (#1324)

ONS road fuel binds its calendar-2024 figure on the frame's own 2024 petrol
and diesel columns (measurement_period 2024, require_matching_fact_period),
under a signed matched_period hold, so the row no longer depends on the
engine's fuel projection. The seven bus and rail outturn rows held without
an index declare the frame's period, which keeps their measurement and makes
it explicit. The England fare-receipt rows and the Northern Ireland bus
receipts row carry 2025 facts and go to the engine at 2025.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tools/check_uk_measurement_periods.py rebuilds each plain-column row's
value route from the run's calibration diagnostics, the resolver's attached
routes in build_record.json and the stored columns of the spine it read, and
writes a receipt of periods and route labels. It names any row whose stored
value was read at a period the frame does not hold, and warns rather than
fails unless --strict is given. On the 19 October national build it names
the 75 rows #1324 lists that are not now declared, and the three bus fare
rows that carry 2025 facts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The national graph route and the dense line's full-measure route resolve the packaged OBR statutory maternity pay row, whose value the frame stores at 2024, through the engine at the calibration period, and leave the stored column as it was.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#1278, #1095. The national 0-4 bands by sex bind, but the single years
inside them were free to move. Three national rows now bind ages 0, 1
and 2 for all persons, on the 15-19 split's precedent (2da556a). Each
reads the ONS mid-year estimate for the UK at one single year of age,
the series the 0-4 bands sum, and counts people with age >= n and
age < n + 1.

Unlike the 15-19 split, the 0-4 sex bands keep their incumbent mapping
and stay bound: the new rows cover both sexes and stop at age 2, so
neither band is their sum.

Registry accounting: the three rows are contract-only declarations. The
contract has 421 targets (418), the registry scope 362 (359) and the
unmapped declarations 176 (173); the mapped targets stay at 186. The
compile-parity tool gains the rows' ledger-only rationale.

Not regenerated here: the target references, the membership report and
the 2025 compile-parity receipt go to the PR's pin pass, which takes the
active references from 1,319 to 1,322. Until then the reference-count
tests read the committed 1,319. On a cut of the pinned feed (abf0921)
each new row resolves at K02000001 with one member at 2025.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…kings (#1278)

María's ruling of 2026-10-20 on the pre-Budget PR: the two plan-2
student-loan borrower exclusions are renewed, not made diagnostics.
slc.borrowers.plan_2_liable and slc.borrowers.plan_2_above_threshold
carry approved_on 2026-10-20 and expires_on 2026-11-26, the November
PR's renewal date; María confirmed both on 2026-10-20. They keep
tracking #1278.

Their reasons quoted SLC's academic year 2025/26 figures, but since
#868 the rows bind 2024/25 (Table 6a: 8.94M liable, 3.985M above the
threshold). They are restated on those, and on the 18 October spine,
whose student_loans stage exhausts its top-up pool at 41 percent of the
liable stock. That spine carries 3.70M liable and 1.56M above the
threshold at design weights; the 19 October national file carries 3.54M
and 1.51M at calibrated weights. All four are weighted totals over
thousands of rows. The adjudications record the renewal and the
restatement.

Two tracking fields named closed issues and now track #1278, with
nothing re-signed:
- the SLC in-year-snapshot uprating hold (#868);
- the obr.vat exclusion (policyengine-uk#1996). Its adjudication notes
  that policyengine-uk#2187 closed #1996 in 2.135.7, and that the
  rebind waits on a lock bump past 2.135.7.

The A16 block of test_packaged_exclusions_load and the consumption-tax
tracking pin follow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1278)

uk-data#482 (open) adds eight columns, band_A_amount to band_H_amount,
to storage/la_council_tax.csv: each English authority's 2026-27 amount
for one band from MHCLG Table 9, its Band D amount times the band's
statutory ratio. Like Band D they are rates, not additive household
controls. local_council_tax_band_d_rate's reason now names them, so the
next re-pin does not port them as targets.

The register is regenerated with tools/census_uk_data_target_parity.py
and its --check passes; this is the one generated file this commit
carries. test_uk_data_target_parity pins the named columns.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
policyengine-uk reads state_pension_reported back to a weekly amount at
52 weeks a year before comparing it with the full rate, so the spine's
365.25/7 put every full-rate pensioner 0.34% above the full rate and
made the excess a protected payment. In 2026-27 the full new State
Pension is £12,547.60 a year at 52 weeks, below the £12,570 personal
allowance, but it read above the allowance.

The State Pension alone now annualises at STATE_PENSION_WEEKS_IN_YEAR
(52), María's ruling of 2026-10-20 and an exception to #1095's weekly
amounts. WEEKS_IN_YEAR and the HMRC replay's SRP leaf keep 365.25/7.
The frs_spine stage declares the exception in a new
weeks_in_year_exceptions field of its annualize_periodic_amounts
operation, in sources.yaml, its source_stages.json projection and the
sources schema.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1278)

From 2024-25 the FRS no longer asks how much council tax a household
pays; DWP derives CTANNUAL from the band. Some households that answer
yes to CTPaid ("Does anyone living here pay Council Tax?") and report
no reduction still carry a zero CTANNUAL, and frs_council_tax kept the
zero, leaving England's share of zero bills well above Wales's and
Scotland's. Such a household (CTPAID 1, CTREB not 1, CTANNUAL 0) now
takes its region, band and single-adult cell mean, as a missing bill
does. A non-recipient's zero bill otherwise stays zero: CTPAID 3 (the
household does not have to pay) or CTPAID 2 (the council pays, with no
reduction reported).

frs_council_tax reads CTPAID and refuses a tab without it, and the
impute_cell_means declaration's missing rule changes in sources.yaml
and source_stages.json together. The spine test fixture and the H2
fixture tool carry CTPAID, and the tool adds one paying household with
a zero bill. The bill level waits until after the Budget.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ALLPAY2, the FRS 2024-25 "Amount of allowance: organisation", is one
amount, but the spine summed it into both miscellaneous_income and
private_transfer_income, so every total adding the two counted it
twice. It stays in miscellaneous_income only (María's ruling of
2026-10-20); private_transfer_income sums APAMT, APDAMT, PAREAMT,
ALLPAY3 and ALLPAY4.

A new test copies a fixture household once per field of its adult,
child, household, odd-job and pension rows, raises that field in its
copy, and refuses any field that moves more than one person income
column; on the old lists it names ALLPAY2. The spousal-maintenance
split in #1320 waits on uk-data#627.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
would_claim_carers_allowance followed reported receipt (#882), and a carer
above the earnings limit cannot report the allowance, so no reform that
widens it reached a new claimant. That includes the April 2025 rise of the
limit from £151 to £196 a week, which the Independent Review of Carer's
Allowance Overpayments (July 2025, chapter 7) estimates brought about 61,000
more people into eligibility.

A new ungated spine stage, carers_allowance_take_up, runs after
child_benefit_take_up and before uc_deduction_attributes. It keeps reported
receipt as the anchor (frs_spine sets it, the SPI income chain refreshes it
on the rows it redraws) and refuses a frame whose flag no longer follows it.
One consumed-only engine materialization at the survey year reads
is_gainfully_employed_for_carers_allowance on the frame's own values, at the
2024-25 limit of £151 (the engine reads a year's parameters at 30 April);
a frame at any other period is refused. A person outside Scotland whose
care_hours reach the engine's gov.dwp.carers_allowance.min_hours, who
reports no allowance and whom that test holds gainfully employed claims at
the take-up contract's new carers_allowance_newly_entitled rate, on a
uniform keyed on person_id (seed 0, salt would_claim_carers_allowance):
would_claim = reported OR (eligible AND u < rate). Reporters stay true,
carers under the limit who report no allowance stay false, and Scotland
stays closed, since Carer Support Payment reads the same flag and the engine
applies it no earnings test. The stage uses the existing
assign_binary_from_rate operation, so the schema and the operation
vocabulary do not move.

The earnings test is read before salary_sacrifice moves some
contributors' pension amounts out of their pay, which lowers the earnings
the engine tests by about half the amount moved. A drawn carer whom that
conversion takes under the survey year's limit keeps the flag, although
the released file then shows them entitled and not reporting. Reading the
converted pay would need the stage after salary_sacrifice, with a
lineage-keyed draw for the CGT clones; the module and the notes say so.

The rate, 0.6 with a sensitivity of 0.45 to 0.75, is María's ruling of
2026-10-20. No published take-up rate exists for newly entitled carers, so
the contract entry is frozen_by_adjudication, with the earnings-limit
back-test against the 61,000 as its follow-up. The receipt bands support on
distinct household_source_id, gives a weighted total only where ten or more
sources carry it, and gives the realised share only where neither side rests
on one to nine. The stage has no health gate, and the flag stays out of the
terminal take-up signal gate.

As for the Pension Credit and Child Benefit redraws, the graph opens an
isolation boundary before the rewrite and declares an open input surface.
The kernel tables, the spine registration, the stage JSON, the stale notes
on the anchor and in the policyengine-uk concept mapping, the roster pins
(45 to 46 stages) and the H2 fixture tool move with it. The fixture gains
35-hour carers on both sides of the limit, in Scotland and outside it, and
Carer's Allowance rows, so the stage draws there. The coverage manifest
and the H2 fixture are left for the pin pass to regenerate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The SPI band-donor receipt records a band's mean realised total
  income only where its carriers drew at least ten distinct tape
  records, and "<10" otherwise.
- Twenty-three measure-exclusion reasons (the 1m-and-over and
  500k-1m HMRC band cells, and the benefit-cap amount bands) are
  restated in aggregate terms. Their meaning, decisions and windows are
  unchanged.
- hmrc_spi_income_spine binds spi_income as a helper module, so an edit
  there moves the stage's implementation hash, as it already did for
  property_components. A test holds both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
)

The uk.full.problem kernel and the rotated holdout kernel reviewed the
local binding adjudications on the wall clock. The target compilation
already reviews the measure exclusions on the build's review_date, and
#1306 (E) let prepare_uk_full_solve and rotated_uk_local_holdout take a
date (77febc5, 67e54c4); the two kernels now pass it.

- uk.full.problem carries review_date in its parameters, and its kernel
  passes it to prepare_uk_full_solve as now.
- uk.full.holdout inherits it with the problem's parameters, and its
  kernel passes it to rotated_uk_local_holdout, so every fold reviews on
  it.

Tests: a prepared build puts the solve's review date on both nodes. At
the kernel level, the problem kernel passes its date and refuses one
past the committed register's windows; the holdout kernel refuses such a
date, and every fold it runs receives the build's date. These three
fail on the old kernels. The review-date changelog fragment loses its
"is a follow-up" sentence.

No effect on a build before the Budget: the earliest local
adjudication expires 2026-12-15. A changed review date now re-keys the
problem and the holdout as well as the target compilation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's ruling of 2026-10-20: run a national arm with
orr.rail_government_support held out beside the calibration-period
fix's arm (#1324). Commit the hold-out only if the gates pass and the
share of rows within 10 percent holds; otherwise keep the row bound and
label the distributional tables. The arm passed: 7/7 terminal gates,
and 99.1% of the rows both arms bind within 10 percent (98.8% with the
row bound).

The row binds ORR table 7270's broad total of government support to the
railway. The frame's household rail subsidy sits far below it at design
weights, so the solve closes the gap by moving weight onto rail-using
households. That also moves untargeted columns that correlate with rail
use. The 7270-versus-7271 basis ruling stays open (#890 G).

The entry carries approved_on 2026-10-20 and expires_on 2026-12-11,
which María confirmed on 2026-10-20, and tracks #1278. The exclusion census gains 'orr.'
and the register 58 entries (57). The national arm reads the same file
by --measure-exclusions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Regenerated with tools/generate_uk_target_references.py from the pinned feed abf0921 (source_fact_feed chronicle-uk-abf0921-consumer-facts). The active references go from 1,319 to 1,322 with the ONS single years of age 0, 1 and 2. The contract's measurement_period 2024 is mirrored onto ONS road fuel and the seven held bus and rail rows, and road fuel's selector reads its calendar-2024 observation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e fixture (pin pass)

The national compile-parity receipt (tools/build_uk_ledger_compile_parity_signed_differences.py --surface national) adds the three ONS age rows as ledger-only and reads road fuel's calendar-2024 figure; the local receipt does not move. The release input coverage manifest re-hashes the resources this PR changes, and the charter-H2 spine fixture is regenerated with the rows the fixture tool gains here. Both --check runs pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vahid-ahmadi

Copy link
Copy Markdown
Contributor

@juaristi22 thanks, this is my review at 7e5c8b37cd40737b91cc775b0c6b4b60aa6022c7.

Verdict. I found nothing that blocks the 22 October build. The #1324 period change does what the body says, and I traced the engine route end to end: a stored column read at 2025 goes to a pe-uk simulation that extends the 2024 dataset with extend_single_year_dataset, so employment_income and the other stored inputs come back uprated, not carried forward. Other builds are unaffected. The Carer's Allowance draw is sound in its mechanics, and the 61,000 figure checks out against the Independent Review. But the 0.6 rate has no empirical anchor yet, and the eligible pool is wider than the people the 61,000 counts. So any costing of a Carer's Allowance earnings-limit change on this build carries that assumption. I have six "should" items and three nits below. Only finding 1 is a code gap, and it does not bite any row today.

What I ran

  • Worktree at the head, with a fresh uv sync --all-packages --locked --extra uk (policyengine-uk 2.134.4, policyengine-core 3.33.3).
  • The touched tests and their neighbours: 16 files, engine-free plus the two touched engine tests (CA take-up, loss relief). 408 passed, 6 skipped.
  • tools/ci_test_plan.py verify: ok.
  • CI at the head: all 10 checks green.
  • The new period tests against the merge base: they import symbols the base does not have (ENGINE_PERIOD_COLUMN_PREFIX), so they cannot run there. This shows the tests are new, not that they would catch a regression.
  • Read the whole diff of the runtime code, the contract, the holds and the take-up contract. Read the policyengine-uk code the PR relies on: carers_allowance_pre_overlap, is_gainfully_employed_for_carers_allowance, carers_allowance_earnings, carer_support_payment_pre_overlap, is_carer_for_benefits, the three State Pension variables, extend_single_year_dataset and pe-core's carry-over and uprating branch.
  • Sources checked: the Independent Review of Carer's Allowance Overpayments (gov.uk, dated 21 July 2025), DWP benefit statistics February 2026, mygov.scot Carer Support Payment "If you work", and FRS 2024-25 background information and methodology.
  • Not run: any national or spine build, tools/check_uk_measurement_periods.py on a real run, and the back-tests. They need the licensed FRS/SPI inputs, which I did not use. So I have not checked the build figures in the body (98.8%, 0.61 realised share, the −11.6% to −10.1% gap).

Findings

  1. should target_materialization.py:971 (_refuse_ignored_measurement_period) and ledger_targets.py:1633 (column_at). A binding that declares measurement_period equal to the frame's period passes the guard whether or not its value variable is stored. If the variable is engine-computed (not stored), column_at returns the injected value. The resolver computes that value at the run's calibration period (2025), because it does not read the binding's period. So the row is measured at 2025 while it declares 2024, and nothing refuses. uk_value_read_routes (measurement_period.py:77) then labels it provider:... and does not flag it. This bites none of the nine rows that declare 2024 today: petrol, diesel and the bus and rail columns are stored stage cells (graph.py etb_* cells), and cgt_2024_tax carries its year in its name through _cgt_model_measure. But the next row that declares the frame's period on an engine variable will be wrong silently. Suggest the guard also refuses a declared frame period whose value variable is not in stored_columns (bar the CGT year-named measures), with a test.

  2. should carers_allowance_take_up.py, the eligible pool. The pool is persons outside Scotland with care_hours ≥ 35, no reported CA, and over the 2024-25 limit. Two things make it wider than the population the Review's 61,000 counts:

    • The FRS hours field counts care for anyone. Neither the stage nor pe-uk (carers_allowance_pre_overlap) checks that the cared-for person gets a qualifying disability benefit. pe-uk's own is_carer_for_benefits documentation says so.
    • The pool includes every over-limit carer, not just those between £151 and the 2025-26 £196. So the 0.61 "realised share" in the receipt is a share of all over-limit carers, most of whom never become entitled. It is not a take-up rate among the newly entitled (see nit A).

    Neither point changes who is paid for carers who stay over the limit, since the engine still applies the earnings test. But the number paid at baseline 2025 equals 0.6 times the weighted count of pool members who end up under £196, and that count is never reported. The Review gives no take-up estimate (I confirmed: chapter 7 has the 61,000 and nothing on claims). So 0.6, and its 0.45 to 0.75 range, is a judgement call. Ask: add to the receipt the weighted count of pool members entitled at the calibration year, as an aggregate under the existing small-cell rule. That is the earnings-limit back-test the take-up contract already promises. Compare it with the 61,000 before anyone publishes a costing of a CA earnings-limit change on this build.

  3. should carers_allowance_take_up.py (module docstring), stage order. The docstring says the stage reads earnings before salary_sacrifice moves pension amounts out of pay. A drawn carer whom that conversion takes under the limit keeps the flag. Because carers_allowance_pre_overlap needs only hours, no gainful employment and would_claim, that person is paid CA in the released file even though they reported none. That changes baseline spending, not just how far reforms reach. The receipt does not count these people. Ask: count them in the receipt now (aggregate), and keep the move after salary_sacrifice on UK Carer's Allowance: would_claim follows reported receipt, so earnings-limit reforms reach 4,600 new carers against an official 61,000 #1319's list. Draw UK QRF stages from uniforms keyed on each record's id, and tie pension contributions to pay #1332 ties pension contributions to pay and also touches frs_spine.py, so this set may move again when it lands.

  4. should Scotland stays closed, and the reason given is only half the story. The body and CARERS_ALLOWANCE_ELIGIBILITY say the engine applies Carer Support Payment no earnings test. That is true of carer_support_payment_pre_overlap, but in law CSP has a take-home pay limit: mygov.scot gives £204 a week now, the same as CA. So keeping Scotland closed is the right workaround, but the cause is a pe-uk gap: the CSP earnings test is missing. That gap also lets the engine pay CSP to any reported Scottish recipient over the limit. Suggest rewording to "pe-uk does not yet apply CSP's earnings limit", plus a pe-uk issue for the test. Until it lands, a Scottish CSP earnings-limit reform reaches no new claimant on this file.

  5. should England bus fare receipts end +9.6% (+10.1% with rail bound) after the engine route. The body's diagnosis matches the engine. pe-uk #2546 (bus fares by area and year, with a fitted fare table) looks like the fix it is waiting on. Please link #2546 from the body's "Not in this PR" item. Also note in the release notes that cap-change costings on this build likely overstate their cost, as the body says, so nobody reads them as final.

  6. should merge order with your other open PRs:

    Suggest this PR first, as it has tonight's deadline. Then Bump the UK lock to the newest policyengine-uk (2.141.1, then the #2523 fix release) #1333, with the measurement-period receipt, the CA receipt and the 52-week State Pension check re-run on that lock. Then Draw UK QRF stages from uniforms keyed on each record's id, and tie pension contributions to pay #1332.

Nits

A. carers_allowance_take_up.py receipt: rename realized_share to something like drawn_share_of_over_limit_pool, so nobody reads it as a take-up rate (see finding 2).

B. The DWP benefit statistics February 2026 page, which the take-up contract cites as caseload_url, disagrees with itself on CA: the text says entitled cases at August 2025 were 4.7% lower than a year earlier, but Figure 2 shows +62,000. That page also gives 1,376,400 entitled, of which 380,000 are entitled but not paid. Use Stat-Xplore, not that page, when the caseload target binds after the Budget.

C. frs_council_tax.py docstring: the FRS 2024-25 methodology confirms that the band and "some other Council Tax block questions were removed from the field interview from April 2024" and that the other answers "are then imputed during DWP's validation processes". It does not say CTANNUAL is derived from the band. Suggest quoting that wording instead of "DWP derives CTANNUAL from the band", and saying whether CTPAID is asked or imputed.

What looks right

  • Period routing. Only an adapter that declares stored_columns and column_at (UKFrameTargetAdapter) changes route. Every other adapter, including the US route, reads columns as before. Engine values for stored names go under engine_period__, so filters, conditions and bands keep their stored reads, and every injection site (national_calibration, full_measure per block and whole frame, the scoring path) uses the shared inject and drop. stored_columns is snapshotted before injection in each of them.
  • Uprating on the engine route. I checked this because several routed inputs (employment_income, employee_pension_contributions, loss_relief) have no variable-level uprating. The resolver's simulation loads the H5 and pe-uk's _extend_dataset_with_cgt_provenance calls extend_single_year_dataset, which uprates through uprating_indices.yaml (employment_income is listed there). So 2025 reads are projected values, not the 2024 values relabelled.
  • Bands. The SPI band rows band on total_income_after_loss_relief, which the provider computes, so band and value are both read at 2025.
  • The refusal. MeasurementPeriodError is a RuntimeError, so it escapes the per-spec except (KeyError, TypeError, ValueError) and refuses the run rather than skipping the row. All nine declared rows pass it.
  • The loss-relief guard. The frame read now applies only at the frame's period.
  • State Pension at 52 weeks. pe-core's WEEKS_IN_YEAR is 52, and new_state_pension, basic_state_pension and additional_state_pension all divide state_pension_reported by it. So 365.25/7 did put full-rate pensioners 0.34% over the rate.
  • ALLPAY2. It is now in miscellaneous_income's field list and gone from private_transfer_income, so it is counted once, not zero times. The new test refusing a field that feeds two income columns is a good guard.
  • Council tax. The CTPAID 1 zero-bill rule is narrow (paying, no reduction, zero bill) and reuses the existing cell-mean donors.
  • Carer's Allowance mechanics.
    • The draw is deterministic and keyed on person_id.
    • The stage refuses an incoming flag that does not follow reported receipt.
    • Reporters stay true.
    • It refuses any period other than the survey year.
    • The receipt is aggregate-only, with "<10" handling on both sides of the share.
  • 61,000. Verified in the Independent Review, chapter 7: "raising the earnings threshold from £151 to £196 per week, bringing an estimated 61,000 more people into eligibility".
  • Disclosure. Dropping the SPI band min and max, and returning "<10" for a mean drawn from fewer than ten distinct records, closes a real disclosure route.
  • Graph kernels. Reviewing the local binding on the build's review date, not the wall clock, makes the kernels reproducible. Adding spi_income to the SPI stage's helper modules fixes a stale-hash gap.
  • ONS ages 0, 1 and 2. These follow the 15 to 19 precedent and stay beside the 0 to 4 sex bands without double-binding a sum.
  • Childcare. This PR does not touch Tax-Free Childcare or the funded-hours take-up draws. The is_child_receiving_* names in the diff are only in a new test's list of population variables that keep their stored reads. The new age 0, 1 and 2 rows will move the weight on under-3s, so any count of under-3s will move a little in the 22 October build.

🤖 Generated with Claude Code

juaristi22 and others added 5 commits October 21, 2026 12:26
A plain-column binding that declares the frame's period (2024) while the
run calibrates at 2025 reads the stored column. An engine-computed value
variable has no stored column, so the resolver computed it at 2025 and the
row was measured there under the declared period, with nothing refusing
(Vahid's review on #1329, finding 1).

The guard now asks the adapter which period each value variable holds
without the engine. UKFrameTargetAdapter.value_period gives the frame's
period for a stored column, the year in the name for a dated CGT measure
such as cgt_2024_tax, and none otherwise. The guard refuses unless every
value_variable or value_expression term holds the declared period. The
dated-measure rule moves to cgt_calibration.UK_CGT_DATED_MEASURE, so the
measure provider and the adapter share one copy.

No packaged row changes: the plain-column rows that declare 2024 read
stored columns (petrol, diesel, bus and rail) or cgt_2024_tax.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…#1319)

Scotland stays closed in the Carer's Allowance draw because Carer Support
Payment reads the same flag. The reason given was that the engine applies
CSP no earnings test, which read as if CSP had none. In law it has an
earnings limit, as the allowance does (mygov.scot, "If you work"), and
policyengine-uk's carer_support_payment_pre_overlap does not yet apply it
(Vahid's review on #1329, finding 4).

The module, the stage's eligibility and notes in sources.yaml and
source_stages.json, and the changelog fragment now say so. The coverage
manifest and the H2 fixture follow in the pin commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The docstring and the changelog fragment said that from 2024-25 the FRS
no longer asks the amount and that DWP derives CTANNUAL from the band.
Section 1.3 of the FRS 2024-25 background information and methodology
says that the band and some other Council Tax block questions left the
field interview from April 2024, that editing teams input the band, and
that the other answers are imputed during DWP's validation. It does not
say that CTANNUAL is derived from the band, or whether CTPAID is asked or
imputed. Both texts now quote it (Vahid's review on #1329, nit C).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
England's bus fare receipts now read the engine's 2025 fares and end
about 10% above their target, because policyengine-uk gives every single
ticket in England outside London one uncapped fare. The fragment says so
and points to policyengine-uk#2542 and microcosm#1331, so that a costing
of a bus fare cap change on this release is not read as final (Vahid's
review on #1329, finding 5).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pin pass)

The Carer Support Payment rewording changes the carers_allowance_take_up
stage's eligibility and notes in source_stages.json, so the release input
coverage manifest re-hashes the stage manifest, and the charter-H2 spine
fixture carries the new text. Both --check runs pass. The compile-parity
receipts and the target references do not move.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

@vahid-ahmadi thank you, this is a careful review. The changes are on top of 7e5c8b37c in 5 commits, and CI is re-running.

1. A declared frame period on an engine value. Fixed in 9d54588.

  • _refuse_ignored_measurement_period now asks the adapter which period each value variable holds without the engine (UKFrameTargetAdapter.value_period):
    • a stored column holds the frame's period;
    • a CGT measure dated by its name (cgt_2024_tax) holds the year in its name. That rule moves to cgt_calibration.UK_CGT_DATED_MEASURE, so the measure provider and the adapter share one copy;
    • any other variable holds none.
  • When a binding declares the frame's period and the run calibrates at another period, the guard now refuses unless every value_variable or value_expression term holds that period.
  • New tests cover an engine value, an engine term in an expression, the resolver path, a dated CGT measure and a misdated one, and a run calibrated at the frame's own period.
  • All 24 bindings that declare 2024 pass. The plain-column rows read stored columns (petrol, diesel, bus and rail) or cgt_2024_tax, and the rest use provider kinds.

2. The eligible pool and the 0.6. Agreed on both points. The pool counts care for anyone, and it holds every carer over the 2024-25 limit, not only those who come under £196.

  • The stage receipt cannot carry the count you ask for. The stage reads the earnings test on the frame's own values at the survey year, and it refuses any other period, so it cannot see entitlement at 2025.
  • So the back-test runs on the built national file instead. It reports two weighted counts: the drawn carers entitled at 2025, and the whole pool under the 2025-26 limit. Both are compared with the Review's 61,000, with banded support, and no figure where either side rests on one to nine.
  • It goes on UK Carer's Allowance: would_claim follows reported receipt, so earnings-limit reforms reach 4,600 new carers against an official 61,000 #1319. No costing of a Carer's Allowance earnings-limit change on this build should be published before it.
  • And yes: realized_share is the drawn share of the over-limit pool, not a take-up rate.

3. Salary sacrifice. Agreed: such a carer is paid in the baseline, so this is not only about how far reforms reach. The same back-test counts them, as drawn carers whom the released file holds under the survey year's limit. The move after salary_sacrifice stays on #1319's list, with #1332 in mind.

4. Carer Support Payment. Reworded in 3c3b473. Scotland now stays closed "since Carer Support Payment reads the same flag and policyengine-uk does not yet apply its earnings limit". The new wording is in the module, the stage notes and eligibility, and the changelog. The coverage manifest and the H2 fixture are regenerated in 2279fe8. The engine gap is PolicyEngine/policyengine-uk#2548.

5. Bus fares. The body now links PolicyEngine/policyengine-uk#2542, the draft fix PolicyEngine/policyengine-uk#2546 and #1331. The #1324 changelog fragment now says that a costing of a bus fare cap change on this release likely overstates its cost and gains (3d765aa).

6. Merge order. Agreed: this PR tonight, then #1333 with the measurement-period receipt, the Carer's Allowance receipt and the 52-week State Pension check re-run on its lock, then #1332.

Nits

🤖 Generated with Claude Code

@juaristi22
juaristi22 marked this pull request as ready for review October 21, 2026 13:09
@vahid-ahmadi

Copy link
Copy Markdown
Contributor

@juaristi22 thanks, this is my second review, at 2279fe8211ee48d26672d0783a581f400e116e21. It covers the five commits on top of 7e5c8b37c and your reply.

Verdict. Nothing in the delta blocks the 22 October build. The new period guard closes finding 1 properly, and the Carer Support Payment, bus fare and council-tax wording are fixed. Findings 2 and 3 and nits A and B are deferred to #1319, and I accept your reason why the stage receipt cannot carry the count. One thing I got wrong in round 1: the merge order. #1333 is stacked on #1332, so the order has to be #1329, then #1332, then #1333 (see item 6). I have one new ask and one nit.

What I ran

  • Fresh worktree at the head, uv sync --all-packages --locked --extra uk (policyengine-uk 2.134.4, policyengine-core 3.33.3).
  • The engine-free tests of the touched areas, 18 files (measurement period and its receipt and routes, Carer's Allowance take-up, council tax, the CGT calibration, observation period and targets, measure simulation, coverage, source stages, ledger and population targets, the FRS spine, national calibration, full measure, and the shared target materialization): 536 passed, 1 skipped.
  • The engine tests the delta touches: Carer's Allowance take-up, CGT observation period and the release input coverage manifest: 9 passed.
  • Checked that the new guard tests bite: with target_materialization.py put back to its 7e5c8b37c version, four of the new tests fail (both engine-value cases, the resolver case and the misdated CGT case). I restored the file afterwards.
  • tools/ci_test_plan.py verify: ok.
  • CI at the head: all 10 checks green.
  • Listed every binding that declares measurement_period in uk_population_targets.json: 24, all 2024. Five are plain-column rows (road fuel, the bus and rail rows, cgt_2024_tax); the other 19 use parameter_gated_threshold. That matches your count.
  • Re-read the FRS 2024-25 background information and methodology: the council-tax wording you quote is in section 1.3, "Recent improvements to the FRS".
  • Not run: any national or spine build, or the back-tests. They need the licensed FRS/SPI inputs.

Round-1 items

  1. Fixed (9d54588). The guard now refuses the frame's period on any value term the frame does not hold at that period, and value_period is fail-closed: anything that is not a stored column or a name-dated CGT measure returns None. I checked the two things that could break it. frame_period is always a string on the UK adapter (ledger_targets.py:1629, from the national frame's string time_period), so the string comparison does not refuse stored rows by type. And UKFrameTargetAdapter is the only adapter with a frame_period, so no other route skips the new check. Sharing UK_CGT_DATED_MEASURE between the provider and the adapter is the right way to keep them in step.

  2. Deferred, agreed. Your reason holds: the stage reads the earnings test at the survey year and refuses any other period, so the 2025 entitled count has to come from the built file. Running both counts against the 61,000 there is better than what I asked for.

  3. Deferred, agreed. Counting the salary-sacrifice cases in the same back-test is fine.

  4. Fixed (3c3b473, 2279fe8). The wording is right in the module, the stage notes, the eligibility text and the changelog, and policyengine-uk#2548 is open for the engine gap.

  5. Fixed (3d765aa). The body links policyengine-uk#2542 and #2546 and UK bus fares: keep in step with the engine's per-year fares, take the single-ticket share from cap-regime years, and check cap savings against NTS fares paid #1331, and the changelog carries the caveat. One update: #2546 was approved today but is still a draft and not released, so it is not in Bump the UK lock to the newest policyengine-uk (2.141.1, then the #2523 fix release) #1333's 2.141.1 lock. The caveat stays true after the bump, until a later pe-uk bump picks #2546 up. The body still calls it "the draft fix"; "approved, not yet released" would be more accurate.

  6. Withdrawn, my order was wrong. Bump the UK lock to the newest policyengine-uk (2.141.1, then the #2523 fix release) #1333's base is uk-keyed-draws-pension-coherence, which is Draw UK QRF stages from uniforms keyed on each record's id, and tie pension contributions to pay #1332's branch. So Bump the UK lock to the newest policyengine-uk (2.141.1, then the #2523 fix release) #1333 cannot reach main before Draw UK QRF stages from uniforms keyed on each record's id, and tie pension contributions to pay #1332 without retargeting, and retargeting would carry Draw UK QRF stages from uniforms keyed on each record's id, and tie pension contributions to pay #1332's commits with it. My Draw UK QRF stages from uniforms keyed on each record's id, and tie pension contributions to pay #1332 review already said Draw UK QRF stages from uniforms keyed on each record's id, and tie pension contributions to pay #1332 then Bump the UK lock to the newest policyengine-uk (2.141.1, then the #2523 fix release) #1333. The one order is:

    1. UK pre-Budget data: measure stored values at their period, draw Carer's Allowance claims, and the #1278 items (#1324, #1319, #1278) #1329 tonight.
    2. Draw UK QRF stages from uniforms keyed on each record's id, and tie pension contributions to pay #1332, rebased on main after UK pre-Budget data: measure stored values at their period, draw Carer's Allowance claims, and the #1278 items (#1324, #1319, #1278) #1329. It shares frs_spine.py, spi_income.py, calibration_measure_exclusions.json and the parity fixtures with this PR, so it regenerates the H2 fixture and fixes the register count.
    3. Bump the UK lock to the newest policyengine-uk (2.141.1, then the #2523 fix release) #1333, rebased on that, with the measurement-period receipt, the Carer's Allowance receipt and the 52-week State Pension check re-run on its lock.

    Your reply agreed to my round-1 order, so please treat this as the correction.

  • A. Deferred, agreed. One receipt change with the back-test fields is better than two.
  • B. Noted.
  • C. Fixed (d47b48c). The quoted wording and the section number match the methodology, and saying it does not state whether CTPAID is asked is the honest answer.

New

  1. should UK Carer's Allowance: would_claim follows reported receipt, so earnings-limit reforms reach 4,600 new carers against an official 61,000 #1319 does not yet carry what this thread settled. Its body still has only the general back-test item, and it has no comments. Please add one comment there listing: the two weighted counts on the built national file (drawn carers entitled at 2025, and the whole pool under the 2025-26 limit) against the 61,000; the salary-sacrifice count; the realized_share rename; the stage move after salary_sacrifice; and "no costing of a Carer's Allowance earnings-limit change on this build before the back-test". Otherwise they live only in this PR's thread, which closes when it merges tonight.

  2. nit target_materialization.py:1002-1012. The guard checks the value terms only. A filter, condition or band on an engine-computed variable would still be read at the calibration period under a declared frame period. None bites today: the five plain-column rows filter only on country or region, which are stored. A one-line note in the docstring would stop the next person assuming the guard covers filters too.

What looks right

  • The guard splits a value_expression on + exactly as _expression does, and takes the entity the same way as _prepared_column_values (from_entity, then the spec's entity), so it checks the columns the read actually uses.
  • The early return when the declared period equals the calibration period is right: the engine then computes every value at that period. The new test covers it.
  • The misdated CGT case (cgt_2025_tax declared at 2024) refuses, and the resolver path refuses too, so both routes are covered.
  • The regenerated coverage manifest and H2 fixture change only the Carer Support Payment wording and their hashes.

🤖 Generated with Claude Code

@juaristi22
juaristi22 merged commit 211483f into main Oct 21, 2026
10 checks passed
anth-volk pushed a commit that referenced this pull request Oct 21, 2026
Merges origin/main at 37c1684 ("Merge pull request #1297 from
PolicyEngine/orrery-graph-reconstruction"), which brings in #1297 as
merged (head 065a434) and #1329 (211483f, with the #1278, #1319,
#1320 and #1324 work it carried).

No file conflicted. #1297's merged head 065a434 has the same tree as
the pre-rebase #1297 head 94a5bff that this branch had merged, so
every #1295 and #1297 file now equals main's version.
docs/uk-full-build-graph.md merged automatically: it keeps #1329's
46-stage spine count and this branch's per-epoch telemetry sentence.

After the merge, this branch's diff against main is the same nine-file
patch it had against the #1297 head 94a5bff.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anth-volk pushed a commit that referenced this pull request Oct 21, 2026
…-schema

Merges #1296's branch at 0b766e7, which merged origin/main at
37c1684 (#1297 merged as 065a434, and #1329 at 211483f).

No file conflicted. packages/microcosm-build/tests/engine_free/uk/
test_uk_graph_terminal.py merged automatically: it keeps #1329's new
tests and this branch's JSON Schema test of the final validation-check
kernel's diagnostics. docs/agent-guide.md keeps #1297's version from
main plus this branch's JSON Schema section.

Nothing was regenerated. main has not changed microcosm-diagnostics,
any pyproject.toml or uv.lock since this branch last merged #1296, so
generate_calibration_diagnostics_json_schema.py --check reports the
committed schema current, uv lock --check passes, and
APPROVED_UV_LOCK_SHA256 still equals the SHA-256 of uv.lock
(751721b231ab8cc6b3d10164f61ca429f5171661951494503fb0873e3f67b117).

After the merge, this branch's diff against #1296's branch is the same
22-file patch it had against b0406fd; only the hunk offsets in
test_uk_graph_terminal.py moved, because #1329 added tests above them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Oct 21, 2026
The keyed draws change the hermetic H2 fixture's CGT asset-type facts and
property purchase targets, and main (#1329, #1295) moved the same two
files, so they are regenerated on the rebased tree with
tools/graph_uk_spine_fixture.py; --check then passes. They are re-derived
synthetic aggregates with no record-level data.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

UK national calibration reads stored input columns at the frame period: 76 £ rows compare 2024 values with 2025 targets

2 participants