Repository navigation
Write policyengine-uk's alcohol, tobacco and gambling duty inputs from the LCFS stage - #1381
juaristi22 wants to merge 24 commits into
Conversation
|
@juaristi22 thanks, this is my review at Verdict. The engineering is careful. The three shares are appended to the chain, so 17 of main's 20 columns stay byte-identical. Every level is recomputed from the vendored rows, and every unit and divisor is checked against the engine. The disclosure handling is clean. I found one must, and it's a merge gate, not a method error: this has to land after #1383 and be regenerated on top of it. The shoulds are about what the levelled inputs do at household level. Duty is levelled to receipts, but tobacco and gambling spending stay at diary level and diary incidence is kept, so every tobacco household now pays about 2.1 times its recorded tobacco spending in duty, and every gambling household about 1.05 times its recorded stakes. That duty sits on a much smaller share of households than smoke or gamble. The national totals can be read off today without the measurement arms, and I give them below. Alcohol comes out 4% above OBR and tobacco 3% above. Gambling matches by construction, so that row validates nothing. What I ran
What is generated and what I read
Findings
What looks right
🤖 Generated with Claude Code |
e07424d to
352af5e
Compare
|
@juaristi22 round 2, at What I checked
Status of round 1
New findingA. should: the restack cuts the frame's gambling stakes by a third, so every gambling household now pays 1.6 times its stakes in duty.
Merge order#1383 first, and this PR after it. Nothing in #1383 since 🤖 Generated with Claude Code |
|
@vahid-ahmadi thanks for the review. This round is three commits on the restacked branch:
Tests, on the touched surfaces:
A stage-only run of the split on a real spine leaves 33 of the 46 columns byte-identical to the restacked head. Only the 13 gambling inputs move, and all eight LCFS gates pass. 🤖 Generated with Claude Code |
microcosm#1359, step A1. The chain draws two shares after its twenty targets (the eighteen spend targets and the two heating fuels): tobacco's share of the off-trade alcohol and tobacco that make up P602, and on-trade alcohol's share of restaurants and hotels (P611). The derive step declares them and computes them on the donor from the adult diary lines (LCF 2024-25 Volume F), with refunds as zero, a zero denominator as zero and the drinks past P611's one-decimal rounding held at one. A household's tobacco, off-trade and on-trade alcohol spending is its share of the total it drew, so the parts sum to the totals and no total, the VAT base included, moves; appended last, the shares leave the twenty draws bit-identical (tested against the chain without them). The shares and the spending they give are scratch, on no table and outside the support clip. A report-only excise_shares receipt sets each share on the frame beside the donor's, in bands and withheld shares only. The spec, schema and source_stages.json declare the shares; the coverage manifest's source hash and the H2 fixture tool's synthetic donor follow. The H2 fixture itself is regenerated once, at the end of the PR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ared microcosm#1359, step A2. The step gives each household its part of each national excise quantity in proportion to its spending at prior weights: alcohol volumes by its off- and on-trade alcohol spending (in proportion to the two, or each channel's published share of the volume to that channel's spenders), tobacco clearances by its tobacco spending. Each policyengine-uk input it writes names its published row, unit, share of the row and, for hectolitres of alcohol, the strength (with the engine parameter it mirrors) that turns them into litres of the product. The fiscal year is the one policyengine-uk charges excise over, so no bridge is needed; weekly inputs are the annual quantity over the engine's 52 weeks. Outputs that share a row must take all of it. The one-factor scaling level_road_fuel used is now _scale_to_level, shared by both and unchanged for road fuel. The excise_quantities stage-health check recomputes every level from the vendored rows through the stage's own declaration and holds the frame totals, factors, incidence, basis, share range, share support and the published total's residual beside its lines to the receipt. Nothing declares the step yet: its facts wait for PolicyEngine/chronicle#381, so its tests read synthetic rows (test_support/microcosm_build/uk_excise_facts.py), never publisher ones. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1359, step A3. The export-surface gate checks a column only against the enhanced-FRS reference, so an engine test holds the excise step to policyengine-uk 2.145.0: every input it can write is a household, yearly, float input with no formula; the engine's 52 weeks are the step's; each declared strength equals the engine parameter it names; and alcohol and tobacco duty are positive, proportional to the quantities and in household tax for 2024, 2025 and 2026. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1359. Chronicle applies the publisher's scale, as its hydrocarbon-oils package does: OBR's thousand hectolitres of alcohol, million sticks and thousand kilograms arrive as litres of pure alcohol, sticks and kilograms, with the published unit kept only as a dimension (PolicyEngine/chronicle#381, branch uk-excise-facts-381). The step declared the published units and scaled by 1e5 and 1e6, which would have put the alcohol volumes 1e5 too high. It now declares the stored units (litre_of_pure_alcohol, litre, stick, kilogram, scaled by one or a thousand) and refuses a row whose own unit is not the declared one, so a value can no longer be read in the wrong scale. The synthetic facts move to the stored units with the same levels. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1359. María ruled on 2026-10-23 to keep spending weights for the alcohol volume: no on-trade share of volume can be archived, since MESAS is a dashboard of NielsenIQ and Circana data with no fixed file. The rule sets no level, but a pound buys the same volume in a pub as in a shop, where a unit costs about three times more. The rule's comment records the measured size on the LCFS 2024-25 diary against a quarter of the volume sold on-trade: a pub pound carries about 1.6 times its volume and a shop pound about 0.8 times, about a tenth of the duty sits with other households, and the bottom three income deciles bear about 7 % too little of it and the top decile about 7 % too much. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1359, step C1. A third share follows the two excise shares in the chain: gambling stakes over recreation (P609), from the seven COICOP 09.4.3 stake lines (football pools, bingo, other lotteries, bookmaker and tote betting with gaming machines, the Irish Lottery, National Lottery scratchcards and draws; LCF 2024-25 Volume F, Part 2 rows 1631-1649). Like the others it is computed on the donor from the weekly lines, clipped at one because the tab stores P609 to one decimal place, appended last so no earlier draw moves, and scratch: a household's stakes are its share of the recreation it drew, so the total and the VAT base keep their draw. The synthetic donors and the H2 fixture tool carry stake lines. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…m#1359) level_excise_quantities now also writes: - beer_draught_share and cider_draught_share, each the clearances under draught relief (alone or with small producer relief) over all of that product's clearances, in litres of pure alcohol; - policyengine-uk's thirteen gambling duty bases (PolicyEngine/policyengine-uk#2590, from 2.151.0). A single-duty base is HMRC's receipts for the duty, or Machine Games Duty's liabilities for the band, over the engine's rate, so the engine charges back the published receipts. General Betting Duty's base, its receipts over the fixed-odds rate, is split over its five inputs by the Gambling Commission's gross gambling yield for Great Britain. Every base goes to households by their drawn gambling stakes at prior weights, the alcohol and tobacco rule. A gambling series may pin dimensions of its own, such as the cash accounting basis. The vendored resource is renamed excise_facts.json: it carries HMRC's clearances, receipts and liabilities and the Commission's yield. The stage-health gate checks the receipt's draught shares against the declared rows. The engine test checks every new input and rate against policyengine-uk, and that gambling duty is linear in the bases and reaches household tax; it needs policyengine-uk 2.151.0 or later. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…microcosm#1359) The feed moves from abf0921 to 200bf6d55, Chronicle main after PolicyEngine/chronicle#384 (1,502,291 rows, up 2,990; the schema digest is unchanged). It brings HMRC's alcohol and tobacco clearances, betting and gaming duty receipts and Machine Games Duty liabilities, the Gambling Commission's industry statistics for Great Britain and OBR's excise lines. It also replaces the June 2026 Consumer Trends edition with September's. - excise_facts.json vendors the 98 FY2024-25 rows the LCFS excise step reads. Every other vendored resource carries the same rows under the new feed identity, except ons_household_expenditure_facts.json, whose 93 rows take the September values. - Five OBR rows join the national contract, held out of the objective: beer and cider, wine and spirits duties accrued, and tobacco and betting and gaming duties cash. Each is measured at FY2024-25 (measurement_period 2024, require_matching_fact_period, a matched_period hold), and diagnose_uk_consumption_taxes.py measures them at that period. The exclusions expire on 2027-01-08 and the hold on 2026-12-11. - The September revisions move three compiled values. Calendar-2024 household road fuel (07.2.2, bound) goes from GBP 37.025bn to GBP 35.460bn; the measured calendar-2025 electricity and gas rows also move. - The local surface is byte-identical but for the membership's feed label. The national references and parity were regenerated from a cut of the pinned artifact without its constituency, MSOA and county rows. The local ones came from a cut to the constituency, local-authority and non-small-area statistical-scope rows. docs/uk-chronicle-feed-repin.md records the entry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…icrocosm#1359) The lcfs_consumption stage now declares level_excise_quantities on the vendored excise_facts.json. It reads HMRC's FY2024-25 clearances of beer, cider, wine, spirits and other fermented products at the engine's strengths, the beer and cider draught shares, the four tobacco products and the thirteen gambling duty bases. Each is shared out by the drawn spending at prior weights, with alcohol volume following spending. - The stage writes 24 more household outputs, all nonnegative, after its existing ones and in declared order. The schema, the allowed operation kinds and the graph's typed cells know the step, and the export allow-list (code and gates.json) carries the inputs. - The stage-health gate uk_stage_lcfs_consumption_excise_quantities recomputes every level from the vendored rows. It joins the spine gate scope and the microcosm-data contract mirrors, whose digests are re-pinned. - consumption_drift reports each quantity and base at design and final weights. - The stage notes document the spending rule's lean towards on-trade drinkers and the gambling mapping. The engine checks of the gambling inputs need policyengine-uk 2.151.0 or later. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e-pin (microcosm#1359) lcfs_consumption declares 24 more outputs, which flow into the inputs of the stages after it. Every stage that declares a re-vendored resource moves its stage_contract_sha256. The synthetic LCFS donor carries the alcohol, tobacco and gambling diary lines the shares read. The sources are unchanged, and H2 parity passes on policyengine-uk 2.151.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
experiments/1359-excise-quantities-receipts.md gives: - the national figures the step levels to; - a stage-only run on a real spine against microcosm#1383's head; - the re-pin's value moves; - the tests. docs/evidence/uk-excise-quantities-1359/ holds the scripts and the sanitized outputs: aggregates only, with no column digests or donor extremes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…crocosm#1359) Vahid's review: every gambling base followed one stakes total, so National Lottery players carried remote gaming and betting duty. The chain now draws two more shares after the gambling stakes, appended so every earlier draw stays bit-identical: - the National Lottery's share of the stakes (c94316, c94319); - the betting and gaming share of the rest (c94311, c94312, c94314). Lottery Duty's base follows the National Lottery stakes and the other twelve bases follow the betting and gaming stakes. Society lotteries (c94313) and the Irish Lottery (c94315) pay no UK gambling duty and carry no base. Each share rests on 30 or more donors. The stage notes also state the known biases the review asked for: - each duty as a multiple of the spending that shares it out; - incidence against published prevalence, and where the frame loses it; - vaping inside other tobacco; - the national draught shares; - the overstated remote UK horse-racing base. A stage-only run on a real spine leaves 33 of the 46 columns byte-identical to the restacked head. Only the 13 gambling inputs move, and all eight LCFS gates pass. source_stages.json, the coverage manifest and the H2 fixture re-derive. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ocosm#1359) Vahid's review: nothing noticed if a rate or strength change moved the duty the levels give. A new engine test runs the committed declaration's national levels through policyengine-uk as one household. Every duty is linear in its inputs, so this is the frame's duty at prior weights. The test pins each OBR row's gap at FY2024-25: - beer and cider +8.1%, wine +2.6%, spirits +2.2%; - tobacco +3.0%; - betting and gaming +0.1%. The proportionality tests also check that household_net_income loses the duties. The five measure exclusions now state those figures. Betting and gaming gets a reason of its own: its bases are receipts over the engine's rates, so the row is an identity, measured for weight drift and not as evidence. Its gap is Machine Games Duty's liabilities against its cash receipts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The receipts answer Vahid's review with aggregates only: - the engine's duty on the national levels against OBR's FY2024-25 rows, with betting and gaming shown to be an identity; - each duty as a multiple of the spending that shares it out, and the share of households carrying it, against ONS and Gambling Commission prevalence; - where the frame loses incidence against the donor: in the share draws, not the parent totals; - the gambling split's stage run, and how it concentrates the betting and gaming duties on the diary's incidence. The changelog names household_net_income, which falls by the three duties (about £24.8bn at FY2024-25), and states the multiples. docs/evidence/uk-excise-quantities-1359/ gains the reconciliation and incidence scripts and their outputs, and the split run with its comparison. Every donor share in them rests on 30 or more donors. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#1383 merged with these open, so they land here. Item 33: the withholding check refuses a local row that names no contract target. Such a row used to join no family block, so nothing was withheld beside a thin cell of its family. A contract test keeps each parentless local family free of a higher-grain row of its measurement. Today those are the two DESNZ kWh, three equivalised income and housing cost, and two private rent families. The synthetic local rows in the gate, terminal and Orrery tests now name a contract target that no family holds. Item 32: a build computes the withheld rows once. - The national path's diagnostics take the set that the national gates flagged in their calibration evidence. - The Orrery result summary takes the set recorded by a gate that reads the same result, when that gate saw every summary row; otherwise it counts. - A local surface's check logs its blocks, block checks and seconds to stderr, so the first real local surface times it. Item 34: on the FRS 2024-25, fewer than ten code-6 amounts sit above Widowed Parent's Allowance's rate at no Bereavement Support Payment rate. The stage comment says so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…m#1359) #1383's last commits level each COICOP division on the spend the chain drew against the donor's same spend. After the rebase onto main, recreation comes out at £151.0bn rather than £148.7bn. The gambling stakes are a share of it, so they rise to £2.31bn. - Lottery Duty is now about 0.55 times National Lottery stakes (was 0.56), with a base 4.6 times them (was 4.7). - The other gambling duties are about 6.3 times betting and gaming stakes (was 6.4). Incidence, duty per carrying household, and the alcohol and tobacco figures do not move. The stage notes, changelog and receipts say so, and a stage run on the rebased head is in the evidence. All eight LCFS gates pass, and 26 of the 46 columns match the pre-rebase head. The others are the seven levelled divisions and the 13 gambling inputs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
03427be to
e80a424
Compare
|
@juaristi22 round 3, at First, a correction. Round 2 reviewed What I ran
Status of earlier findings
New findingsB. should: the National Lottery share draw loses 45% of the betting and gaming households.
C. nit: the national build trusts the gates' set without checking which rows it covers ( Decisions still open
Merge orderThis PR, then my #1373. #1373 already conflicts with main in the same four files (the coverage manifest, 🤖 Generated with Claude Code |
|
Posting the completed local duty-by-income-decile measurements and isolated accounting check. These results describe the measured snapshot, not the latest PR head. Measured feature: Measured accounting effect. On the actual post-stage frame, I compared populated excise inputs with an in-memory zero-excise control, holding demographics, recorded spending and weights fixed. This isolates the duty accounting effect; it is separate from a parent-branch comparison involving the Chronicle re-pin.
At 2024 policy, the duty total comprises £13.062bn alcohol, £8.167bn tobacco and £3.619bn gambling. The roughly £25bn fall in Duty by income decile versus ONS ETB. Core specification: income-policy 2025, duty/quantity period 2024. Each cell is model / ONS, rounded to whole pounds of annual household duty. Whole-household amounts are attributed to each individual and averaged with person weights. Deciles rank equivalised disposable income after student-loan repayments and before indirect taxes, using the modified-OECD couple-normalised scale.
The comparison shows substantial incidence differences: alcohol in decile 10 is £1,249 versus £695 (about +80%); tobacco in decile 1 is £358 versus £173 (+107%), while decile 10 is £94 versus £148 (−36%); gambling in decile 9 is £228 versus £84 (+171%) and decile 10 is £292 versus £114 (+156%). These descriptive gaps do not isolate their causes. A separate income-policy-2024 re-ranking sensitivity preserves the national duty totals and changes some decile means. Validation and limits. One actual LCFS stage run; all eight stage gates passed. All 46 stored-column totals reproduce the saved split-stage totals within relative tolerance 1e-8 / £0.01. Historical row hashes were unavailable, so this does not establish identical household allocation. Independent recomputation from captured actual arrays verified both income-year specifications, weights, equivalence scales, student loans, tie/boundary allocation, row-order invariance, source support and accounting. Duty means agree within £4.3e-10 and decile totals within £0.0032. Each reported duty cell passes the declared minimum source-support check, with a 30+ distinct positive-duty source-household band. No household records, IDs or exact carrier counts are included here. The source/result frame period is 2024 and its weight kind is IMPORTANCE. No fresh national calibration was performed. The ONS Table 2b benchmark covers April 2023–March 2024 (FYE 2024; fiscal-start 2023). Frame values are supplied at each requested policy period without inflation adjustment. These are nominal comparisons across unmatched years at existing stage weights. Final calibrated incidence and a matched-year ONS replication remain unmeasured. Alcohol combines beer/cider with wines/spirits; gambling uses the ONS betting-tax row, excluding the separately reported Allwyn National Lottery Fund contribution. The previously missing stage-only decile measurement is now available for this snapshot. Final-weight national solves and measurements at subsequent heads remain separate. |
B.1 to B.4's measures now carry footnote i, which defines "excluding lottery draw only" (only people whose sole gambling was lottery draws are left out), and the non-response footnotes, and their rows guard those cells. Every fact carries an age >= 18 constraint, and each age band its bounds, as the ASHE packages do. The docs say what the tables can and can't answer for PolicyEngine/microcosm#1381, with the Commission's caveats, and the attribution names NatCen, the University of Glasgow and the authors. Values and counts are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d (microcosm#1359) The LCFS diary records a stake for about a fifth of the households that GSGB's four-week participation implies, so the excise levelling shares each gambling base over too few households at too high an amount. A new top_up_gambling_participation step takes how many households play from the Gambling Commission's adult rates by sex and age band, combined within a household at a declared theta (0: one player per household, 1: adults who play independently), and which households and how much from the diary: - within nine strata (adults, youngest adult's age band), households without a stake are added by the gas connection walk's weight-fitting rule, in an identity-keyed order drawn in proportion to a regularised donor propensity times the household's published chance, first to the four-week share (regular players), then to the twelve-month share (occasional players); - an added household's stake comes from a positive-only QRF of the donor's stakes at a quantile keyed on its id; an occasional player's is the declared share of it; - households the diary saw keep their stakes, and level_excise_quantities shares each base by the topped-up keys, so every base still sums to its published figure. The receipt gives shares only, and a stratum's figures only where each of its cells rests on ten or more source households. The stage-health check gambling_participation recomputes the adult rates from the vendored rows and checks the settings, each stratum's shares, the stakes and that the levelling read the topped-up keys. No stage declares the step yet: it reads gambling_participation_facts.json, which waits on the GSGB tables in Chronicle and a feed re-pin. The gate is registered with the declaration. Synthetic facts and a synthetic population in test_support exercise the step, the stage and the gate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…m#1359) The stage-health check gambling_participation becomes the release-blocking gate uk_stage_lcfs_consumption_gambling_participation: gates.json, the spine gate scope, the microcosm-data contract mirrors and the battery's parameter vocabulary (maximum_stratum_shortfall), with the gate digests re-pinned. The kind joins the source manifest and the sources schema, and the vendor selection names gambling_participation_facts.json: the 426 Great Britain rows of the Gambling Commission's GSGB 2025 participation concept (PolicyEngine/chronicle#391). The step is declared once the feed carries them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # packages/microcosm-build/src/microcosm/build/uk/release_input_coverage_manifest.json # packages/microcosm-graph/tests/fixtures/parity/uk_spine/uk_spine.json
GSGB 2025 gambling participation by sex, age, area and activity, for PolicyEngine/microcosm#1381
… (microcosm#1359) The top-up scaled every added household's drawn stake by the frame's mean stake over the donor's. The walk picks households whose predictors draw larger stakes, so on the real spine the added regular players' mean came out about 1.9 times the seen households' for betting and gaming (about 1.0 for the National Lottery). The scale now brings the added regular class's mean to the seen households' (the donor's positive mean where the frame drew none); occasional players keep a tenth of it. Who is added does not move. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…bles (microcosm#1359) PolicyEngine/chronicle#391 adds the Gambling Commission's Gambling Survey for Great Britain 2025 tables (1,024 facts; 1,503,315 rows). The merge commit's tree is the pull request head's, merged onto an unmoved base, so the artifact was built from that head's bundle and carries the merge commit's digests. The new vendored gambling_participation_facts.json holds the 426 Great Britain participation rows the LCFS top-up reads, and the package declares it. The other vendored resources keep their rows under the new feed identity. The national and local reference files are byte-identical, the membership moves only its feed label, the local census restates the pin, and both compile-parity receipts are unchanged; both surfaces were regenerated from the national and local cuts. The pin tests restate the new identity, and the runbook records the re-pin. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…m#1359) The stage now brings its two gambling keys to the Gambling Commission's GSGB 2025 participation before the excise levelling, as María ruled on 2026-10-25. The declaration reads the 426 vendored participation rows: the population rates from tables B.3 (four weeks) and B.1 (twelve months), each activity's own row from table B.9 (the all-channels row, or the online row of the two online-only instant-win games), seven age bands, theta 0.9 for the National Lottery and 0.08 for betting and gaming, nine household strata, an occasional stake share of a tenth, and the propensity's settings. Theta is fitted on the LCFS donor so that the declared rates, applied to the donors' own adults, give the diary's ratio of the two-adult to the one-adult household rate: 0.897 and 0.081, beside them in the evidence (scripts/theta_calibration.py, aggregates only). On a stage-only run on the 78,440-household spine, every LCFS gate passes, and every stratum lands on its share: 62.7 percent of households carry Lottery Duty and 29.8 percent the other gambling duties, against 10.7 and 2.0 percent on the diary's draws. The stage notes and the excise changelog fragment describe the step and its known biases; the lockstep tests list its operation and seed, and the stage tests expect its four fits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… top-up (microcosm#1359) The vendored headers restate the 272c987 feed identity, and the LCFS stage declares top_up_gambling_participation and its resource, so the fixture's LCFS declaration and its contract hashes move. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@vahid-ahmadi this is ready for another look. Since your round 3 at What changed
Results on the real spine (78,440 households, stage-only, policyengine-uk 2.151.0). All nine LCFS gates pass, and every stratum lands on its share. 62.7% of households now carry Lottery Duty and 29.8% the other gambling duties, against 10.7% and 2.0% before. Betting and gaming duty is now about £430 a year per regular household, where it was about £4,700 on 2.0% of households. Where your review would help most
Still open
Could you re-review? 🤖 Generated with Claude Code |
…e-pin (microcosm#1359) The evidence for the top-up, aggregates only: - stage_run_gsgb.py runs the stage on microcosm#1333's spine twice in one process, as declared and without top_up_gambling_participation, on the final head. All nine LCFS gates pass; every stratum lands on its share; 62.7 and 29.8 percent of households carry Lottery Duty and the other gambling duties, against 10.7 and 2.0 without the step. The added regular players' mean stake equals the seen households', and the occasional players' is about a tenth of it. The output also holds each equivalised-income decile's share of households and of the base, and the regional shares beside GSGB's regional rates (B.2, B.4), the out-of-sample check. - repin_272c987.sh (with quiet_footprint.sh) is the re-pin as run. The README names the two results left for review: carrying rises with income, which moves the betting and gaming base towards the top decile, and outside London the regional pattern follows GSGB only weakly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@juaristi22 round 4, at What I ran
Status of earlier findings
New findings
Decisions still open
VerdictReady to merge once findings 1 and 2 are reflected in the stage notes, whatever you decide on the gradient. Nothing here needs code changes before merge, unless the reruns in finding 1 change your mind about the feature. Merge order is unchanged: this PR, then #1373. 🤖 Generated with Claude Code |
Closes #1359.
policyengine-uk charges alcohol, tobacco and gambling duty on household inputs that Microcosm never wrote, so all three duties scored zero on Microcosm data. With this PR the LCFS stage writes those inputs, levelled to published FY2024-25 quantities and duty bases. The gambling inputs sit on the households the Gambling Commission's survey says play, not only on the fifth the diary records. No spending column moves.
What changes
Five scratch shares in the LCFS chain. The chain draws them on the donor's diary lines, appended after its twenty targets (the eighteen spend targets and #1383's two heating fuels), so every existing draw is bit-identical:
A household's parts are its shares of the totals it carries.
The new
level_excise_quantitiesstep. It shares each FY2024-25 national figure out to households in proportion to the spending that buys it, at prior weights. The figures are vendored asexcise_facts.json:Sparkling cider, heated tobacco and spread betting have no published quantity and keep the engine's zero.
The new
top_up_gambling_participationstep (María's ruling of 2026-10-25). The diary records a stake for about a fifth of the households that GSGB's four-week participation implies. Under any steady rate of play, a two-week diary sees at least half of the four-week players, so the gap is play the diary misses, not its window. Before the levelling, the step brings the two gambling keys to GSGB 2025:The keys only re-share the bases, so every base still sums to its published figure, and the OBR rows below do not move.
Gate, export and diagnostics.
uk_stage_lcfs_consumption_excise_quantitiesrecomputes every level from the vendored rows.uk_stage_lcfs_consumption_gambling_participationrecomputes the GSGB rates from the vendored rows and checks every stratum's shares, the stakes, and that the levelling read the topped-up keys.consumption_driftreports their totals at design and final weights.Five measured OBR rows, held out of the objective. At prior weights, the engine's duty on the national levels against OBR's FY2024-25 receipts:
An engine test pins each gap. Each row is measured at FY2024-25 (
measurement_period: 2024,require_matching_fact_period, amatched_periodhold), because policyengine-uk holds these inputs flat after 2024 (PolicyEngine/policyengine-uk#2596).Two feed re-pins:
200bf6d(UK excise facts for chronicle#381: OBR excise lines, HMRC alcohol, tobacco and betting and gaming statistics, Gambling Commission GGY, and the September Consumer Trends release chronicle#384): 1,502,291 rows, 2,990 more;272c987(GSGB 2025 gambling participation by sex, age, area and activity, for PolicyEngine/microcosm#1381 chronicle#391): 1,503,315 rows, adding GSGB 2025's 1,024 facts. The new vendoredgambling_participation_facts.jsonholds the 426 Great Britain participation rows. No compiled target moves: the reference files are byte-identical, the membership moves only its feed label, and both compile-parity receipts are unchanged.Read before reviewing
Net income falls by the three duties.
household_taxrises by about £24.8bn at FY2024-25, sohousehold_net_income, and every net-income, decile and inequality output read from it, falls by the same on Microcosm data. HBAI incomes and the VAT base don't move.Alcohol and tobacco duty is a fixed multiple of recorded spending. The levels are national while spending and incidence stay at diary level (the ruling of 2026-10-23), so each household's duty is one multiple of its own spending: tobacco 2.22 times tobacco spending, alcohol 0.41 times alcohol spending.
Tobacco duty sits on fewer households than smoke. 8.2% of households carry it, against 11.9% of UK adults who smoked in 2023 (ONS). The parent totals keep the donor's incidence; the loss is in the share draws.
Gambling duty now sits on GSGB's participation. On a stage-only run on the real spine, every stratum lands on its target. 62.7% of households carry Lottery Duty and 29.8% the other gambling duties, against 10.7% and 2.0% on the diary's draws. At prior weights:
The top-up carries the diary's income gradient. Carrying rises with equivalised income: the National Lottery from 52% of households in the bottom decile to 70% in the top, and betting and gaming from 24% to 36%. Age and sex come from GSGB; income, tenure, region and spending come from the donor propensity. If the diary misses online play more, and online play rises with income, the top-up leans low in income.
Regions, the out-of-sample check, match only partly. Region is not a target. Comparing each region with its own national level, London is the lowest in both keys, as in GSGB (frame 0.94 and 0.89, GSGB 0.81 and 0.83). Elsewhere the pattern follows GSGB weakly: the correlation across the 11 GB regions is 0.41 for the National Lottery and 0.11 for betting and gaming, whose propensity rests on about 190 donors. Northern Ireland takes GSGB's Great Britain rates.
Alcohol volume follows spending. No on-trade volume share can be archived (MESAS has no fixed file), so a pound buys the same volume in a pub as in a shop. Against a published on-trade share of about 27 percent, measured on the 2024-25 diary:
This follows the ruling of 2026-10-23.
Other known biases, in the stage notes:
Gambling bases are receipts over rates, not the Commission's yield, a deviation from the plan. The yield covers Great Britain only and includes undutiable betting; here it only splits General Betting Duty.
The re-pin moves road fuel. Chronicle Remove the stale TANF degenerate-exclusion (P0) + cross-register and take-up preflights #384 replaced the June Consumer Trends edition with September's. Calendar-2024 household road fuel (07.2.2) falls from £37.025bn to £35.460bn (−4.2%); the stage levels petrol and diesel to that figure, and
ons.household_road_fuel_expenditurebinds it. The measured calendar-2025 electricity and gas rows move to £28.8bn and £19.0bn. No other compiled target moves, and the local surface is byte-identical.Evidence
experiments/1359-excise-quantities-receipts.mdanddocs/evidence/uk-excise-quantities-1359/(aggregates only) hold:scripts/theta_calibration.py,outputs/theta_calibration.json): fitted on the donor with the committed declaration and vendored rows, 0.897 for the National Lottery and 0.081 for betting and gaming. The three-or-more-adult ratio is the check, and it fits less well (fitted 1.97 and 1.89 against the diary's 1.83 and 1.66).b2035eb30), the participation is unchanged, the added regular players' mean stake equals the seen households', and the occasional players' is about a tenth of it (outputs/stage_run_gsgb.json).ci_test_plan verify;272c987re-pin: the stage, gambling, stage-health, spec, contract, battery, gates and FRS spine files with the two pin tests, 1,161 passing, andci_test_plan verify. Run alone afterwards: the H2 parity and excise engine tests, 11 passing; the pin and local census tests, 27 passing; and the high-value home stock's feed fingerprint check, passing. The full vendoring drift check, which loads the 7.5 GB feed twice, was not run, as at the200bf6dre-pin; the vendored files come from that tool and feed.Vahid's third review of #1383
#1383 merged with three items of that review open, so they land here (
98a09827a):Open items
🤖 Generated with Claude Code