[juaristi22]

Summary

PolicyEngine/microcosm#1381 tops up gambling incidence on the frame to the share of adults who gamble, which the LCFS diary under-records. This PR adds the Gambling Commission's published participation rates: gambling-commission-gsgb-annual-report-2025, 1,024 facts from the Gambling Survey for Great Britain, Annual Report 2025 data tables (published 16 July 2026; 20,775 adults; fieldwork 13 January 2025 to 22 January 2026). scripts/build_uk_excise_packages.py (build_gsgb_participation) regenerates it, and docs/data-sources/uk-gambling-participation-gsgb-2025.md sets out every choice.

• B.1 and B.3: participation in the past 12 months and the past four weeks, any gambling and excluding lottery-draw-only players, for all adults, by sex, by seven age bands and by sex crossed with age band. Each row comes with its unweighted base.
• B.2 and B.4: the same by country and English region.
• B.9 and B.10: past-four-week participation by activity, with the online and in-person sub-rows: by sex and age band, and by country and English region.
• B.13: the number of activities among people who gambled in the past four weeks, by sex and age band, with the mean.

Bundle

Against main at 66114a6:

• facts 1,652,983 to 1,654,007 (+1,024);
• packages 419 to 420; source tables 361 to 362;
• calendar 2025 +1,024; Great Britain +544, England +40 (and 40 for each of Scotland, Wales and the nine regions); people +1,024;
• no new period, geography or entity keys; no duplicate keys;
• warnings 132 to 133: B.9 and B.10 word one activity row differently, and both labels are kept as published.

The full bundle test (53 minutes, 15 GB) was not run locally; CI's bundle job checks the pins. The UK suite was built at d3a11d0 the way Microcosm re-pins, with build-bundle --suite uk and then build-consumer-artifact:

• valid, with 0 errors;
• 1,503,315 rows: the 1,502,291 of the UK feed at 200bf6d (Microcosm's current pin) plus these 1,024;
• consumer-fact schema digest unchanged (d953306d…);
• the one new warning is the groupby label above; the package also joins the existing sex=male and sex=female label warnings.

Review notes

• The workbook is stored in R2 only. The Commission states no licence for it: not on the workbook, the publication page or data.gov.uk, which lists no GSGB dataset. Its website terms require prior approval to copy its material. So the manifest has storage.local: false, .gitignore keeps the bytes out of the repository, and the licence, attribution and evidence are recorded. Builds read the file from the source-artifact cache, or with LEDGER_SOURCE_ARTIFACT_FETCH=1, as CI sets, from source_url.
• Period. Calendar 2025, with survey_reference coverage from 13 January 2025 to 22 January 2026; the windows count back from each interview.
• Values are the stored cells: proportions the workbook displays as percentages to 0 dp (unit: share), and bases with the file's floating-point noise (for example 1231.999999999998).
• Bases are facts, not a dimension. Each table's unweighted bases are gambling_commission.gsgb.unweighted_base facts with the row's dimensions.
  • B.1 to B.4 print the base of their first measure, and B.9 and B.10 that of any gambling activity. Their footnotes say other bases are of a similar magnitude.
  • B.13's base is its own denominator: people who gambled in the past four weeks.
• Restated rows. B.9 and B.10's "Any gambling" row equals B.3 and B.4's past-four-week column cell for cell (asserted), so it is emitted once, from B.3 and B.4.
• source_sheet on every fact. B.9 and B.10's bases differ from B.3 and B.4's by one in three cells, so each table keeps its own, told apart by source_sheet (as the Consumer Trends package does).
• Left out: weighted bases; B.13's standard deviation (Chronicle has no aggregation for it); tables A, B.5 to B.8, B.11, B.12, C and D; - cells, of which these tables have none.
• The adult universe. Every fact carries an age >= 18 constraint, and each age band its bounds, as the ASHE packages state theirs. age_band: all keeps the shared "All ages" label.
• Footnotes in the notes. B.1 to B.4's measures carry footnote i, which defines "excluding lottery draw only": only people whose sole gambling was lottery draws are left out. They also carry the non-response footnotes, and their rows guard those cells.
• What the facts can and can't answer for Write policyengine-uk's alcohol, tobacco and gambling duty inputs from the LCFS stage microcosm#1381 is a section of the docs page. It covers people rather than households, no union cells, the windows, and the Commission's caveats.
• Labels. sex, age_band and source_sheet reuse the existing dimension and value labels; the other dimensions are new.
  • Row labels are each table's own text, with whitespace collapsed and footnote markers dropped.
  • B.9's row 6 reads "Tickets National Lottery draws bought in person" where B.10's reads "Tickets for …". Both are kept, which adds the one groupby-label warning.
• R2-only storage for a licence reason. docs/agent-source-package-harness.md allowed R2-only storage only for files too large to commit. It now also allows it when the publisher states no licence that lets Chronicle redistribute the file.

Chronicle Governance

• Approved Chronicle agent role: ledger-source-ingestor, for the package.
• Outside the role's allowed paths (the roles file is stale, as in UK excise facts for chronicle#381: OBR excise lines, HMRC alcohol, tobacco and betting and gaming statistics, Gambling Commission GGY, and the September Consumer Trends release #384): chronicle/source_package.py and chronicle/bundle.py (registration), db/data/**, .gitignore, scripts/build_uk_excise_packages.py, docs/** and changelog.d/gsgb-participation-2025.added.md.
• Deterministic checks run:
  • validate-package and build-suite: valid, 0 agent-acceptance errors; the only warning is concept_alignment_validation_skipped.
  • publish-raw, then verify-raw: the R2 object matches its pin.
  • inventory-artifacts over db/data: valid, 493 artifacts, 41 R2-only, no checksum mismatches.
  • The generator rebuilds the package byte for byte, and a full run leaves every other package unchanged.
• LLM judge verdicts (an independent agent, at 687996c): ledger-source-fidelity PASS, ledger-boundary PASS.
  • It checked all 1,024 facts against its own read of the workbook: values, floating-point noise included, every dimension, geography, period and provenance. It also checked the 520 row and 152 column header guards. Re-running the generator gave the same YAML byte for byte.
  • It confirmed the claims behind the choices: the restated rows are equal in all 22 cells, the bases differ in exactly the three cells named, and no '-' appears in the seven sheets. It also confirmed the licence evidence on the cited pages.
  • Five minor findings, all fixed in d3a11d0:
    • B.13's base note wrongly cited a footnote;
    • the doc claimed every family carries every dimension;
    • the window notes mentioned interviews, but GSGB is self-completed;
    • B.9 row 6's label had been normalised to B.10's wording;
    • R2-only storage for a licence reason needed a line in the harness doc.
  • Vahid's round 1 changes, in 6269a88, came after this review. They add notes, docs, universe constraints and the attribution, and change no value or count.

Tests

• New: tests/test_source_gambling_commission_gsgb_2025.py, 7 tests. They cover registration, R2-only storage and the licence record; validity, labels and unique keys; values against the cells for every table; the area codes; the restated rows and bases; and B.13. They skip when the workbook is neither cached nor fetched.
• Touched existing files: alias drift, artifacts, geography names, the UK excise facts for alcohol, tobacco and gambling: the OBR duty and clearance lines already archived, the revised Consumer Trends release, Gambling Commission GGY and HMRC betting and gaming receipts (PolicyEngine/microcosm#1359) #381 tests and the light bundle tests: 137 pass with the new file at 6269a88.
• ruff check: passed.

🤖 Generated with Claude Code

[vahid-ahmadi]

@juaristi22 thanks, this is my review at d3a11d0. The PR is a draft, so this is a comment only.

Verdict. The facts are right. Every cell I checked against a workbook I downloaded myself matches, and the generator rebuilds the package byte for byte from scratch. The geography, period, population and licence records are accurate. I found nothing at must level. There are three shoulds. Two are about what the package tells microcosm#1381, and they matter more than anything in the code. The third is a repository policy question for us, not a change for you. Before this leaves draft it needs the bundle job green at this head (still running when I looked) and an answer to finding 3.

What I ran

• Source bytes. I downloaded the workbook from the manifest's source_url: 200, 484,303 bytes, SHA-256 ccee2269…, matching the manifest. I also fetched the publication page, the Commission's guidance on using GSGB statistics, its terms of use and a data.gov.uk search.
• Regeneration. I deleted packages/gambling_commission/gsgb_annual_report_2025/source_package.yaml and ran scripts/build_uk_excise_packages.py with LEDGER_SOURCE_ARTIFACT_FETCH=1. It came back byte for byte and git status was clean, so the other five packages the script writes are unchanged too.
• Values. I built the 1,024 facts with load_source_package(...).build_facts(2026) and compared 42 of them with cells I read separately with openpyxl, choosing each fact by its dimensions, not its cell key. All 42 match exactly, floating-point noise included. (My first pass flagged one, B.9 F12, but that was my filter: the row is "Other scratchcards bought in person" and the fact rightly carries gambling_channel: in_person.)

table	cells checked
B.1 (12 months, sex and age)	B3 59.0% all adults; C19 male 75 plus excluding draw-only; B10 aged 55 to 64; C5 all females; D20 base, female 18 to 24 (754)
B.2 (12 months, area)	London B6, Scotland B13, England C3, Wales C14, North East base D7 (900)
B.3 (4 weeks, sex and age)	B3 47.4% and C3 27.0% all adults; C6 aged 18 to 24; B25 female 65 to 74; D14 base, male 25 to 34
B.4 (4 weeks, area)	London B6, Scotland C13, South West C10, Yorkshire and the Humber base D12
B.9 (activity, sex and age)	betting B16; male sports betting online C17; National Lottery scratchcards K11; casino at a casino E26; National Lottery draws in person I6; other scratchcards F12; machines in a venue H27; private betting C33; any gambling online B35; bases D37 and E37 (1231.999999999998)
B.10 (activity, area)	London betting, Wales National Lottery draws in person, North East football pools, England bingo at a venue, East of England any gambling in person, South East other online instant win, Scotland base
B.13 (number of activities)	18 to 24 seven or more E6; female one D3; male two to three C4; mean B7; base K9

• Checks on the choices. B.9 row 34 equals B.3 column B in all 10 cells, and no B.9 or B.10 all-channel "any gambling" fact is emitted. No fact carries B.13's standard deviation or a weighted base. None of the seven sheets has a - cell. The counts by sheet and concept add up: 72 + 36 + 72 + 36 + 340 + 408 + 60 = 1,024, of which 870 participation, 104 bases, 40 shares by number of activities and 10 means. Geographies are Great Britain 544 and 40 for each country and region, which matches the bundle pins.
• Metadata. The coversheet says "Publication date: 16 July 2026" and "Fieldwork took place between 13 January 2025 and 22 January 2026", and the generator asserts both. The publication page says official statistics, push-to-web, 20,775 adults aged 18 and over living in Great Britain. Stamping Great Britain (K03000001), calendar 2025 with survey_reference coverage, and entity: person are all right. The licence record is accurate: neither the workbook nor the page states a licence, data.gov.uk lists no GSGB dataset (only the November 2024 industry statistics), and the terms of use say "Copying any of this material is not permitted without prior approval".
• Tests. tests/test_source_gambling_commission_gsgb_2025.py with the UK excise facts for alcohol, tobacco and gambling: the OBR duty and clearance lines already archived, the revised Consumer Trends release, Gambling Commission GGY and HMRC betting and gaming receipts (PolicyEngine/microcosm#1359) #381, alias-drift, artifacts and geography-names files: 113 passed. ruff check and ruff format --check with ruff 0.14.10 are clean on the touched Python. I did not run the 53-minute bundle test. CI's bundle and surface jobs were both in progress at d3a11d0 when I looked.
• What I read. In full: the GSGB part of the build script (lines 3323 to 4380), the manifest, the docs page, the harness doc change, the bundle test diff, the registration lines and the new test file's fixtures and first two tests. The 18,419-line package YAML is generated; I read its header and grepped it, and checked its output through the facts above rather than reading it. I didn't read the LLM judge's output.

Findings

1. should: the package doesn't say what "excluding lottery draw only players" leaves in, and that is the cell microcosm#1381 is likely to use.

   • The workbook's footnote i (B.1 row 28, and the same in B.2 to B.4) reads: "Excluding participants who had only played National Lottery or other charity lottery draw based games."
   • So the 27.0% (past four weeks) and 40.2% (past 12 months) still count people whose only other activity was a National Lottery scratchcard or online instant win game, both Lottery Duty products. They also drop people who only played society lottery draws.
   • Neither the package notes nor the docs page carry the footnote. The scope label is just "Any gambling, excluding lottery draw only players".
   • Please put the footnote text in the concept_evidence_notes of the excluding_lottery_draw_only measures and say it in the docs, as UK excise facts for chronicle#381: OBR excise lines, HMRC alcohol, tobacco and betting and gaming statistics, Gambling Commission GGY, and the September Consumer Trends release #384 did with OBR's 3.12 footnotes.
   • The non-response footnotes (95, 149 and 299 participants with no derived answer) could go in the same notes. That part is a nit.

2. should: say in the docs what these rates can and can't give microcosm#1381's top-up, including the Commission's own caveats. I couldn't find the top-up written down anywhere. The #1381 body at e80a424 has no such step and no microcosm issue mentions GSGB, so this PR is ahead of its consumer. Against what #1381 needs to fix (round 3, finding B: 2.0% of frame households carry betting and gaming stakes and 10.7% the National Lottery), the tables leave three gaps:

   • Unit and coverage. These are shares of adults aged 18 and over in Great Britain. The frame counts households in the UK. The workbook has no household table, so a person rate is a floor for household incidence, not a target. Northern Ireland isn't covered.
   • Scope. No cell gives "betting and gaming, excluding every National Lottery product", which is the base the other twelve duties follow. From B.9 (past four weeks, all adults), betting alone is 9.8%. So the non-lottery person rate lies between 9.8% and the 27.0% of finding 1. Likewise no cell gives "any National Lottery product": draws alone are 31.0%, scratchcards 11.0% and online instant win games 6.2%, with no union. Even the 9.8% floor is about five times the frame's 2.0%, so finding B holds under any reading of these tables.
   • Window. The LCFS diary is two weeks; the duty is annual. The tables give past four weeks (47.4% and 27.0%) and past 12 months (59.0% and 40.2%), but by activity only past four weeks. Which window stands for "carries duty in the year" is a design call for #1381, and having both here is right. Fieldwork is calendar 2025, against the stage's FY2024-25 levelling.
   • The Commission's caveats. The docs page already says the guidance "advises against direct comparisons with other gambling or health surveys", which is what a top-up to an LCFS frame is. It leaves out the next part of that guidance: the LSE and NatCen experiment "found that mentioning gambling in the survey invitation significantly increased estimates of gambling participation", and "could not conclude whether estimates of gambling participation were more or less accurate as a result". The guidance also says highlighting differences between surveys is acceptable if the different methods are made clear. So a top-up to GSGB rates may be a top-up towards the high end, and #1381 should record that. Please add that sentence to the docs page.
   • A short section on the docs page would do: which cell answers which question, and which doesn't.

3. should, and a decision for us, not you: the harness doc now allows R2-only storage when the publisher states no licence.

   • docs/agent-source-package-harness.md used to allow R2-only storage only for files too large to commit. This PR adds a second reason. That's a repository policy change inside a data PR.
   • Keeping the bytes out of git doesn't settle the licence question. R2 still holds a copy that we made, and the 1,024 values go out in the bundle and Microcosm's consumer feed.
   • For UK excise facts for chronicle#381: OBR excise lines, HMRC alcohol, tobacco and betting and gaming statistics, Gambling Commission GGY, and the September Consumer Trends release #384, the Commission had at least published an OGL listing for its industry statistics. For GSGB there's none, and the terms require prior approval to copy.
   • Whether that's acceptable for published official statistics, or whether we write to the Commission first, is the same call round 2 of UK excise facts for chronicle#381: OBR excise lines, HMRC alcohol, tobacco and betting and gaming statistics, Gambling Commission GGY, and the September Consumer Trends release #384 left with us, and it's sharper here. I'm raising it on our side. Please don't change anything until we've answered.

4. nit: a stale comment. scripts/build_uk_excise_packages.py, lines 3421 to 3422, still says B.9 row 6's "label is B.10's". d3a11d0 keeps each table's own label, as the comment at line 3734 says.

5. nit: age_band: all is labelled "All ages". The ASHE packages use the same label, so it isn't wrong by convention, but here the population is adults aged 18 and over. Only the measure label ("Proportion of adults who gambled") and the comments say so, and nothing in the filters does. A consumer that joins these to a frame with children could misread the denominator.

6. nit: the attribution names the Commission but not the producers. The publication page credits NatCen and the University of Glasgow, with authors Ridout, Roop, Wardle, Moore and Sadler. Worth adding to attribution for a full citation.

What looks right

• Selectors. The coversheet, notes, every title, every row and column header and the base rows are guarded, and every emitted cell is asserted to be a number. B.10's misspelt headers ("(percentgage)", the missing bracket) are matched as published, not cleaned up.
• Bases. Keeping each table's bases, with source_sheet on every fact, is right: they differ between B.3/B.4 and B.9/B.10, and B.13's base is a different population. The notes say that B.1 to B.4 print only the first measure's base.
• Units. Proportions are kept as the stored values with unit: share, not rounded to the displayed 0 dp, and the docs give the Commission's confidence interval formula. That's why the bases are worth having.

🤖 Generated with Claude Code

[juaristi22]

@vahid-ahmadi thanks. Items 1, 2, 4, 5 and 6 are addressed in 6269a88, now pushed. Item 3 is unchanged, as you asked. Values and fact counts don't move, so the bundle pins stand.

Should

1. What "excluding lottery draw only" leaves in.

   • Footnote i is now in the notes of all 72 excluding_lottery_draw_only measures, word for word. A sentence after it says what follows: only people whose sole gambling was lottery draws are left out, and National Lottery scratchcards, online instant win games and every other activity stay in.
   • The non-response footnotes (95 and 149 for 12 months, 299 for four weeks) are in the notes of every B.1 to B.4 measure. B.1 and B.3's bases also carry their note that sex could not be derived for 20 participants.
   • B.1 to B.4's rows now guard both footnote cells, so a change to either stops the build.
   • The docs page says the same.

2. What the tables can and can't give #1381. The docs page has a new section, "What these facts can and can't answer". It covers:

   • people, not households;
   • Great Britain only;
   • what the exclusion leaves in;
   • that no cell gives a union, neither betting and gaming excluding every National Lottery product, nor any National Lottery product;
   • that the activity rows can't be summed (B.13's mean is 2.4 activities);
   • that by activity there is only the past four weeks;
   • the non-response counts;
   • the Commission's caveats, including the LSE and NatCen invitation experiment.

   Two things I left out on purpose:

   • "A person rate is a floor for household incidence." That holds only if gambling isn't concentrated in households. Where it is, household incidence can fall below the person rate, so the page says only that household incidence isn't published.
   • The 9.8% to 27.0% range. That's a derivation for #1381 to make. The page names the cells it would use.

   I paraphrased the Commission's guidance and linked it rather than copying it, given its terms of use.

3. R2-only storage and the harness line. Understood, and nothing is changed. I chose R2-only for this file on 25 October. The harness line records that choice and can be dropped or reworded, whatever you decide; nothing else depends on it.

Nits

4. Fixed: the comment on B.9 row 6 now says each table keeps its own label.
5. The adult denominator. I kept age_band: all as "All ages", since a different label would add a cross-package label warning. Instead, every fact now carries an age >= 18 constraint, and each age band its bounds (age >= 18 and age < 25 for 18 to 24), as the ASHE packages state theirs. A consumer joining to a frame with children can read the denominator from the constraints.
6. Done: the attribution names the National Centre for Social Research and the University of Glasgow, and the five authors.

CI and the artifact. The run at 6269a88 replaces the one at d3a11d0. I'm rebuilding the UK consumer artifact at 6269a88 for #1381, because the new constraints change the facts' keys.

🤖 Generated with Claude Code

[vahid-ahmadi]

@juaristi22 thanks, this is round 2 at 6269a88. The PR is still a draft, so this is a comment only.

Verdict. Items 1, 2, 4, 5 and 6 are done. I have two new nits on the docs page, and neither blocks. The values haven't moved. Only item 3 now keeps this in draft, and that decision is ours. The other condition is CI. At 16:04Z the bundle and surface jobs at this head were still running, after 65 minutes, so neither has passed yet. Round 1's run was cancelled before it finished, so neither job has gone green on this PR.

Round 1 items

1. Footnote i: fixed. All 72 excluding_lottery_draw_only facts carry footnote i word for word, plus the sentence on what stays in. Every B.1 to B.4 fact carries its table's non-response footnote (95 and 149, or 299). B.1 and B.3's bases carry the note on sex for 20 people. I checked each against the workbook's footnote rows. Both footnote cells are now guarded on every row.
2. What #1381 can and can't take: fixed. The new docs section covers what I asked for. On your two omissions, you're right on both. I was wrong that a person rate is a floor for household incidence. If gamblers cluster in larger households, the household rate can fall below the person rate, so "not published" is the correct thing to say. The 9.8% to 27.0% range is #1381's to derive. Paraphrasing the guidance is fine too, but it isn't linked yet (nit 8). The 2.4 mean (B.13 B7, 2.357) and the publication's sentence on comparability both match the source.
3. R2-only storage and the licence: still open, waiting on Vahid's decision. Thanks for leaving it alone.
4. Stale comment: fixed (scripts/build_uk_excise_packages.py:3422).
5. Adult denominator: fixed. Keeping "All ages" and adding constraints works. 618 facts carry age >= 18 and each age band carries its own bounds (58 facts per band). This matches the ASHE "Age lower bound" and "Age upper bound" labels.
6. Attribution: fixed. The manifest names NatCen, the University of Glasgow and the five authors.

New nits

7. nit: the docs say every fact carries age >= 18, but only the 618 all facts do. A banded fact carries only its band's bounds, for example age >= 25 and age < 35, so "adds its bounds" isn't quite right either. The bands imply adults, so nothing is wrong in the data. Perhaps: "every fact carries a lower age bound of 18 or more".
8. nit: link the guidance you paraphrase. Lines 120 and 161 to 167 cite "the Commission's guidance" without a URL. It is https://www.gamblingcommission.gov.uk/about-us/page/guidance-on-using-statistics-from-the-gambling-survey-for-great-britain.

What I ran at 6269a88

• Regeneration. I deleted the GSGB source_package.yaml and reran scripts/build_uk_excise_packages.py with LEDGER_SOURCE_ARTIFACT_FETCH=1. The output was identical byte for byte, and git status was clean.
• Values. I built the 1,024 facts and matched them to round 1's on concept, geography and filters. Every one pairs up with the same value, and the concept counts are unchanged (870, 104, 40, 10). I reran the 42 cell checks against my own download of the workbook, and all 42 match (F12 under its in-person channel, as in round 1).
• Tests. The GSGB test file with the UK excise facts for alcohol, tobacco and gambling: the OBR duty and clearance lines already archived, the revised Consumer Trends release, Gambling Commission GGY and HMRC betting and gaming receipts (PolicyEngine/microcosm#1359) #381, alias-drift, artifacts and geography-names files: 114 passed (113 before plus your new test). ruff check and ruff format --check (0.14.10) are clean.
• What I read. The full diff d3a11d0..6269a88 apart from the generated YAML, which I checked through the facts above. I didn't run the bundle test locally.

🤖 Generated with Claude Code