Skip to content

Show UK programmes from policyengine-uk's registry, and UK terms on the Rules hub - #57

Merged
juaristi22 merged 4 commits into
masterfrom
uk-rules-pages
Oct 27, 2026
Merged

juaristi22 merged 4 commits into
masterfrom
uk-rules-pages

Conversation

@juaristi22

@juaristi22 juaristi22 commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

The UK Rules pages showed a static paragraph for coverage and US terms on the hub. This PR changes two things.

  • Coverage tracker (/uk/model/rules/coverage). It now shows policyengine-uk's own programme registry (policyengine_uk/programs.yaml), which the build already snapshots to public/programs-uk.json.
    • The page has the snapshot's provenance line, status counts, search, a verified-year filter and the programme list.
    • The registry comes from the latest model code on main. When that is ahead of the production API, the provenance line says a programme marked complete may not be in the live calculator yet.
    • Programmes sit under UK government, devolved and local authority headings, then under the body that runs them (HMRC, DWP, DfE, Revenue Scotland, Social Security Scotland, Welsh Government, and so on).
    • Cards for programmes that apply only in some nations say where (England, Scotland, England and Wales, and so on).
    • A card opens in place to show the registry's notes, verified years and variable. It also links the parameters on GitHub at the snapshot's commit, but only where the parameter tree built with that snapshot has the folder or file.
    • There are no US states, state chips or "By state" view.
  • Rules hub (/uk/model/rules). The cards said "federal, state, and local" programs and variables "at the person, family, tax unit, and household level". The UK site now says "UK government, devolved and local tax-and-benefit programmes" and "the person, benefit unit and household level", in British spelling. The US copy is unchanged.

This PR is separate from #56 (UK data pages): no files overlap and a trial merge is clean. Max's draft #51 has an earlier UK tracker. This PR borrows its shared-parts split (coverageParts.tsx and the RulesOverview.tsx change are identical to #51's) and leaves out its Enhanced FRS data parts.

Details

  • Shared parts. The status dot, stat card and year helpers move from RulesOverview.tsx to coverageParts.tsx unchanged.
  • No US fallback for the UK. fetchProgramsWithSource('uk') no longer falls back to the bundled US registry when the snapshot and the API both fail. The UK page then links programs.yaml instead.
  • Provenance. CoverageProvenance takes an optional note for a snapshot that is ahead of the production API. Only the UK tracker passes one.
  • Proxy. It now rewrites param-tree-*.json under /us and /uk, as it does programs-*.json, so standalone previews (/uk/model/...) load the tree. On policyengine.org, /uk/model/param-tree-uk.json already reaches this app.
  • Build script. fetch-metadata.js deletes a stale param-tree-*.json along with the stale programs-*.json before a refresh. Whatever is on disk after a run then comes from one commit.
    • The tree still needs five GitHub API calls per country.
    • Unauthenticated builds can hit GitHub's 60-requests-an-hour limit and then show no parameter links. The Vercel build should set GITHUB_TOKEN (or GH_TOKEN).
  • Agencies. The tracker maps the registry's 11 agencies, plus DESNZ for a proposed registry correction (the Energy Price Guarantee). An agency it doesn't know goes under "Other programmes", named.
  • No computation-tree links on UK cards. The flowchart service runs an older policyengine-uk: 7 of the registry's 51 variables are missing there, for example lifetime_isa_bonus and pension_age_winter_heating_payment.
  • Rendering. /rules now renders per request, because it reads the country, like /rules/coverage.

Checks

  • At fa00059: bun run test (184 passed), bun run lint and tsc. bun run build passed at the first push, and Vercel builds every push. This repo has no test workflow, so the tests run locally.
  • New tests cover:
    • the UK tracker over a pinned copy of the UK registry snapshot: every programme in exactly one known section, counts, search, year filter, details opening in place, checked parameter links, the live-calculator note and no US fallback;
    • the provenance note's conditions;
    • the Rules hub copy for both countries, with the country from the query string or the header;
    • the proxy rewrites;
    • deleting a stale parameter tree;
    • the loader's UK fallback.
  • US pages unchanged. At the first push, /us/rules, /us/rules/coverage, /us/rules/parameters and /us/rules/variables had the same text and element geometry as master at 375 and 1440 px, with the same registry snapshot on both. Later commits don't change the US render path. The US provenance line takes no note, and tests pin its exact text.
  • UK Rules pages at 375 and 1440 px have no page-level horizontal overflow and no console errors.

🤖 Generated with Claude Code

juaristi22 and others added 2 commits October 27, 2026 14:20
…acker

The UK coverage tracker showed a fixed paragraph. It now reads the
programs.yaml snapshot that the build already writes to programs-uk.json:
status counts, search, a verified-year filter, and the programmes grouped
into UK government, devolved and local authority programmes, then by the
body that runs them. A card opens in place to show the registry's notes,
verified years and variable. It links the parameters only where the
parameter tree built from the same commit has that folder or file.

- Move the status dot, stat card and year helpers that both trackers use
  into coverageParts.tsx, unchanged.
- Keep parameter_prefix and the registry's own notes on each program.
- Return no programmes, rather than the bundled US registry, when the UK
  snapshot and the API both fail.
- Let the proxy rewrite param-tree-*.json under /us and /uk, as it does
  programs-*.json, so standalone previews find the tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Rules hub said "federal, state, and local" programs and variables "at
the person, family, tax unit, and household level" on both sites. On the
UK site it now names UK government, devolved and local programmes and the
person, benefit unit and household entities, in British spelling. The US
copy is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
policyengine-model Ready Ready Preview Oct 27, 2026 4:57pm UTC

Request Review

@vahid-ahmadi

Copy link
Copy Markdown

@juaristi22 round 1, at 4c0e1c9d2.

Verdict: works and reads well; no must items. Three should items, two of them about where the data comes from rather than the UI.

The programme list matches the registry. src/test/fixtures/programs-uk.snapshot.json is policyengine-uk main @ b9f55f8 (2.162.12), which is still main HEAD today. I compared it with policyengine_uk/programs.yaml at that commit: 55 programmes, same ids, none missing. There are 11 agencies, all in AGENCIES (src/components/rules/UkCoverageTracker.tsx:28-40), and 51 variables, as the description says.

Findings

  1. should: the tracker shows policyengine-uk main, not what policyengine.org runs. scripts/fetch-metadata.js:35-54 resolves the default branch HEAD at build time and :85-97 snapshots its registry. Nothing pins it, so every deploy picks up whatever main has. When I built it, the provenance line read "policyengine-uk 2.162.12 ... The production API served 2.132.6" (src/components/rules/CoverageProvenance.tsx:9-11). So a programme marked complete here may not be in the live calculator yet. The line does say this, but only in passing. Either pin to the API's version tag, or add a plain sentence that coverage is for the latest model code, which can be ahead of the live calculator.

  2. should: parameter links depend on unauthenticated GitHub API calls and can quietly disappear. My first bun run build had no GITHUB_TOKEN. It wrote public/programs-uk.json but not public/param-tree-uk.json. A rerun got HTTP 403 from api.github.com, so this was probably the 60-requests-per-hour limit. With no tree, parameterUrl returns nothing (UkCoverageTracker.tsx:110) and no UK card shows "Parameters on GitHub". There is no warning on the page. Please check the Vercel build sets GITHUB_TOKEN. Also, fetchMetadataFiles deletes a stale programs-*.json before it refreshes (fetch-metadata.js:154) but never deletes a stale param-tree-*.json. After a failed refresh, links can be checked against an old tree but point at the new commit. That script predates this PR, but the UK links now rely on it.

  3. should: some labels in the registry are wrong, and the new headings make them more visible. The tracker shows agency and coverage as the registry has them, now under named bodies. In policyengine_uk/programs.yaml @ b9f55f8:

    • Scottish income tax, agency: Revenue Scotland (line 627). HMRC collects it; Revenue Scotland runs LBTT and Scottish Landfill Tax. It now shows under "Devolved programmes > Revenue Scotland".
    • Council Tax, coverage: UK (line 564). Northern Ireland has domestic rates, so this should be Great Britain. As written, the card has no nation chip (UkCoverageTracker.tsx:174).
    • Tax credits, agency: DWP (line 367). HMRC ran them until they closed in April 2025.
    • Energy Price Guarantee, agency: Ofgem (line 598). It was a government scheme (BEIS, then DESNZ). Ofgem only set the price cap.
    • Winter Fuel Payment, coverage: UK (line 392). Scotland has had Pension Age Winter Heating Payment instead since 2024–25, and that has its own entry.

    I would fix these in a policyengine-uk issue or PR, not with overrides here, so the registry stays the only source. It is worth doing before this goes live.

  4. nit: Complete and Partial use two shades of the same teal. These are statusColors.complete = primary[600] and partial = primary[400] (src/designTokens/colors.ts:60-65). The text label carries the meaning, so this is fine for accessibility. But the dots and stat-card numbers are hard to tell apart in the screenshot. This was the case before the PR, so it doesn't need to change here.

UK terms are right throughout: "programmes", "benefit unit" (src/data/rulesSection.ts:54), "Universal Credit", "Council Tax", "Child Benefit" and the department names. These pages don't use ITL1 regions. Headings and labels are in sentence case. The new UI takes every value from src/designTokens and has no hard-coded colours. The early return for the UK at RulesOverview.tsx:339 comes after every hook, so the order of hooks is safe.

What I ran

  • Fresh clone, bun install --frozen-lockfile, then bun run test (16 files, 180 passed), bun run lint (clean), bunx tsc --noEmit (clean) and bun run build (passes). I then reran scripts/fetch-metadata.js with a token and built again, so the UK tree was present.
  • I ran next start and took headless Chrome screenshots of /uk/rules/coverage at 1440 and 375 px, and looked at the 1440 one. Sections, bodies, counts, nation chips and the provenance line all render. The faded stat numbers in the capture are the entry animation caught mid-way.
  • curl on /uk/rules: the UK hub copy is served, and /uk/param-tree-uk.json returns 200 through the proxy.
  • I checked the prose and UI against the PolicyEngine writing and design checklists myself, not with the plugin agents.

Not checked: the claim that 7 of 51 variables are missing from the flowchart service, the US pages at both widths (the description says they are unchanged), and #51.

🤖 Generated with Claude Code

juaristi22 and others added 2 commits October 27, 2026 16:55
…ulator

The UK provenance line now ends: "The registry is read from the latest
model code, so a programme marked complete here may not be in the live
calculator yet." It shows only when the snapshot's version differs from
the one the production API served, or that version is unknown. The US
line is unchanged.

The tracker also maps DESNZ to the UK government. That is the agency
proposed for the Energy Price Guarantee in policyengine-uk's registry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… a refresh

fetch-metadata.js already deleted programs-*.json before refreshing it,
but kept param-tree-*.json. If the tree fetch then failed, UK parameter
links were checked against a tree from an earlier build while pointing
at the new commit. Both files are now deleted first, so whatever is on
disk after a run comes from one commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

@vahid-ahmadi thanks. Round 1 is in e481e50 and fa00059.

  1. main versus the live calculator. The UK provenance line now ends: "The registry is read from the latest model code, so a programme marked complete here may not be in the live calculator yet."
    • It shows when the snapshot's version differs from the production API's, or when the API version is unknown (CoverageProvenance.tsx, tested in coverageProvenance.test.tsx).
    • I didn't pin to the API's tag, because fetch-metadata.js also feeds the US tracker and this PR keeps US output unchanged. The US page has the same gap.
  2. Unauthenticated GitHub calls. fetch-metadata.js now deletes a stale param-tree-*.json along with programs-*.json before a refresh (fa00059, covered by two tests). A tree on disk therefore always comes from the snapshot's commit.
    • Whether the Vercel project sets GITHUB_TOKEN still needs checking by someone with access to its settings.
    • Production served /uk/model/param-tree-uk.json (200) today, so its last build did get the tree.
  3. Registry labels. Agreed, these belong in policyengine-uk: Correct the agency or coverage of six programmes in programs.yaml policyengine-uk#2696 has your five plus Council Tax Reduction, which also says coverage: UK but has no Northern Ireland scheme. The tracker now maps DESNZ, the agency proposed there for the Energy Price Guarantee, so that entry stays under UK government after the correction.
  4. Teal shades. I've left these as they are: they predate this PR, and the labels carry the status.

On the items you didn't check:

  • Flowchart. GET https://flowchart.fly.dev/api/variables?country=UK returns 924 variables. These seven registry variables aren't among them: lifetime_isa_bonus, isa_cash_interest_flat_rate_charge, gambling_duty, winter_fuel_payment, wales_childcare_offer, warm_home_discount and pension_age_winter_heating_payment.
  • US pages. I compared /us/rules, /us/rules/coverage, /us/rules/parameters and /us/rules/variables against a master worktree serving the same registry snapshot, in headless Chromium at 375 and 1440 px. innerText matched, and so did the position and size of every element: 0 differences across the 8 page-and-width pairs. Round 1 doesn't touch the US render path.

bun run test (184 passed) and bun run lint pass at fa00059.

@vahid-ahmadi

Copy link
Copy Markdown

@juaristi22 round 2, at fa0005903.

Verdict: all three round 1 items are dealt with. No new must or should items. Fine to take out of draft from my side; merging is Vahid's call.

Round 1 items

  1. should, main versus the live calculator: fixed. The UK provenance line now adds the note when the snapshot version differs from the API's or the API version is unknown (src/components/rules/CoverageProvenance.tsx:20, text at src/components/rules/UkCoverageTracker.tsx:20-21). Not pinning because the script also feeds the US tracker is a fair reason. On my build the line read "policyengine-uk 2.162.12 (main @ b9f55f8) ... The production API served 2.132.6 ... may not be in the live calculator yet." The API still serves 2.132.6 today, so the note is needed.
  2. should, parameter links and GITHUB_TOKEN: fixed in the code, one check left. A stale param-tree-*.json is now deleted with the snapshot before a refresh (scripts/fetch-metadata.js:158), with tests for both failure paths. The Vercel GITHUB_TOKEN check is still open, but that needs someone with access to the project settings, not a change here.
  3. should, registry labels: moved upstream. Correct the agency or coverage of six programmes in programs.yaml policyengine-uk#2696 is open with the five I listed plus Council Tax Reduction. DESNZ is mapped ahead of it (UkCoverageTracker.tsx:36), so the Energy Price Guarantee stays under UK government once that lands. Until #2696 merges the live page still shows the old labels, so I would still land #2696 first.
  4. nit, teal shades: left as is, agreed.

Thanks for checking the flowchart list and the US pages too.

New findings

None.

What I ran

Fresh clone at fa0005903: bun run test (16 files, 184 passed), bun run lint and bunx tsc --noEmit clean, bun run build with a token (both UK files written). next start plus a headless Chromium screenshot of /uk/rules/coverage at 1440 px, which I looked at: the provenance line, counts (55, 38 complete, 17 partial) and sections render.

🤖 Generated with Claude Code

@juaristi22
juaristi22 marked this pull request as ready for review October 27, 2026 17:54
@juaristi22
juaristi22 merged commit 8435f47 into master Oct 27, 2026
2 checks passed

This branch was successfully deployed

1 active deployment
Preview — fa000590 Deployed Oct 27, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants