Skip to content

Test that a fact's period is the tax year its source table names - #333

Open
MaxGhenis wants to merge 3 commits into
mainfrom
w2-source-table-year-tests
Open

MaxGhenis wants to merge 3 commits into
mainfrom
w2-source-table-year-tests

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds tests that compare a fact's period with the tax year its source table's title names. No package, parser or schema changes.

The case behind it: the microcosm US feed carried five facts with period 2023 and vintage tax_year_2023 whose source is IRS SOI Table 4.B for Tax Year 2020 (20in04w2all.xlsx). #292 fixed the package on main. These tests check the result against the publisher's own title, and they catch a wrong literal year, which #292's guard cannot see.

What I checked at the source (10 October 2026)

  • IRS's Form W-2 statistics page (last reviewed 27 January 2026) links Table 4 for 2020 and 2019 only. 21in04w2all.xlsx through 24in04w2all.xlsx return 404. There is no TY2023 Table 4.B to add. IRS's prior-releases page carries tax years 2008 to 2018 in one workbook (18inallw2.xls), so 2020 is the latest year published.
  • A fresh download of 20in04w2all.xlsx has the sha256 the manifest pins (1178d776…cfc442f).
  • Cell A1 of sheet "Table 4.B" reads "Table 4.B. Summary of Items for Taxpayers with Form W-2, by Return and Earner Type, Tax Year 2020".
  • The five values are in that sheet (money amounts in thousands):
Fact Cell Published value
Box 7 social security tips, returns B13 6,038,613
Box 7 social security tips, taxpayers C13 6,105,713
Box 7 social security tips, amount D13 $26,786,522,000
Box 12 D 401(k) elective deferrals, amount D22 $277,859,181,000
Box 12 AA designated Roth contributions, amount D41 $32,302,509,000

How the ty2023 rows came about

At c5e5bf8, the commit microcosm pins, packages/irs_soi/w2_statistics_2020/source_package.yaml set artifact_year: 2020 and rendered period, record_set_id, source_record_id_prefix, vintage and legal_vintage from {year}. Microcosm's scope built the package at --year 2023 for three record sets, so the Tax Year 2020 cells went out with ty2023 labels.

On main today the package builds the same five TY2020 facts at every --year from 2019 to 2025 (a test below asserts it).

What this adds

tests/test_chronicle_source_table_year.py:

  • Registry test. For every pinned package whose artifact.source_table names exactly one tax year, every tax-year fact built at the artifact year must carry that year as its period, in its tax_year_<year> vintage and in the ty<year> tokens of its record ids. One package qualifies today (irs_soi/w2_statistics_2020). HMRC titles such as "tax year 2023-24" name two years and are left alone.
  • W-2 title. The declared source_table equals cell A1 of the workbook, so the year the registry test reads is IRS's.
  • W-2 values (differential). The five facts equal a direct openpyxl read of the sheet, and both equal the published values above.
  • W-2 at every build year. --year 2019, 2021, 2022, 2023, 2024 and 2025 build facts identical to the 2020 build.
  • The title reader. Sixteen example titles, plus every combination of 3 prefixes, 5 suffixes, 5 years and 7 connectors: one named year reads back, and two different years never read as one.

docs/agent-source-package-harness.md gains a paragraph on the new check under "Pinned artifacts and year labels".

Invariant

When a package's source table title names exactly one tax year, every tax-year fact the package builds at its artifact year has that year as its period, in its tax_year_<year> vintage and in its record ids.

Mutation checks (run locally, then reverted)

Mutation of the W-2 package New tests #292's test_chronicle_artifact_year_restamp.py
The c5e5bf8 file ({year} labels) 6 fail (every non-2020 build year) not run
period: '2023' as a literal 2 fail all pass
vintage: tax_year_2023 as a literal 2 fail not run

The second row is the gap these tests close: a literal wrong year is the same file read under the same labels at every build year, so a guard that compares one build with another has nothing to flag.

Related

Tests

  • uv run pytest tests/test_chronicle_source_table_year.py: 27 passed (10.6 s).
  • uv run ruff check chronicle policyengine_chronicle db scripts tests: clean.

🤖 Generated with Claude Code

soi-w2-statistics-2020 reads one workbook, IRS SOI Table 4.B for Tax Year
2020. Until #292 it rendered its labels from --year, and the microcosm US
feed built it at --year 2023: five TY2020 cells went out as ty2023 facts.
#292 made the labels literal and added the artifact-year restamp guard, which
compares one build with another. A literal label that is simply the wrong
year passes that guard.

These tests compare the labels with what the publisher printed. For every
pinned package whose artifact.source_table names exactly one tax year, each
tax-year fact built at the artifact year must carry that year as its period,
in its tax_year_<year> vintage and in its record ids. For the W-2 package
they also check that the declared title is the workbook's own A1 cell, that
the five facts equal a direct read of the sheet, and that every --year from
2019 to 2025 builds the same TY2020 facts.

Tax Year 2020 is the latest Table 4.B IRS has published (checked 2026-10-10:
21in04w2all.xlsx through 24in04w2all.xlsx return 404), so there is no later
table to add.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis force-pushed the w2-source-table-year-tests branch from e891f48 to f5bd1d5 Compare October 10, 2026 20:40
MaxGhenis and others added 2 commits October 10, 2026 17:33
- Check the legal vintage too: a wrong literal legal_vintage on the W-2 tips
  amount passed every test, because it is the same at every build year. The
  registry test now reads any tax_year_<year> in the source vintage or the
  legal vintage, and the W-2 test pins the tips amount's legal vintage.
- Require the table's year to be the only ty<year> token in each record id
  and record set id; an id with no year token used to pass.
- Choose packages from the decoded artifact block, not from the YAML text, so
  a quoted key or an escaped title cannot hide one. Only the part before the
  top-level record_sets key is parsed (all 329 packages in about 2 s, where a
  full parse takes about 50 s).
- Make the title reader's year list possessive and refuse any separator
  followed by a digit after it. "Tax Years 2019 and 2020/21" used to read as
  2019 by giving back the years the lookahead refused, and "2020; 2021" and
  "2020 to 21" read as 2020. ";" and "/" now separate full years.
- Say in the harness doc exactly what the test enforces.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Refuse a title only when a separator is followed by a one- or two-digit
  fragment that ends the word (an abbreviated year such as 2023-24). Any
  other number after a separator ("Tax Year 2020, 1040 Returns", "Tax Year
  2020 - 1st Release") no longer drops the title.
- Read "or", "vs.", "thru" and U+2212 as separators, so "Tax Year 2020 or
  2021" names two years, not 2020.
- Say in the doc that the package must build a fact and that every fact's
  source_table must name the year, and that the year tokens are matched in
  lowercase.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant