Repository navigation
Conversation
soi-w2-statistics-2020 reads one workbook, IRS SOI Table 4.B for Tax Year 2020. Until #292 it rendered its labels from --year, and the microcosm US feed built it at --year 2023: five TY2020 cells went out as ty2023 facts. #292 made the labels literal and added the artifact-year restamp guard, which compares one build with another. A literal label that is simply the wrong year passes that guard. These tests compare the labels with what the publisher printed. For every pinned package whose artifact.source_table names exactly one tax year, each tax-year fact built at the artifact year must carry that year as its period, in its tax_year_<year> vintage and in its record ids. For the W-2 package they also check that the declared title is the workbook's own A1 cell, that the five facts equal a direct read of the sheet, and that every --year from 2019 to 2025 builds the same TY2020 facts. Tax Year 2020 is the latest Table 4.B IRS has published (checked 2026-10-10: 21in04w2all.xlsx through 24in04w2all.xlsx return 404), so there is no later table to add. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
force-pushed
the
w2-source-table-year-tests
branch
from
October 10, 2026 20:40
e891f48 to
f5bd1d5
Compare
- Check the legal vintage too: a wrong literal legal_vintage on the W-2 tips amount passed every test, because it is the same at every build year. The registry test now reads any tax_year_<year> in the source vintage or the legal vintage, and the W-2 test pins the tips amount's legal vintage. - Require the table's year to be the only ty<year> token in each record id and record set id; an id with no year token used to pass. - Choose packages from the decoded artifact block, not from the YAML text, so a quoted key or an escaped title cannot hide one. Only the part before the top-level record_sets key is parsed (all 329 packages in about 2 s, where a full parse takes about 50 s). - Make the title reader's year list possessive and refuse any separator followed by a digit after it. "Tax Years 2019 and 2020/21" used to read as 2019 by giving back the years the lookahead refused, and "2020; 2021" and "2020 to 21" read as 2020. ";" and "/" now separate full years. - Say in the harness doc exactly what the test enforces. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Refuse a title only when a separator is followed by a one- or two-digit
fragment that ends the word (an abbreviated year such as 2023-24). Any
other number after a separator ("Tax Year 2020, 1040 Returns", "Tax Year
2020 - 1st Release") no longer drops the title.
- Read "or", "vs.", "thru" and U+2212 as separators, so "Tax Year 2020 or
2021" names two years, not 2020.
- Say in the doc that the package must build a fact and that every fact's
source_table must name the year, and that the year tokens are matched in
lowercase.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds tests that compare a fact's period with the tax year its source table's title names. No package, parser or schema changes.
The case behind it: the microcosm US feed carried five facts with period 2023 and vintage
tax_year_2023whose source is IRS SOI Table 4.B for Tax Year 2020 (20in04w2all.xlsx). #292 fixed the package on main. These tests check the result against the publisher's own title, and they catch a wrong literal year, which #292's guard cannot see.What I checked at the source (10 October 2026)
21in04w2all.xlsxthrough24in04w2all.xlsxreturn 404. There is no TY2023 Table 4.B to add. IRS's prior-releases page carries tax years 2008 to 2018 in one workbook (18inallw2.xls), so 2020 is the latest year published.20in04w2all.xlsxhas the sha256 the manifest pins (1178d776…cfc442f).How the ty2023 rows came about
At
c5e5bf8, the commit microcosm pins,packages/irs_soi/w2_statistics_2020/source_package.yamlsetartifact_year: 2020and renderedperiod,record_set_id,source_record_id_prefix,vintageandlegal_vintagefrom{year}. Microcosm's scope built the package at--year 2023for three record sets, so the Tax Year 2020 cells went out with ty2023 labels.On main today the package builds the same five TY2020 facts at every
--yearfrom 2019 to 2025 (a test below asserts it).What this adds
tests/test_chronicle_source_table_year.py:artifact.source_tablenames exactly one tax year, every tax-year fact built at the artifact year must carry that year as its period, in itstax_year_<year>vintage and in thety<year>tokens of its record ids. One package qualifies today (irs_soi/w2_statistics_2020). HMRC titles such as "tax year 2023-24" name two years and are left alone.source_tableequals cell A1 of the workbook, so the year the registry test reads is IRS's.openpyxlread of the sheet, and both equal the published values above.--year2019, 2021, 2022, 2023, 2024 and 2025 build facts identical to the 2020 build.docs/agent-source-package-harness.mdgains a paragraph on the new check under "Pinned artifacts and year labels".Invariant
When a package's source table title names exactly one tax year, every tax-year fact the package builds at its artifact year has that year as its period, in its
tax_year_<year>vintage and in its record ids.Mutation checks (run locally, then reverted)
test_chronicle_artifact_year_restamp.pyc5e5bf8file ({year}labels)period: '2023'as a literalvintage: tax_year_2023as a literalThe second row is the gap these tests close: a literal wrong year is the same file read under the same labels at every build year, so a guard that compares one build with another has nothing to flag.
Related
feed/us-c5e5bf8-w2-2020(f98acf4, tagmicrocosm-us-feed-w2-ty2020-v1) isc5e5bf8plus only the W-2 package file as Refuse pinned-artifact builds that only relabel another year (#117) #292 left it. Stamp the W-2 item facts with Tax Year 2020; the 2024 tips target moves from $28.28B to $34.29B microcosm#1192 re-pins the US feed to it: the 2024 W-2 tips target moves from $28,280,884,269 to $34,287,530,779, and no other target changes. It is a pin base, not for merging.3dce46a6), ten pairs of microcosm's pinned scope are no longer declared: these three W-2 pairs, the congressional-district file, the two IRA tables and fourstate_2022rows. The W-2 pairs move now; the others are microcosm#1030's.Tests
uv run pytest tests/test_chronicle_source_table_year.py: 27 passed (10.6 s).uv run ruff check chronicle policyengine_chronicle db scripts tests: clean.🤖 Generated with Claude Code