Repository navigation
Carry industry and worked-last-year on every US person record (ASEC restore + ACS crosswalk) - #1039
Merged
Merged
Conversation
…year The #720 restoration now appends the ASEC longest-job industry recodes WEIND (detailed, 0-23) and WEMIND (major, 0-15) to every vintage from its pinned Census person member. No H5 vintage carries them, so they sit outside the 2026-08-23 offline-fix receipt and are recorded as such. After restoring, each vintage is checked against two identities that hold in all three members: WEMIND is the major group of WEIND (the pinned WEIND_TO_WEMIND function), and WEIND carries a worker code (1-22) exactly where WKSWORK > 0. The release-time org_wages carry reads them next to POCCU2 and emits detailed_industry_recode, major_industry_recode and worked_last_year (WKSWORK > 0), refusing missing or out-of-range codes and checking the halves of the identities that hold on every spine. Gate bands follow the measured shares. Part of #719. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The release predictor join now also materializes WEIND, WEMIND and WKSWORK for ACS-spine people, so the org_wages carry gives every released person record an industry and a worked-last-year indicator, the same way ACS OCCP already reaches detailed_occupation_recode through POCCU2. ACS_INDP_TO_WEIND (267 codes) reads every civilian code's group from the pinned 2025 ASEC member, where INDUSTRY determines WEIND exactly and the 2022 Census code list matches ACS 2024 INDP. The eight ACS-only codes are reviewed: the seven Armed Forces branch codes collapse to CPS 9890's group 22, and 9920 is the did-not-work group 23. Blank INDP mirrors the OCCP treatment: 0 below age 16 (age 15 keeps the out-of-universe sentinel), 23 from age 16 where only ESR 6 may be blank. WEMIND follows WEIND by the pinned nesting; WKSWORK is WKWN (weeks in the past 12 months) or 0. Positive weeks without a worker industry code is refused. Crosswalk version 2; digest re-pinned. Part of #719. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…release source_stages.json's org_wages stage names WEIND, WEMIND and WKSWORK as derive_from_asec inputs and detailed_industry_recode, major_industry_recode and worked_last_year as outputs (the recodes nonnegative). The three join the post-reference required release inputs with notes that today's published defaults lack them by design, and the coverage manifest regenerates (170 required). The bundle regenerates from the manifest authority (sources.yaml, calibration.yaml tail contracts) with the frozen source_stages digest re-pinned. Adds docs/us-work-experience-columns.md (sources, crosswalk, identities, real-data evidence on the pinned ASEC members and all 3,422,888 ACS 2024 persons) and the changelog fragment. Part of #719. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hypothesis property tests state and check the #719 invariants over generated inputs: WEMIND nests WEIND, positive weeks carry a worker industry code, the ASEC-only converse, the release carry's round trip, and the ACS crosswalk's universe rules. A differential test runs the same rows through the ASEC restoration and the release carry and requires each to accept exactly when its stated identities hold (the restoration is the stricter site by design). The spec-engine inventory vector moves with the org_wages manifest change: source-manifest, resource-semantics, full-checkpoint and tail digests, and 937 tail control fields (+3 tail-contract columns). Part of #719. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ields +11 configuration/authored fields (3 derive inputs, 3 outputs, 2 nonnegative outputs, 3 tail-contract columns); the calibration (312 -> 315) and source_stages (1,712 -> 1,720) pointer claims and the pointer inventory digest move with them. Coverage evidence regenerated by tools/spec_engine_coverage.py: 42,191/42,191 fields, 41/41 inventory checks. Part of #719. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Preserve the exact ASEC industry restore and shared ACS crosswalk across current fiscal and native ACS paths. Require complete canonical job inputs on preserved donors and final staging artifacts. Regenerate source bundles, input coverage, and spec-engine attestations with reproducible tooling. Retain main's stage/graph architecture, dependency locks, and test layout.
Brings in main through 1eb0b1b. The us-f0-coverage.json conflict takes main's version. The L0 export fixtures gain the job fields the work-experience columns need, and the import-graph pin moves to 77 modules. Rebuilt from the cancelled lane's salvage (refs/subfleet-salvage/ work-experience-inputs-719-v2-20261024T011240Z-a1). The lane had resolved this merge in a private git dir but was stopped before committing; the tree is that working tree without the private git dir. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…merge The merge of main took main's sources.yaml, dropping this branch's work-experience entries (WEIND, WEMIND, WKSWORK inputs; industry and worked_last_year outputs; the stage note). Regenerating the bundle from the constants restores them exactly, and the documented re-pin sequence (tools/repin_us_spec_engine_contracts.py --legacy-only, the bundle generator, the re-pin, tools/spec_engine_coverage.py) moves the field counts to 42_228 configuration / 32_448 authored and re-pins the stage, calibration, tail and checkpoint digests. Both --check modes pass and the coverage tool reports 41/41 inventory checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The repinner hashed the compiled bundle's calibration, but the loader writes integral floats as integers (5.0 becomes 5), so its digest (080ec479...) never matched the semantics test, which hashes the constants builder's raw contract (aa65317d..., failing engine-free CI). The repinner now builds the contract exactly as that fixture does, and a new test holds its derivation to the committed pin. Stripping this branch's three new tail names reproduces main's pin (05cb099e...) from the raw contract, not from the bundle. The work-experience differential test now also draws nested industry pairs with any weeks, so the forward and converse worker checks decide the outcome; removing either check in either site now fails it. The docs and crosswalk comment say nonzero INDUSTRY determines WEIND: in the pinned 2025 member INDUSTRY = 0 maps to both WEIND 0 and 23. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
marked this pull request as ready for review
October 24, 2026 06:11
Contributor
Author
|
Round-2 delta review (GPT-6.1 Sol, Subfleet 20261024-015826-lwi-r2-1039): APPROVE at 558d5b6, no findings.
Merging on green CI; if #1376 lands first, this branch takes main and re-pins with the fixed tool. |
The repinner moved the field-usage tests' mode and generation-0 counts by value transition, taking each old value from the committed coverage JSON. A merge can break that: in a trial merge of #1376 into this branch, git auto-merged #1376's compiler_semantic 27718 into the JSON while the conflicted test kept 27_717. The tool then saw no transition, left the stale literal, and --check passed while the test failed. The five asserted count dictionaries (source, mode and generation-0 counts in the engine-free field-usage test; mode and generation-0 counts in the coverage-tool test) are now rewritten by key from the ledger. The keys must match the observed ledger exactly. Totals still move by value transition from the module's own pins. On this branch --check is unchanged. On the trial merge the tool now flags and fixes the stale count, and both tests pass (21 passed). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Round-3 review noted the scalar totals still moved by value transition from the module's own pins, so a merge that had already moved EXPECTED_CONFIGURATION_FIELD_COUNT could leave a stale 42_228 in a test that --check would not see. The engine-free field-usage test and the coverage-tool test are now refreshed only by asserted expression: - `len(field_ledger.fields)` and the distinct-pointer count; - the coverage report's field totals and claim count; - the inventory item counts and the four asserted inventory counts; - the keyed breakdowns. Planting stale values in three of these makes --check fail and the tool restore them. The value-transition helper, now unused, is removed with its test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#1376 and this branch both moved the F0 spec pins and the pool tool's import graph. The conflicts resolve as follows: - agent-guide.md keeps both paragraphs, in the one running paragraph main uses. - The pool build tool's runtime graph is 80 modules: this branch's two crosswalk modules plus #1376's county_vintage, puma_county_vintage and block_ladder_sources. - Every other conflicted pin was regenerated by the documented sequence (repinner --legacy-only, bundle generator, repinner, coverage tool). The ledger is now 42,229 configuration fields, which is this branch's +11 plus #1376's +1, and coverage passes 41/41 inventory checks. The US bundle spec_sha256 is 53fb8de6. The repinner's keyed rewrite caught #1376's compiler_semantic 27718, which git had auto-merged into the coverage JSON but not into the conflicted tests. A final --check passes. 885 targeted engine-free and engine-contract tests pass, plus the pool tool's 7 constants-adapter tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Round-4 review found that the identity test's `r"87 unclaimed normative field"` regex has no refresh path. Every field has exactly one primary claim, so dropping the selection claim leaves exactly its observed expansion (87) unclaimed. The repinner now rewrites that one expectation from the expansion it already computes. With a stale 86 planted, --check fails and the tool restores 87. The main merge also dropped #1376's note that the puma_county_vintage draw module joins the seed protocol's attested kernel modules; it is restored above the seed hashes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Reviews since 558d5b6 (GPT-6.1 Sol via Subfleet), all APPROVE with no findings:
Merging when CI is green at 7ab3a77. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #719. Replaces #745 (closed: it predated the #720 restoration and the ACS release join, and could not rebase across 1,600 commits). Delivers #922 item 5 at the code level.
What
Every released US person record carries three new PolicyEngine-US inputs (engine side: policyengine-us#9311, in the locked 2.2.1), next to
detailed_occupation_recode:detailed_industry_recodeWEINDINDP→ACS_INDP_TO_WEINDmajor_industry_recodeWEMINDWEIND_TO_WEMINDof the aboveworked_last_yearWKSWORK > 0WKWN > 0The design reuses existing boundaries rather than adding a stage:
asec_census_person_columns(Pooled ASEC 2022/2023 vintages lack the NOW_* coverage recodes: reported Medicaid at interview thins to 24.6M vs ~58M survey under 65 on the certified artifact #720) now also restoresWEIND/WEMINDfor every vintage from its pinned Census person member by exactPERIDNUMjoin before pooling. No H5 vintage carries them, so they are recorded as beyond the 8/23 offline-fix receipt.acs_release_predictors, crosswalk v2, digest re-pinned) maps nativeINDP/WKWNonto the same three raw columns, exactly as it mapsOCCPontoPOCCU2.org_wagescarry emits the three columns beside occupation;source_stages.jsondeclares them; they join the post-reference required release inputs (coverage manifest: 170 required).Why
WEIND/WEMINDandWKSWORK > 0: they describe the longest job last year, the same job asPOCCU2and the same year as the income; the survey-weekA_DTIND/A_MJINDwould describe the current job. In all three pinned membersWEINDcarries a worker code exactly whereWKSWORK > 0, whileWORKYN = 1misses ~600 allocation rows per year.The ACS crosswalk takes each civilian code's group from the pinned 2025 ASEC member (same 2022 Census code list as ACS 2024
INDP;INDUSTRYdeterminesWEINDexactly there) and reviews the eight ACS-only codes (seven Armed Forces branches → 22, as CPS's single 9890; 9920 → did-not-work 23). BlankINDPmirrors theOCCPrule (0 below 16, age 15 kept out of universe; 23 from 16, only ESR 6 may be blank).Invariants (stated and executed)
WEMIND == WEIND_TO_WEMIND[WEIND]— checked in the ASEC restoration, by construction in the ACS join, and in theorg_wagescarry and gate.WKSWORK > 0⇒WEINDin 1–22 — all three sites, every spine.WEINDin 1–22 ⇒WKSWORK > 0— checked only in the restoration, where rows are known to be ASEC. Intended asymmetry: ACSINDPcovers a job in the past five years, so the release carry does not check it.worked_last_year == (WKSWORK > 0).INDPfrom 16 → 23; observedINDP→ table;WKSWORK = WKWNor 0.test_us_work_experience_properties.pychecks all five with Hypothesis (200 examples each) plus a differential test running the same generated rows through the ASEC restoration and the release carry, requiring each to accept exactly when its stated identities hold. The differential draws both independent codes and nested industry pairs with any weeks, so each worker check decides some cases; deleting any of the three checks fails it.Real-data evidence (in
docs/us-work-experience-columns.md)pppub23/24/25.csv: nonzeroINDUSTRY→WEIND/WEMINDfunctional (263/263/259 codes;INDUSTRY = 0covers bothWEIND0 and 23),WEIND→WEMINDthe same function in all three, zero universe violations; weighted nonzero-industry shares 0.822–0.824, worked 0.520–0.523.INDPcode mapped,WKWN>0iffWKL=1, the 264,806 people with an industry but no weeks equal theWKL=2count exactly; weighted nonzero-industry 0.812, worked 0.538. Gate bands bracket both spines.Pins moved (all explained)
ACS crosswalk digest (v2); frozen
source_stages.jsondigest; spec-engine inventory (source-manifest, resource-semantics, full-checkpoint and tail digests; tail control fields 934 → 937 = the three tail-contract columns); field-usage counts (+11 configuration fields = 3 derive inputs + 3 outputs + 2 nonnegative + 3 tail-contract) and the affected pointer claims; coverage evidence JSON regenerated by its tool.Repinner (
tools/repin_us_spec_engine_contracts.py)The documented re-pin sequence regenerates every F0 pin. Three bugs were fixed during review:
Calibration projection. The tool hashed the compiled bundle, whose loader writes 5.0 as 5. It now builds the contract exactly as the semantics-test fixture does (pin
aa65317d…). Stripping this PR's three tail names reproduces main's05cb099e…from the raw contract, but not from the bundle.Count pins found by old value. Pins were located by their old value, taken from the evidence JSON or the module's own pins, so a merge that moved one copy hid a stale other while
--checkpassed. That happened in a trial merge with Add 2023 county-equivalent geography to US household ladders #1376. Every field-usage and coverage-test count is now refreshed by its asserted expression or key:"<N> unclaimed normative field"expectation.Planting stale values makes
--checkfail and the tool restore them.Merge with #1376
Merged after #1376, #1377 and #1378. The combined ledger has 42,229 configuration fields (main +12: 11 here, 1 from #1376) and passes 41/41 inventory checks. The pool tool's runtime graph has 80 modules. The bundle spec hash is
53fb8de6….Not in this PR
Released files carry the columns once the pooled base and multispine pool are rebuilt and a release runs (#922 sequencing step 1). Until then the release input-coverage gate is red on these three columns by design (notes in the manifest).
axiom: n/a: data pipeline only (no policy rule changed)
🤖 Generated with Claude Code