Skip to content

Run the ESI premium stage in the stacked pool (#454) - #1199

Open
MaxGhenis wants to merge 6 commits into
mainfrom
esi-premiums-pool-operator
Open

MaxGhenis wants to merge 6 commits into
mainfrom
esi-premiums-pool-operator

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The stacked ASEC + ACS pool now carries employer_sponsored_insurance_premiums on every row, so a pool-lineage (--exact-k) release has the input that us_release_input_coverage_gate requires and us_esi_premiums_anchor_gate grades. Follow-up to #1189 for #454; docs/us-esi-employer-premiums.md named this gap under "What this does not do".

Stacked on #1189 (esi-employer-premiums-454). This repo's CI runs only on PRs that target main, so checks start when #1189 merges and this PR is retargeted. Until then the test evidence is local; see the test plan.

axiom: n/a: data/microsim pipeline

The two design questions

1. Scaling. A cps_source operator sees only the rows with raw CPS evidence, at pool weights. assemble_stacked_spine gives each source half of ASEC's household mass, so those rows hold half the population. Run unchanged, the stage would hold half the policyholders to the whole anchor and double every premium.

The pool passes the stage anchor_share: the household mass of the rows it runs on over the pool's household mass, read from the live frame (us_esi_premiums_household_mass_share). The factor is anchor x share / sum(w x raw). Assembly multiplies every ASEC household weight by that same share, so the factor equals the single-source factor and each ASEC person carries the dollars a single-source build assigns. anchor_share defaults to 1, which leaves the base build as it was.

2. ACS rows. ACS has no policyholder flag, payment status, tier or firm size, so the stage cannot run there. The column enters the pool's QRF transfer as family source_operator_esi_premiums, on the transfer's fixed predictor surface (age, sex and State always; wages, self-employment, Social Security, retirement and investment income, household head and tenure where both sources observe them).

A QRF draw keeps no total and can put a premium on a non-worker. with_us_esi_premium_pool_anchor runs in the derive stage, once every row is filled. On transferred rows only, it:

  1. gives every support clone its clone-0 source record's draw, so a person carries one premium whether the pool transferred before or after it cloned;
  2. clears the premium where that source record reports zero wages; then
  3. scales the rest by one factor so those rows carry the same weighted premium per unit of household mass as the source-derived rows.

A transferred person's wage must be evidence. The anchor resolves the ACS earnings universe over the transferred rows. An eligible person (15 or older) needs a reported raw WAGP, even if a producer filled the mapped wage. A person below 15 needs the named universe zero. A missing or non-numeric mapped wage is refused too. In the stacked pool nothing real is refused: the late ACS earnings-universe producer writes the zeros below 15 before the derive stage, and Census reports WAGP for everyone else.

It tells the two kinds of row apart by whether the raw ASEC coverage columns are present, corroborated by the CPS record id (PERIDNUM), not by a source channel. So esi_premiums.py stays spine-blind and moves to the registered operator set. A CPS record that lost its coverage codes is refused by the anchor and fails both gates; it is never graded as a transfer.

Where this departs from the brief

The brief suggested a post-clone operator, since the draw is keyed by source identity. I placed the stage pre-clone:

  • A pre-clone operator's outputs are early gap-fill targets. The donors are ASEC rows before the PUF clone, so the fill sees the measured CPS wage on donors and the measured ACS wage on recipients. A post-clone operator would fill from PUF-detail clones, whose wages are PUF-imputed.
  • The clone copies one value per source person, so clone agreement holds by construction.
  • The late producer DAG is unchanged: 38 producers, 19 transfer groups, 70 targets, 71 edges, and the same schedule and payload digests.

Invariants

For every valid pool, stated in the module and property-tested with Hypothesis in tests/engine_free/us/test_us_esi_premiums_pool.py:

  • Conservation. The pool-wide weighted column equals the source-derived rows' total divided by their household mass share, which is the single-source column total.
  • Equal intensity. Source-derived and transferred rows carry the same weighted premium per unit of household mass.
  • Source rows are immutable. No source-derived cell changes by a byte.
  • One premium per person. Every support clone of a transferred person carries its source record's draw.
  • Structural zeros. A transferred person whose source record has zero wages carries nothing, on every support clone.
  • Proportionality. Every other transferred value is one common multiple of what the transfer drew for the source record.
  • Idempotence. A pool already on that scale is returned as the same object.
  • Weight homogeneity. Rescaling every household weight moves no value.

The strategies generate zero-weight households and support clones (index 1 and the tail's index 2), with per-clone draws on the transferred side.

Differential tests: the pool's CPS rows equal a single-source build's dollars row for row, and the pool total equals the single-source total for any mass split. The anchor-universe total the release gate recomputes equals the anchor on the pooled fixture, before and after the PUF clone.

Assumption. The anchor (BEA NIPA 7.8 line 17 since #1189's d1317 change) counts every policyholder, and ACS rows have no policyholder flag to price the non-employed ones from. The gates take the pool's anchor-universe total as the ASEC rows' total plus the transferred column over the ASEC rows' employed share. That assumes employed policyholders carry the same share of employer premiums on the ACS half as on the ASEC half. The anchor gate records the assumption and fails if the transferred rows' column per unit of household mass leaves 0.8 to 1.25 times the source rows'.

Measured on the pinned inputs

experiments/us-esi-454/run_pool_fill_on_pinned_inputs.py runs the part of the pool that decides the column on the real inputs (pooled 2023-2025 ASEC, ACS 2024 1-year PUMS, the real assembly, stage, early gap-fill QRF, anchor operator, by-origin battery and both gates) on a household sample. It is not a pool build, and no artifact was built or published. Receipts hold aggregates only.

1% of households 5% of households
Person rows, ASEC / ACS 4,177 / 34,293 21,133 / 171,381
ACS rows filled / left null 34,293 / 0 171,381 / 0
ACS-over-ASEC incidence, after the fill / after the anchor (band 0.8 to 1.25) 0.933 / 0.930 0.975 / 0.969
Conditional quantile distance, after the fill / after the anchor (limit 0.25) 0.072 / 0.172 0.043 / 0.084
Transferred rows cleared for zero wages 34 ($1.7B) 255 ($2.8B)
Scale factor on the other transferred rows 1.105 1.046
Mean per holder after the anchor, ASEC / ACS $10,956 / $12,071 $11,766 / $12,325
Pool column total (single-source: $863.9B) $884.0B $863.0B
Transferred wages that were not evidence 0 0
By-origin battery, signal gate, anchor gate pass pass

Figures are at #1189's BEA anchor ($977.0B for 2024).

The fill reproduces ASEC's premium per holder and gives slightly fewer ACS people one, and ACS rows carry fewer weighted people at equal household mass, so the factor is above one and ACS holders end about 5% above ASEC holders in the 5% sample. The factor fell from 1.10 to 1.05 as the donor sample grew fivefold. A full build was not run. In the 5% sample the earnings-universe producer wrote the named zero wage on the 25,511 ACS records below 15, and Census reported WAGP for all 145,870 others.

Sealed pools and pinned surfaces

  • Legacy v11 authority. Adding a family changes the live stacked authority. _legacy_stacked_authority_receipt rebuilds the version-11 authority from live surfaces to score attested schema-9 pools, so a new family would have changed it. The reconstruction now excludes families added after v11 (_POST_V11_AUTHORITY_TARGET_FAMILIES), and a new test pins the legacy authority digest and its components to their previous values.
  • Legacy publication golden. test_legacy_entrypoint_publication_matches_origin_main_golden moves only in checkpoint identity digests. Run against the base branch with the same fixture, the pool H5 and agreement bytes are identical and the manifest differs only in stage_checkpoints.*identity_sha256.
  • H3 graph. us_runtime/graph.py declares the premium, the seven raw coverage columns, the CPS record id (a string cell), the mapped and raw ACS wages as source cells, and the premium as a derive rewrite. The derive kernel's implementation hash now covers esi_premiums.py. The parity fixture was regenerated with tools/graph_us_post_transfer_fixture.py, whose helper path was stale since the test split. Every pre-existing source and expected cell is byte-identical. Writing a real-shaped 21-digit PERIDNUM into the fixture exposed a defect in the generic csv-tables source codec. It inferred numbers before casting to the declared dtype, so a declared string column of long numeric ids came back rounded. It now reads declared string columns as strings, and a codec test covers long ids, leading zeros, blanks and whitespace. The new expected column shows the transferred rows moving from the drawn $6,000 to $8,000, the ASEC row's premium at equal household mass.
  • Re-pins. Counts and digests the new early family and derive consumer move are re-pinned at computed values: 135 battery targets, 49 early targets in 14 families, 119 pool transfer targets, a 1,079-row remaining-stage manifest, the stacked authority and its components, the gap-fill schedule, the checkpoint identity, the spec bundle, field-usage claims and the coverage evidence. The seed protocol and seed map digests do not move.
  • No calibration spec. No post-transfer calibration spec is added for this column. Terminal validation replays a calibrated target's bytes, which would refuse the later rescale.

Independent review

GPT-6.1 Sol reviewed this twice, and both reviews asked for changes. The last commit answers the second.

Round 1

Finding Answer
A missing clone-0 wage was read as no wages and the premium cleared Wages must be evidence (round 2 tightened this: see below).
A CPS record with every coverage field null passed as a transferred row, in a pool or a single-source frame Row kind is corroborated by PERIDNUM. The anchor refuses such a record and both gates fail on it.
The legacy two-spine path transfers after it clones, so support clones could draw different premiums Every transferred clone takes its clone-0 record's draw, and the signal gate fails on transferred clone disagreement. I kept the stage on the legacy path rather than freezing separate legacy surfaces: every pool operator is shared with it, and a pool lineage without the input fails the release gates.
A pool whose transferred rows hold no household weight failed the gates and the anchor The anchor keeps the factor at one, and the gates skip the two-sided comparison.

Round 2

Finding Answer
The legacy PUF pass fills missing mapped wages with zero, so the round-1 wage check could not see missing evidence there The anchor resolves the ACS earnings universe over the transferred rows. An eligible person with a blank raw WAGP is refused whatever the mapped column holds, and a person below 15 must carry the named universe zero. WAGP and the universe inputs are declared anchor inputs.
The target used 1 - share, which loses a tiny side to rounding, so the factor depended on weight scale Each side's household mass is summed on its own. The target is source total x transferred mass / source mass, and the gates compare each side against its own mass. The property strategy now spans weights from 1e-9 to 1e10.
The H3 graph declared PERIDNUM as a float It is a string cell, and the graph's CSV source codec now reads declared string columns as strings (it rounded long numeric ids).

My own widened property tests found one more defect in round 1: the mass share depended on floating-point summation order. Round 2's separate masses replace that code.

Not in this PR

  • ACS local lane. declared_acs_transfer_target_families still does not name the column, and fill_reviewed_nulls refuses that frame. A separate branch, esi-premiums-acs-local-lane-454, is stacked on this one.
  • ACS-native predictors (HINS1, COW, ESR) for the fill.
  • ACS usual hours. The pool's operator boundary refuses the ACS-native usual-hours column on main; the experiment drops it. A separate session is fixing that.

Test plan

Local, on the pushed head:

  • uv run ruff check . and python3 tools/ci_test_plan.py verify
  • tools/generate_us_bundle_from_constants.py --check, tools/spec_engine_coverage.py --check, tools/repin_us_pool_engine_contracts.py --check
  • the test files the change touches, across the engine-free, engine-contract and engine-workflow tiers and the H3 graph parity test (rerun on the final head)
  • full engine-free and engine-US jobs on the final head (running)
  • third-round independent review

CI, once this targets main:

  • engine-free and engine-US jobs green on both Python versions
  • lint and wheels

🤖 Generated with Claude Code

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

The base of this PR, #1189, moved to head 3cba55254. Decision d1317 (2026-10-11) set the stage's scale anchor to BEA NIPA 7.8 line 17 (B4923C) instead of CMS NHE Table 24, and the US feed is re-pinned. What this PR needs on its next rebase:

  • esi_premiums.py. EMPLOYER_PREMIUM_ANCHOR is now BEA, with values for 2023, 2024 and 2025 and a new feed_record_set key. EMPLOYER_PREMIUM_CROSS_CHECK is now NHE, with values for 2023 and 2024. _anchor keeps its signature and returns BEA values, and 2025 is now a valid build year. The anchor gate's cross_check_ratio is the total over NHE. There is a new us_esi_premiums_anchor_feed_mismatches(facts), which the release tool calls after the feed pin check. Beyond the constants, messages and docstrings, _person_raw_shares, the handler's scale line, us_esi_premiums_summary and the gate structure are unchanged.
  • Numbers. The single-source column is $863.9B on the default pool (scale factor 0.9708), not $925.7B. Pool fill receipts move by the same factor, 977.034 / 1,047.0.
  • Feed. us/chronicle_feed.json pins Chronicle cf9ece2 (40,181 rows, facts_sha256 054e5684…). The scope has 640 pairs and 66 runs, and target_parity_manifest.json has 82 feed families.
  • Spec bundle. spec_sha256 is b7311566… and coverage is 42,217 fields with 41 inventory checks. The source_stages claim is 1,749 / d0979722…, and the inventory digests changed. Seed sites are unchanged at 53. The digest re-pins in this PR need one more pass.
  • Main. It was merged in at 909586128, which brings in Bind state × AGI-band SOI targets and re-pin the US feed for TY2023 state bands (#940) #1102.

A round-4 independent review is running on #1189.

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

One more commit on the base, #1189, now at head ebfb05582 (answers to its fourth review). On top of the notice above, a rebase needs these:

  • Release tool. The feed check is now _check_us_esi_anchor_against_feed(facts, allow_unpinned_feed=..., allow_esi_premium_gaps=...) in tools/build_us_fiscal_refresh_release.py. esi_premiums.json is schema_version 3, with an anchor_feed block from _esi_anchor_feed_evidence.
  • Spec bundle. spec_sha256 is now 0a5ecc7f… and the source-stage manifest digest is 559062a1…: a note in us/source_stages.json changed wording. Three inventory digests in spec_engine/inventory_coverage.py moved again (full_checkpoint, late_resource_semantics, source_manifest). Field counts and the source_stages claim are unchanged from the notice above.
  • Target parity. tools/build_us_target_parity_manifest.py has a new _ESI_NHE_ORIGIN, and the manifest is regenerated.

esi_premiums.py itself did not change in this commit.

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

The base, #1189, is now at head dc8578d05 and is no longer conflicting with main. It merges main at 123927679 and answers its fifth review's nits. A rebase onto it needs these:

esi_premiums.py is unchanged since 3cba55254. A round-6 delta review is running on the merge.

MaxGhenis and others added 4 commits October 12, 2026 09:50
The stacked ASEC + ACS pool did not run the meps_esi_premiums stage, so a
pool-lineage release carried no employer_sponsored_insurance_premiums and
failed us_release_input_coverage_gate and us_esi_premiums_anchor_gate.

Stage on the CPS rows. with_us_esi_premium_inputs joins the pool's
pre-clone source operators. The pool passes anchor_share, the household
mass of the rows the stage runs on over the pool's household mass, read
from the live frame. The factor becomes anchor x share / sum(w x raw), so
each ASEC person carries the dollars a single-source build assigns.
anchor_share defaults to 1 and leaves the base build as it was.

ACS rows. The column enters the early gap-fill as family
source_operator_esi_premiums, so the QRF sees measured wages on donors and
recipients. with_us_esi_premium_pool_anchor then runs in the derive stage.
On rows without the raw ASEC coverage columns it clears the premium where
the person's clone-0 source record has no wages and scales the rest by one
factor, so transferred rows carry the same weighted premium per unit of
household mass as source-derived rows. It never rewrites a source-derived
row and refuses a row with only some raw columns.

Invariants, property-tested in test_us_esi_premiums_pool.py: the pool-wide
weighted column equals the source rows' total over their mass share; both
kinds of row carry equal premium per unit of mass; source rows are
immutable; transferred people without source wages carry nothing on every
clone; the rest move by one common multiple; a pool already on scale is
returned unchanged; rescaling every weight moves no value. Differential
tests hold the pool's CPS rows to a single-source build's dollars.

Gates. Both ESI gates grade a pooled frame by row kind and band the
transferred rows against the source rows. The anchor gate records the
assumption it makes about the ACS half's employed share.

Sealed pools. The legacy v11 stacked authority reconstruction excludes
families added after v11, and a test pins its digest and components to
their previous values.

H3 graph. The post-transfer graph declares the premium, the raw coverage
columns and the wage column as source cells and the premium as a derive
rewrite, and the derive kernel's implementation hash covers esi_premiums.
The parity fixture is regenerated; every pre-existing cell is
byte-identical. The fixture generator's helper path, stale since the test
split, is fixed.

Pins. Counts and digests the new early family and derive consumer move
are re-pinned at observed values: 135 battery targets, 49 early targets in
14 families, 119 pool transfer targets, a 1,074-row remaining-stage
manifest, the stacked authority and its components, the gap-fill schedule,
the checkpoint identity, the spec bundle, field-usage claims and the
coverage evidence. The late producer DAG, the seed protocol and the seed
map do not move. The legacy publication golden's pool H5 and agreement
bytes are unchanged; its manifest differs only in checkpoint identity
digests.

Evidence. experiments/us-esi-454/run_pool_fill_on_pinned_inputs.py runs
the stage, the real early gap-fill and the anchor on the pinned ASEC and
ACS inputs for a household sample. Receipts for 1% and 5% samples are
committed (aggregates only) and a test pins the documented figures to
them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/us-esi-employer-premiums.md gains a stacked-pools section: the mass
share rule, the transfer to ACS rows, the derive-stage anchor and its
invariants, the anchor-universe assumption, why the column has no
post-transfer calibration spec, and what the pinned-input runs measured.
docs/us-multispine-operator-ordering.md adds the stage and the anchor to
the ordering, the hard-consumer audit and the operator registry, and
moves the surface counts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Missing wages. The anchor read a missing or non-numeric clone-0 wage as no
wages and cleared the premium. It now refuses a transferred row whose
source record has no finite wage. The stacked pool's ACS earnings-universe
producer writes the explicit zeros below age 15 before the derive stage,
and the QBI reconciliation that follows the anchor already refuses a blank
ACS earnings value there, so the rule adds no refusal a pool would
otherwise survive.

Row kind. Nulls in every raw coverage field made a row transferred, so a
CPS record that lost its codes passed as a transfer, in a pool or in a
single-source frame. The CPS record id (PERIDNUM) now corroborates it: a
row with the id must carry every field, a row with the fields must carry
the id, and a frame without the id column cannot hold transferred rows.
PERIDNUM is a declared anchor input (remaining-stage manifest, 1,075 rows;
H3 graph cell).

One premium per person. The legacy two-spine order transfers after it
clones, so its transfer draws each support clone separately. The anchor
now gives every transferred clone its clone-0 record's draw before it
clears and scales, and the signal gate fails on transferred clone
disagreement.

No transferred mass. The anchor no longer refuses a pool whose transferred
rows hold no household weight, and both gates skip the two-sided
comparison then.

Mass share. Widening the property tests to zero weights and support clones
found that the share was taken against the sum of all household weights,
so "no transferred mass" depended on summation order and rescaling every
weight could turn a valid pool into a refusal. Each side is now summed on
its own.

The experiment runs the earnings-universe producer before the anchor, as
the pool does. Rerun at 1% and 5%, it reproduces every documented figure:
no eligible ACS record was missing a wage and no clone needed resetting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Wage evidence. The legacy two-spine PUF pass fills missing mapped
earnings with zero, so the round-1 wage check could not tell a filled
zero from a measured one there. The anchor now resolves the ACS earnings
universe (resolve_acs_pums_earnings_universe) over the transferred rows:
an eligible person (15 or older) with a blank raw WAGP is refused
whatever the mapped column holds, and a person below 15 must carry the
named universe zero. WAGP and the universe inputs (age, tax-unit
membership, support provenance) are declared anchor inputs: the
remaining-stage manifest grows to 1,079 rows (54 derive), and the H3
graph gains the raw wage cell.

Masses. The anchor and the gates took the transferred side's mass as one
minus the source share, which loses a tiny side to rounding, so the
factor depended on weight scale. Each side's household mass is now summed
on its own; the target is source total x transferred mass / source mass,
and the gates compare each side against its own mass. The property
strategy spans weights from 1e-9 to 1e10.

PERIDNUM in the H3 graph is a string cell. Writing a real-shaped 21-digit
id into the fixture showed that the generic csv-tables source codec
inferred numbers before casting to the declared dtype, so long numeric
ids came back rounded. It now reads declared string columns as strings;
a codec test covers long ids, leading zeros, blanks and whitespace.

The base now anchors the stage to BEA NIPA 7.8 line 17 (d1317). The
pinned-input experiment was rerun at that anchor; the documented figures
and their pinning test follow it. In the 5% sample every transferred wage
was evidence: the earnings-universe producer wrote the named zero on the
25,511 ACS records below 15, and Census reported WAGP for the 145,870
others.

Re-pinned at observed values: the remaining-stage manifest and engine ABI
lock, the spec bundle, field usage and coverage evidence, the checkpoint
identity, the H3 fixture and graph counts, and the legacy publication
golden (pool H5 and agreement bytes unchanged).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis force-pushed the esi-premiums-pool-operator branch from db5f656 to 1677937 Compare October 12, 2026 15:32
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

The base, #1189, is now at head acc316427. It merges main at a90eec3cb (510 commits) and is no longer conflicting. A rebase onto it needs these:

esi_premiums.py is unchanged since 3cba55254. Two executed reviews (round 7) are running on #1189.

@MaxGhenis
MaxGhenis changed the base branch from esi-employer-premiums-454 to main October 24, 2026 02:51
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Base PR microcosm#1189 merged into main as 1eb0b1b4f81c8ae5323a0024d558322080e941c1 (2026-10-24 02:51Z). This PR's base is now main instead of esi-employer-premiums-454.

Since this branch's last update (167793776), #1189 picked up these changes:

  • the BEA B4923C anchor (d1317);
  • the feed re-pin to chronicle cf9ece2;
  • the anchor-feed check;
  • two merges of main, the latest at dec45a439.

The last merge re-pinned these digests:

  • seed map 3cf71bdd, seed protocol 15e9b110, source manifest 629a0712;
  • loader golden vector 44e3f5f2;
  • US bundle spec_sha256 97f41a9f.

Merge origin/main into this branch (no rebase: #1201 is stacked on it), then recompute any pins your change moves with tools/generate_us_bundle_from_constants.py --check and tools/spec_engine_coverage.py.

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Adopting this PR to land it. Max ruled d1318 on 2026-10-24: build the next US release once this lands. The session that opened it no longer exists, so its last pushed head 167793776 is the starting point. The local branch backup/esi-acs-lane-on-0e5f700dd and the worktree at 986f69ea8 are older drafts that this head supersedes, and I'm leaving them untouched. Plan: merge origin/main into this branch (no rebase, since #1201 is stacked on it), carry #1189's BEA anchor and feed check through the pool path, recompute the pins with the generators, run the local tests, push, get an executed independent review, then merge on the usual gates.

MaxGhenis and others added 2 commits October 24, 2026 14:22
Brings this PR up to main after microcosm#1189 merged (1eb0b1b) and
main moved past it. All nine conflicts were pins, counts or prose:

- field_usage counts: main's value plus this PR's own delta from its
  base dc8578d (configuration 42,437; authored 32,495; resolved
  9,942), confirmed by the ledger.
- inventory_coverage: this PR's authority, early-family and gap-fill
  digests (main does not move them) and a new full-checkpoint identity
  78a33bba, read off the inventory's observed values.
- the pointer inventory hash (6c3377be) and the US bundle spec_sha256
  (24ca0447), from the ledger and generate_us_bundle_from_constants.py
  --check; us-f0-coverage.json regenerated (42437/42437, 41/41).
- the pool tool's runtime graph is main's 80 modules, with
  esi_premiums.py classified as a spine-blind operator, as this PR
  intends; the stale "77" in main's comment now says 80.
- the experiment README keeps main's registry-receipt note and this
  PR's pool-fill section.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…olute-mass gates

Round 3 (review_r3.out.md) left two should-fix findings and two nits.

1. The pool anchor multiplied the source total by the transferred mass
   before dividing by the source mass, so finite weights whose correct
   totals are finite could overflow (an infinite target at 1e152, an
   infinite factor at 1e153) or underflow (a zero factor at 1e-200). It
   now divides first: intensity = source total / source mass, target =
   intensity x transferred mass, pool target = source total + target. It
   refuses a non-finite intensity, target or pool target, a non-finite or
   negative factor, and non-finite scaled outputs, and the total check
   needs a finite total before isclose. Tests: equal masses at 1 through
   1e300 give factor 2; pools whose correct total overflows are refused;
   the weight-rescaling property now spans 1e-150 to 1e150. All fail on
   the old arithmetic.

2. Entity CSVs read declared string ids as text, but a separate weights
   table still inferred numbers, so a csv-tables source with string ids
   such as 00042 failed "Weight table ids do not align". Both now read
   through _read_declared_csv, and a weights table that repeats an entity
   id column without declaring it takes the entity's dtype. Test with a
   leading-zero id and a 21-digit numeric id; it fails without the fix.

3. Both ESI gates decided whether the transferred side is weighed from its
   mass share, which underflows to zero at masses 1e340 apart. The summary
   now reports each side's absolute household mass and both gates read
   transferred_household_mass > 0. Test: a tenth of the premium on a side
   whose share rounds to zero still fails both gates.

4. run_pool_fill_on_pinned_inputs.py accepted --target-year values it does
   not price; it now accepts only the pool's own time period.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis added a commit that referenced this pull request Oct 24, 2026
…e check (#454)

#1199's rounds 2 and 3 changed the kernel this lane calls,
with_us_esi_premium_pool_anchor. Before trusting a transferred row's zero
wage, the kernel now resolves the ACS PUMS earnings universe
(resolve_acs_pums_earnings_universe), and that needs raw WAGP and the clone
index beside the wage. After the rebase onto #1199's head, the lane's view
dropped the clone index, so every lane build refused.

- The anchor view keeps the clone index. It still drops the support source
  id: lane ACS rows keep pre-remap source ids that can repeat a donor's, so
  without that id the kernel reads each row's own wages.
- The kernel's resolver scopes on the stacked pool's ACS support channel
  ("acs"). This lane tags its ACS rows with the late spine's channel
  ("acs_2024_1yr"), so on the lane that check sees no rows.
  _wages_under_the_acs_universe now applies the rule itself, refusing as
  the resolver does:
  - a blank raw WAGP at 15 or older, or at an unknown age, however the
    mapped wage was filled (round 2's zero-fill case);
  - below 15, a raw WAGP that is not blank, or a mapped wage that is
    neither null nor zero.
  The receipt counts the rows it checked and says why the kernel's own
  receipt counts none. A test pins that, so the docs are revisited if the
  owner module ever learns the lane's channel.
- The test frames carry raw WAGP. New tests cover the zero-filled adult,
  the child with a nonzero mapped wage, a child already holding the named
  zero, and missing WAGP or clone index.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant