Skip to content

G8-r2c B: transport.quantile_map@2 with a fitted grouped Pareto top band - #1459

Merged
MaxGhenis merged 8 commits into
mainfrom
nzhub/g8r2c-b-pareto-map
Nov 3, 2026
Merged

MaxGhenis merged 8 commits into
mainfrom
nzhub/g8r2c-b-pareto-map

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

G8-r2c shared hunk B (#1183): transport.quantile_map@2 and grouped_pareto_mle@1. B's base is main at 312dc4d6. #1452 merged into main, carrying the w4d head fcdb4739 that the merge-up below brought in; the "Merge-up" section stays as history.

Head: b20948d9e83347968c7ab0a154c1cbdb722ba875, after the fix round below.

Landing note. Belgium's hub measured a tree holding hunks A, B, C and D, and both transport.quantile_map@2 and measure.combine@1 read differently there than in this PR's table. So when A and D land after B and C, they move those two rows again. This body quotes no @2 value from that measurement, because F3 changed @2's hash after it.

Merge-up

  • Head: c9fa6639f5ebc3886fd948f7938f941429195a4a (was a401e22f).
  • Merged: w4d fcdb47395293955347ec2697feef1a736a76cc23, in merge commit 9ededbe1, with git merge --no-ff. Nothing was rebased or force-pushed.
  • What the merge changed. Git merged it with no conflicts: B's edits to T/compose.py sit apart from the fix round's. Every line of code the fix round owns keeps its text: quantile_support and inside_quantile_support in T/evidence_contract.py, the support and closure kernels in T/evidence_kernels.py, and _support_roster, support_roster_slices, _check_structural_columns, _requests, _check_unreached_requests, _check_closure_relation_slices and _check_structural_rows_written in T/compose.py.
  • Checked. After the merge, the changed lines of git diff fcdb4739 9ededbe1 were identical, line for line, to those of git diff d81280bb a401e22f. So the merge commit adds B's hunks to w4d and nothing else.
  • Two tests the merge broke, fixed in bcc9c4c2. Both are in test_transport_quantile_map_v2.py.
    • test_the_w4_support_producer_reads_v2_params_with_an_open_top (:721) unpacked positive, negative = quantile_support(...). Since w4c, quantile_support returns the frozen QuantileSupport dataclass (T/evidence_contract.py:569), which has no __iter__, so the line raised TypeError. The test now binds support, compares support.positive with a tuple of tuples, asserts support.negative == support.positive (a bare list feeds both signs), and calls quantile_bounds(support.positive, support.negative).
    • test_v2_support_bounds_both_signs_for_a_bare_band_list (:803) XPASSed strictly once E1 arrived, so bcc9c4c2 removed its xfail(strict=True) mark. The assertion is unchanged, and the test passes.

Fix round

The review in #1183 comment 6590677926 (APPROVE_WITH_NITS) owed B one word of source and a body update. Every file:line is at b20948d9.

Fix Commit Where Red test Shared hunk touched
F3: a refusal, not a receipt, names the heavy-tail reason b20948d9 B/tail_estimators.py:25: "and the receipt names that reason" becomes "and the refusal names that reason". The word has the same length, so no line moves Text only. TailEstimatorRefusalError raises at :343 and :361, and its message and code name the reason (:62-67) B/tail_estimators.py:25, B's own new file. It moves transport.quantile_map@2's hash, which digests tail_estimators (T/quantile_kernels.py:121-131); no other row moves
F4: this body this body the first lines, "Landing note", "Each property tests its full budget", "Implementation hashes" and "Tests" Text only none

Review fixes, earlier round

The review (#1183 comment 6585875953, APPROVE_WITH_NITS) owed B three fixes and two text changes. Every file:line is at this head.

Fix Commit Where Red under its mutant
Two properties counted skipped draws as passes 334f5a19 test_transport_quantile_map_v2.py:278 and :534; also :361, which had the same guard No test goes red: the defect was a lost example budget, not a wrong assertion. With the early returns, 30 and 44 of 60 counted examples reached the assertion; with assume, 60 of 60 do (see "Each property tests its full budget")
Review fix 4: half-even at a tie a5dd62a8 test_six_decimal_rounding_is_half_even_at_an_exact_tie, test file :258, on _round_half_even (B/tail_estimators.py:163-171) B-M1, remainder >= half at :169: that test, alone
Review fix 5: name the reason heavy tails refuse c9fa6639 B/tail_estimators.py:22-25, and "The change" below Text only
Review fix 6: the scaling invariant's tolerance this body "Invariants" below Text only

The change

  • grouped_pareto_mle@1, in the new B/tail_estimators.py (B/ is packages/microcosm-build/src/microcosm/build/, T/ is B/transport/).
    • It fits α by maximum likelihood to contiguous anchor bands (a_k, b_k] with counts n_k and the open top above c with count n_T, as design §2 writes it. grouped_pareto_score (B/tail_estimators.py:314-322) computes ℓ′(α) = Σ n_k (d_k / expm1(α d_k) − u_k) − n_T t.
    • grouped_pareto_mle (:325-375) bisects the bracket for the iteration count. Both come from the node's estimator declaration (parse_grouped_pareto_declaration, :192-247), as exact decimal text and an integer. It rounds the midpoint once to 6 decimals, half-even, in exact rationals (_round_half_even, :163-171), and carries α as p/10^6.
    • It refuses bracket_without_sign_change and alpha_not_above_one by name (TailEstimatorRefusalError, :62-68; raised at :343 and :361).
    • Heavy tails refuse as bracket_without_sign_change. Under the declared bracket [1 + 2^−20, 64], the rounded α is at least 1.000001, so alpha_not_above_one can't fire: data whose root lies at or below 1 refuses as bracket_without_sign_change instead. It still refuses, so R2C-9 holds. The module docstring says so (B/tail_estimators.py:22-25).
    • ParetoFit.receipt (:112-146) writes the estimator id and version, the anchor, each anchor's fact reference, cuts and count, the bracket, the iterations, ℓ′ at both ends of the final bracket, α as p/10^6 and as decimal text, and the implied top-band mean αc/(α − 1) as an exact fraction. The sensitivity variants are the NZ session's.
  • transport.quantile_map@2, in the new T/quantile_kernels.py.
    • estimated_alpha (T/quantile_kernels.py:54-112) decodes the band surface, runs _surface_bands (imported from T/population_kernels.py, unedited) for the same validation the map's shares get, and reads each anchor's count from that surface's specs.
    • run (:133-149) puts the fitted α into interpolation as a literal and calls TRANSPORT_QUANTILE_MAP.run, which is @1's own code path. It then adds the fit under receipt["tail_estimator"].
    • @2 accepts a bare band list only, and refuses a literal α (that's @1's), signed or component bands, and anchors that aren't the band list's top contiguous suffix ending at the open top.
    • Its hash (:121-131) digests its own module, tail_estimators, and every module @1's hash digests.
  • Registration. T/compose.py lists @2 beside @1 in SKELETON_KERNELS, in the money producer sets used by _money_producer_gaps and _check_single_money_normalization, and in _SUPPORT_PRODUCERS as quantile_mapped. T/registry.py installs it with the population kernels.

Invariants, and the test that checks each

All tests are in packages/microcosm-build/tests/engine_free/shared/test_transport_quantile_map_v2.py.

Invariant Test
ℓ′ is nonincreasing on the bracket and changes sign at most once test_property_the_score_changes_sign_once_on_the_bracket (Hypothesis)
Exact Pareto expected counts recover α within 1e-6, for α in [1.2, 5] and random anchors test_property_recovers_alpha_within_the_declared_tolerance (Hypothesis), test_recovers_a_known_alpha_from_exact_pareto_counts
Scaling every count by k leaves α unchanged: bitwise for k = 2^j, and within one unit of the 6th decimal otherwise. The second clause relaxes the design's "unchanged", and the six-decimal rounding causes it: scaling by any other k rounds each binary64 count once, and a root that sits near a six-decimal rounding boundary can then round to the neighbouring value test_property_scaling_every_count_leaves_alpha_unchanged (Hypothesis)
Moving mass upward lowers α test_property_moving_mass_upward_lowers_alpha (Hypothesis)
One finite band plus the top gives α = ln((n₁ + n_top)/n_top)/ln(U/L) test_belgium_rsz_counts_give_the_closed_form_alpha (3.348955), test_property_one_band_and_the_top_match_the_closed_form (Hypothesis)
@2 with its fitted α is bitwise @1 with that α as a literal test_v2_equals_v1_with_its_estimated_alpha_as_a_literal, test_property_v2_is_bitwise_v1_at_the_fitted_alpha (Hypothesis differential)
The anchor counts are the band rows' surface counts; changing one anchor count changes α, and a body count doesn't test_anchor_counts_are_the_band_rows_counts
Six-decimal rounding is half-even at an exact tie: 1.0078125 gives 1.007812, and 1.0078135 gives 1.007814 test_six_decimal_rounding_is_half_even_at_an_exact_tie
Each refusal fires by name test_refuses_a_bracket_without_a_sign_change_by_name, test_refuses_an_alpha_at_or_below_one_by_name, test_refuses_malformed_anchors, test_the_declaration_is_parsed_exactly, test_v2_refusals
The w4 support producer accepts @2's params, reads a bare band list on both signs and reports expected_min and expected_max as null test_the_w4_support_producer_reads_v2_params_with_an_open_top, test_the_support_bounds_kernel_reports_a_null_expected_max_for_v2, test_v2_support_bounds_both_signs_for_a_bare_band_list
Composition accepts @2 as a quantile_mapped money producer test_composition_accepts_v2_as_the_quantile_mapped_money_producer, test_v2_is_registered_beside_v1

The recovery tolerance, 1e-6. Rounding to 6 decimals contributes at most 5e-7. Sixty-four halvings of [1 + 2^−20, 64] leave a bracket of about 3e-18. The binary64 score's summation error moves the root by far less than 1e-9 here. Rounding alone rules out any tolerance below 5e-7.

Belgium's support request. test_v2_support_bounds_both_signs_for_a_bare_band_list was xfail(strict=True) on d81280bb, where quantile_support fed a bare band list to the positive side only. The w4 fix round's E1 (T/evidence_contract.py:591, signs() :629-641 at fcdb4739) reads it on both signs, so after the merge-up the test XPASSed strictly. Commit bcc9c4c2 removed the mark; the assertion didn't change, and the test passes.

Each property tests its full budget. Three properties used to return early on a draw they couldn't use, and Hypothesis counted each return as a pass. They now call hypothesis.assume, so Hypothesis draws a replacement and its health check would report a filter that rejects too much. With --hypothesis-show-statistics and a temporary hypothesis.event at each final assertion (not committed), all 60 of 60 counted examples reach the assertion in each property. The counts of replaced draws below come from one run, and they vary by run, because Hypothesis draws differently each time: the review's run (#1183 comment 6590677926) gave 102, 42 and 9. In every run, each property reaches its 60 examples.

Property Before, at bcc9c4c2 After, at 334f5a19
test_property_one_band_and_the_top_match_the_closed_form 30 of 60 reached the assertion 60 of 60; Hypothesis replaced 109 invalid draws
test_property_v2_is_bitwise_v1_at_the_fitted_alpha 44 of 60 60 of 60; 14 replaced
test_property_the_score_changes_sign_once_on_the_bracket 60 of 60 (its strategy's filter already guarded the return) 60 of 60; 7 replaced

Mutants

Each mutant ran against the test file above with -n 4. All 14 turn tests red. B-M1 is the review's (#1183 comment 6585875953) and ran at a5dd62a8; the other 13 ran at a401e22f.

Mutant Red tests (a selection)
Truncate α instead of rounding half-even test_belgium_rsz_counts_give_the_closed_form_alpha, test_the_receipt_records_the_fit
Drop the α ≤ 1 refusal test_refuses_an_alpha_at_or_below_one_by_name
Drop the sign-change refusal test_refuses_a_bracket_without_a_sign_change_by_name, test_degenerate_counts_fall_out_as_no_sign_change, test_v2_refusals
Hardcode the bracket instead of reading the declaration 9 tests, including test_property_recovers_alpha_within_the_declared_tolerance
Hardcode 20 iterations test_recovers_a_known_alpha_from_exact_pareto_counts, test_belgium_rsz_counts_give_the_closed_form_alpha and 3 more
Halve the open top's score term 7 tests, including test_property_moving_mass_upward_lowers_alpha
@2 reads the first band's count for every anchor test_anchor_counts_are_the_band_rows_counts, test_v2_equals_v1_with_its_estimated_alpha_as_a_literal
@2 passes the unrounded midpoint to @1 test_v2_equals_v1_with_its_estimated_alpha_as_a_literal, test_property_v2_is_bitwise_v1_at_the_fitted_alpha
Drop the top-suffix anchor check test_v2_refusals
Drop @2 from _SUPPORT_PRODUCERS test_v2_is_registered_beside_v1
Drop @2 from the money producer sets test_composition_accepts_v2_as_the_quantile_mapped_money_producer
Drop @2 from SKELETON_KERNELS test_v2_is_registered_beside_v1
Drop @2's registration test_v2_is_registered_beside_v1
B-M1: half-up in place of half-even (remainder >= half, B/tail_estimators.py:169 at this head, :166 at a401e22f) test_six_decimal_rounding_is_half_even_at_an_exact_tie, alone

Shared hunks (file:line at head b20948d9)

Every hunk below is B's; git diff fcdb4739 b20948d9 shows these and nothing else, and so does git diff origin/main...b20948d9. F3 changed one word at B/tail_estimators.py:25 and moved no line.

  • New B/tail_estimators.py:1-375 and T/quantile_kernels.py:1-158.
  • T/compose.py:184 (SKELETON_KERNELS), :219-220 (_QUANTILE_MAPS, _MONEY_PRODUCERS), :847-850 (docstring), :946-956 (_money_producer_gaps's expected producers and message; changed lines :946-948, :952-954 and :956), :1019 (_check_single_money_normalization's kernel-to-route map), :1636 (_SUPPORT_PRODUCERS). None of these is in the code the fix round owns.
  • T/registry.py:56 (import), :535 (docstring count), :542 (registration).
  • New test packages/microcosm-build/tests/engine_free/shared/test_transport_quantile_map_v2.py:1-806.
  • packages/microcosm-build/tests/engine_free/shared/test_transport_population_kernels.py:143 and :156: the registration pin now lists transport.quantile_map@2 and says fifteen. It pins refs, not hashes.

Judgment calls

  • Delegation, not a copy. @2 calls TRANSPORT_QUANTILE_MAP.run on a dataclasses.replace of its context, with the fitted α in interpolation. So @1's own code maps, and @1's refusals keep naming transport.quantile_map@1.
  • Anchors as references. The estimator declares anchor_references, which must be the band list's top contiguous suffix ending at the open top. That ties each anchor to the fact key it counts, and the receipt lists them.
  • Bare band lists only. @2 fits one α to a bare band list. A signed or component band document is refused by name, since one α across two distributions is a choice nobody has made yet.
  • The bracket is decimal text (["1.00000095367431640625", "64"]), so a declaration states an exact rational and no binary float sits in a parameter. MAX_BISECTION_ITERATIONS is 200, because float bisection stops narrowing after about 64 halvings.
  • Counts. The anchors read TargetSpec.value for the same references _surface_bands resolves. _surface_bands itself returns shares, not counts, so the kernel calls it for validation and reads the counts from the same decoded surface.
  • Left out: the sensitivity variants (VPT's re-anchored fit and the two-point estimator), which are the NZ session's.

Implementation hashes

I computed implementation_hash() for every kernel the engine-free toy registry installs: make_composed_fixture in test_support/microcosm_build/transport_composed.py, whose build_transport_registry call installs the population, geography, target, gate, take-up, reporting and evidence kernels plus the three simulate.* rule kernels for its toy engines. That's 43 kernels at main and at fcdb4739, 44 at B and 45 at C. The earlier tables left out the three simulate.* rows; none of them moves. For 312dc4d6 I installed microcosm-emitter-interface==0.7.0, from its uv.lock, into a scratch target on PYTHONPATH, not into the venv. Each worktree's src directories came first on PYTHONPATH, and I printed microcosm.build.__path__ to confirm it.

Against the intake heads, only transport.quantile_map@2 moves (F3). Against fcdb4739, B adds @2 and C adds measure.combine@1.

gates.battery@1 moves on main, not here. Its hash digests the source closure of its gate bindings (binding_identity.describe, T/gate_kernels.py:740, hashed at :771-775), and three main merges after fcdb4739 changed modules in that closure: #1410's a6bcd1ab (B/staging.py, B/staging_storage.py, B/us_runtime/worker_identity.py), #1428's e226192a (B/source_manifest.py) and #1051's 4b1b1b10 (B/us_runtime/multispine_pool.py). I found them by diffing the closure's per-module digests at fcdb4739 and 312dc4d6 and running git log --first-parent --oneline fcdb4739..312dc4d6 on those five files.

Kernel main 312dc4d6 w4d fcdb4739 B c9fa6639 B b20948d9 C 10694ec0 C acfa6b94 Why
calibrate.ordered_adam@1 0d4577effa147ebe 0d4577effa147ebe 0d4577effa147ebe 0d4577effa147ebe 0d4577effa147ebe 0d4577effa147ebe unchanged
calibration.prepare@1 c13e76497b8d2dfb c13e76497b8d2dfb c13e76497b8d2dfb c13e76497b8d2dfb c13e76497b8d2dfb c13e76497b8d2dfb unchanged
concepts.encode@1 7815fae8ed573af6 7815fae8ed573af6 7815fae8ed573af6 7815fae8ed573af6 7815fae8ed573af6 7815fae8ed573af6 unchanged
concepts.encode_groups@1 062f88411ad605ab 062f88411ad605ab 062f88411ad605ab 062f88411ad605ab 062f88411ad605ab 062f88411ad605ab unchanged
diagnostics.calibration@1 d38f8fad075a9ace d38f8fad075a9ace d38f8fad075a9ace d38f8fad075a9ace d38f8fad075a9ace d38f8fad075a9ace unchanged
export.prepare@1 af31fa6e1690b90c af31fa6e1690b90c af31fa6e1690b90c af31fa6e1690b90c af31fa6e1690b90c af31fa6e1690b90c unchanged
export.readback@1 96be393c760ca768 96be393c760ca768 96be393c760ca768 96be393c760ca768 96be393c760ca768 96be393c760ca768 unchanged
gates.battery@1 5893a7fbe1976856 29f0505b6a0093f8 29f0505b6a0093f8 29f0505b6a0093f8 29f0505b6a0093f8 29f0505b6a0093f8 main's own move: #1410's a6bcd1ab, #1428's e226192a and #1051's 4b1b1b10 changed first-party modules in the battery's binding source closure. B and C move nothing
geography.assign_atomic@1 d60f7ec388903678 d60f7ec388903678 d60f7ec388903678 d60f7ec388903678 d60f7ec388903678 d60f7ec388903678 unchanged
geography.derive@1 d60f7ec388903678 d60f7ec388903678 d60f7ec388903678 d60f7ec388903678 d60f7ec388903678 d60f7ec388903678 unchanged
geography.gate@1 d60f7ec388903678 d60f7ec388903678 d60f7ec388903678 d60f7ec388903678 d60f7ec388903678 d60f7ec388903678 unchanged
geography.person_location@1 3efcbe3741d04497 3efcbe3741d04497 3efcbe3741d04497 3efcbe3741d04497 3efcbe3741d04497 3efcbe3741d04497 unchanged
geography.support_from_facts@1 60273ea8388098b8 60273ea8388098b8 60273ea8388098b8 60273ea8388098b8 60273ea8388098b8 60273ea8388098b8 unchanged
geography.support_import@1 d60f7ec388903678 d60f7ec388903678 d60f7ec388903678 d60f7ec388903678 d60f7ec388903678 d60f7ec388903678 unchanged
measure.combine@1 — — — — 8636ac082235b979 8636ac082235b979 new in C; unchanged by this round
money.annual_transfers@1 c8a067cd750ed2fe c8a067cd750ed2fe c8a067cd750ed2fe c8a067cd750ed2fe c8a067cd750ed2fe c8a067cd750ed2fe unchanged
money.minor_units@1 c8a067cd750ed2fe c8a067cd750ed2fe c8a067cd750ed2fe c8a067cd750ed2fe c8a067cd750ed2fe c8a067cd750ed2fe unchanged
money.normalize@1 f10fbeeaf87467b6 f10fbeeaf87467b6 f10fbeeaf87467b6 f10fbeeaf87467b6 f10fbeeaf87467b6 f10fbeeaf87467b6 unchanged
reporting.carry_income_decile@1 80335f0183c27410 80335f0183c27410 80335f0183c27410 80335f0183c27410 80335f0183c27410 80335f0183c27410 unchanged
reporting.income_decile@1 80335f0183c27410 80335f0183c27410 80335f0183c27410 80335f0183c27410 80335f0183c27410 80335f0183c27410 unchanged
reporting.receipt_group@1 80335f0183c27410 80335f0183c27410 80335f0183c27410 80335f0183c27410 80335f0183c27410 80335f0183c27410 unchanged
simulate.counterfactual@1 d54638a3d73a25f1 d54638a3d73a25f1 d54638a3d73a25f1 d54638a3d73a25f1 d54638a3d73a25f1 d54638a3d73a25f1 unchanged
simulate.rules_by_ref@1 5806634ec3d9c944 5806634ec3d9c944 5806634ec3d9c944 5806634ec3d9c944 5806634ec3d9c944 5806634ec3d9c944 unchanged
simulate.solve_zero@1 45fd2ffdc8cdcabe 45fd2ffdc8cdcabe 45fd2ffdc8cdcabe 45fd2ffdc8cdcabe 45fd2ffdc8cdcabe 45fd2ffdc8cdcabe unchanged
takeup.assign@1 53caf4ae91fc4acb 53caf4ae91fc4acb 53caf4ae91fc4acb 53caf4ae91fc4acb 53caf4ae91fc4acb 53caf4ae91fc4acb unchanged
takeup.bands@1 1ceca3ba7c7c9b92 1ceca3ba7c7c9b92 1ceca3ba7c7c9b92 1ceca3ba7c7c9b92 1ceca3ba7c7c9b92 1ceca3ba7c7c9b92 unchanged
takeup.cell_diagnostics@1 1ceca3ba7c7c9b92 1ceca3ba7c7c9b92 1ceca3ba7c7c9b92 1ceca3ba7c7c9b92 1ceca3ba7c7c9b92 1ceca3ba7c7c9b92 unchanged
takeup.compare@1 a318a19b0d147ec9 a318a19b0d147ec9 a318a19b0d147ec9 a318a19b0d147ec9 a318a19b0d147ec9 a318a19b0d147ec9 unchanged
takeup.gap@1 3e15ad5e2903c31c 3e15ad5e2903c31c 3e15ad5e2903c31c 3e15ad5e2903c31c 3e15ad5e2903c31c 3e15ad5e2903c31c unchanged
targets.compile@1 f3d372c40418665b f3d372c40418665b f3d372c40418665b f3d372c40418665b f3d372c40418665b f3d372c40418665b unchanged
targets.problem@1 aed111cadf163873 aed111cadf163873 aed111cadf163873 aed111cadf163873 aed111cadf163873 aed111cadf163873 unchanged
transport.boundary@1 fc9d10670f6a092f fc9d10670f6a092f fc9d10670f6a092f fc9d10670f6a092f fc9d10670f6a092f fc9d10670f6a092f unchanged
transport.create@1 82dc0880aefba156 82dc0880aefba156 82dc0880aefba156 82dc0880aefba156 82dc0880aefba156 82dc0880aefba156 unchanged
transport.create@2 82dc0880aefba156 82dc0880aefba156 82dc0880aefba156 82dc0880aefba156 82dc0880aefba156 82dc0880aefba156 unchanged
transport.currency@1 9c3457ddb908aa96 9c3457ddb908aa96 9c3457ddb908aa96 9c3457ddb908aa96 9c3457ddb908aa96 9c3457ddb908aa96 unchanged
transport.donor_ranges@1 62def362139a52e4 62def362139a52e4 62def362139a52e4 62def362139a52e4 62def362139a52e4 62def362139a52e4 unchanged
transport.donor_receipt@1 62def362139a52e4 62def362139a52e4 62def362139a52e4 62def362139a52e4 62def362139a52e4 62def362139a52e4 unchanged
transport.input_closure@1 23134112061495b7 23134112061495b7 23134112061495b7 23134112061495b7 23134112061495b7 23134112061495b7 unchanged
transport.package@1 e86be552287a6eb3 e86be552287a6eb3 e86be552287a6eb3 e86be552287a6eb3 e86be552287a6eb3 e86be552287a6eb3 unchanged
transport.quantile_map@1 0966a4eb8a492216 0966a4eb8a492216 0966a4eb8a492216 0966a4eb8a492216 0966a4eb8a492216 0966a4eb8a492216 unchanged
transport.quantile_map@2 — — aa0fd5e4e532264a 40b17ba56c16b60f aa0fd5e4e532264a 40b17ba56c16b60f new in B. F3's one word in B/tail_estimators.py:25 moved it from aa0fd5e4e532264a, because @2 digests tail_estimators
transport.scenario_override@1 062f88411ad605ab 062f88411ad605ab 062f88411ad605ab 062f88411ad605ab 062f88411ad605ab 062f88411ad605ab unchanged
transport.support_bounds@1 2303c60a7ec6135c 2303c60a7ec6135c 2303c60a7ec6135c 2303c60a7ec6135c 2303c60a7ec6135c 2303c60a7ec6135c unchanged
transport.unit_attributes@1 f50ec74a1bfe35bb f50ec74a1bfe35bb f50ec74a1bfe35bb f50ec74a1bfe35bb f50ec74a1bfe35bb f50ec74a1bfe35bb unchanged
transport.weights_audit@1 1f3d475013456127 1f3d475013456127 1f3d475013456127 1f3d475013456127 1f3d475013456127 1f3d475013456127 unchanged

Tests

Closed list at head b20948d9, with -n 4 and no --maxfail. All files are in packages/microcosm-build/tests/engine_free/shared/.

File Passed Skipped Failed
test_transport_quantile_map_v2.py 38 0 0
test_transport_population_kernels.py 32 0 0
test_transport_graph.py 15 0 0
test_transport_evidence_kernels.py 151 0 0
test_country_spec.py 143 0 0
Total 379 0 0

The run reported 379 passed, with no skip and no xfail; I took the per-file counts from --collect-only on each file. test_transport_graph.py's guards against a float literal or a country name in any T/*.py pass. At this head, python tools/ci_test_plan.py verify prints verification=ok, and uv run --no-sync ruff check . prints "All checks passed!".

At c9fa6639, the previous round ran a wider list of 52 files: 4124 passed and 86 skipped, each skip for a pinned NZ feed, pinned geography bytes, the Axiom dense engine or the real donor. This round didn't rerun those files.

https://claude.ai/code/session_01LwQq6FfhfP7EBWSvpa8Wv4


Generated by Claude Code

grouped_pareto_mle@1, in the new microcosm.build.tail_estimators, fits a
Pareto alpha by maximum likelihood to contiguous anchor bands and the open
top. It bisects the declared bracket for the declared iteration count,
rounds once to six decimals half-even, carries alpha as p/10**6, and
refuses a bracket without a sign change and an alpha at or below one by
name. Its receipt records the anchors, their fact references and counts,
the bracket, the iterations, l' at the final bracket, alpha and the
implied top-band mean alpha*c/(alpha - 1).

transport.quantile_map@2, in the new transport/quantile_kernels.py, reads
the anchor counts from the same compiled band surface _surface_bands
reads, then delegates to quantile_map@1's own run with the fitted alpha as
a literal. The new module keeps @1's source hash, and every registered
kernel's implementation hash is unchanged. Composition lists @2 beside @1
in SKELETON_KERNELS, the money producer sets and _SUPPORT_PRODUCERS
(quantile_mapped), and the registry installs it with the population
kernels.

Invariants: l' is strictly decreasing, so the bisection finds the one
root; exact Pareto expected counts recover alpha within 1e-6; scaling the
counts leaves alpha unchanged; moving mass upward lowers alpha; @2's map
is bitwise @1's at the fitted alpha; Belgium's RSZ counts give 3.348955.
Hypothesis tests check each, and each of 13 mutants turns a test red.
The bare-band-list support test is xfail(strict) until the w4 fix round's
E1 lands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N2tyHSTU4AFqsmoXwaTAXH
register_transport_population_kernels installs fifteen kernels now that
transport.quantile_map@2 joins them. The docstring said thirteen on the
base, while the test pinned fourteen; both now say fifteen. The pin
lists the refs, not hashes, so no implementation hash moves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N2tyHSTU4AFqsmoXwaTAXH
This merges the G8-w4 fix round's w4d head
fcdb473 into G8-r2c hunk B. Git found no
conflicts, so T/compose.py carries both sides' additions and the fix
round's text in the code it owns.

Invariant: the merge adds B's hunks to w4d and nothing else. The changed
lines of `git diff fcdb473 HEAD` equal those of `git diff d81280b
a401e22`, line for line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Since w4c, quantile_support returns the frozen QuantileSupport dataclass, which has no __iter__, so the old tuple unpack in test_the_w4_support_producer_reads_v2_params_with_an_open_top raised TypeError after the merge. The test now binds the dataclass, compares support.positive with a tuple of tuples, asserts that a bare band list feeds both signs (support.negative == support.positive), and passes both fields to quantile_bounds.

The fix round's E1 makes a bare band list's support read on both signs, so test_v2_support_bounds_both_signs_for_a_bare_band_list XPASSed strictly. This removes its xfail mark; the assertion is unchanged.

Checked: test_transport_quantile_map_v2.py passes, 37 of 37.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_property_one_band_and_the_top_match_the_closed_form returned early when the closed-form alpha fell outside 1.01 to 63, and test_property_v2_is_bitwise_v1_at_the_fitted_alpha returned early when the estimator refused. Hypothesis counted each return as a passing example. Each now calls hypothesis.assume, so Hypothesis replaces the draw and its health check would report a filter that rejects too much. test_property_the_score_changes_sign_once_on_the_bracket had the same guard, and it calls assume too.

Checked with --hypothesis-show-statistics and a temporary hypothesis.event at each final assertion (not committed). Before, 30 of 60 counted examples reached the closed-form assertion and 44 of 60 reached the differential. After, all 60 of 60 reach it in each of the three properties (109, 14 and 7 invalid draws replaced), and no health check fired.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The review's mutant B-M1 (half-up in place of half-even in _round_half_even, B/tail_estimators.py:166) left every test green, because no test rounds a value lying exactly on a tie. test_six_decimal_rounding_is_half_even_at_an_exact_tie rounds Fraction("1.0078125") to 1.007812 and Fraction("1.0078135") to 1.007814, and two values just off the tie to their nearer neighbour.

Checked: with the condition changed to remainder >= half, the new test fails and the other 37 tests in the file pass; with the code restored, all 38 pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Under the r2c bracket [1 + 2**-20, 64] the rounded alpha is at least 1.000001, so alpha_not_above_one cannot fire, and heavy-tail data whose root lies at or below one is refused as bracket_without_sign_change. The module docstring now says so. The code doesn't change; both cases still refuse, so R2C-9 holds.

The docstring is part of the module source that transport.quantile_map@2's implementation_hash digests, so that new row's hash moves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis changed the base branch from nzhub/g8w4d-nz-wiring to main November 2, 2026 23:11
The tail estimator module docstring said the receipt names the
bracket_without_sign_change reason. A refusal writes no receipt:
TailEstimatorRefusalError raises from grouped_pareto_alpha before
transport.quantile_map@2 delegates to @1, and its message and code carry
the reason. This changes that one word and nothing else; the replacement
has the same length, so no line moves.

The change moves transport.quantile_map@2's implementation hash, because
that hash digests tail_estimators. I recomputed implementation_hash() for
all 44 kernels the engine-free toy registry installs, before and after;
only transport.quantile_map@2 moved.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6FfhfP7EBWSvpa8Wv4
@MaxGhenis
MaxGhenis marked this pull request as ready for review November 3, 2026 00:02
@MaxGhenis
MaxGhenis merged commit 5238b72 into main Nov 3, 2026
10 checks passed
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Merged by the NZ hub (session local_e407a527) at b20948d with --merge. Gates: gh pr checks exit 0; MERGEABLE; not draft; no CHANGES_REQUESTED review. Reviewed head: b20948d. Reviews on #1183: 6590677926 (APPROVE_WITH_NITS; fixes landed at b20948d) and delta review 6591604580 (APPROVE_WITH_NITS; both nits are body text, fixed before this merge). Belgium's hub OK'd this head (notes/1102/from-belgium-1471-control.md in the hub's folder). The one word moved transport.quantile_map@2 to 40b17ba56c16b60f.

MaxGhenis pushed a commit that referenced this pull request Nov 3, 2026
Brings in #1459 and #1461 (ea8f409), which landed after the first merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6Jmpqc2fSZAM219btF
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants