Repository navigation
Add Stats NZ household net worth for the year ended June 2024 - #329
Conversation
Register ird-wage-salary-distribution-2026, following the merged I1 package. Preserve the complete 198,542-byte staged workbook unchanged and pin the hub receipt: fetched 2026-10-10T08:31:07Z, SHA-256 d0526b2092f9fa2bec44e11984c0ef17fda788a742c1c7054d416f6a0cd9e750. Use the stable publisher URL, CC BY 4.0 licence, and Chronicle build_r2_key. R2 upload/remote checksum verification remain for the hub before merge. Emit 510 administrative person observations for the ending 2026 tax year: 472 band facts (235 bands plus separate total, count and income), 19 decile facts and 19 top-percentile facts. Preserve all 20,631 worksheet cells and verbatim explanatory drawing notes and population footnotes. Historical and revised columns remain source cells only. Rename checklist row 62 from ird/wage_salary_distribution_2025 to ird/wage_salary_distribution_2026; tick artifact/package boxes and leave R2 unchecked. Update the stale I1 sentence and add the changelog and source evidence note. Invariants and checks: - Every numeric fact reads one publisher cell. Audit all 510 unique selected coordinates against the unchanged workbook; direct-cell lineage/unit-scale regression and build-suite acceptance pass. All facts are administrative observations for persons, without PolicyEngine-computed values. - Separately published totals remain independent BB242/BC242 facts, never sums of bands. Exact-cell and no-range-selector tests pass; total-kind YAML mutation fails its regression before the accepted YAML passes. - Upper boundaries are NZD cut-points with aggregation quantile, never shares. Exact BB255/BB278 anchors and all boundary aggregation tests pass; changing quantile to sum fails its regression. - Decile 10 BB264 and percentile 100 BB279 have no upper boundary. Both raw blanks remain source cells, both boundary facts stay absent, and their income facts remain present. Adding the blank percentile-100 boundary to scratch YAML fails the regression; no value is imputed. - TY2026 means 1 April 2025 to 31 March 2026, the ending year. Header guards, coverage-date tests and build at bundle year 2023 retain tax_year:2026; changing the package period to 2025 fails its regression. - $M cells retain exact cached publisher precision with scale 1000000 and no rounding; boundaries have scale 1. All 510 values equal the core's existing normalization of their cached source cells. Scale, rounding and divisor mutations each fail the regression. Source XML numeric lexemes remain in unchanged workbook bytes; no means, shares or uprated values are derived. - Open bands retain literal publisher labels and categorical equality constraints; no endpoints are closed and no I1 bands are merged. Changing the top band label in scratch YAML fails the regression. - Every dimension and value carries the explicit authored label. Complete consumer-contract/label tests pass; removing a dimension declaration fails. - The drawing sentence naming revisions to 2022–2025 and row-2 R flags naming 2023–2025 are both retained verbatim without reconciliation. Raw revision and explanatory-note tests pass; only New TY2026 facts are selected. - Every required publisher guard has a regression: 273 independent coordinates cover all row/measure headers, years, New flag and population footnotes. Guard removal yields 274 expected failures; all 273 guards individually reject mutated source cells in the final green run. Validation: validate-package CLI PASS (5 sets, 274 rows, 6 measures, 510 records, 5 regions, 0 errors/warnings). Targeted test file: 295 passed, 12 dependency warnings; 10 scratch YAML red cases: 283 expected failures. Build-suite PASS, all 12 acceptance gates true, 510 facts/records, 20631 cells, 510 constraints, lineage coverage 1.0. Axiom validation was not run; acceptance records the standard concept_alignment_validation_skipped warning. Ruff check/format and git diff --check PASS. Local ledger-source-fidelity and ledger-boundary reviews PASS. Full bundle/full-suite tests were not run; CI owns them. Measured bundle delta (single-source build_bundle, year=2023): - facts: +510; by_source ird: +510 - geographies: +0; new geography keys: []; by_geography country:NZ: +510 - publisher sources: +0 (ird already present) - packages: +1 (ird-wage-salary-distribution-2026) - tables: +1; by_source_table ird:Wage and salary distributions for individuals 2026: +510 - warnings: +0; errors: +0; skipped sources: +0 - period count: +0 (tax_year:2026 already present); by_period tax_year:2026: +510 - entity count: +0 (person already present); by_entity person: +510 - aggregate duplicate keys: +0; semantic duplicate keys: +0 Single-source standalone counts: 510 facts, 1 geography country:NZ, 1 source ird, 1 package, 1 table, 1 period tax_year:2026, 1 entity person, 0 warnings/duplicates. Pins use baseline 36c0358 plus this measured delta: 1001258 facts, 319 packages, 314 tables, ird 4362, person 495562, country:NZ 4581; geography 20547, source 56, period 511, entity 12, warning 117 counts unchanged. Hub can re-pin if parallel packages land first. The 46,758,982-byte bundle output and pytest temp directory were deleted; remaining scratch contains only small scripts and red evidence. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Preserve the unchanged September 2025 Stats NZ XLSX and emit 1,413 HES observations across Table1.01, Tables2.01–2.04, Tables3.01–3.04, Table5.01 and Tables7.01–7.04. Added for microcosm#592; outside #176's 15. Artifact: 386,546 bytes, fetched 2026-10-10T08:31:08Z, SHA-256 2fb942322e5131824a15007e7b0742c4f8220822c7e8f9adee83bae154913f9e. Receipt URL and CC BY 4.0 / Stats NZ attribution are recorded in manifest. R2 key uses chronicle.artifacts.build_r2_key and requires hub upload: raw/nz/stats_nz/household_net_worth_2024/2024/2fb942322e5131824a15007e7b0742c4f8220822c7e8f9adee83bae154913f9e/household-net-worth-statistics-year-ended-june-2024.xlsx Invariant checks: - Every value is publisher-asserted: all 1,413 facts equal a distinct cached 2024 source cell, with SHA/size/URL/R2 provenance and explicit label/note lineage. Whole workbook bytes (39 sheets) are unchanged; 31,534 source cells preserve Contents and all fourteen scoped worksheet used ranges. Tests: artifact_receipt_storage_and_licence_are_pinned, preserves_contents_and_complete_scoped_sheets, every_fact_is_one_unscaled_published_estimate, exact_publisher_cells. - Medians are cut-points and quintiles are publisher groupings/counts: median aggregation, exact literal quintile headings, published count cells and every scoped asset/group cell are checked. No percentile bounds or counts are derived. Tests: quintiles_and_age_groups_remain_publisher_groupings, frozen_publisher_boundaries; changing median to sum is rejected in YAML. - Table1.01 asset means/medians are among non-zero holders: literal A5 population note is a fact dimension, concept evidence and guarded lineage. All-household Table5.01 C37=525 remains separate from holders-only Table1.01 C36=529. Tables2/3 .01/.02 and Tables7 .01/.02 retain their publisher-assumed zeros; count sheets retain holders notes. Total sheets without a population note get no inferred note. Tests: population_notes_distinguish_holders_and_all_households, every_publisher_note_is_verbatim_in_fact_lineage; population YAML mutant and every population-note source mutant reject. - Separately published totals remain separate single-cell observations: no end_address, divisor, range, sum of rows or reconciliation. Tests: every_fact_is_one_unscaled_published_estimate, frozen_publisher_boundaries; mutated range/divisor YAML rejects. - Stats NZ rounding and units are preserved: value_scale=1, no round_to; values remain NZD_thousands, household_thousands or person_thousands. All 1,413 numeric cells compare exactly; scale/rounding YAML mutants reject. - Reference period is July 2023–June 2024: fiscal_year:2023 uses opening year, survey_reference coverage 2023-07-01..2024-06-30, artifact_year fixed 2024. Other requested build years yield identical facts; period YAML mutant rejects. Tests: requested_year_cannot_change_reference_period and frozen_publisher_boundaries. No uprating, aging or period alignment. - Every dimension/value is labelled: bundle dimension-label report has zero errors/warnings; source spelling and footnote markers preserved. Chronicle display labels trim outer whitespace; source cells/layout remain verbatim. Tests: dimensions_values_and_consumer_contract_have_labels; missing-label YAML mutant rejects. Package-specific label dimensions avoid collisions. - Suppression stays missing: seven S estimate cells and literal definitions stay raw text, with no facts, substitutions or implied zeros. All seven source mutations reject. Percentage changes and their errors are omitted; RSEs remain source cells because the active AggregateFact/spec/consumer contract has no sampling-error field (legacy SourceFact.margin_of_error does not apply). Tests: suppression_is_raw_evidence_and_never_an_estimate. - Household/person quantities come only from their published tables; no conversions, cash/liquid-asset computation, imputation, donor handling, reconciliation, target activation or model-measurement bindings. Verified by complete one-cell coverage, frozen YAML and independent ledger-boundary judge. All facts assertion=observation, provenance_class=survey_aggregate, survey_instrument=Household Economic Survey. - Determinism: source-cell, fact and consumer-row serialized SHA-256 outputs match in fresh interpreters under PYTHONHASHSEED=1 and 2. Validation: - validate-package: PASS, 82 record sets, 1,413 records, zero errors/warnings. - build-suite: PASS, all agent-acceptance checks true; source-cell, facts, consumer-contract and raw-facts boundary gates pass; lineage coverage 1.0. Final single-source suite: 31,534 cells, 1,413 facts, 4,925 constraints, 82 source regions, 3 source-label concept alignments, 1 raw artifact. Optional Axiom validation was skipped (normal nested concept/acceptance warning); source-only mappings remain source_label. - First red: 752 independently enumerated mutated publisher guard coordinates rejected; 8 mutated YAML cases rejected (scale, divisor, range, rounding, period, labels, population, median). Green regressions follow each guard. - pytest tests/test_chronicle_nz_household_net_worth.py -p no:cacheprovider --basetemp=.hub-scratch/pytest -o tmp_path_retention_policy=failed -q: 793 passed in 75.11s; 12 dependency deprecation warnings. Basetemp deleted. Includes 16 exact-cell anchors and all 752 guard regressions. - Ruff check + format check of changed Python files: PASS. git diff --check: PASS. Full-bundle test node and all full bundles left for CI. - Independent ledger-source-fidelity: PASS; ledger-boundary: PASS. Final follow-up confirms no unresolved findings. Exact measured single-source bundle contribution (build year 2023): source alias=stats-nz-household-net-worth-2024; facts=1413; geographies=1; geography keys/by_geography={country:NZ:1413}; sources=1 (stats_nz); packages=1; tables=1; warnings=0; errors=0; periods=1; by_period={fiscal_year:2023:1413}; entities=2; by_entity={household:940,person:473}; by_source={stats_nz:1413}. by_source_table={stats_nz:Household net worth statistics: Year ended June 2024, Tables 1.01, 2.01–2.04, 3.01–3.04, 5.01 and 7.01–7.04:1413} Aggregate/semantic duplicate keys=0; skipped sources=0. Bundle deleted. Default-bundle net delta relative to assigned base 36c0358: +1413 facts; +0 geography keys (country:NZ already exists), +0 publisher names (stats_nz already exists), +1 package, +1 source table, +0 warnings, +0 period keys (fiscal_year:2023 already exists), +0 entity keys (household and person exist), +0 duplicate keys, +0 skipped sources. by_geography increments={country:NZ:+1413}; by_period increments={fiscal_year:2023:+1413}; by_entity increments={household:+940,person:+473}; by_source increments={stats_nz:+1413}; source-table increment is above. Pins: facts1000748→1002161; packages318→319; tables313→314; stats_nz85→1498; country:NZ4071→5484; household153696→154636; person495052→495525; geographies20547, periods511, entities12, sources56, warnings117 unchanged. Hub can re-pin these additive counts after another NZ package lands. No commits, pushes, PR edits or network calls performed; existing staged hub sources preserved. R2 upload/remote verification remains with the hub. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
The hub published the manifest-declared objects with chronicle publish-raw and verified each by streaming it back from ledger-raw (SHA-256 matches) on 2026-10-10. Tick the checklist's raw/nz column and replace the lane's upload-pending sentences. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The hub published the manifest-declared objects with chronicle publish-raw and verified each by streaming it back from ledger-raw (SHA-256 matches) on 2026-10-10. Tick the checklist's raw/nz column and replace the lane's upload-pending sentences. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r1 (nit 2): the fragment still said the upload was pending. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dditively Stack chronicle#328 on #330 so both land without a second two-hour bundle run. tests/test_chronicle_bundle.py pins are main 36c0358 plus both measured single-source deltas: facts 1000748 + 510 (IRD I2) + 112 (Treasury) = 1001370; packages 318 + 2 = 320; tables 313 + 2 = 315; country:NZ 4071 + 622 = 4693; ird 4362, treasury 139, person 495562, government 3562. Git merged the identical 318->319 and 313->314 edits silently; corrected to 320 and 315. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…erification Review r1 (finding 2): the row ticked raw/nz verified but still said the key awaited upload. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Declare an independent 510-observation map from the committed workbook's literal row ranges and fixed BB/BC columns. Check exact record IDs, sheet, cell address, unit, scale and aggregation, plus emitted fact metadata. Use the independent coordinates and scales for every value/lineage check. Add four in-memory row-override regressions: count to income, boundary to income, total to band, and wrong-year percentile column. All four mutants fail the new positive test in its body, with no setup errors. Tracked YAML and all 510 fact IDs, value types and exact values remain unchanged. Validation: 300 targeted tests pass; Ruff check and format pass; git diff --check passes. Only the wage/salary test file is changed. No full suite or full-bundle node ran. Leave changes uncommitted for the NZ hub. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
End Table1.01 Shares and other equity at its published row 23 subtotal. Attach rows 24–27 to the publisher-backed Assets parent and A9 guard, preserving all 1,413 numeric facts. Audit every section, quintile and age group across all 14 scoped sheets; document exact publisher boundaries. Add independent exact-membership tests for fact dimensions/constraints and exported consumer universe_constraints. Existing independent XLSX coordinate coverage remains intact. An in-memory original-defect mutant kills both new tests; a constraint-only mutant also kills the facts test. Coordinate mutants fail the existing completeness test, with 6 total call-phase assertion failures. Validation: 795 targeted tests pass; validate-package passes with no errors/warnings; Ruff check/format and git diff --check pass. Before/after record/value SHA256: 2fdaaf83226b148adb5922616e158c5e7b77ab952a5b7dd9fbaa77811a94cf0d. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
…tively Stack chronicle#329 on #328 (which contains main at 3dce46a, including the Budget 2026 Estimates package) so both land without another two-hour bundle run each. tests/test_chronicle_bundle.py pins are that branch's plus this package's measured single-source delta: facts 1001370 + 1413 = 1002783; packages 321; tables 316; stats_nz 1498; country:NZ 4693 + 1413 = 6106; household 154636; person 495562 + 473 = 496035; fiscal_year:2023 + 1413. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Merged by the NZ hub (session local_e407a527) at d287ab5 with --merge. Gates: |
Added for PolicyEngine/microcosm#592, outside #176's original 15 packages. This is B6 part 2 of the NZ v0 plan. Part 1 was the MBIE rents package, merged in #320.
What this adds
packages/stats_nz/household_net_worth_2024holds 1,413 Household Economic Survey observations from Stats NZ's "Household net worth statistics: Year ended June 2024" (released 26 September 2025). The reference period is July 2023 to June 2024,fiscal_year:2023under the opening-year convention.It covers 14 tables:
Across these tables, the package retains median, mean, total and count estimates, in $000s or in thousands of households or people. Tables 2.01–2.04, 3.01–3.04 and 7.01–7.04 each publish one statistic per sheet.
What it leaves out:
Sstay as raw text. They are never treated as facts or zeros.Source
https://www.stats.govt.nz/assets/Uploads/Household-net-worth-statistics/Household-net-worth-statistics-Year-ended-June-2024/Download-data/household-net-worth-statistics-year-ended-june-2024.xlsx.2fb94232…54913f9e, fetched 2026-10-10.raw/nz/stats_nz/household_net_worth_2024/…. The hub verified it by streaming it back (SHA-256 matches).Invariants and how they're checked
These are scoped to this package's 1,413 facts:
One cell per fact. Every fact equals one cached publisher cell, with no ranges, divisors, rounding or rescaling. The complete workbook bytes are preserved. The Contents sheet plus the 14 scoped sheets' used ranges come to 31,534 cells.
Holders versus all households. Table 1.01's asset-type means and medians cover only households holding that asset. The table's own population note is carried in each fact's dimensions. Two figures make the difference concrete:
Both are kept as separate facts.
Published groupings only. Medians are cut-points, and quintiles are the publisher's groupings with their published counts. Nothing is derived, summed or converted between households and people.
Section boundaries follow the workbook. Each
published_sectionvalue covers exactly the rows under the publisher's bold header, up to its bold subtotal. The test declares each section's member rows from the workbook's structure, not from the YAML, and checks both the facts and the exported consumer constraints.Fixed reference period. Requesting another build year yields identical facts.
Labels. Every dimension and value is labelled (Dimension labels are only checked for the UK suite: 68 of 79 other packages fail the consumer_fact.v2 rules #265).
Regression tests reject 752 mutated publisher guard cells and 8 YAML mutations: scale, divisor, range, rounding, period, labels, population and median. Fact output is identical under
PYTHONHASHSEED=1and2.Tests
tests/test_chronicle_nz_household_net_worth.py: 795 passed. The fix lane ran it, and the hub re-ran it on the final tree.Bundle pins
3dce46a6. Until Add IRD wage and salary distribution for TY2026 (chronicle#210 I2) #328 merges, this PR's diff also shows Add IRD wage and salary distribution for TY2026 (chronicle#210 I2) #328's files. The pins are that branch's plus this delta: facts 1,001,370 + 1,413 = 1,002,783; packages 321; tables 316; country:NZ 6,106.Built by a GPT-6.1 Sol lane from the NZ hub's brief. The hub verified the test runs, artifact hash and R2 object.
Chronicle governance
ledger-source-ingestor. As in the other package PRs, the per-package files outside itsallowed_pathsare the alias entry inchronicle/source_package.py,db/data/**,docs/pe-nz-source-checklist.md, the changelog fragment and the bundle pins. There are no logic changes outsidepackages/.validate-packagepasses, with 0 errors and 0 warnings;ledger-source-fidelity: round 1, at007d2451, recorded FAIL on the section leak. Round 2, atd287ab50, found the leak fixed: all 1,413 values equal their publisher cells, and an independent workbook-derived audit found zero grouping mismatches across the 14 sheets. Round 2 did not restate a named verdict, so PASS is the hub's reading of it.ledger-boundary: round 1 recorded PASS. Round 2 found no arithmetic, period alignment or other boundary change.ledger-contract: no schema or consumer-contract change.Review
An independent GPT-6.1 Sol review at
007d2451returned REQUEST_CHANGES on one defect:df575669). Rows 24–27 now sit under the publisher's "Assets" header (A9), and their lineage guard points at that header. All 1,413 values are unchanged: the sorted (record id, value) pairs hash identically before and after.The review's wording nits are fixed above: the cell count and the per-sheet measures. Its stale "R2 pending" line in the checklist is fixed in
335dbd69.A second review at
d287ab50returned APPROVE_WITH_NITS with no blocking defect:🤖 Generated with Claude Code