Repository navigation
Round 14: confirmatory multi-state analyses (v0.13.0) - #11
Merged
Merged
Conversation
…rt rules census.workflows.import_into_cache holds the CLI 'cache import' logic (validate every file first, atomic write, synthetic into the default cache only with an explicit allow flag); the CLI and the new cache_import tool (also 'tool cache_import') both call it. Routes table in docs/reference_workflow.md updated (PLAN_ROUND_14 D2).
…das4Warning) The only pandas deprecation the offline suite triggers from census/ code (run with FutureWarning, DeprecationWarning and Pandas4Warning shown). The caller's frame is still relabelled in place; the labels come from renaming a zero-row slice, so no data is copied on pandas 2.3 either. Test: same object, same labels as rename(), no warning (PLAN_ROUND_14 D1).
Adds state 26 to Battery.states and re-records tests/fixtures/validation live (--live --record into a fresh directory, so fixtures keyed by the old five-state requests are dropped rather than orphaned). The fixture set grows from 1.51 MB to 1.76 MB (du 2.0M -> 2.2M), under the D14-3 limit of ~5 MB. All ten checks pass live and offline with no tolerance changes. Counts that depend on the state list are updated: the reported F5 2023 income share within 2% (0.8414 -> 0.8589) in the offline test, and the KNOWN_FINDINGS F2 owner-occupied county MOE misses (279 of 1,164 county-years) and F5 income shares.
Live run with the standard command (python scripts/validate_against_census.py --live): all ten checks pass.
… R2/F3) scripts/pums_memory_benchmark.py generates seeded PUMS-shaped person files (100k, 500k, 1.9M rows; no downloads) and measures read, weight matrix and tabulation peak RSS in fresh subprocesses at three chunk sizes, checking that estimates, SEs and the frame are bit-identical across chunk sizes. docs/audits/pums_memory.md: results, about 2.8 KB per record, 5.3 GiB at 1.9M rows, what fits in 16 and 128 GiB, recommendation (no streaming rewrite). Unit test runs the --small (2,000-row) mode.
…ding Found by the Round 14 memory benchmark: _group_sum copied the n x 81 matrix twice (mat[ok] even when every row counts, and pandas 3's DataFrame constructor copying by default); weighted_median gathered the rows twice; tabulate copied W[rows] per median group; read_pums_csv kept the parsed chunks alive through conversion. 1.9M rows: tabulate peak 6,406 -> 5,406 MiB, read 2,261 -> 1,969 MiB. Results are bit-identical (tests compare against the v0.12.0 implementations; the benchmark compares every estimate and SE before and after).
… 14 A1-A3)
- geography.states (list) as an alternative to geography.state; per-state
extraction stacked into one frame; validation errors for both/neither,
duplicates, codes with states, national levels. Single-state manifests and
hashes are unchanged.
- adjustments: [{type: rpp, variables, rpp_year, areas}] applied after
derivations, before exclusions, via the shared place_adjust core (refactored
out of place_adjust_frame; one implementation). Scales estimates, SEs and
supplied replicates. RPP year (default: latest in the table, recorded),
areas, boundary source and RPP table SHA-256 (or custom path/hash) recorded
in run.json, bundle.json and methods.md. A year mismatch is a note; a
failed adjustment is a blocking adjustment_failed deviation.
- Derivations are evaluated on supplied replicates (by manifest name or ACS
code) with exact replicate SEs.
- place_adjust_frame gains an optional rpp_year.
…se study update (Round 14 A4) - examples/manifests/mi_ny_income_education.json: pre-registered H1 (positive pct_bachelors slope on RPP-adjusted median household income, counties of 26 and 36, ACS 2020-2024, explicit state term). - tests/test_manifest_multistate.py: validation, synthetic two-state runs (stacking, state RPP, default year, blocking failures, derivations on replicates, replicate scaling), offline CLI + MCP route on frozen real API responses (64 KB) and a frozen CBSA membership, live test, --record. - docs/case_study_mi_ny.md: confirmatory manifest route first (section 1), exploratory route kept as sensitivity analysis; live run reproduces Round 13 exactly (811.00, 90% CI [684.29, 937.71]). - Case-study live test checks the confirmatory run; preregistered recipe documents states and adjustments.
… MCP cache_import
… export relabels columns without a zero-row slice Round 14 review findings 8 and 10. Group rows are contiguous slices of one stable argsort of the group ids, in original row order, so medians are bit-identical to v0.12.0. format_dataframe assigns [descriptions.get(c, c) for c in df.columns] in place.
…adjustments 1. Derivations on replicates run inside error handling: a data error (ValueError from _units/_rep_cube, KeyError, ...) is a blocking replicates_incomplete deviation; run_manifest no longer raises. The MCP run_manifest wrapper also maps KeyError to an invalid-input result. 2. Inputs given by ACS code are matched after _default_name normalization (B19013_001E, DP04_0046PE). 3. derived_on_replicates lists only derivations that received replicate SEs; derived_on_replicates_rows gives rows_replicate of rows in the analysed frame, and methods.md says "replicate SE for k of n units; approximate SE elsewhere" when coverage is partial. 4. geography.states (and state) require exactly 2-digit FIPS strings. 5. geography.states is rejected for zcta (not state-prefixed, crosses states). 6. rpp_year is required by validation; create_manifest fills in the latest RPP year before hashing, rewrites the manifest file and says so (notes, filled). Example manifests already declare it; hashes unchanged. 7. Exclusions run before adjustments, so a pre-registered geoid_exclude can remove a unit without an RPP area; a remaining one is adjustment_failed. Methods text lists exclusions before the price adjustment. MI/NY frozen result unchanged (811.00, [684.29, 937.71]). 9. default_rpp_areas(level) owns the default areas rule; place_adjust_frame no longer repeats place_adjust's areas check. Tests: tests/test_review_fixes_r14.py; test_manifest_multistate updated for the required rpp_year. Docs: recipes/preregistered.md, case_study_mi_ny.md.
…lls rpp_year; write <name>.completed.json
Full --gate --jobs 0: PASS, 17 min; coverage table unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See PLAN_ROUND_14.md section 9 and the CHANGELOG 0.13.0 entry.
Local: full suite 1021 passed, 34 skipped; full gate PASS (17 min).