Skip to content

v0.15.1: B01003 total fallback; Michigan 2015-2019 population covariance - #14

Merged
cafeTechne merged 3 commits into
defaultBranchfrom
fix/vre-total-fallback
Nov 9, 2026
Merged

cafeTechne merged 3 commits into
defaultBranchfrom
fix/vre-total-fallback

Conversation

@cafeTechne

Copy link
Copy Markdown
Owner

Patch release.

  • census.gov firewall rejects the 2015-2019 Michigan tract B01003 VRE file for every client. B01001 line 1 is the same total (verified on NY 2019: 4,918 tracts, 0 differing cells; re-checked by a live test). fetch_vre now falls back for that verified pair only, never silently, with full provenance (support ID, URLs, SHA-256, verification) flowing into manifests.
  • Covariance study completed for Michigan 2015-2019 population: matches 2020-2024 and NY; no conclusion changes.

Local: 1194 passed (3.14), 1190 passed (3.10).

…ile is rejected or 404

census.gov's firewall rejects the 2015-2019 Michigan tract B01003 VRE file for
every client (support ID 13979921158196382640). B01001 line 1 (Total:) is the
same total population: on the 2015-2019 New York tract files (4,918 tracts)
ESTIMATE, MOE and all 80 replicates are identical (0 differing cells).

fetch_vre now, for this verified pair only (VRE_TOTAL_FALLBACKS), fetches the
B01001 file of the same release/geography/state when the B01003 file is
rejected or 404, and returns its line 1 relabelled as B01003_001 (title
Total). Never silent: a warning, attrs['substitution'] (requested URL, reason,
support ID / HTTP status, source table and URL, SHA-256 of the file read,
verification statement) and attrs['notes']; source_url/sha256 name the file
actually read. Manifest replicate records and methods text, and the
covariance study's file list, carry the substitution. Other B01003 lines
(new lines= argument), other tables, other failures, allow_substitute=False,
or an unavailable B01001 raise the original error.

Tests: offline rejection/404/both-unavailable/non-total/identical-pair cases;
live (CENSUS_LIVE_TESTS=1) re-check of the line-1 identity on NY 2019 tracts.
…B01001 total

Rerun of scripts/acs_error_covariance.py (--tract-years 2024 2019). The
firewall-blocked B01003_26 2019 file now comes from B01001_26 line 1 (fetch_vre
substitute); the report lists it under 'Substituted files' with the B01001
SHA-256 and the verification basis, and no file is unavailable any more.

Michigan 2019 population: order 1 within county -0.075 [-0.103, -0.046]
(2024: -0.081), across county -0.005 [-0.014, 0.004], aggregation ratio
0.000, split-half pair reliability 0.64 (2024: 0.67); same pattern as New
York 2019 (-0.042, 0.000, 0.60). Conclusions unchanged; Findings 6 and the
C2 limits updated. README notes the substitute.
@cafeTechne
cafeTechne merged commit faae550 into defaultBranch Nov 9, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant