Remove temporary ACS WIC assumption after replacement certification

changelog.d/560.fixed.md

@@ -0,0 +1 @@
+Add opt-in verified US state/year preparation and temporarily assume WIC claiming only for missing participation decisions on identified ACS people, preserving donor decisions and country-model eligibility.

changelog.d/561.fixed.md

@@ -0,0 +1 @@
+Remove the temporary assumption that every eligible ACS person claims WIC after certifying a replacement local-area dataset with complete participation inputs.

docs/engineering/runbooks/acs-local-wic-replacement.md

@@ -0,0 +1,62 @@
+# Replace the temporary ACS WIC assumption
+
+## Behaviour before removal
+
+The certified local-area release
+`populace-us-2024-buildo-acs-local-767312d60-20260923T074941Z` stores donor
+participation decisions as `would_claim_wic`, with missing decisions on ACS
+people. The current model consumes the monthly person input
+`takes_up_wic_if_eligible`.
+
+The compatibility parent preserves all valid existing decisions and sets only missing
+decisions on people whose `person_support_channel` is `acs_2024_1yr` to `True`.
+This explicitly assumes that every eligible affected ACS person claims WIC;
+it does not change eligibility. Missing donor decisions, missing identifying
+provenance, and malformed values remain errors. Source H5 files are unchanged.
+
+The existing loader applies this before calculation and year preparation,
+including baseline and reform branches. The fix does not add partitioning,
+state/year preparation APIs, new dataset defaults, or deployment behavior.
+
+## Replacement ownership
+
+[Microcosm #1154](https://github.com/PolicyEngine/microcosm/issues/1154) tracks
+the code repair, a dataset rebuild and publication by a separate dataset owner,
+certification in this repository, and removal of the exception. Merging the
+[code repair](https://github.com/PolicyEngine/microcosm/pull/1156) does not
+publish a replacement population or complete that issue.
+
+The repair generates ACS participation after completing demographic inputs and
+before combining ACS records with donors. It reuses the existing WIC generator,
+preserves donor decisions, writes complete boolean participation inputs, and
+rejects missing decisions. The dataset owner must rebuild the affected outputs
+and satisfy the existing release checks.
+
+## Remove the exception
+
+The separate stacked [removal PR #563](https://github.com/PolicyEngine/policyengine.py/pull/563)
+must remain a draft until the replacement dataset has been published and
+certified. Do not invent its revision or hash or relax its failing check.
+
+This branch removes the exception and rejects all missing legacy decisions,
+including ACS decisions. `test_acs_wic_replacement.py` intentionally fails
+while the known incomplete source remains certified. That failure is a merge
+requirement, not a test to skip. No replacement is currently certified by
+these changes.
+
+Once the replacement is available:
+
+1. Follow the [US certification runbook](build-m-us-populace-certification.md)
+   with its actual immutable regional manifest. Keep national defaults and
+   model pins unchanged unless separately approved.
+2. Verify complete current-name participation on the replacement and record
+   its revision and hashes.
+3. Add the real certification changes to the removal PR and rebase it onto
+   `main` after the temporary compatibility fix merges.
+4. Delete only the ACS missing-value exception. Keep ordinary legacy-name
+   mapping while other certified datasets require it.
+5. Run the focused mapping and country-model integration tests and validate
+   preparation against the actual replacement dataset.
+
+This fix does not rebuild or publish data, deploy services, or introduce
+environment variables or database changes.

src/policyengine/tax_benefit_models/us/legacy_inputs.py

@@ -146,8 +146,8 @@ def apply_legacy_input_renames(
     Every mapped table is checked before anything is set. Its ``{entity}_id``
     column must list the simulation's entity IDs in the simulation's order,
     and its stored values must be complete and, for a boolean live input,
-    boolean. Otherwise this raises ``ValueError`` rather than set misaligned
-    or invented values.
+    boolean. Missing decisions, including ACS decisions, and malformed
+    stored values raise ``ValueError``.
 
     Applying the mapping again sets the same values, so it is idempotent.
 
@@ -356,6 +356,15 @@ def _live_values(stored: pd.Series, variable: Any, context: str) -> np.ndarray:
         return np.asarray(stored.to_numpy(dtype=bool))
     if pd.api.types.is_numeric_dtype(stored.dtype) and stored.isin((0, 1)).all():
         return np.asarray(stored.to_numpy(), dtype=bool)
+    # Nullable legacy H5 columns may contain genuine booleans and numeric 0/1
+    # as objects. Validate each value before casting: bool("False") is True.
+    if stored.map(
+        lambda value: (
+            isinstance(value, bool | np.bool_ | int | np.integer | float | np.floating)
+            and value in (0, 1)
+        )
+    ).all():
+        return np.asarray(stored.to_numpy(), dtype=bool)
     raise ValueError(f"{context}: the stored values are not boolean.")
 
 

tests/test_acs_wic_replacement.py

@@ -0,0 +1,16 @@
+"""Keep the removal PR blocked until the actual replacement is certified."""
+
+from policyengine.provenance.manifest import get_release_manifest
+
+
+def test_certified_local_dataset_no_longer_uses_the_incomplete_wic_release():
+    reference = get_release_manifest("us").datasets["populace_us_2024_acs_local"]
+    # Do not skip this check or invent replacement identifiers. Microcosm #1154
+    # must publish a complete dataset and its real certification must be added
+    # here before the temporary ACS assumption can safely be removed.
+    assert reference.revision != (
+        "populace-us-2024-buildo-acs-local-767312d60-20260923T074941Z"
+    ), "Blocked on Microcosm #1154: certify the replacement ACS-local release."
+    assert reference.sha256 != (
+        "769756c31f3ca646d12c272511744dec04c0e68870c6946dd945fbba65b6a7ec"
+    ), "Blocked on Microcosm #1154: the incomplete dataset is still certified."

tests/test_us_legacy_inputs.py

@@ -322,6 +322,98 @@ def test_a_draw_that_is_not_a_complete_boolean_is_refused(draw, message):
     assert simulation.inputs == {}
 
 
+@pytest.mark.parametrize(
+    "draw",
+    [
+        [False, np.bool_(True), None, False],
+        [0, 1.0, np.nan, np.bool_(False)],
+        pd.array([False, True, pd.NA, False], dtype="boolean"),
+    ],
+)
+def test_missing_acs_wic_decisions_are_rejected_without_mutating_source_or_simulation(
+    draw,
+):
+    ids = [1, 2, 3, 4]
+    person = _person(
+        ids,
+        draw,
+        person_support_channel=[
+            "asec",
+            "puf_tax_detail",
+            "acs_2024_1yr",
+            "acs_2024_1yr",
+        ],
+    )
+    original = person.copy(deep=True)
+    simulation = FakeSimulation(_variables(LIVE), ids)
+
+    with pytest.raises(ValueError, match="missing values"):
+        apply_legacy_input_renames(simulation, _tables({2024: person}))
+    assert simulation.inputs == {}
+    pd.testing.assert_frame_equal(person, original)
+
+
+@pytest.mark.parametrize("channel", ["asec", "puf_tax_detail", "acs", "unknown", None])
+def test_missing_wic_decisions_outside_identified_acs_people_are_refused(channel):
+    person = _person(
+        [1, 2], [None, None], person_support_channel=["acs_2024_1yr", channel]
+    )
+    simulation = FakeSimulation(_variables(LIVE), [1, 2])
+    with pytest.raises(ValueError, match="missing values"):
+        apply_legacy_input_renames(simulation, _tables({2024: person}))
+    assert simulation.inputs == {}
+
+
+@pytest.mark.parametrize("invalid", ["False", "True", "0", 2, -1, np.inf])
+def test_an_invalid_stored_wic_decision_is_rejected_without_compatibility(invalid):
+    person = _person(
+        [1, 2], [invalid, False], person_support_channel=["asec", "acs_2024_1yr"]
+    )
+    simulation = FakeSimulation(_variables(LIVE), [1, 2])
+    with pytest.raises(ValueError, match="not boolean"):
+        apply_legacy_input_renames(simulation, _tables({2024: person}))
+    assert simulation.inputs == {}
+
+
+def test_a_later_invalid_year_prevents_earlier_acs_assignments():
+    valid = _person([1], [False], person_support_channel=["acs_2024_1yr"])
+    invalid = _person([1], [None], person_support_channel=["asec"])
+    simulation = FakeSimulation(_variables(LIVE), [1])
+    with pytest.raises(ValueError, match="2025.*missing values"):
+        apply_legacy_input_renames(simulation, _tables({2024: valid, 2025: invalid}))
+    assert simulation.inputs == {}
+
+
+def test_missing_acs_decisions_are_rejected_before_setting_branch_inputs():
+    person = _person(
+        [1, 2], [False, None], person_support_channel=["asec", "acs_2024_1yr"]
+    )
+    dataset = _multi_year_dataset({2024: person})
+    simulation = FakeSimulation(_variables(LIVE), [1, 2], dataset=dataset)
+    baseline = FakeSimulation(_variables(LIVE), [1, 2], dataset=dataset)
+    simulation.branches["baseline"] = baseline
+    with pytest.raises(ValueError, match="missing values"):
+        apply_legacy_input_renames_to_microsimulation(simulation)
+    assert simulation.inputs == baseline.inputs == {}
+
+
+def test_current_name_wic_decisions_load_without_legacy_mapping():
+    person = _person(
+        [1], [None], person_support_channel=["acs_2024_1yr"], **{LIVE: [False]}
+    )
+    simulation = FakeSimulation(_variables(LIVE), [1])
+    assert apply_legacy_input_renames(simulation, _tables({2024: person})) == {}
+    assert simulation.inputs == {}
+
+
+def test_boolean_values_in_an_object_column_are_preserved():
+    person = _person([1, 2], pd.Series([False, np.bool_(True)], dtype=object))
+    person[LEGACY] = person[LEGACY].astype(object)
+    simulation = FakeSimulation(_variables(LIVE), [1, 2])
+    apply_legacy_input_renames(simulation, _tables({2024: person}))
+    assert simulation.inputs[(LIVE, "2024-01")].tolist() == [False, True]
+
+
 def test_a_non_boolean_live_input_keeps_the_stored_values():
     variables = {LIVE: _variable(value_type=float)}
     simulation = FakeSimulation(variables, [1, 2])

tests/test_us_legacy_inputs_integration.py

@@ -177,6 +177,27 @@ def test_run_keeps_a_stored_false_draw(mapped_run):
     assert mapped_run.release_bundle["legacy_input_renames"] == RENAME
 
 
+def test_native_wic_decisions_preserve_eligibility_without_compatibility(
+    tmp_path,
+):
+    frames = _frames()
+    del frames["person"][LEGACY]
+    frames["person"][LIVE] = [True, False, True]
+    frames["person"]["person_support_channel"] = [
+        "acs_2024_1yr",
+        "asec",
+        "acs_2024_1yr",
+    ]
+    person = _person_outputs(_run(_in_memory_dataset(tmp_path, frames)))
+
+    assert person[LIVE].tolist() == [True, False, True]
+    assert not person["is_wic_eligible"].iloc[0]
+    assert person["wic"].iloc[0] == 0
+    assert person["is_wic_eligible"].iloc[INFANT]
+    assert person["wic"].iloc[INFANT] == 0
+    assert person["wic"].iloc[TODDLER] > 0
+
+
 def test_run_over_a_core_h5_keeps_a_stored_false_draw(tmp_path):
     """Path 1 over a policyengine-core ``variable/period`` file.