[hua7450]

What this fixes

Core 3.33.13 broke a policyengine-us test on main within minutes of its release, and nothing in Core's CI could have shown it first.

Time (UTC, 2026-10-27)	Event
14:48	PolicyEngine/policyengine-us#10400 adds core/test_social_security_input_state_machine.py, whose backend runs Core's installed SimulationBuilder, Simulation, holder and dumper source in namespaces it builds.
16:42	#624 merges: build_default_simulation now calls operator.index, and restore_simulation calls a new _restore_count.
16:43	3.33.13 is on PyPI.
from 16:45	Every policyengine-us PR and release run fails that test with NameError: name 'operator' is not defined; policyengine-us CI installs the newest Core on every run. PolicyEngine/policyengine-us#10504 gives the backend both names.

#624 itself is correct. Core already checks itself against policyengine-us, but SmokeTestForMultipleVersions passed on #624 because it doesn't test enough to catch this:

• It installs a pinned release, policyengine-us==2.42.13, which predates the test.
• It runs Core's smoke tests, which check that the US and UK systems load, not any policyengine-us test.

Nothing checks again after the PR either: Publish in push.yaml has no needs.

What changes

A new PR job, PolicyEngineUSMain ("Test policyengine-us main against this Core"), sits beside the smoke job:

1. It checks out policyengine-us main and installs it the way its own CI does: its lock, with the newest Core release.
2. It runs the policyengine-us files in .github/requirements/policyengine-us-main-tests.txt with that release, with a JUnit report.
3. It installs this checkout's Core over the release, and checks that direct_url.json points at the checkout, because the two versions can be equal before a release bump.
4. It runs the same files again, with the same Hypothesis seed, so both runs draw the same examples.
5. .github/country_main_tests.py compare fails on each test that fails with this Core but did not fail with the release. A test that fails with both is policyengine-us's own failure, so it is only a warning.

That last rule addresses why #546 pinned the smoke bundle: country-side failures kept turning Core's CI red. policyengine-us main is sometimes red for its own reasons. Release run 40544014984, the first on 3.33.13, failed this Core test and also core/test_mt_state_agi_invariants.py, a policyengine-us failure fixed by PolicyEngine/policyengine-us#10496. This job would blame Core only for the first.

The list starts with 13 fast files that drive Core internals: simulations, holders, data storage, tools, populations, parameters and reforms. It includes the state machine test. Together they took about 95 seconds in policyengine-us release run 40544965026, against 96 minutes for its whole tests/core. A listed file that policyengine-us renames or removes is skipped with a warning, so the job cannot fail for that reason.

Tests

• tests/core/test_country_main_tests.py, 10 tests, passed locally. They cover:

  • the comparison: breaks, failures with both, skips and new collection errors;
  • missing reports;
  • file selection;
  • the list's format;
  • the order of the job's steps.

• ruff format --check . passes.

• A local replay of the job's steps against today's policyengine-us main (macOS; each pytest pass took about 45 seconds), with the 13 files and 470 tests:

  Release	Candidate	Release run	Candidate run	compare
  3.33.12	Restore simulations with no persons or empty groups, and refuse non-integer default counts #624's merge commit (bb7660e)	470 passed	1 failed	exit 1: "This Core breaks ...TestSocialSecurityInputStateMachine::runTest"
  3.33.13	current master	1 failed	1 failed	exit 0, with a warning that the test fails with the release too

  So the job would have failed Restore simulations with no persons or empty groups, and refuse non-integer default counts #624's PR, and it doesn't blame later PRs for the test until Give the Social Security state backend the names Core 3.33.13 needs policyengine-us#10504 lands.

• Not run locally: actionlint, which this PR's Lint workflows job runs. The job itself runs on this PR against today's policyengine-us main.

Open questions for review:

• Should the job block merging, or only report until it has proved stable?
• Should Publish also wait for it on the release commit?
• Should the list move to policyengine-us, so the people writing those tests maintain it?