[vahid-ahmadi]

Summary

Split the Python test job over five shards instead of four. The timeout stays at 60 minutes.

The shard count is defined in one place: the shard matrix in .github/workflows/test-suite.yaml. --shard takes ${{ matrix.shard }}/${{ strategy.job-total }}, so N follows the matrix. Test / Result needs the whole Python job and reads every outcomes-*.jsonl, so it needs no list of shards. The release workflow (versioning.yaml) calls the same reusable workflow. Comments in pr_code_changes.yaml and the Makefile, and docs/engineering/skills/testing.md, now say five shards and K/5.

Evidence

• test-suite.yaml runs the Python tests with timeout-minutes: 60 over 4 shards, chosen by a SHA-256 hash of each test's node id (--shard in policyengine_uk/tests/conftest.py), with -n 4 --dist worksteal.
• On 20 and 21 October shard 1/4 took 50 to 52 minutes on unrelated pull requests: Pin benefit cap exemption rules with regression tests and legal references (#1818) #1820 51m25s, Fix the annual allowance on survey data, the UC benefit cap earnings threshold, relevant UK earnings and Scottish water uprating #2536 52m34s, Add PIP points-based assessment scaffold (#1069) #1661 50m14s. The workflow comment records 15 to 20 minutes per shard on 10 October.
• Compute gross council tax from LA Band D amounts and statutory band ratios #1855 adds 382 local authority Band D parameters (the tree goes from 3,358 to 4,108 parameters, and from 53.8k to 65.0k values). Every reform Simulation clones the tree, so its tests run about 15 to 20% slower. Its shard 1/4 was cancelled at 60 minutes twice. In run 39568525017 the last test passed at 59m13s and the coverage upload was cut off; the other three shards took 51m, 48m and 59m.
• So main is close enough to the limit that any pull request adding about 10% more parameters will time out.

Checks

• --shard handles N = 5. On this branch I collected all 7,750 Python tests without --shard, then collected --shard K/5 for K = 1 to 5. Every test is in exactly one shard, and the union equals the full set. Shard sizes: 1,504, 1,590, 1,567, 1,554 and 1,535 (with four shards: 1,949, 1,934, 1,927 and 1,940). A separate SHA-256 assignment matched what pytest selected for every test.
• policyengine_uk/tests/code_health/test_ci_test_shards.py: 46 passed. No test asserts the shard count.
• Required checks: the main ruleset requires Test / Result and Lint only (read from rules/branches/main; classic branch protection has no required checks). The check names change from Test / Python K/4 to Test / Python K/5, which are not required, so the ruleset needs no change.
• No changelog fragment: this changes CI only, and a fragment would trigger a release.

The shards split by test count, not by time. If one slow test module lands in one shard, that shard stays the longest; the CI run on this PR will show the split.

Reproducer: n/a: CI configuration only; model source and dependencies are unchanged
axiom: n/a: CI configuration only

🤖 Generated with Claude Code

[juaristi22]

Approved at c0c010d49b77ddfb8f403c36a47c5b80afcce71d.

The fifth shard is picked up throughout: strategy.job-total supplies N to the existing hash partition, and Result waits for the whole Python matrix and checks every outcome log against the full collection. PR, merge-queue, main and release callers keep using the reusable workflow. The required contexts remain Test / Result and Lint.

Validation: 20 focused static/metadata assertions passed, and git diff --check passed. Exact-commit CI passed all five Python shards and the aggregate result in both the source-tree and release-wheel runs; lint and YAML passed too. The slowest source shard took 39m05s and the slowest release shard 42m16s, below the unchanged 60-minute timeout. No findings or material evidence gaps. The full model suite was not rerun locally for this CI-only change.