Skip to content

Let the Test jobs run for 105 minutes - #737

Merged
anth-volk merged 1 commit into
mainfrom
test-time-limit-105
Nov 3, 2026
Merged

anth-volk merged 1 commit into
mainfrom
test-time-limit-105

Conversation

@anth-volk

Copy link
Copy Markdown
Contributor

Fixes #736

Summary

Raises timeout-minutes from 90 to 105 on the two jobs that run the full test suite with coverage (make test, which is pytest tests --cov=policyengine --cov-report=term-missing):

  • .github/workflows/pr_code_changes.yaml, job Test (the "Test (3.11)" to "Test (3.14)" matrix)
  • .github/workflows/push.yaml, job Test (pushes to main; its comment says it matches the pull-request Test job, so a release run isn't cancelled before Versioning)

Also adds changelog.d/736.changed.md and updates the comments above both limits.

Why

On Python 3.11 to 3.13, coverage.py 7.13.5 (from uv.lock) records coverage with its C trace function; it switches to sys.monitoring only on 3.14. Those jobs now take 65 to 90 minutes, and one of them is often cancelled at 90 minutes with no failing test:

PR Cancelled at 90 min Other 3.11-3.13 jobs 3.14
#730 3.12 (1h30m25s) 3.11 1h29m21s, 3.13 1h06m51s 58m49s
#735 3.11 (1h30m34s) 3.12 1h29m43s, 3.13 1h21m11s 47m22s

Left unchanged

  • Other jobs in both workflows (changelog check, Lint, Mypy, Python-Compat, BundleVerification, Docs, Versioning, Publish, NotifyConsumers) do not run the test suite.
  • pr_docs_changes.yaml (docs build, 30 min), draft-pdf.yml (paper, 15 min) and certify-uk-release.yaml (its jobs run their own checks with their own limits; it starts pr_code_changes.yaml with gh workflow run, so that run picks up the new limit).

Environment variables

This PR changes no environment variable, secret or variable reference. The two changed jobs read:

  • HUGGING_FACE_TOKEN: repository secret (an organisation secret of the same name also exists), passed as secrets.HUGGING_FACE_TOKEN in the env: of the "Run tests with coverage" step, the only step that uses it, in both jobs.
  • CI_RUNNER_X64 and CI_RUNNER_EVENTS: repository variables read in each job's runs-on expression. Neither is set at repository or organisation level, so both jobs run on ubuntu-latest.

Overlap

#731 sets both limits to 150 on its own branch. Whichever merges second will need its timeout-minutes values and comments reconciled.

Possible follow-up (not in this PR)

Measure coverage only on 3.14 and run plain pytest on 3.11 to 3.13. That would bring those jobs down to about the 3.14 time.

Checks run

  • actionlint on both workflows: no new findings (one existing warning about actions/configure-pages@v3 in push.yaml, also on main).
  • Parsed both workflows with PyYAML and confirmed both Test jobs have timeout-minutes: 105.
  • make lint: passes.
  • No tests run locally.

🤖 Generated with Claude Code

On Python 3.11 to 3.13, coverage.py 7.13.5 records coverage with its C
trace function (it uses sys.monitoring only on 3.14), so the Test jobs
there take 65 to 90 minutes, and one of them was often cancelled at the
90-minute limit with no failing test: 3.12 on #730, 3.11 on #735. The
3.14 job takes 47 to 60 minutes. The push-to-main Test job runs the same
suite and keeps the same limit as the pull-request one, so a release run
isn't cancelled before Versioning either.

Fixes #736

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk
anth-volk marked this pull request as ready for review November 3, 2026 19:26
@anth-volk
anth-volk merged commit dd6da4c into main Nov 3, 2026
13 checks passed
@anth-volk
anth-volk deleted the test-time-limit-105 branch November 3, 2026 19:27
anth-volk added a commit that referenced this pull request Nov 3, 2026
Brings in #737, which lets the Test jobs run for 105 minutes instead of
90; the 3.11 and 3.13 Test jobs on this branch stopped at the old limit.
No conflicts: #701, also new on main, changes model metadata and none of
the UK area lists or area results this branch edits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Test jobs on Python 3.11 to 3.13 are cancelled at the 90-minute limit

1 participant