Skip to content

Run the Tests workflow for pull requests into any base branch - #1322

Closed
anth-volk wants to merge 4 commits into
mainfrom
ci-pull-requests-into-any-base
Closed

anth-volk wants to merge 4 commits into
mainfrom
ci-pull-requests-into-any-base

Conversation

@anth-volk

@anth-volk anth-volk commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #1321

Summary

The Tests workflow (.github/workflows/test.yml) ran for pull_request events only when the base branch was main. A stacked pull request targets another pull request's branch, so it got no CI at all. This PR removes the branches filter from the pull_request trigger.

  • Pull requests: any base branch now runs the same jobs.
  • Pushes: still run the workflow only for main.
  • Unchanged: the jobs, the country selection, the concurrency group and the secrets.
  • Runner cost: every push to any pull request's branch now runs the full workflow on the organization's shared runners, whatever its base. Merging one change up this five-PR stack starts five full runs. The agent guide says so.

This PR is the bottom of a stack: main ← this PR ← #1295 ← #1297 ← #1296 ← #1298 (GitHub stack #1323). CI started on every pull request above it after the stack was linked and pushed, which confirms the widened trigger works for stacked bases. It merges first. GitHub reads a pull request's workflow from that pull request's test merge commit, so each pull request above this one only runs CI because its branch includes this commit.

Nothing else in the workflow depends on the base branch

  • Concurrency group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.run_id }}. It is keyed on the pull request number, so each pull request still cancels only its own outdated runs.
  • Country selection: tools/ci_test_plan.py select-countries reads the pull request's own changed files (pulls/<number>/files), which GitHub computes against the pull request's base, whatever it is.
  • No other base dependence: the workflow never reads github.base_ref or github.head_ref, has no paths filters, and checks out the default ref, which is the pull request's test merge commit.
  • Caches: the setup-uv and setup-node caches restore from the pull request's own ref or the default branch, so a non-main base changes nothing.

Environment variables and secrets the workflow reads

I checked secret names only, read-only. Listing the repository, organization and staging environment secrets needs admin access, which this account has, so the listings are complete.

Name Read by Provider Different for a non-main base?
OMP_NUM_THREADS, OPENBLAS_NUM_THREADS, MKL_NUM_THREADS every job (workflow-level env:) literal "1" in the workflow No
GITHUB_TOKEN select-countries step env: (${{ github.token }}), passed to tools/ci_test_plan.py for the pull-request files API GitHub Actions' automatic token, scoped by the job's permissions: (contents: read, pull-requests: read) No. A pull request from a branch in this repository gets the same permissions for any base; a fork's pull request gets a read-only token either way.
GITHUB_EVENT_PATH, GITHUB_EVENT_NAME, GITHUB_REPOSITORY, GITHUB_OUTPUT tools/ci_test_plan.py select-countries GitHub Actions runner No. The event is pull_request for any base, and its payload has the same pull_request.number.
GITHUB_STEP_SUMMARY tools/add_engine_us_timing_summary.sh GitHub Actions runner No
HF_STAGING_READ_TOKEN integration-uk step env: (${{ secrets.HF_STAGING_READ_TOKEN }}), read by tools/run_integration_tests.sh None found. It is not a repository secret, not an organization secret available to this repository, and not a staging environment secret; the job declares no environment: anyway. No. It resolves to an empty string on every run, for main pushes and every pull request, so the script always skips its optional private-repository access check. This predates this PR, which doesn't change it.

Pull requests from branches in this repository get the same secrets whatever their base branch. GitHub withholds secrets only from pull requests opened from forks.

Commits

1. 50406fd59 Run the Tests workflow for pull requests into any base branch

  • Tests, in test_ci_integration_job.py:
    • test_pull_requests_into_any_base_branch_run_the_tests_workflow, with three bases: main, orrery-projection-without-store and stacked/parent-branch (replaced by commit 2);
    • the guard test_only_pushes_to_main_run_the_tests_workflow.
    • Both read the parsed triggers. PyYAML reads the bare on key as the YAML 1.1 boolean true, and the tests account for that.
  • Red: 2 failed, for the intended reason. The branches: [main] filter excluded the two non-main bases. The main case and the push guard passed.
  • Green: 4 passed. The whole file gave 13 passed.
  • Checks: tools/ci_test_plan.py verify ok; ruff check and ruff format --check clean on the changed file; actionlint passed on test.yml.
  • Docs: the agent guide's CI paragraph says pull requests into any base branch run the workflow. The changelog fragment is changelog.d/1321-ci-pull-requests-into-any-base.changed.md.

2. b701a5e49 Assert the pull_request trigger has no base-branch filter

A review finding. Commit 1's test checked three literal branch names against the trigger's branches and branches-ignore lists, but GitHub matches those entries as glob patterns, so a membership check says nothing about other bases. The intended design is no base-branch filter at all.

  • Test: test_pull_requests_into_any_base_branch_run_the_tests_workflow now asserts that the pull_request trigger has neither a branches nor a branches-ignore key. _workflow_triggers is typed dict[str, Any], and the pytest import, now unused, is removed.
  • Red: 1 failed, for the intended reason, run against the workflow as it was before this PR (50406fd59^, with branches: [main]): assert ['branches'] == [].
  • The old test was too weak, checked by editing the workflow temporarily:
    • with branches: [main, orrery-projection-without-store, stacked/parent-branch], which gives pull requests into any other base no run, the old test passed (3 passed) and the new one fails;
    • with branches: ['**'], which runs for every base, the old test failed (3 failed). The new test fails too, because the design is no filter.
  • Green: 1 passed. The whole file gave 11 passed (13 before, less the three parametrized cases, plus the new test).
  • Checks: ruff check and ruff format --check clean on the changed file.

3. 81650f002 Note the runner cost of running CI for pull requests into any base

A review finding. The agent guide now says that every push to a pull request's branch runs the full workflow on the organization's shared runners, whatever its base, with the jobs and country selection a pull request into main gets, so merging one change up a stack of N pull requests starts N full runs. Documentation only, so no red step. test_ci_integration_job.py, which reads the agent guide: 11 passed.

4. 445823919 Merge main

main at a90eec3cb (#1289, #1305), merged with a merge commit and no conflicts; none of its new commits touch this PR's files. No red step. test_ci_integration_job.py gave 11 passed and tools/ci_test_plan.py verify reported verification=ok.

GitHub builds each stacked pull request's test merge on top of the test merge of the pull request below it, starting from main: this PR's refs/pull/1322/merge has main as its first parent, and each later one has the previous one's. Once main gained #1305, #1296's change to rowwise_staging.py conflicted with it, so GitHub could not build #1296's test merge, reported #1296 as conflicting although its branch contained its base, and started no CI for #1296 or #1298. Merging main here, and each branch into the one above, resolved that.

Locally I ran only the changed test file; CI runs the rest. On 2026-10-20 I re-checked the secret names, read-only: test.yml still reads only secrets.HF_STAGING_READ_TOKEN, which is still neither a repository secret nor an organization secret available to this repository, so the table above holds. Commits 2 to 4 change no workflow file.

Merge order

  1. this PR;
  2. Project Orrery artifacts without the content store, and write the execution overlay as its own file #1295;
  3. Reconstruct declaration-only Orrery publications for UK builds before #1144 #1297;
  4. Thin US calibration epoch telemetry to the UK's rule #1296;
  5. Publish a JSON Schema for calibration diagnostics schema 8 #1298.

Each pull request above this one has its parent merged in, so with merge commits each one merges cleanly after its parent.

🤖 Generated with Claude Code

The pull_request trigger was filtered to branches: [main]. A stacked pull
request targets another pull request's branch, so GitHub started no Tests
run for it at all. The filter is removed. Pushes still run the workflow
only for main.

Nothing else in the workflow depends on the base branch. The concurrency
group is keyed on the pull request number, and country selection reads the
pull request's own changed files from the API. Pull requests from this
repository's branches get the same token permissions and secrets whatever
their base.

New tests read the workflow's triggers. Pull requests into main, into
another pull request's branch and into a nested branch name must all run
it, and pushes must run it only for main. The agent guide and a changelog
fragment say so.

Fixes #1321

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anth-volk and others added 3 commits October 20, 2026 16:48
The previous test checked three literal branch names against the trigger's
branches and branches-ignore lists. GitHub matches those entries as glob
patterns, so a membership check proves nothing about other bases: a filter
listing exactly those three names passed it, although pull requests into any
other base would get no run, and a filter of '**', which runs for every base,
failed it.

The intended design is no base-branch filter at all, so the test now asserts
that the pull_request trigger has neither a branches nor a branches-ignore
key. _workflow_triggers is typed as dict[str, Any].

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every push to a pull request's branch now runs the full Tests workflow on
the organization's shared runners, whatever the base, so merging one change
up a stack of N pull requests starts N full runs. The agent guide says so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GitHub builds each stacked pull request's test merge on top of main, so
once main moved on (#1289, #1305), the stack's test merges had to include
it. main's new commits do not touch this pull request's files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk
anth-volk marked this pull request as draft October 20, 2026 17:59
@anth-volk
anth-volk removed this pull request from stack #1323 October 20, 2026 18:01
anth-volk pushed a commit that referenced this pull request Oct 20, 2026
This branch carried #1322 through three merges of its branch
(`ci-pull-requests-into-any-base`). #1322 removed the `branches: [main]`
filter from the Tests workflow so stacked pull requests would get CI.
That is not wanted: CI runs only for pull requests into `main` on purpose,
to limit the number of simultaneous runs on the organization's shared
runners. #1322 is being closed.

This restores the four files #1322 changed to their versions on `main`
(`a90eec3cb`), which is also this branch's merge base with `main`:

- `.github/workflows/test.yml`: the `pull_request` trigger filters on
  `main` again;
- `docs/agent-guide.md`: the CI paragraph no longer says any base runs;
- `packages/microcosm-build/tests/engine_free/shared/test_ci_integration_job.py`:
  #1322's two trigger tests are removed;
- `changelog.d/1321-ci-pull-requests-into-any-base.changed.md`: deleted.

The branch's own commits are not rewritten, so `8d159a8`, which
microcosm-collector#9 records as its source, stays in the history.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anth-volk pushed a commit that referenced this pull request Oct 20, 2026
…-schema

Carries the restored main-only Tests workflow trigger (#1322 is not wanted) up the stack.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk

Copy link
Copy Markdown
Contributor Author

Closing without merging. The Tests workflow runs only for pull requests into main on purpose: it limits the number of simultaneous runs on the organization's shared runners, which are often backed up.

This pull request's changes were already in the branches stacked on it, so they are undone there without rewriting any commit:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Run the Tests workflow for pull requests into any base branch

1 participant