Repository navigation
Run autoharness jobs only in the merge queue - #738
Open
Tianshu-Huang wants to merge 4 commits into
Open
Tianshu-Huang wants to merge 4 commits into
Tianshu-Huang wants to merge 4 commits into
Conversation
Running autoharness only in the merge queue means a contributor's first autoharness result arrives after the PR is queued, and a failure ejects it hours later. workflow_dispatch does not help contributors: it needs write access and runs on a branch of this repo, not on a fork's PR. Add the run-autoharness label as a third way to run the autoharness and metrics jobs: while it is on a PR they run on every push. Adding any label now triggers the workflow, so the partitions skip label events (they already ran for that commit) and the autoharness jobs ignore labels other than run-autoharness.
feliperodri
added
the
run-autoharness
feliperodri
approved these changes
Oct 30, 2026
feliperodri
left a comment
feliperodri
left a comment
Member
There was a problem hiding this comment.
Approving. I pushed two commits on top so contributors can ask for an autoharness run before the merge queue: adding the run-autoharness label runs autoharness and the metrics jobs on the PR, on every push while the label stays. workflow_dispatch doesn't cover this, because it needs write access and runs on a branch of this repo, not on a fork's PR.
Adding any label now triggers the workflow, so the partitions skip label events (they already ran for that commit), and the autoharness jobs ignore labels other than run-autoharness.
Tested on this PR:
- Push without the label: both autoharness checks pass in seconds with their steps skipped; List, Analyzer, Metrics and Build JSON are skipped; the partitions run (run).
run-autoharnessadded: autoharness runsRun Kani Verification, and Metrics, List and Analyzer run; the partitions skip (run).- The merge-queue path gets tested when this PR goes through the queue; I'll check that autoharness actually runs there.
Thanks @Tianshu-Huang!
feliperodri
enabled auto-merge
October 31, 2026 00:18
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Five CI jobs (autoharness, metrics, list, analyzer, build JSON) run on every PR push and take about five runner-hours. Every PR waits on them. #730 asks to run them only before merging.
Fix
Add
if: merge_group || workflow_dispatchto those five jobs.Autoharness is a required check, so it cannot be skipped as a whole job: a skipped job loses the
(ubuntu-latest)part of its name and the required check never turns green. So for that job theif:is on each step. The job still runs, does nothing, and passes in seconds.Testing
On my fork, a manual run passed all 16 jobs (run), and a PR skipped the five jobs with the two autoharness required checks green (PR). Did not test
merge_group.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.