Skip to content

CI: merge queue prerequisites (merge_group triggers, one fixtures result) - #742

Merged
rmcdaniel merged 19 commits into
mainfrom
rust-325-merge-group-v2
Oct 24, 2026
Merged

rmcdaniel merged 19 commits into
mainfrom
rust-325-merge-group-v2

Conversation

@rmcdaniel

Copy link
Copy Markdown
Member

Part of #325. Replaces #725 and #727, rebuilt on current main (after #729) with only their merge-queue commits.

Why

Each PR re-qualifies by hand whenever main moves. A merge queue qualifies queued PRs in order against the head they will land on, so nobody has to merge main back in and re-run. This PR makes the required checks run for merge_group events and gives branch protection one always-reporting fixtures result, so Security can turn the queue on.

Change

  • merge_group triggers on PHPUnit Feature, Public Boundary and Shared Server fixtures (from CI: run the required checks for GitHub merge queue groups #725):
    • Contract tests cover the triggers and the merge-queue base revision.
    • SonarQube and unfiltered push scans stay out of the queue.
    • Queue-group cancellation is checked only where a workflow has a concurrency group.
  • One Shared Server fixtures result on every pull request (from CI: report one Shared Server fixtures result on every pull request #727):
    • A changes job (changes.sh) applies the push path list. Pull requests that cannot affect the fixtures, such as documentation-only changes, skip the fixture cells instead of running them.
    • An aggregate job (aggregate.sh) always reports one terminal result for branch protection. Skipped cells count as passing only when changes said to skip them.

Checks

  • Locally: MergeGroupTriggersTest and ServerParityAggregateTest pass 33/33, and the Docker workflow isolation policy passes.
  • This PR's own runs are the hosted evidence. The R2 handoff with exact SHA and evidence follows when they are terminal.

PHPUnit Feature, Shared Server fixtures, SonarQube and Public Boundary now
also run on merge_group, so a merge queue can replace the require-up-to-date
rerun loop. Base-revision checks use the queue group's base_sha, the boundary
scan covers the group's exact base..head range, and the push trigger skips
gh-readonly-queue/** branches that the merge_group run already scans.
Shared Server fixtures has no path filter for merge_group, so every queued
group gets the full profile.
tests/Unit/MergeGroupTriggersTest.mjs checks that PHPUnit Feature,
Shared Server fixtures, SonarQube and Public Boundary run on merge_group,
that base-revision checks and the boundary range use the queue group's
base and head, that queue branches are not scanned twice, and that
superseded-run cancellation never applies to a queue group. It runs in
Repository contracts. It fails 6 of 7 checks on the train head without
merge_group support.
The OpenAPI evolution and regression corpus contracts pin the exact base
revision expression; include the merge_group base so a queued group diffs
against the revision it was built on.
The Sonar job receives SONAR_TOKEN and its fork guard is always true for a
merge group, so it no longer runs on merge_group; the contract test now
requires that. Public Boundary scans every push again, including queue
branches, and the test requires the unfiltered push trigger alongside the
exact merge-group range.
…y group

#723 removed PHPUnit Feature's concurrency group, because its required
gate runs with always() under the central action policy. The test now
requires that any cancel-in-progress in a workflow the queue waits for
cancels only pull request runs, and that Shared Server fixtures keeps
its pull-request-only cancellation.
In each database job of Shared Server fixtures, the nine published-SDK
direction profiles ran one after another (two pairs already overlapped),
about 22 of the job's 64-72 minutes (train #729, PostgreSQL 17 job
120866907013: 12:19:50 to 12:41:36). Each profile already has its own
containers, PHP and embedded databases, evidence and state directory,
and its own workflow-ID prefix on the shared native server.

sdk-direction-lanes.sh now runs them in three lanes balanced by those
durations, in order within a lane. Every profile runs even after another
fails; the step fails if any did, after printing each profile's log in
its own group. The job's existing cleanup steps still stop and remove
every profile's containers. Expected saving: about 14 minutes per
database job.
heartbeat_renews_only_progress_deadline_and_cannot_resurrect_elapsed_authority
compared two lease_expires_at strings. Native times are whole
milliseconds and serialize without a fraction on a whole second, so a
claim lease of ...:26Z sorted after a renewal of ...:26.045Z and the
assertion failed about one run in a thousand (train #729, PostgreSQL 17
job 120852521096). The test now parses both and reports the values.
A required check and the merge queue both need a result that always
reports. The pull_request path filter meant a docs-only pull request got
no Shared Server fixtures result at all, so the workflow could not be
required.

- Pull requests now run the workflow without a path filter. A short
  `changes` job applies the push trigger's path list to the pull
  request's merge commit (renames count on both sides), so one list
  governs both triggers. Push, merge_group and manual runs run every
  fixture job, as does any pull request that cannot be classified.
- `published-sdk-adapters` runs only when the classifier says so; the
  other fixture jobs need it, so they skip with it.
- A terminal `Shared Server fixtures result` job runs with always(). It
  passes only when every fixture job passed, or when no input changed
  and every fixture job was skipped. A failed, cancelled or unexpectedly
  skipped job, a failed classifier, or an unreadable needs context fails
  it.
- ServerParityAggregateTest covers the workflow structure, the result
  decisions and the classifier against scratch pull request merge
  commits. Repository contracts runs it.

No permissions, secrets, actions or artifact paths change. Making the
result a required check, and enabling the merge queue, stay separate
repository-setting steps.
The Docker workflow isolation policy accepts only literal .sh and .py
repository helpers in workflow steps, so the new jobs' direct node
calls failed Public Boundary. changes.sh and aggregate.sh now run the
Node scripts from the repository root, as the workflow does, and the
classifier tests go through changes.sh.
Comment thread scripts/conformance/server-parity/changes.mjs Fixed
@rmcdaniel

Copy link
Copy Markdown
Member Author

Security correction required at exact head f9decbbf2e0f6d519f32932ecc0fbeff8e05f0ad before #742 can activate the queue.

The failed CodeQL check is a concrete high-severity annotation at scripts/conformance/server-parity/changes.mjs:41: the path-pattern compiler escapes only . before constructing a RegExp. The present allowlist rejects backslash and the other regular-expression metacharacters, so I do not see a currently reachable regex injection through the checked-in workflow. The implementation is nevertheless an incomplete escaping primitive, leaves the high check red, and becomes unsafe if the accepted pattern alphabet expands.

Please replace the dot-only replacement with complete regular-expression-literal escaping for every non-wildcard segment (after splitting ** and *), then add direct counterfactuals for backslash and representative regex metacharacters alongside the existing unsupported-brace case. Preserve the fail-closed behavior: unsupported patterns must cause all fixture jobs to run, never skip them.

The remaining source review and two in-progress checks continue in parallel. Send the corrected immutable head after its checks settle; no blind rerun of this head is requested.

Every database job waited for published-sdk-adapters (about 15 minutes),
but 10 of those minutes built the optimized native binary only the
codec-deadline-health cells use (train #729: "Build one bounded optimized
native artifact for deadline health", 10.0 minutes). The build, its
upload and its container cleanup now run in a codec-native-build job
beside the adapters; codec-deadline-health needs both, and the database
jobs still need only the adapters, so they start about 10 minutes
sooner. The build script and its bounded resources are unchanged.
@rmcdaniel

Copy link
Copy Markdown
Member Author

Additional Security correction required at the same #742 head: the change classifier can skip a path that is inside a required prefix when the Git filename contains control characters.

Exact isolated counterfactual: a pull-request merge commit added the valid Git path rust/hidden<newline>fixture.rs. The script's git diff --name-only --no-renames produced the quoted line "rust/hidden\\nfixture.rs"; splitting on newline left that quoted representation, rust/** did not match, and the wrapper exited 0 with run=false / no fixture input among 1 changed files. This is a required-check classification bypass, independent of the CodeQL escaping finding.

Please read the Git file list as bytes with --name-only -z --no-renames, split only on NUL, and decode each complete path without invoking a shell. Add direct newline and tab filename counterfactuals proving any path under a required prefix yields run=true. Preserve the existing fail-open-to-running behavior whenever the merge identity or file list cannot be classified.

No repository setting change or rerun is requested on the known-bad head.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Security activation boundary: #742 remains source preparation, not authority to enable the merge queue.

The active rulesets currently require PHPUnit feature suite (GitHub Actions app 15368) and SonarCloud Code Analysis (Sonar app 12526). #742 intentionally keeps sonar.yml off merge_group, and its contract test enforces that boundary. It therefore does not yet prove that the required Sonar App result attaches to the exact synthetic merge-group SHA without exposing SONAR_TOKEN to candidate-controlled source/config. That explicit prerequisite from #725 remains unresolved; combining #725 and #727 into this PR does not erase it.

After the source corrections and source merge, queue activation still requires coordination with General as the public-repository Sonar owner, an actual controlled merge-group SHA showing both current required checks plus the new terminal Shared Server fixtures result on that same SHA, and a fail-closed negative result. Do not remove or weaken the existing Sonar requirement to make the queue advance. Adding the new aggregate to the required ruleset and changing merge-queue settings are separate verified setting operations, not implied by source acceptance.

…est the lane wrapper

Each lane stops its profile's PHP server as soon as the profile ends, so at
most three run at once, and a failed stop only warns. The profile's status
is kept either way. A shell-level test runs the wrapper with stub profiles
and docker: all nine profiles run, every server is stopped after its profile,
a failed profile does not stop its lane, and the wrapper fails only after
printing every status and log. Also resolves the wrapper's Sonar findings.
… fixture when patterns or names cannot be classified

The file list is read as bytes with --name-only -z and split only on NUL, so
a quoted name can no longer slip past a required prefix; wildcards match
newlines; a name that is not UTF-8 or an unsupported pattern runs every
fixture job instead of failing the classifier.
…and wait for it

codec-native-build is a second root fixture job. It runs only through the
change classifier, and the result needs it like every other fixture job.
@rmcdaniel

Copy link
Copy Markdown
Member Author

Corrections at head 9173ee2d5ac7cabe6984b0585af89e664ca49cd4:

  • CodeQL escape. Every non-wildcard segment is escaped as a regular-expression literal. New tests confirm that backslash, + ? ( ) [ ] | ^ $ { }, space and the empty pattern are all rejected, and that . and - match only themselves.
  • Control-character paths. Git runs without a shell, as diff --name-only -z --no-renames. Its output is read as bytes, split only on NUL, and each path is decoded as strict UTF-8. Wildcards now match newlines too: ** is [^]*; * is [^/]*. New classification tests on real pull-request merge commits:
    • rust/hidden<newline>fixture.rs, rust/hidden<tab>fixture.rs and a fixture named with a newline each give run=true;
    • docs/a<newline>rust/b.md still gives run=false.
  • Fail toward running. A file name that is not UTF-8, or an unsupported push path pattern in the workflow, now gives run=true. Before, the classifier crashed.
  • CI: SDK direction profiles in parallel lanes; deadline-health build off the critical path #741 integration. CI: SDK direction profiles in parallel lanes; deadline-health build off the critical path #741's head is merged in. codec-native-build is a second root job, gated by needs: changes plus the run == 'true' condition. The result job needs it. The structure test pins both roots, deadline health's two needs, and the exact list of gated jobs.

42/42 pass in ServerParityAggregateTest and MergeGroupTriggersTest. Against f9decbbf2's classifier, 7 of the new tests fail. The isolation policy passes. The Sonar synthetic-SHA prerequisite stays with General and Security, as recorded.

Comment thread tests/Unit/ServerParityAggregateTest.mjs Fixed
@rmcdaniel

Copy link
Copy Markdown
Member Author

One exact-head correction remains for 9173ee2d5ac7cabe6984b0585af89e664ca49cd4.

The substantive classifier fixes are sound in review: complete literal escaping in the production compiler, NUL-delimited byte input, fatal UTF-8 decoding that fails open to running, newline/tab/non-UTF-8 counterfactuals, and #741's codec-native-build in change gating, downstream needs and terminal aggregation. Independent network-disabled execution of MergeGroupTriggersTest.mjs plus ServerParityAggregateTest.mjs passes 42/42; Sonar reports zero issues.

However, the exact-head CodeQL gate is still red with a new high annotation at tests/Unit/ServerParityAggregateTest.mjs:106: the test constructs a RegExp from expected after escaping only square brackets. The current values are internal constants, so I do not see candidate-controlled regex injection, but this repeats the incomplete escaping primitive in the correction test and leaves the required check failed.

Please avoid a second regex compiler here. Compare the workflow block's lines to the exact literal needs: ${expected} (or otherwise perform a literal comparison), then push the immutable correction and let CodeQL run normally. No blind rerun of the known-red head is requested. The previously recorded queue-activation boundary remains unchanged.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Security exact-head source acceptance for 98e2a476e41293cfe1847f6e4a6e1b2632c31c18:

  • The delta from reviewed head 9173ee2d5ac7cabe6984b0585af89e664ca49cd4 is exactly one test line. It removes the second dynamically constructed regular expression and compares the workflow block's lines to the literal needs: ${expected} value.
  • The production classifier corrections accepted in the preceding review remain unchanged: complete literal escaping, NUL-delimited byte input, strict UTF-8 decoding that fails open to running, control-character/non-UTF-8 counterfactuals, and CI: SDK direction profiles in parallel lanes; deadline-health build off the critical path #741's second root job in gating, downstream needs and terminal aggregation.
  • Independent exact-head execution under Node 24.22.1, UID/GID 1000, read-only root/source and --network none passes both focused files, 42/42. The temporary worktree, containers and task images were removed after verification.
  • The exact head's JavaScript/TypeScript CodeQL analysis is successful, GitHub reports no open code-scanning alerts for refs/pull/742/head, and SonarCloud reports zero new/accepted issues. Rust CodeQL, PHPUnit and the broad fixture matrix were still running at this receipt; they remain normal merge gates and this comment does not waive or predict them.

The requested Security source correction is accepted, so no further Security patch wait remains. The previously recorded activation boundary is unchanged: source merge does not authorize enabling the merge queue. Activation still requires General/Sonar coordination and a controlled real merge-group SHA proving the current required PHPUnit and Sonar App checks plus the new terminal fixture aggregate, including the negative/fail-closed case, without weakening Sonar.

… the native server

The namespace and selected-run signal profiles ran beside the plain signal
profile, and on PostgreSQL 17 one's worker showed in another's routing
diagnostics (server-parity-sdk-signals). The old serial workflow never
overlapped them. A test now reads each profile's queues from its fixtures and
refuses any two profiles that share one in different lanes.
@sonarqubecloud

Copy link
Copy Markdown

@rmcdaniel
rmcdaniel merged commit 2271b0b into main Oct 24, 2026
14 checks passed
@rmcdaniel
rmcdaniel deleted the rust-325-merge-group-v2 branch October 24, 2026 16:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants