Conversation
|
Claude finished @nicobytes's task in 3m 16s —— View job PR Review — strict-gate CI wiringI reviewed the current diff ( New Issues
ResolvedEvery finding from the prior Copilot and @oidacra rounds is addressed in the current diff:
Notes (non-blocking)
Verdict: No blocking issues in the diff. The one Medium above is a "confirm the assumption" item on the
|
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved critical and moderate findings remain in CI wiring, reporting, and pre-push behavior.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Activates the diff-scoped strict TypeScript gate for pull requests and adds local pre-push enforcement.
Changes:
- Wires the gate into Maven CI for pull requests.
- Adds pre-push enforcement.
- Adds duration reporting, tests, documentation, and specifications.
File summaries
| File | Description | Review findings |
|---|---|---|
specs/37536-wire-strict-gate-ci/spec.md |
Requirements and acceptance criteria | No final review comments. |
specs/37536-wire-strict-gate-ci/data-model.md |
Outcome and duration model | Nit (2 votes): Documents report.files.length, which the implementation does not expose and cannot use for the documented diff-size calculation. |
specs/37536-wire-strict-gate-ci/contracts/ci-hooks.md |
CI and hook contracts | No final review comments. |
core-web/tools/scripts/strict-gate/README.md |
Gate documentation and status | Nit (1 vote): Contradictory “spike” and “wired and live” status wording. |
core-web/tools/scripts/strict-gate/lib/format.mjs |
Duration and summary formatting | Moderate (1 vote): Omits unmapped files from both summary branches. Moderate (3 votes): Reports assignment counts rather than unique changed files, excluding unmapped files from the measurement. |
core-web/tools/scripts/strict-gate/format.duration.test.mjs |
Duration summary tests | No final review comments. |
core-web/pom.xml |
Pull-request Maven gate execution | Critical (1 vote): Hard-codes origin/main, producing incorrect or failed comparisons for master pull requests.Moderate (1 vote): GitHub output lacks the scope/dependency-discard preamble. |
core-web/.husky/pre-push |
Local strict-gate enforcement | Moderate (3 votes): Runs the toolchain without a changed-path preflight for non-frontend pushes. Moderate (1 vote): Mislabels exit-2 execution failures as violations. Moderate (1 vote): Relies on potentially stale origin/main, diverging from CI. |
Review details
Suppressed comments (6)
core-web/.husky/pre-push:52
- The hook ignores the refs supplied on pre-push stdin and always checks
HEAD. A validgit push origin other-local-branch:remote-branch(or a multi-ref push) can therefore be allowed or refused based on a different commit than the one being sent. Parse the local ref/SHA from stdin, skip deletions, and pass that SHA as--head, or explicitly restrict the hook to current-branch pushes.
pnpm exec node tools/scripts/strict-gate/run.mjs \
--base origin/main \
--head HEAD \
core-web/.husky/pre-push:68
- For exit
2(for example, an unresolved base or unreadable project graph), the harness may print no violations, but this wrapper still tells the developer to “Fix the violations above.” That misclassifies a harness failure and obscures the recovery path; distinguish findings (1) from a gate execution error before printing this guidance.
echo "Push refused by the strict-gate check (exit $status)."
echo " Fix the violations above, or push anyway with: git push --no-verify"
core-web/.husky/pre-push:51
- This local invocation assumes
origin/mainis current, but the hook never refreshes the remote-tracking ref; CI explicitly fetchesorigin/mainimmediately before Maven. A developer with a stale ref can therefore get a different result from the pull-request run, violating the documented claim that local and remote answers agree. Refresh the target base (or make the freshness prerequisite explicit and enforce it) before running the check.
pnpm exec node tools/scripts/strict-gate/run.mjs \
--base origin/main \
core-web/pom.xml:518
--format=githubemits only::errorannotations;formatGithub()does not emit the required scope/dependency-discard preamble. The new CI log therefore starts with findings without telling agents that only changed lines are in scope (the separate Markdown summary does not change the formatter output). Add the scope preamble to the GitHub formatter before enabling this invocation.
<argument>--format=github</argument>
core-web/tools/scripts/strict-gate/README.md:80
- This new section says the gate is "Wired and live", but the unchanged introduction still says "This is spike output, not production tooling" (line 9). After adding CI and pre-push enforcement, that description is contradictory and can lead contributors to treat the live hooks as unsupported or removable spike artifacts. Update the introduction/status wording to reflect that this is now temporary production tooling.
**Wired and live since #37536.** Two invocation points, deliberately asymmetric:
core-web/tools/scripts/strict-gate/lib/format.mjs:146
formatMarkdownis the job summary used by the new--format=githubCI path, but the pass branch never rendersreport.unmapped(and the findings branch omits it too). A changed TypeScript file that no project claims therefore appears as a clean pass with no indication that it was unexamined, contrary to the gate's edge-case contract; include the unmapped files/reasons in both summary branches.
costLine(report)
- Files reviewed: 8/8 changed files
- Comments generated: 4
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
e4cebb7 to
c7ada9a
Compare
The strict-gate harness merged with #37403 was wired into nothing and has never executed once. Since 2026-09-08 no pull request has been annotated, reported on, or blocked by it, and main has kept accumulating non-strict TypeScript exactly as before. This wires it into CI as a blocking gate: a change that adds a strict-mode violation on a line it wrote does not merge. A `strict-gate` execution in core-web/pom.xml, inside the existing `validate` profile beside lint-test and format-test. `successCodes` lists 0 and nothing else -- findings (1) and a harness that could not run (2) both fail. A 180s timeout fails the same way, bounding the harness's unshallow git-fetch fallback. It runs in two contexts, and both are load-bearing for different reasons. `pull_request` is where the author reads it: the harness emits `::error file=,line=,col=` lines that GitHub renders inline on the changed lines, plus a job summary. `merge_group` is where it is ENFORCED. This repository declares no required status checks on main -- the ruleset requires a pull request, one approval, thread resolution and signed commits -- so a red check on a pull request does not by itself stop a merge. A job that fails in the merge queue ejects the pull request, and that does. Wiring only the first would have produced a gate that goes red and merges anyway. Trunk and nightly stay skipped: HEAD equals origin/main there, so the diff is empty. Maven property activation has no OR, hence two profiles rather than one condition. No workflow file changes; activation reads the runner's own GITHUB_EVENT_NAME. `useMavenLogger` is pinned false with a comment even though it is the default, because that default is the only reason the `::error` lines reach the log at column 0 where GitHub can render them. Setting it true kills every annotation while leaving the build green. One output-only harness change: every run states its cost in the job summary. The gate now sits on the critical path of every frontend merge, so the real runtime distribution matters; the harness already measured every run and simply never printed it. What it decides is unchanged. Known gap, accepted knowingly: there is no escape hatch. A frontend change that legitimately must add a violation cannot merge until it is fixed or the gate is switched off repository-wide. The spike measured 0 false positives across its corpus, so the expected frequency is low, and inventing a bypass before anyone needs one tends to produce the bypass everybody uses. Scope: CI only. No local git hook; core-web/.husky/ and lint-staged.config.mjs are untouched, so no contributor's local workflow changes. No new dependency. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
c7ada9a to
973da59
Compare
…typecheck-gate-into-ci-merged-on-main-but-never-executes
…hook reference (#37536) Addresses review on #37545. The cost line counted (config -> file) assignments, not changed files. `selectConfigs` claims a source under EVERY eligible config, so a project whose lib and spec configs both include a file produced two target entries for one changed file, and `targets[].files.length` summed them -- reporting a one-file diff as two. Unmapped files were dropped from the count entirely. Both defeat the only reason that line exists: to be the cost-versus-diff-size evidence for the runtime question. Now counts distinct paths across targets and unmapped. The original test never caught this because it used distinct files across targets. Two new cases pin the two failure modes directly. The job summary also never mentioned unmapped files, so a changed TypeScript file that no project compiles read as a clean pass with nothing said about it -- the edge case the spec names, and a worse one now the gate blocks. The summary now lists them with the reason, in both the passing and the failing branch. Not a failure signal: the difference between "nothing was wrong" and "nothing was looked at". Also removed a comment in core-web/pom.xml pointing at core-web/.husky/pre-push, a file this pull request does not add. It was left behind when the local hook was cut, and would have sent a developer looking for something that was never there. Reconciled two related staleness bugs: the harness README called itself "not production tooling" next to a section saying it is wired and live, and data-model.md documented `report.files.length` as the diff-size source when buildReport exposes no such field. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Thanks — triaged all of it. Fixed in db56f5b. Important context for anyone reading the bot reviews above: both were produced against earlier commits. Copilot reviewed the first push, which still contained Fixed🟠 Dangling 🟡 You were also right that the existing test never exercised it — it used distinct files across targets. Two new cases pin the two failure modes directly, and both were confirmed failing before the fix. 🟡 🟡 🟡 README contradiction — correct. "This is spike output, not production tooling" sat directly above a section saying it is wired and live. Reconciled. Not changed, with reasons🔴/🟡 Two things make me want to leave it: the property is the pre-existing 🟡 That is a real concern and it is exactly what the harness's own design notes worry about. I am leaving it because changing the shape of the annotation output is more than the output-only carve-out this PR is operating under, and it is a harness design question from #37403 rather than something this wiring introduced. Worth its own issue if it bites. Verification
|
…omments (#37536) Second round of review on #37545. Nothing here changes the gate's decision logic; it is all output correctness, test strength, and comments that taught mechanisms that do not exist. The test suite was weak in a way worth describing, because the reviewer found it by mutation testing rather than by reading. Four separate mutations of the implementation passed the whole suite: replacing the config count with a distinct-project count, splitting the duration onto its own line (defeating the stated purpose of pairing it with the diff size), removing the blank line that makes the findings table render, and dropping the unmapped note from the findings branch entirely. The common cause was whole-document regexes: formatMarkdown returns a multi-line document, and the unmapped note renders the same number-then-"file" shape the assertions searched for, so an assertion could be satisfied by a line other than the one under test. One assertion -- `doesNotMatch(/\b2\s*file/i)` -- could never fail for the bug it targeted, because the real output reads "2 changed file" and `\s*` does not match " changed ". Assertions are now equality against an extracted cost line, which is how the sibling suites already work. Two real bugs came out of that. The sub-second test asserted a value three rounding steps clear of the boundary; `toFixed(1)` renders anything under 50ms as "0.0s", and the measured no-op path is ~175ms -- a factor of four, reachable on a faster runner. Sub-second runs now render in milliseconds. And `durationMs.total` was unguarded: formatMarkdown is exported and buildReport's default only fires on undefined, so a partial object rendered "NaNs" in the job summary. The cost line also read "Checked N changed file(s)" directly above a note saying some of them were not examined -- two adjacent lines asserting opposite things about the same file. The verb is gone; the count is what it always was. Three POM comments taught mechanisms that are not there: - the skip was described as trunk/nightly having an empty diff. It is by event name. Trunk and nightly also accept workflow_dispatch from a feature branch, nightly disables change detection entirely, and cicd_5-lts.yml runs the frontend suite on release-* pushes -- so "it would be a harmless no-op" is false for cases that are nonetheless skipped. - the timeout was described as failing through successCodes. The watchdog throws directly and never consults them, so adding a success code to "allow timeouts" would do nothing. - the "no required status checks" premise holds today, but the ruleset providing it is an org-level `2026-08-24_incident-response`, and a separate `Default Merge Queue` ruleset does define required checks with enforcement disabled. Whoever re-enables it should see that this design assumed the opposite. Also removed the plugin-level <configuration> duplicated into the validate profile: verified against `help:effective-pom` that the base <build> entry already supplies executable, workingDirectory, PATH and NODE_OPTIONS -- the copy omitted NODE_OPTIONS and forked the PATH comment, so a future edit to the base would have silently missed it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This is the finding that would have made the gate unusable, caught in review on #37545 and reproduced in this PR's own CI run (job 104166954128): strict-gate: warning — no merge base between 'origin/main' and head; comparing against the tip of 'origin/main' instead. The cause is structural, not incidental. cicd_comp_test-phase.yml checks out at fetch-depth: 1 and compensates with `git fetch --depth=1`, so both sides of the comparison are truncated, `git merge-base` cannot traverse, and the harness takes its documented base-tip fallback. That fallback is survivable for `nx affected` -- over-including projects for lint is harmless on a lint-clean main -- which is why it went unnoticed while that was the only consumer. It is not survivable for a BLOCKING strict check, because main is deliberately not strict-clean while #37198 is in flight: every commit main moved ahead of the branch gets attributed to the pull request and strict-checked, so authors would be rejected for violations they never wrote. The harness's own comment on that code path records it producing 50 findings, essentially none of them the branch's own. The spike's "0 false positives" figure was measured with a correct merge base and does not transfer. Fix: resolve the base from the event payload (pull_request.base.sha, or merge_group.base_sha), fetch that one commit shallow, and hand it to the gate through a new `strict.gate.base` property. With the base already AT the divergence point, the base-tip fallback stops being a degradation and becomes exactly right -- no deepening, no extra fetch cost. `strict.gate.base` is deliberately its own property rather than a change to ${git.origin.branch}: lint-test and format-test tolerate an approximate base, this gate does not, and widening the blast radius to the two neighbouring checks is not something this change should do quietly. It defaults to ${git.origin.branch}, so a local run behaves exactly as before. Side effect worth naming: this also settles the hardcoded-origin/main concern raised separately in review. A pull request targeting a branch other than main now compares against its own base rather than against main. If the base commit cannot be fetched the step warns and leaves the gate on origin/main, which is today's behaviour rather than a new failure mode. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Both of the things I owed you are moving. Probe: #37564 — draft, DO NOT MERGE. Carries this branch plus one deliberate Scope recorded on #37536 — #37536 (comment). It states plainly that the blocking flip was a direct decision during implementation and was not re-approved against the issue's stated scope, lists all four deviations from the acceptance list, and carries the merge-base finding including the part I think matters most for anyone reading the original evidence: the "0 false positives" figure was measured with a correct merge base and does not transfer to the configuration the gate was about to ship in. That framing is yours. Deliberately not in this pull request, tracked for a follow-up, each answered on its own thread: exiting Thank you for the mutation testing — four of my assertions were passing for the wrong reason and one could never have failed at all. That is not something I would have found by rereading. |
|
Probe evidence is in: #37536 (comment) (see the following comment). All three confirmed — base from the event payload, annotation anchored to #37564 closed and its branch deleted. One thing the probe turned up that I would not have predicted: the base-tip warning still fires, and now fires on every run, because both sides remain shallow so From my side this is ready. Holding for your call on whether any of the three follow-ups should block here. |
…7536) The gate was handed github.event.pull_request.base.sha as its base. GitHub pins that field when the pull request is opened and never moves it as the base branch advances, so on this branch it was 276 commits stale. CI checks out refs/pull/N/merge at depth 1, merge-base cannot traverse, and the gate fell back to diffing the stale base against that merge commit -- scoping in 190 files, 97 of them .ts/.html belonging to other people's merged work, and reporting ~50 pre-existing strict violations against this author. Use the checkout's own first parent instead. The merge commit CI builds has the current base tip as parent 1 and the branch as parent 2, so parent1..HEAD is exactly the pull request's contribution. Deepening the fetch by one makes that parent visible; requiring a second parent keeps it honest, since on a non-merge checkout HEAD^1 would be the previous commit rather than the base. Verified against the merge commit that failed (3568f08): the first parent is 6fe9551 and the scoped diff is precisely this pull request's 8 files, none of them .ts/.html, so the gate reports PASS. Re-running it with the stale base reproduces the CI findings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…typecheck-gate-into-ci-merged-on-main-but-never-executes
…typecheck-gate-into-ci-merged-on-main-but-never-executes
…typecheck-gate-into-ci-merged-on-main-but-never-executes
Proposed Changes
Fixes #37536. Supersedes #37448.
PR #37403 merged the strict-gate harness to
mainon 2026-09-08 and wired it into nothing. Since then no pull request has been annotated, reported on, or blocked by it, andmainhas kept accumulating non-strict TypeScript exactly as before.This wires it into CI as a blocking gate: a change that adds a strict-mode violation on a line it wrote does not merge.
core-web/pom.xmlstrict-gateexecution in thevalidateprofile + two activation profilescore-web/tools/scripts/strict-gate/lib/format.mjscore-web/tools/scripts/strict-gate/format.duration.test.mjscore-web/tools/scripts/strict-gate/README.mdCI only. No local git hook —
core-web/.husky/andlint-staged.config.mjsare untouched, so no contributor's local workflow changes. No workflow file changes. No new dependency.It runs in two contexts, and both are load-bearing
pull_request::error file=,line=,col=lines render inline on the changed lines, plus a job summary.merge_groupThat second row is the part worth reviewing. This repository declares no required status checks on
main— the ruleset requires a pull request, one approval, thread resolution and signed commits, and no checks. So a red check on a pull request does not by itself stop a merge. A job that fails in the merge queue ejects the pull request, and that does.Wiring only the
pull_requesthalf would have produced a gate that goes red and merges anyway. Trunk and nightly stay skipped —HEADequalsorigin/mainthere, so the diff is empty and the run could only ever pass.Maven property activation has no OR, hence two profiles rather than one condition. Activation reads the runner's own
GITHUB_EVENT_NAME, which is why no workflow file needs to change.Semantics
successCodeslists0and nothing else. Findings (1) and a harness that could not run (2) both fail. A gate nobody has to obey is a report; a gate that reports "clean" without having looked is worse than no gate, because it is indistinguishable from a clean pull request.<timeout>fails the same way, bounding the harness's unshallow git-fetch fallback (a full fetch here is ~1.1 GB / ~20 min).useMavenLoggeris pinnedfalsewith a comment, even though it is the default — that default is the only reason the::errorlines reach the log at column 0 where GitHub can render them. Setting ittruekills every annotation while leaving the build green.Known gap, accepted knowingly
There is no escape hatch. A frontend change that legitimately must add a violation — porting legacy code, an unavoidable third-party shape — cannot merge until the violation is fixed or the gate is switched off repository-wide. No label, no per-file suppression.
The spike measured 0 false positives across its corpus (11 findings across 5 pull requests, all real), so the expected frequency is low, and inventing a bypass before anyone needs one tends to produce the bypass everybody uses. If this bites, say so on #37536.
Reviewer check
Verified locally with the real CI command, on a branch carrying a deliberate
TS7006:BUILD SUCCESS, no gate output at allGITHUB_EVENT_NAME=pull_request::error …dot-utils.ts,line=270,col=35::at column 0, summary table writtenGITHUB_EVENT_NAME=merge_group, violation presentBUILD FAILURE—Failed to execute goal …:exec (strict-gate) … Process exited with an error: 1Activation measured with
help:evaluate -Dexpression=skip.strict.gate:GITHUB_EVENT_NAMEpull_requestfalsemerge_groupfalsepushtruescheduletruetrueThree deliberate deviations from the issue's acceptance list
lint-staged.config.mjshook, and no local hook of any kind.<skip>keys off its ownskip.strict.gate, not${skip.validate}. The issue asks for the latter verbatim. Deliberate: a blocking gate that the documented way to silence its two neighbours also silences is not much of a gate. The flip side is that a plain local./mvnw -Pvalidateruns lint and prettier but not this.Worth recording for whoever picks up a local hook later — the harness compares two committed points, so a commit-time hook would examine the previous commit and report "clean" on the very violation being committed. That was verified, not theorised (
checked 0 project config(s)against an uncommitted change). A local hook has to sit at push time, or the harness needs a notion of staged content. Reasoning preserved inspecs/37536-wire-strict-gate-ci/spec.mdunder C-001/C-002.Out of scope
Checklist
Additional Info
The gate runs — confirmed.
exec:3.0.0:exec (strict-gate)appears at line 1448 of job 104166954128, so the two-profile activation works and the harness is no longer dead code.But this run proves less than that sentence suggests, and one finding from review changed the picture materially. That same run showed the gate had no merge base and was comparing against the tip of
origin/main— which, withmaindeliberately not strict-clean, would have rejected authors for violations they never wrote. Fixed in 61f3fde by resolving the base from the event payload. Details on the review thread.Still owed before merge: a throwaway branch carrying a deliberate
TS7006, confirming exit 1, the annotation rendering inline on the diff, and themerge_groupbehaviour. This branch changes no.ts, so none of that is exercised yet.What lands on the team: from the merge onward, a frontend pull request with a strict violation on a line it wrote will not merge. The spike's rate was ~2 findings per frontend pull request, all of them real.
Follow-up owed on the issue: observed duration and diff size from at least five real pull requests. The gate now sits on the critical path of every frontend merge, so if the real distribution is worse than the corpus suggested (8.4–9.4 s average, 12 s tail), the conversation is about making the harness faster.
Known unrelated redness:
corpus.acceptance.test.mjsneeds network access to the GitHub API and carries machine-dependent timing assertions. It is red offline both before and after this change; the other 10 test files pass with zero failures.🤖 Generated with Claude Code