Skip to content

Commit 5836083

Browse files
dmealingclaude
andcommitted
ci(conformance): the second environment only looked at a cut
The cross-port conformance workflow ran on release tags and manual dispatch only. That was a cost decision made when push-to-main coverage moved to the self-hosted local-ci runner, and on scope it is sound — local-ci's `java-slow` lane runs the same full reactor (`gate_java_reactor`, `mvn clean install`), so "the reactor is not run between releases" was never true. The gap is ENVIRONMENTAL, and it is measured. The two Java test defects fixed in 1c24b8f and cd01580 were both found by the `v1.0.0-rc.5` tag's run of this workflow. The 08:17 nightly local-ci run that preceded both fixes reported `java-slow: success` with both defects live on `main`. One asserts on an NPE message HotSpot's fast-throw is entitled to stop producing once the path is hot; the other is two test classes sharing files with nothing declaring their surefire order. JIT heat and filesystem ordering are precisely the defect classes a single environment cannot see on its own — and the two runs differ in exactly that: hosted ubuntu-latest with jacoco ON here, self-hosted with `-Djacoco.skip=true` there. So the second environment existed, and a tag-only trigger meant it looked only at a cut, which is the most expensive moment to find out. A nightly schedule at 04:41 UTC (offset from local-ci's 08:17 so the two full-reactor runs do not overlap) makes it look every day. Standard runners are free for a public repo; the second look costs wall-clock and nothing else. Both live statements of the old trigger set are corrected in the same commit — AGENTS.md's CI paragraph and fixtures/validation-conformance/README.md — so the workflow and its documentation cannot disagree about when it runs. The api-contract README's equivalent sentence is about integration-tests.yml, which is unchanged, and the plan/spec/CHANGELOG mentions are historical records. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NTcEKXTQMYt84fAjuw5A2M
1 parent cd01580 commit 5836083

3 files changed

Lines changed: 26 additions & 6 deletions

File tree

.github/workflows/conformance.yml

Lines changed: 22 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,36 @@
11
name: conformance
22

33
# COST: the cross-port conformance matrix + full Java reactor + drift/mutation gates
4-
# are heavy, so they NO LONGER run on every push/PR — release gate (tag `v*`) +
5-
# on-demand (workflow_dispatch) only. Run them LOCALLY before opening/merging a PR:
4+
# are heavy, so they do NOT run on every push/PR — release gate (tag `v*`), a NIGHTLY
5+
# schedule, and on-demand (workflow_dispatch). Run them LOCALLY before opening/merging
6+
# a PR:
67
# scripts/ci-local.sh # full parity (this + integration suite + drift)
78
# scripts/ci-local.sh --quick # everything except the docker integration suite
89
# The .githooks/pre-push hook also runs the TS build+typecheck gate locally, and the
910
# cheap public-repo SECURITY gate (hygiene / leak-scan) still runs on every PR.
10-
# Push-to-main coverage now comes from local-ci.yml on the self-hosted runner.
11+
# Push-to-main coverage comes from local-ci.yml on the self-hosted runner.
12+
#
13+
# WHY A SCHEDULE, given local-ci already runs nightly: local-ci's `java-slow` lane runs
14+
# the same full reactor (`gate_java_reactor`, `mvn clean install`) — so "it does not run
15+
# the reactor" is NOT the gap. The gap is environmental, and it is measured. Two Java
16+
# test defects (a test asserting on an NPE message HotSpot's fast-throw is entitled to
17+
# stop producing once the path is hot; two test classes sharing files with nothing
18+
# declaring their surefire order) were found by the `v1.0.0-rc.5` tag's run of THIS
19+
# workflow and fixed in 1c24b8f9d / cd0158073. The 08:17 nightly local-ci run that
20+
# preceded both fixes reported `java-slow: success` with both defects live on `main`.
21+
# The two runs differ in environment, not in scope — hosted ubuntu-latest with jacoco ON
22+
# here, self-hosted with `-Djacoco.skip=true` there — and JIT heat and filesystem
23+
# ordering are exactly the defect classes one environment cannot see on its own. A
24+
# tag-only trigger meant the second environment looked only at a cut, which is the most
25+
# expensive moment to find out. Standard runners are free for a public repo, so the
26+
# second look costs nothing but wall-clock.
1127
on:
1228
push:
1329
tags:
1430
- 'v*'
31+
# Offset from local-ci's 08:17 so the two full-reactor runs do not overlap.
32+
schedule:
33+
- cron: '41 4 * * *'
1534
workflow_dispatch:
1635

1736
jobs:

AGENTS.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -116,7 +116,7 @@ bypasses branch protection. This hook closes that hole locally: when a push touc
116116
… typecheck` gate CI runs and **blocks the push when it is red** (~6s on a clean tree;
117117
skipped entirely for non-TS pushes). Bypass in an emergency with `git push --no-verify`
118118
or `SKIP_TS_TYPECHECK=1 git push`. The Java/C#/Python compile+conformance gates do NOT run on PRs (hosted CI runs
119-
them on release tags + manual dispatch only, for cost). Instead, every push to
119+
them on release tags, a nightly schedule, and manual dispatch, for cost). Instead, every push to
120120
`main` triggers `local-ci.yml` on the maintainer's self-hosted runner: affected
121121
ports only (via `scripts/ci-affected-ports.sh`), parallel per-port jobs, each
122122
running `scripts/ci-local.sh --only <port> --strict-toolchains`; a nightly

fixtures/validation-conformance/README.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -158,8 +158,9 @@ non-Docker conformance job) — TS/C#/Java/Python under the `conformance` matrix
158158
Kotlin under `conformance-kotlin` — asserting byte-identical boolean verdicts
159159
across all five generated validation artifacts.
160160

161-
That workflow runs on release tags and on `workflow_dispatch`, not on every PR
162-
(pull requests get the leak scan only, for hosted-minutes cost). Push-to-`main`
161+
That workflow runs on release tags, nightly on a schedule, and on
162+
`workflow_dispatch` — not on every PR (pull requests get the leak scan only, for
163+
hosted-minutes cost). Push-to-`main`
163164
coverage comes from the self-hosted `local-ci.yml`, and `scripts/ci-local.sh`
164165
reproduces the gate locally. See [`docs/CONFORMANCE.md`](../../docs/CONFORMANCE.md).
165166

0 commit comments

Comments
 (0)