fix(release): wait for npm before changeset publish, and decide the EQL assets from npm and the tags - #1026
Conversation
|
freshtonic
left a comment
There was a problem hiding this comment.
I reviewed cb58a7b9...d7a13ff8. The change is correct, and I approve it.
What I checked
- The
publishedPackagesclaim is true. Inchangesets/action@v1.9.0,runPublishcallsgetExecOutputwithignoreReturnCode: true. It parses theNew tag:lines, andindex.tssetspublishedandpublishedPackagesbefore it examines the exit code. Thus a failedchangeset publishstill exports the packages that it published. Also,createGithubReleasesis on by default, so the action pushes the@cipherstash/eql@<version>tag thateql-assetsreads. - The wait asks the same question as
changeset publish.npmVersionsrunsnpm view <name> versions --json. It returnsnullon E404, and the wait retries on a 404. A re-run also exports the versions that were published before, but npm lists them at once, so they do not cause a delay. - No two runs can decide at the same time.
concurrency: ${{ github.workflow }}-${{ github.ref }}queues the runs and does not cancel them. Thus two pushes cannot both findneeded=trueand both makeeql-<version>. latestcannot move back to an older version.eql-assetsreads only the tree's EQL version, which is the newest final version onmain. A repair cannot dispatchupdate_floating_tags=truefor an older version.- The job graph is correct. All four EQL jobs use
!cancelled()and name the results that they need.eql-docsandeql-imagealso requireeql-sqlandeql-docsto succeed. The job-graph model in the tests copies the transitivesuccess()rule from actions/runner#2205. The "cancelled duringrelease" case is a good test of the choice of!cancelled()instead ofalways(). - Inputs to the tag and dispatch are safe.
VERSIONis validated before it goes into a tag name or a-fargument.refcomes from the GitHub API.eql-assetsinheritscontents: readand requests noid-token. - Changeset and skills are not necessary. The change is to repository tooling only.
AGENTS.mdhas the two new notes.
A gap to know about (not a blocker)
The repair key is "the eql-<version> tag does not exist". _build-eql-sql.yml makes the tag and the release in one softprops/action-gh-release step. If eql-sql succeeds and eql-docs or eql-image then fails, the tag exists. No later push builds the missing docs or image. A re-run of the failed jobs in that run still works, because the eql-assets outputs stay. A later push is not a recovery path for those two jobs. This is a smaller window than the fault that this PR fixes. A short sentence in the eql-assets comment or in the AGENTS.md note can tell the next person to re-run the failed jobs, and not to wait for the next push. Write it in this PR or in a follow-up.
The PR body also identifies the open GHCR write-access risk for eql-image. I agree that it is outside this change.
Move the GitHub Actions expression evaluator out of workflow-dispatch-job-conditions.test.mjs into lib/expressions.mjs, so a second guard can walk the release job graph with the same reading of a condition rather than a copy of it. Model `cancelled()` from a run state the caller passes, false by default. The release EQL jobs are about to be written `!cancelled() && ...`, and the evaluator must not refuse them. `success()` and `failure()` still throw: they depend on the whole needs chain, which these contexts do not carry. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PaY5xYydZUWhv8Nex9Sw8a
…et publish publish-ffi and publish-auth publish their seven tarballs each with `npm publish`. The release job then runs `changeset publish`, which publishes every version `npm info` does not list. npm lists a publish minutes after accepting it, so changesets published some native packages a second time, from the workspace and with restricted access, and npm refused them with E402. That failed the release job for protect-ffi 0.33.0 (run 36978691809) and @cipherstash/auth 0.44.1 (run 37071820363). Both publish jobs now export the name@version list they write to published.txt. A new step in the release job, before changesets/action, polls `npm view <name> versions` until every listed version appears, for up to 15 minutes. A skipped publish job exports an empty list, so an ordinary JS release does not wait. A timeout fails the job before `changeset publish`, so nothing is published, and a re-run waits again. Refs: CIP-4276 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PaY5xYydZUWhv8Nex9Sw8a
The SQL, docs and image jobs ran only when the release job's `eql_published` output was true. A step after `changeset publish` set it, so when the publish failed the step never ran, even though EQL had been published, and a re-run found nothing to publish. That is why EQL 3.0.6 has no eql-3.0.6 tag, GitHub release or image. A new eql-assets job runs scripts/eql-release-assets.mjs after the release job, whatever its result. The assets are owed when npm carries the tree's EQL version and the eql-<version> tag does not exist. This run's publishedPackages also counts, because npm lists a new version minutes after accepting it; the release job now exports it straight from the changesets step. The assets are built at the commit the @cipherstash/eql@<version> tag names, which is where npm's tarball came from, so a later run repairs an old release from that release's source. The four EQL jobs now use `!cancelled()` instead of the implicit `success()`. That is false when any job up the needs chain was skipped (actions/runner#2205), and publish-ffi and publish-auth are skipped on most releases, so an EQL release without an FFI and an auth release in the same run would also have skipped every EQL asset job. Refs: CIP-4276 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PaY5xYydZUWhv8Nex9Sw8a
d7a13ff to
d4a9555
Compare
This pull request fixes the two faults in
release.ymlthat Linear issue CIP-4276 describes. It also fixes a third fault that would have skipped the EQL assets on most releases. Once it merges, the next push tomainbuilds the missing EQL 3.0.6 tag, GitHub release and image.The release job now waits for npm before
changeset publishchangeset publishis the Changesets command that publishes every workspace package whose version npm does not list yet. Thepublish-ffiandpublish-authjobs publish the native packages first, withnpm publish. npm accepts a publish a minute or more before its package document lists the new version. The package document is npm's record of a package, with its list of versions.So
changeset publishsaw some native versions as unpublished and published them again. It usesrestrictedaccess, so npm refused each one withE402 Payment Required, and thereleasejob failed. This happened in Release JS run 36978691809 for protect-ffi 0.33.0. It happened again in Release JS run 37071820363 for@cipherstash/auth,auth-darwin-x64andauth-win32-x64-msvc0.44.1. Both runs were on 2 October 2026.Now both publish jobs export a
publishedoutput, which holds the samename@versionlist that they write topublished.txt. A new step in thereleasejob runsscripts/wait-for-npm-versions.mjsbeforechangesets/action. The script asksnpm view <name> versionsfor each package until npm lists every version, for up to 15 minutes. That is the same question, through the same npm, thatchangeset publishasks next.A skipped publish job exports an empty string, so the script returns at once for an ordinary JavaScript release. A registry error during the wait is retried until the time limit. If the wait times out, the job fails before
changeset publish, so nothing is published, and a re-run waits again.The EQL asset jobs now ask npm and the tags
Three jobs build the EQL assets: the
eql-3.0.6tag and GitHub release with the SQL bundle, the docs bundle, and thepostgres-eqlimage. They ran only when thereleasejob seteql_published. A step afterchangeset publishset it, so a failed publish never set it, even when the publish had already reached npm. A re-run found nothing left to publish, so it did not set it either.A new job,
eql-assets, now runsscripts/eql-release-assets.mjsafter thereleasejob, whatever its result. The script reports that EQL needs its assets when npm has the tree's EQL version and theeql-<version>tag does not exist. It also counts@cipherstash/eqlin this run'spublishedPackages, because npm can take minutes to list a version that it just accepted. Thereleasejob exportspublishedPackagesstraight from the Changesets step, so the output survives a failed publish.The script builds the assets at the commit that the
@cipherstash/eql@<version>tag names.changesets/actionpushes that tag at the commit it published from. So a run that repairs an older release builds the source that npm shipped, not whatevermainholds now. If npm has a version and neither tag nor this run says which commit published it, the script fails and asks for a manual build.The script also reports nothing needed while EQL is a frozen publisher in
scripts/release-gate.mjs. Without that check, it would fail every push tomain, because a frozen EQL publishes from another repository and leaves no Changesets tag here.The EQL jobs no longer depend on every earlier job succeeding
GitHub adds a hidden
success()check to a job condition that calls no status function. That check is false when any job anywhere up theneeds:chain was skipped or failed, as actions/runner issue 2205 records.publish-ffiandpublish-authare ancestors of the EQL jobs, and most releases skip at least one of them.The new tests include a job-graph model: it decides which
release.ymljobs run, given the results of the jobs before them. I ran it overmain'srelease.ymlin three cases. An EQL release with no FFI or auth release in the same run skipped all three EQL asset jobs. Only a run that also released both FFI and auth would have built them.The four EQL jobs now start with
!cancelled()and name the results they need, such asneeds.eql-sql.result == 'success'. I chose!cancelled()overalways()so that a person who cancels the run still stops these jobs.The next push to main builds the EQL 3.0.6 assets
Merging this pull request is a push to
main, so it startsRelease JS. Unless another push lands first, that run does the following.gatejob finds nothing unpublished, sopublish-ffiandpublish-authare skipped. The wait step gets two empty lists and returns at once.releasejob runschangesets/action. With no changesets onmain, it publishes nothing.eql-assetsjob reads version 3.0.6 frompackages/eql/packages/eql/package.json. It finds noeql-3.0.6tag, finds 3.0.6 on npm, and finds@cipherstash/eql@3.0.6at23e9af3f. That commit is the merge of Version Packages pull request Version Packages #938. It reportsneeded=trueandref=23e9af3f.eql-sqljob checks out23e9af3fand runsmise run --force build --version 3.0.6. It creates the tageql-3.0.6at23e9af3fand a GitHub release that is not a prerelease. The release holdscipherstash-encrypt.sqlandcipherstash-encrypt-uninstall.sql.eql-docsjob builds the docs at23e9af3fand attaches the.zipand.tar.gzbundles to that release.eql-imagejob dispatchesrelease-postgres-eql-image.ymlagainst the tageql-3.0.6, withupdate_floating_tags=true. That workflow builds images for PostgreSQL 14, 15, 16 and 17, tagged<pg>-3.0.6and<pg>. It then pointslatestand3.0.6at the PostgreSQL 17 image. Today the newest image tags in the registry are for 3.0.5.I ran
scripts/eql-release-assets.mjsread-only against npm and GitHub on 3 October 2026. It printed@cipherstash/eql@3.0.6 is on npm and eql-3.0.6 does not exist; building at 23e9af3f8e28c6d0495fdafcff096f58cbf0f4f7.Between
23e9af3fandmain, the three EQL workflows are identical, andpackages/eqldiffers only in itsAGENTS.md. The tageql-bindings-v3.0.6, which release-plz made for the crate, also points at23e9af3f. So the new tag, the npm package and the crate all name the same commit.One risk is outside this change.
AGENTS.mdlists write access toghcr.io/cipherstash/postgres-eqlfrom this repository as an open cutover step, because the package is still linked tocipherstash/encrypt-query-language. If that access is missing, the image workflow fails when it pushes. The SQL and docs release would still exist, and a person can dispatch the image workflow again after granting access.Tests cover each case the issue names
changeset publish: the job-graph model runseql-sql,eql-docsandeql-imagewhenreleasefails. The script owes the assets when this run published EQL and npm does not list it yet.npmandghreplaced onPATH.FFI_PUBLISHEDempty.scripts/__tests__/lib/expressions.mjsnow holds the GitHub expression evaluator thatworkflow-dispatch-job-conditions.test.mjsused. The new job-graph tests use the same evaluator, and it now modelscancelled().AGENTS.mdgains a short note on each fix.Each new test fails when the code it covers breaks
A mutation check breaks the code on purpose and confirms that a test fails. I ran 14 mutations, and each one failed at least one test. I restored the code after each one.
releasepublish-authexports no listeql-<version>tag checkeql-sqluses the hiddensuccess()eql-assetsuses the hiddensuccess()eql-assetsusesalways()releasepassespublishedPackagesthrough a later stepChecks
pnpm test:scripts: 64 files and 1,193 tests pass, with 1 skipped as onmain.pnpm run code:check: no errors, and no new warnings.actionlint1.7.7 withshellcheck0.11.0 on the release workflows: no findings.lint:workflow-cache,lint:runnersandlint:package-paths: all pass.Linked issues
This PR fixed #1035.
🤖 Generated with Claude Code
https://claude.ai/code/session_01PaY5xYydZUWhv8Nex9Sw8a