Skip to content

fix(ci): give the WebAssembly gate a measured budget, shared links and its own build cache - #1334

Merged
HuiJun merged 3 commits into
developfrom
fix/wasm-gate-shared-builds-timeout
Oct 21, 2026
Merged

HuiJun merged 3 commits into
developfrom
fix/wasm-gate-shared-builds-timeout

Conversation

@devin-ai-integration

Copy link
Copy Markdown
Contributor

What and why

The GitHub Actions job "WebAssembly gate" (tests/wasm) finishes in 7-11 minutes on every pull-request head, under go test's default -timeout 10m, so a run that lands on a slow hosted runner dies with panic: test timed out after 10m0s in whichever test is running — three unrelated heads failed that way on 2026-10-20/21, none on an assertion. The same gate is green on CircleCI, whose config already passes -timeout 30m.

Where the time goes, from the -v output of the last twenty GitHub runs (one representative 574 s run; the slowest passing run took 600 s of package time, the failing ones were killed at 600 s during TestWasmRuns):

step seconds
TestWasmBuilds/wasip1/tree_compiles (GOOS=wasip1 go build ./...) 123
TestWasmBuilds/wasip1/tree_vets (GOOS=wasip1 go vet ./...) 113
TestWasmBuilds/js/tree_compiles 87
TestWasmBuilds/js/tree_vets 115
the fourteen links (7 commands × 2 targets, 0.2-2.4 s each) 17
TestWasmRuns/wasip1 (relinked all 7 commands, then ran them) 22
TestWasmRuns/js (relinked all 7, ran them, gzip budgets) 40
TestBrowserREPLWalkthroughs (-tags sysml_re js build, walkthroughs, gzip budget) 54
TestRunnerDeadline and the dependency/parity tests 4

So 440 of the 574 s were compiling and vetting the two trees — work the Go build cache would absorb, except that every run of this job was cold: setup-go's cache: true entry is keyed on go.sum alone and shared by every Go job, and the job that finishes first saves it, so the entry never held the wasm objects (Cache hit occurred on the primary key …, not saving cache in every wasm-gate log). The gzip steps are small (5-10 s together). The duplicated linking cost about 17 s per run.

Changes:

  • A measured budget. make wasm-check now runs go test with -timeout $(WASM_GATE_TIMEOUT), 25 m by default: about twice the cold 7-11 minute run, and the Makefile comment says so. The GitHub job calls make wasm-check rather than its own go test line, so there is one budget, and the job's timeout-minutes rises from 20 to 30 so a timeout is go test's, which names the running test, not the runner's.
  • Each command linked once and shared. link now builds a given target, command and tag set once per run (a sync.Once per key, under a directory TestMain owns and removes) and returns the same binary to every test that asks; a test asking after a failed link gets the same failure report. TestWasmBuilds, TestWasmRuns and TestBrowserREPLWalkthroughs share the binaries, so the seven relinks per target in TestWasmRuns and the standalone -tags sysml_re build are gone. Run alone (-run TestWasmRuns), the run gate still links what it needs.
  • Every link is a timed step. TestWasmRuns/<target>/links the commands and TestWasmBuilds/js/links sysml -tags sysml_re are new subtests; link logs each link's duration. A future timeout therefore names the step.
  • The gate's own build cache. The job turns setup-go's cache off and restores ~/.cache/go-build and ~/go/pkg/mod from a wasm-gate-<os>-go<version>-<go.sum hash>-<sha> key by prefix, nearest first. Pull-request caches are private to their branch, so a new workflow, wasm-cache.yml, runs the same make wasm-check on every push to develop that touches Go sources and saves the cache it leaves behind (also after a red gate: the objects are as good). Each develop push saves one entry and the older ones age out under the repository's cache quota.

No assertion was dropped or weakened and both targets still build, vet, link and run.

In-flight overlap

Checked the 21 open pull requests and the 43 remote branches without a pull request (git ls-remote --heads origin, each diffed against origin/develop) for changes under tests/wasm/, to the wasm-gate job in .github/workflows/pr.yml, or to the Makefile: none touch the gate. #1324 edits pr.yml only to rename the MDK plugin job.

How it was verified

  • go build ./..., go vet ./..., gofmt -l . (empty), make lint (staticcheck + gosec), python3 scripts/changelog.py check.
  • make wasm-check with the new settings, warm cache: 118.9 s for the package; TestWasmRuns/<target>/links the commands and links sysml -tags sysml_re report 0.00 s because TestWasmBuilds linked the binaries first. go test -run 'TestWasmRuns|TestBrowserREPLWalkthroughs' alone: 67 s, with the links then done inside TestWasmRuns (the linked cmd/… in … log lines).
  • Before, on this machine with an empty GOCACHE: 290.6 s for the package (TestWasmBuilds 205 s), 102.6 s warm.
  • The GitHub Actions duration of this job on this branch, before and after the cache has a develop-side entry, is in the pull request comments.

Checklist

  • make lint passes locally; the suite touched (make wasm-check) passes with the new settings, the rest of make test runs in CI
  • Tests added or updated for the change
  • Documentation extended where it already covers the surface (see CONTRIBUTING.md)
  • Changelog entry added as changes/unreleased/<slug>.<section>.md, not as an edit to CHANGELOG.md
  • baselines regenerated and make docs-counts run if a gate count moved (compliance rows need nothing: the census is counted at docs build)
  • No internal work-item labels (waves, slices, F4, K5) in the body, docs, or changelog

Link to Devin session: https://nasa-jpl-demo.devinenterprise.com/sessions/cf14ed1555b240a3be47852cfa6e3f80
Open in Devin Desktop: https://nasa-jpl-demo.devinenterprise.com/desktop/session/cf14ed1555b240a3be47852cfa6e3f80?variant=devin
Requested by: @HuiJun

…d its own build cache

The gate compiles and vets the whole tree for both wasm targets, 7-11 minutes on a hosted runner, under go test's default ten-minute timeout, so a slow run died with "test timed out after 10m0s" in whatever test was running. make wasm-check now sets WASM_GATE_TIMEOUT (25m, about twice the cold run) and the job's limit sits above it. Each command is linked once per run and shared by TestWasmBuilds, TestWasmRuns and the browser REPL walkthrough, every link is a timed subtest, and the GitHub Actions gate restores its own Go build and module cache, saved on each develop push by wasm-cache.yml, instead of setup-go's shared entry, which another job saved without the wasm objects.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

WebAssembly gate durations on GitHub Actions, -v package times:

run cache TestWasmBuilds TestWasmRuns browser REPL package job
before, typical passing head (job 118935747922) setup-go's shared entry, no wasm objects 455 s 62 s (14 relinks included) 54 s 574 s 10 min 05 s
before, the slowest heads same 477 s killed at 600 s — timed out at 600 s —
this head 4adcf76 (job 118974785266) own key, nothing to restore yet 291 s 30 s (links the commands 0.00 s) 33 s 357 s 7 min 01 s

The run on this branch is still a cold compile — pull-request caches are private to their branch, so the first wasm-gate-… entry appears when wasm-cache.yml runs on develop after the merge; the drop from 574 to 357 s is a faster runner plus the shared links, not the cache. Locally the same cache takes the package from 290.6 s cold to 118.9 s warm. The budget is 25 min against a 6-11 min cold run, so the slowest runner observed (600 s+) keeps more than twice its time.

@devin-ai-integration
devin-ai-integration Bot marked this pull request as ready for review October 21, 2026 01:36
devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration Bot and others added 2 commits October 21, 2026 01:38
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
@HuiJun
HuiJun merged commit fce39e1 into develop Oct 21, 2026
27 checks passed
@HuiJun
HuiJun deleted the fix/wasm-gate-shared-builds-timeout branch October 21, 2026 06:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant