Skip to content

Run the PR gate's workspace Rust tests through cargo-nextest - #698

Draft
ciaranra wants to merge 2 commits into
devfrom
ci-rust-gate-nextest
Draft

Run the PR gate's workspace Rust tests through cargo-nextest#698
ciaranra wants to merge 2 commits into
devfrom
ci-rust-gate-nextest

Conversation

@ciaranra

@ciaranra ciaranra commented Sep 4, 2026

Copy link
Copy Markdown
Member

Draft until the local full-lane run and the CI timing are in.

Why

After #683/#684 the Python side of the PR gate finishes in ~14 min and pr-core-rust (~30 min) is the critical path. Inside that job the workspace test phase is 11 min, and the per-binary times from a green run add up to 656 s across 323 test binaries with the longest at 314 s: cargo test runs the binaries one after another, so the phase is serial regardless of how many cores the runner has. nextest schedules tests from all binaries at once, so on the 4-core runner the phase is bounded by the longest single test (~5 min) instead of the sum.

What

  • scripts/ci/ensure-nextest.sh: installs the pinned cargo-nextest 0.9.143 prebuilt release for x86_64 Linux into the cargo bin directory, verified against the sha256 nextest publishes with the release, idempotent, curl retries. Any other platform fails loudly. A cargo install would spend minutes compiling on every run and cache-bin: false keeps it out of the Rust cache.
  • pecos rust test --nextest: new flag. The workspace phase runs cargo nextest run followed by cargo test --doc, both with exactly the same package selection and features (nextest does not run doctests). The three follow-up phases (pecos-cli with runtime, zlup with cli, pecos-decoders) are unchanged. The invocation list is built by a small pure function with unit tests pinning the nextest/cargo profile mapping: nextest's own --profile selects a nextest profile, so the cargo profile goes through --release / --cargo-profile native.
  • just rstest <mode> <runner>: runner is cargo (default, unchanged) or nextest. Local use is untouched.
  • pr-core-gate.yml: installs nextest after Rust and runs just rstest debug nextest.

Verification

  • cargo clippy -p pecos-cli --all-targets -- -D warnings, cargo fmt --check, the two new unit tests, pre-commit on all changed files: clean.
  • scripts/ci/ensure-nextest.sh: fresh install into an empty CARGO_HOME succeeds and prints the version; a second run is a no-op; a copy with a wrong pinned checksum fails at sha256sum -c and installs nothing.
  • Full just rstest debug nextest on a cold worktree (14 cores, RTX 4090 present): workspace phase Starting 10352 tests across 270 binaries (60 tests skipped) -> 10352 passed, 60 skipped in 524 s, then cargo test --doc over the same 51 doc-test binaries, 818 doctests passed; the pecos-cli, zlup and pecos-decoders phases passed. The only failure was in the GPU-only pecos-gpu-sims phase, which this machine runs because it has a GPU and CI skips (GPU not detected): gpu_density_matrix_two_qubit_roots_preserve_rotation_channels fails on the untouched dev base too (GpuDensityMatrix64 SXX: max density-matrix error 1.5e-7 exceeds 1e-12), and that phase still runs through plain cargo test. Unrelated to this change; reported separately.
  • CI timing of pr-core-rust on this PR (run 33841091323, warm Rust cache): 25 min total; compile 9m02, nextest phase Summary [626.409s] 10352 tests run: 10352 passed, 60 skipped, doctests 7 s, follow-up phases 3 min.

Result: no gain, recommend closing

The serial cargo test phase was 656 s; nextest is 626 s. The per-test times nextest reports add up to 2486 CPU-seconds, which on a 4-core runner is 10.4 min regardless of scheduling, and neo_surface_ler_test alone takes 560 s while everything else runs alongside it (314 s when it had the machine to itself). cargo test was already saturating the cores because tests within each binary run on all threads, so running binaries one after another cost nothing. The workspace test phase is CPU-bound, not scheduling-bound, and the ~5 min the job did lose came from the now-warm Rust cache (#683), not from nextest. Adding a pinned tool and an install step for a 30 s difference is not worth carrying; the levers that would move this job are the CPU cost of the debug-profile simulators under test (e.g. [profile.test] opt-level) or sharding the job. Left as a draft for the record.

Review

Independent correctness review of the first commit: no blocker. It confirmed the workspace selection is identical between cargo test and cargo nextest run + cargo test --doc (ignored tests stay ignored, no non-libtest harness targets in the selection, required-feature targets keep cargo's selection, the later pecos-cli phase stays authoritative for the pecos binary), that --locked / --release / --cargo-profile native are valid for nextest 0.9.143 and that --cargo-profile native lands in target/native/ like the doctest command after it, and that the default cargo runner is unchanged. One robustness finding, fixed in the second commit: the installer wrote to ${CARGO_HOME}/bin but only ensure-rust.sh put a cargo bin directory on PATH, and that one is $HOME/.cargo/bin regardless of CARGO_HOME; the installer now appends its own directory to GITHUB_PATH on both the fresh and already-installed paths (verified with a scratch CARGO_HOME and GITHUB_PATH).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant