Repository navigation
Clippy linting is very slow #528
Description
Activity
Important Note: You have a quite custom build script. Don't run
cargo gpu clippyor anything like this, instead, the build script will delegatecargo clippyto also run clippy with rust-gpu, and otherwise build with rust-gpu like normal.- methodology:
- tested on nazar-pc/abundance@e69b77a
- Hardware: DGX Spark with 20 arm cores on stock ubuntu nvidia variant
- clean between every run
- cargo-gpu has a cached build of
rustc_codegen_spirv - NOT using cargo-gpu, but regular cargo build / clippy and have the build script delegate the rust-gpu build / check / clippy
- build is a dev build (not release)
- testing on
ab-proof-of-space-gpu:- build: 32.73s
- clippy with rust-gpu check (before PR): 16.92s
- clippy with rust-gpu clippy (after PR): 16.97s
- testing on
ab-farmer:- build: 1m 04s
- clippy with rust-gpu check (before PR): 32.75s
- clippy with rust-gpu clippy (after PR): 32.70s
- Notes:
- build fails, was missing
libtool, docs appreciated :D - check variant slowest component is compiling
hwlocality-sys, not rust-gpu!
- build fails, was missing
- regression test on
ab-farmerwith older Rust-GPU/cargo-gpu@7016069 before 2 PRs- requires backporting the build script due to some API changes, but should be otherwise equivalent
- build: 1m 01s
- clippy with rust-gpu check (before PR): 32.18s
- clippy with rust-gpu clippy (after PR): 32.42s
- regression test on
ab-farmeron commit nazar-pc/abundance@87fe6a8- build: 59.63s
- no check or clippy available
TLDR: not really able to figure out what's wrong on my local machine
- methodology:
At least on Windows there is certainly a massive difference.
To demonstrate I made a simple change on top of
main: nazar-pc/abundance@93cdea0This is the latest
mainbranch wherewindows-2025 individuallyjob takes 20m 59s to complete: https://github.com/nazar-pc/abundance/actions/runs/21930624164
This is another run with above commit on top wherewindows-2025 individuallyjob takes 9m 51s: https://github.com/nazar-pc/abundance/actions/runs/21946267916There is literally a single line change that either calls
SpirvBuilder::clippy()or doesn't.P.S. As for hwlocality - it is awesome, but hwloc C dependency takes a while to compile indeed with extra environment dependencies. I hope to replace it with pure Rust https://github.com/WildPixelGames/gdt-cpus once it matures with more features. But either way my concern is only about rust-gpu causing CI delays on top of whatever baseline is.
I guess downloading/installing of the custom toolchain is another difference on top of running clippy as such. And Windows seems to be particularly horrible with the file system operations, but it still doesn't take quite as long to install the toolchain of the workspace itself.
Are you comparing running clippy with rust-gpu toolchain vs
A) not running anything or
B) running cargo check with rust-gpu?Cause I've been testing for B, but your latest comment sounds like you're testing for A.
Also are you caching
~/.cache/rust-gpu/codegen/*/*.so, so the codegen backend doesn't need to be rebuild every time?I'm always calling
cargo clippywith the up to date nightly toolchain. Internally insidebuild.rsit either callsSpirvBuilder::clippy()or it doesn't. I never callSpirvBuilder::check()because it is not distinguishable from regular build insidebuild.rsto the best of my knowledge.The only things cached in CI are
~/.cargo/registryand~/.cargo/git.Sidenote: I've just released spirv-tools 0.13.1 which may affect future measurements
In commit nazar-pc/abundance@54bff2f you're (effectively) doing
cargo gpu checkin your build script and CI times have already worsened to 20minMeaning this is not a clippy specific problem. The problem is that you're now building and using rust-gpu in your clippy job, which is making it take ~10min a lot longer. cargo-gpu CI also takes 10min to run on github provided runners, so I'd call this very much expected.
Interesting. Why does it need to take a whopping 10 minutes to build though? Is it compiling a huge amount of code with crazy optimizations or something?
I guess I can try caching the codegen backend so it is not rebuilt every time, but at the same time 10 minutes is a reeeally long time.
When I do a local build on my old Notebook,
spirv-toolsC++ compile alone takes ~2min and all of rust only takes 1min. Which is why I started to question whether we actually needspirv-tools, sincespirv-optisn't that great andspirv-valis in theory optional. And what is driving #521Would it be possible to skip building C++ stuff when only clippy is called? There would be no SPIR-V produced in that case anyway.
Well we used to have such an optimization... and I just patched it out:
[0.13.2] - 2026-02-16
- PR#26 never skip C++ compile on clippy, caused issues downstream
Comment on that issue:
Projects that use
spirv-builderdirectly from a build script fail. See https://github.com/Rust-GPU/rust-gpu-template/actions/runs/22034408133/job/63664791318?pr=12I was throwing out the idea of having spirv-tools be feature gated, but that would also require cargo-gpu support for codegen backends with various features. Currently, it just builds all codegens with default features. But even if you have separate features, you'd be forcing everyone who already has a codegen backend with spirv-tools to compile it again without spirv-tools. And we can't coerce the two builds, since spirv-opt affects build output significantly.
Isn't spirv-opt a CLI program? If so then it should be possible to compile it optionally, while the codegen backend has its support built-in. If it is compiled as a library then it is a bit of a problem though.
True, and
spirv-toolsdoes have the two featuresuse-compiled-toolsanduse-installed-tools, which we make use of in rust-gpu CI. The main problem is that cargo-gpu does not support forwarding features to the codegen backend, and may need a new caching strategy to support that. For now we've decided to hardcode the default features (aka.use-compiled-tools), which also got rid of issues where people used newer spirv tools that introduced some new validation, suddenly making their compiles fail.
I switched to using clippy for linting GPU code in CI and I think it increased CI time by ~8 minutes.
I don't think it should take 8 minutes to run
cargo clippythough.Here is CI run where
build.rsfor GPU crate was skipping all the work under Clippy: https://github.com/nazar-pc/abundance/actions/runs/21787689837/job/62861720690#step:11:1022Here is CI run where
SpirvBuilder::clippy()is called: https://github.com/nazar-pc/abundance/actions/runs/21905750349/job/63245490660#step:11:1023Clippy time for
ab-farmercrate that includesab-proof-of-space-gpuas a dependency increased from ~2 minutes to over 10 minutes.Dependencies for SPIR-V are fairly minimal too and should not take nearly as much time to compile and lint: https://github.com/nazar-pc/abundance/blob/c55eae7482783aea52f0f1e9510e71378c9f10b8/crates/farmer/ab-proof-of-space-gpu/Cargo.toml#L19-L23
I do not know what could it possibly be doing for such a long time, but I do not like it.