CI: ccache the SD CUDA leg

[danielhanchen]

Problem

Run 31289281107 took 72m43s. One job is the whole run:

job	duration
Linux-Ubuntu-22.04-x86_64-cuda12	4121s (68.7 min)
Darwin-macOS-x86_64	424s
win-cpu-x64	219s
Darwin-macOS-arm64	199s
Linux-Ubuntu-24.04-aarch64	189s
Linux-Ubuntu-22.04-x86_64	175s
Assemble + publish	116s
Resolve tag + stamp source	36s

Inside that job, one step: Build sd-cli + sd-server (CUDA) at 3903s. Toolkit install was 82s, packaging 57s, upload 31s.

The cause is that this pipeline has no ccache anywhere. unsloth-sd-prebuilt.yml has zero mentions of ccache, COMPILER_LAUNCHER or actions/cache, so the CUDA leg recompiles all seven architectures from scratch every time. The only ccache in the repo is in build.yml, keyed windows-rocm-* and ubuntu-rocm-cmake-*.

Three things confirm it rather than one:

• the repo's Actions cache list has 20 entries totalling 9.68 GiB, and every one is a ROCm key from build.yml. Nothing was ever written for this job.
• consecutive runs show no warm-cache effect: 31271264076 took 4942s, then 31289281107 took 4121s.
• at 9.68 GiB the repo is far from the 50 GB cache limit, so this is not eviction. It simply never caches.

Change

Ports the pattern already used by the llama.cpp CUDA legs, which run 407-665s each with 90-94% hit rates on a larger codebase.

• ccache action pinned to the same SHA the llama.cpp workflows use, max-size: 2G.
• The key carries the CUDA version and the architecture list, because both decide the objects. ccache hashes compiler identity into every entry, so a cache written by another toolkit can never hit. A version-less key is exactly what held the llama.cpp ROCm legs at a 0% hit rate and made them that pipeline's critical path (prebuilt: make the ROCm ccache usable, and hold one ROCm alpha per week llama.cpp#88).
• restore-keys drops the tag so a new tag starts from the previous generation instead of from nothing.
• Both the key and the cmake invocation read one pair of job-level env vars, so the key cannot silently drift from what it is keying on.
• All three compiler launchers, including CMAKE_CUDA_COMPILER_LAUNCHER. That one carries this build: nvcc is nearly all of the 65 minutes, and Jimver installs it outside the default search, so caching only C and CXX would leave the expensive half uncached.
• Save in an explicit step on always(), so a failed or capped job keeps what it compiled (CI: keep ccache on failed builds, bound the CUDA Windows timeout, stop uploading the debug bundle on green runs llama.cpp#81). This leg is continue-on-error, so a failure here is routine rather than exceptional.
• ccache --evict-older-than 14d on !cancelled(), so the short teardown window goes to the save rather than to housekeeping.
• ccache --show-stats after the build, so the hit rate is visible in the log instead of having to be inferred.

Behaviour is otherwise unchanged: same toolkit, same architecture list, same targets, same bundle.

Expected effect

Cold cache is unchanged at around 65 minutes. Warm runs should drop that leg well under 10 minutes and take the pipeline off a 70 minute critical path.

Not in this PR

The leg still builds all seven architectures (75;80;86;89;90;100;120) serially in one job, where llama.cpp spreads the same span across parallel matrix profiles. That is the other half of the 65 minutes and a larger change, worth deciding separately.

-j "$(nproc)" is also left alone. llama.cpp pins its multi-arch nvcc builds to -j 3 because they peak at about 3 GB host RSS, and 4 parallel nvcc processes on a 16 GB runner is close to the edge, but this job has not been failing on memory so I have not touched it.

Testing

The workflow parses, and I checked every step's resolved inputs rather than just the step list, which caught the new steps having been spliced between path: and if-no-files-found: on the upload step. Confirmed against GitHub's context availability table that the env context is valid in steps.*.with, since cuda: now reads from it.

The real check is the next run's ccache --show-stats and the second run after that, which is the first one able to hit.