CI: tier the test matrix into PR / main / nightly to cut Actions minutes #432

The Intel OSPO flagged ScalableVectorSearch as using ~80k GitHub Actions minutes per month. Measured over 2026-09-16..10-16: 79.7k runner-minutes across 1,158 workflow runs. About 90% of that comes from pull_request events: each push to a PR runs ~25 jobs, and PRs average ~5 pushes.

This issue proposes splitting CI into tiers. Coverage policy is a team decision, so it is up for discussion here before any PR.

Where the minutes go

The cost of a push went from 346 to 521 runner-minutes after #369 (2026-10-01).

What each matrix entry uniquely covers

• ivf=OFF is a strict subset of ivf=ON. IVF only adds sources (tests/CMakeLists.txt IVF lists, utils, benchmark) and the MKL link. Nothing in include/ checks SVS_ENABLE_IVF. The build-without-MKL path is already covered by the clang-18 ASAN job, which is ivf=OFF. That makes the three RelWithDebInfo ivf=OFF jobs redundant.

• Compilers:

  • There is almost no compiler-specific code (meta.h:260, preprocessor.h:58/64).
  • g++-11 is the release toolchain; the LTO archive requires GCC 11.2.
  • clang-15 is the oldest supported clang.
  • clang-18 is ASAN.
  • macOS clang-20 is libc++.
  • g++-12 adds little, and builds with -Wno-uninitialized (cmake/options.cmake).

• ARM has no NEON code, so it runs the generic kernels. Its unique value is the weak memory model for the concurrent index, and one compiler is enough to cover that. Today it runs three.

• Not covered by any job today:

  • the TSan targets (SVS_EXPERIMENTAL_ENABLE_CONCURRENT_TSAN)
  • integration tests under ASAN (they are not compiled in Debug)
  • SVS_NO_AVX512=YES
  • NUMA
  • the full-size concurrent stress tests, after test(concurrent): run concurrency.cpp tests at small size by default #433

  AVX-512 coverage depends on which runner a job lands on: 16 of the 26 sampled runners lacked AVX-512, and 14 of those were AMD EPYC 7763.

Proposal

A failure that only the main or nightly tier can catch shows up after merge rather than before. Two ways to get that coverage before merge when it matters:

• a full-ci label that runs the main-push tier on a PR;
• running macOS automatically when a PR touches platform-sensitive paths (allocator, datatype.h).

The per-push minute figures are estimates computed from median job durations, not measurements.

Related

• test(concurrent): run concurrency.cpp tests at small size by default #433: concurrent tests at small size by default. Measured: those tests take 107 s instead of ~550 s on 4 cores.
• test: fix three flaky tests seen on ARM and macOS CI #434: three flaky tests that cost ~2.8k min/month in failed jobs and reruns.
• ci: skip PR runs of workflows the change cannot affect #435: paths-ignore filters. Replayed on the last 30 days of PRs, they skip ~12k PR minutes.
• Not in this issue, for separate follow-ups:
  • prebuilt Docker image in GHCR (the image is rebuilt 8–9 times per push at 3.8 min each);
  • pinning FAISS and caching its build;
  • the static c-api variant failing on main since 10-09 because of skew against the nightly LTO archive (see Add ABI diff check for C API and C++ bindings #387);
  • no required status checks are configured on main.