meta: cold builds 50% faster than PlatformIO (performance audit)

[zackees]

Goal

fbuild's cold build is at least 50% faster than PlatformIO's on the nightly Blink benchmark (SVG, benchmark-build-comparison.yml, ubuntu-24.04). Put simply: fbuild_cold_median ≤ 0.5 × platformio_cold_median, sustained for 7 consecutive nightly runs, with the benchmark's cold definition unchanged or changed equally for every tool.

Baseline (2026-09-25, sha 0433e96)

	cold	warm
PlatformIO	745 ms	551 ms
fbuild	2111 ms (2.8× PIO)	46 ms
target	≤ ~372 ms	—

Root cause of today's gap (#1464): the embedded zccache's post-miss include scan uses ~85% of CPU during a cold build, against ~11% for the compiler. LTO and flags match PlatformIO's.

Plan (order matters)

Phase 0: measure

• perf audit: per-phase cold-build timing breakdown in the Blink benchmark #1465 per-phase cold timing breakdown + raw-compiler baseline in the benchmark

Phase 1: zccache (fix upstream first)

• perf: skip recursive include scan on cache miss when the compiler wrote a depfile zackees/zccache#1668 skip recursive include scan when a depfile exists (biggest win)
• perf: memoize per-file include scans across TUs zackees/zccache#1669 memoize per-file include scans across TUs
• perf: single-pass include scanner + miss-path overhead benchmark zackees/zccache#1670 single-pass scanner + per-TU miss-overhead benchmark

Phase 2: fbuild adoption

• perf: cold build 3x slower than PlatformIO — zccache miss-path include scan dominates CPU #1464 repin zccache and verify the benchmark reaches about parity with PlatformIO
• perf: probe toolchain once per daemon, not per TU #1466 probe the toolchain once per daemon, not per TU
• compile_commands.json omits -flto and rewrites the compiler as clang++ --target=avr #1467 make compile_commands.json reflect the real flags (-flto missing)

Phase 3: beat PlatformIO by 50%

• perf: structural cold-build wins to reach 50% faster than PlatformIO #1468 structural wins (critical-path scheduling, link overlap, spawn overhead, PCH, a fairness ruling on prebuilt core)

Expectations

Phases 1–2 should remove about 1–1.5 s and bring fbuild to roughly PlatformIO parity. Measured locally at 4 cores, the raw avr-gcc floor is already ~340 ms, close to the 372 ms target, so Phase 3 is required and Phase 0's data decides which Phase 3 items are worth doing.

Performance-audit rules

• Every PR in this effort includes before/after benchmark numbers (median of 3, 4 cores).
• Profile with a symbolized release build: CARGO_PROFILE_RELEASE_DEBUG=line-tables-only CARGO_PROFILE_RELEASE_STRIP=none, plus perf record --call-graph dwarf.
• No regressions to warm builds (currently 46 ms).

[zackees]

Progress: zackees/zccache#1670 landed (zackees/zccache#1701: 8.4× faster single-pass include lexer, plus a per-TU miss-overhead bench and CI budget). #1464 landed (#1474, zccache repinned to 8475316). CI Blink benchmark (4 vCPU, 3 trials): fbuild cold 451 ms vs PlatformIO 868 ms (0.52×), warm 46.3 ms (no regression); the baseline was 2111 ms (2.8×). Next: #1466 (zackees/zccache#1708 single-flights the probes), #1467, #1468.