Metal: ggml_metal_library_init_from_source still fails on Apple M5 / macOS 26.6 in 3.20.0 despite full Xcode + Metal Toolchain (regression of #629 for embed path)

[natea]

Issue description

ggml_metal_library_init_from_source: error compiling source / the tensor API is not supported in this environment - disabling still occurs on node-llama-cpp 3.20.0 — the version #629 was marked "resolved" in — with a full Xcode install (26.6) and the Metal Toolchain component explicitly installed, not just Command Line Tools.

This directly contradicts the hypothesis in #629's last comment ("My guess is the Metal compiler toolchain bundled with just Command Line Tools doesn't yet include support for this new Metal4 tensor API, while a full Xcode install might") — in my case, full Xcode + the separately-downloaded Metal Toolchain component (xcodebuild -downloadComponent MetalToolchain, confirmed xcrun metal --version reports Apple metal version 32023.883, and compiles a trivial .metal kernel fine standalone) is present, and the failure is identical.

Environment

• Apple M5 (unified memory 24.96GB reported by inspect gpu)
• macOS 26.6.2 (Build 25G83)
• Xcode 26.6 (full install, not just CLT), Metal Toolchain 17F109 explicitly installed
• node-llama-cpp 3.20.0 (prebuilt Metal binaries, b10361)
• Node.js 22.22.3
• Discovered via qmd CLI (qmd embed), which calls node-llama-cpp to embed with embeddinggemma-300M-Q8_0.gguf

Error output

node-llama-cpp inspect gpu:

Metal: available

Metal device: Apple M5
Metal used VRAM: 0% (448KB/24.96GB)
Metal free VRAM: 99.99% (24.96GB/24.96GB)
Metal unified memory: 24.96GB (100%)

ggml_metal_library_init_from_source: error compiling source
ggml_metal_device_init: - the tensor API is not supported in this environment - disabling
CPU model: Apple M5

qmd embed (the actual failing operation — embedding generation, not context creation):

Model: embeddinggemma-300M-Q8_0.gguf
[node-llama-cpp] ggml_metal_library_init_from_source: error compiling source

qmd embed hard-aborts here rather than degrading gracefully the way inspect gpu does.

Repro

xcodebuild -downloadComponent MetalToolchain   # confirmed already installed + working standalone
npx --no node-llama-cpp inspect gpu            # logs the tensor-API warning but exits cleanly
qmd embed                                       # same underlying condition, but the embedding call aborts

CPU-only fallback also doesn't work on this platform

Per #629's verified workaround (NODE_LLAMA_CPP_GPU=false / QMD_LLAMA_GPU=false), I tried the equivalent and got:

QMD Warning: CPU-only llama.cpp prebuilt not available (NoBinaryFoundError); using packaged backend with GPU offloading disabled.
[node-llama-cpp] ggml_metal_library_init_from_source: error compiling source

So on mac-arm64 there's still no CPU-only prebuilt binary shipped, meaning the GPU-disable escape hatch requires a local source build (cmake) as noted in #629 — worth confirming that's still the only path, or considering shipping a CPU-only mac-arm64 prebuilt given how common this M5-family failure mode appears to be (#629, #595, this issue).

Question

Is the 3.20.0 fix for #629 scoped to context-creation only (where it now degrades gracefully with the "disabling" log line) while the embedding-generation code path (model.createEmbeddingContext() / getEmbeddingFor, as used by qmd embed) still hits an unhandled instance of the same underlying Metal4-tensor-API-unavailable condition? Happy to run further diagnostics on this M5 machine — same offer as in #629.

Related: #629 (same error signature, marked fixed in 3.20.0, not actually fixed for this call path), #595 (different symptom — sporadic hangs rather than a hard abort — but same qmd tool, same M5 hardware class).