perf: sparse-aware dot/matmul against a mostly-zero constant

[FBumann]

Note

The following content was generated by AI (Claude Code), prompted and reviewed by @FBumann.

Closes #748.

LinearExpression.__matmul__ is (self * other).sum(common_dims), which stacks one term per element of the contracted dimensions regardless of zeros in the operand — a sparse constant (e.g. PyPSA's cycle-incidence matrix in the KVL constraint) densifies the result to full _term (284× more cells than needed, the largest single allocation in a SciGRID-DE build, see #748/#749).

This adds a sparse-aware kernel using the reframe from the issue: each nonzero entry of the operand belongs to exactly one output group, so the contraction is a clean sparse-entry groupby(free_dim).sum(). The kernel selects the nonzero entries (isel with a flat entry dimension), multiplies each with its slice of the expression, and group-sums via the existing scatter kernel (#802). Only contributing terms are materialized; the represented expression is identical to the dense path's — only the positional term layout (count, order, padding) differs, which linopy already treats as non-semantic (densify_terms, zero-coeff filtering at export).

When it engages (all other cases fall back to the dense path unchanged):

• constant operand with at most one free (non-contracted) dimension, unique labels,
• contracted labels match the expression exactly — alignment semantics are never re-implemented, the sparse path only fires when alignment is a no-op,
• nonzero density ≤ linopy.options["sparse_dot_max_density"] (new option, default 0.5; 0 disables the sparse path),
• numpy-backed data (dask keeps the dense path).

NaN entries in the operand are kept as terms (!= 0), matching the dense path's NaN propagation under both current and future (v1) semantics. The constant part is reduced with the same skipna semantics as _sum.

Benchmark (memray, KVL-shaped contraction: 852 branches × 268 cycles, 24 snapshots, 3 branches/cycle)

Op-only measurement (memray.Tracker around just the expr @ C call, native traces):

	result _term	coeff+var cells	op peak	op total allocated
dense path (sparse_dot_max_density=0)	852	10,960,128	189.2 MB	288.9 MB
sparse kernel (this PR)	3	38,592	3.9 MB	4.7 MB

284× fewer result cells (matching the figures in #748), ~48× lower op peak. Whole-process peak (interpreter + model build + op) drops 282.3 MB → 97.1 MB.

Tests: equivalence against the dense path via a canonical (term-layout-insensitive) comparison — incidence matrix, vector contraction, expression with constant, NaN entries, aux coords on the free dim, coordinate-less free dim — plus fallback tests (reordered labels, dense operand, duplicate free labels, >1 free dims, option disabled) asserting the dense path's exact layout is unchanged. Full suite: 3800 passed, 45 skipped.

Sibling issues #745/#749/#756 (groupby padding, ragged merge, long-format umbrella) are not touched; this is the self-contained first step of the #756 plan.

🤖 Generated with Claude Code

[codspeed-hq]

Merging this PR will not alter performance

✅ 173 untouched benchmarks
🆕 2 new benchmarks
⏩ 173 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
🆕	Memory	test_op[expr_matmul_dense]	N/A	39.7 MB	N/A
🆕	Memory	test_op[expr_matmul_sparse]	N/A	1 MB	N/A

---

_{Comparing perf/sparse-dot (e03337f) with master (2afe5d0)²}

Footnotes

1. 173 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. No successful run was found on master (f268bff) during the generation of this report, so 2afe5d0 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[FBumann]

Note

AI-generated note (Claude Code, prompted by @FBumann).

Benchmark coverage: #868 adds a matmul op group to ops.py (expr_matmul_dense / expr_matmul_sparse) — the suite previously had no benchmark exercising @ at all (kvl_cycles uses the expanded (flow * C).sum(...), which bypasses this kernel). It should merge before this PR so the ops have a master baseline; this PR's CodSpeed run should then show expr_matmul_sparse dropping sharply while expr_matmul_dense stays put. Since CI runs on the merge commit, no rebase of this branch is needed once #868 lands.