Skip to content

simd: _sym group sum — symmetric range with a reserved empty code - #321

Merged
AdaWorldAPI merged 2 commits into
masterfrom
claude/fold-distillation-pr-wave-s57uj7
Sep 23, 2026
Merged

AdaWorldAPI merged 2 commits into
masterfrom
claude/fold-distillation-pr-wave-s57uj7

Conversation

@AdaWorldAPI

@AdaWorldAPI AdaWorldAPI commented Sep 23, 2026 •

Copy link
Copy Markdown
Owner

What

  • masked_group_sum_sym_i32 and masked_group_sum_sym_i32_via: grouped sums that can tell an empty group from a group whose rows sum to zero.
  • SYM_EMPTY_I64 = i64::MIN: the one value these functions reserve to mean "no row reached this group".

All three are re-exported from ndarray::simd.

Full range vs _sym (the naming rule)

Only functions with the _sym suffix reserve a value, and the reserved value is fixed (the caller does not choose it).

  • No _sym suffix: the function uses the full two's-complement range. Every i64 it writes is a real value, i64::MIN included, and it never treats i64::MIN specially. Callers that treat the output as plain integers are always right.
  • _sym suffix: real sums live in the symmetric range ±(2⁶³−1), and SYM_EMPTY_I64 means "empty". This is the 4-bit analogue of reading a nibble as −7..+7 and keeping −8 as NaN, rather than using the full −8..+7.

The rule is written once, in the docs of SYM_EMPTY_I64.

An earlier revision of this PR (masked_group_sum_seeded_i32(…, empty, …)) let the caller pass any marker. That meant a caller expecting full-range behaviour could reserve a real value by accident, so it was removed.

How it works

  • The caller fills out with SYM_EMPTY_I64 first.
  • The first selected row that lands in a group replaces the marker. Every later row adds with wrapping_add, the same as the plain sum.
  • Both functions are named closures over the shared group_walk, and one fold closure serves both key addresses. No new bit loop.

Row limit: the marker can be hit by a real sum only at 2³² or more rows. Exactly 2³² rows of i32::MIN sum to i64::MIN. Callers that fold that many rows into one group must cap the row count.

Why

This is for SQL HAVING over grouped results in lance-graph's quack layer (AdaWorldAPI/lance-graph#1266). There the reserved code stays internal: quack turns each raw result into "empty slots set to 0" plus a K-bit presence mask before anything downstream sees it.

Tests

  • sym_sum_matches_the_reference_and_marks_empty_groups: both key addresses against a scalar reference, at every length in LENS.
  • a_cancelling_group_is_present_and_an_unreached_group_is_empty: a group whose rows cancel to 0 is present, a group no row reached is empty, and both lookup steps of the _via path drop rows correctly.
  • full_range_sum_never_treats_the_sym_code_as_empty: the plain sum adds to a slot holding i64::MIN, while the _sym sum replaces it.
  • Doctest.

Disable run: turning the _sym fold into a plain wrapping_add makes all three _sym-sensitive tests fail. clippy -D warnings and fmt are clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GXUahz73MZxtxWcfpHp9dG

masked_group_sum_seeded_i32 and _via: a slot still holding the caller's
`empty` marker has seen no row, so the first selected row REPLACES the
marker and later rows add (wrapping_add, as the plain sum). The plain sum
cannot separate an empty group from one whose rows cancel to 0.

`empty` = i64::MIN is the intended marker: a sum of n i32 stays strictly
above it for every n < 2^32, which leaves a symmetric real range
±(2^63 - 1). The doc names the one reachable case (exactly 2^32 rows of
i32::MIN) so a caller with that row count bounds it or picks another marker.

One walker, as the rest of the family: two named closures over
group_walk. Tests: both addresses against the scalar oracle at every
length, plus the cancel-vs-empty case and both VIA drops.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GXUahz73MZxtxWcfpHp9dG
@coderabbitai

coderabbitai Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Pull request is closed - (🔄 Check again to try again)

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Comment @coderabbitai help to get the list of available commands.

@cursor

cursor Bot commented Sep 23, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_5cae7d3d-2e65-4c9b-a841-812e1ffd8a53)

…hosen marker

masked_group_sum_seeded_i32{,_via}(…, empty, …) let any caller pick any
marker, so a full-range caller could reserve a real value by accident.
The reservation is now opt-in by NAME only:

- masked_group_sum_sym_i32{,_via}: symmetric range ±(2^63−1), with
  SYM_EMPTY_I64 = i64::MIN reserved as "no row reached this group" — the
  4-bit −7..+7 + NaN reading, never the −8..+7 one.
- Every unsuffixed reduction keeps the full two's-complement range and never
  treats i64::MIN specially. A new test pins that: the full-range sum adds to
  an i64::MIN slot where the _sym sum replaces it.

The rule is stated once, on SYM_EMPTY_I64. One fold closure serves both key
addresses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GXUahz73MZxtxWcfpHp9dG
@AdaWorldAPI AdaWorldAPI changed the title simd: seeded group sum — tell an empty group from a zero sum simd: _sym group sum — symmetric range with a reserved empty code Sep 23, 2026
@AdaWorldAPI
AdaWorldAPI marked this pull request as ready for review September 23, 2026 08:54
@cursor

cursor Bot commented Sep 23, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_2446c836-9c84-4b71-9a94-8dfe2b270785)

@AdaWorldAPI
AdaWorldAPI merged commit 58a9fd2 into master Sep 23, 2026
26 of 27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants