Skip to content

mask-risc: the terminal elects materialization — fused Range ∩ plane → Count/Any - #1268

Merged
AdaWorldAPI merged 4 commits into
mainfrom
claude/fold-distillation-pr-wave-s57uj7
Sep 23, 2026
Merged

AdaWorldAPI merged 4 commits into
mainfrom
claude/fold-distillation-pr-wave-s57uj7

Conversation

@AdaWorldAPI

Copy link
Copy Markdown
Owner

Summary

This PR implements capstone wave W2b-A ("mask expression → terminal").

The core idea: a mask expression denotes a set of rows. Whether that set is ever written out as a bitmap should be decided by the terminal, not by the op that computes it.

For Range[lo,hi) ∩ resident plane folded to Count or Any, the executor now computes the answer directly from the resident plane:

  • It reads only the plane words the range touches, plus two edge words masked in a register.
  • It carves no scratch slot, writes no derived membership word, and never produces row ids.

Keep is never fused. It is the explicit request for a bitmap, and it still writes the demanded Out::Mask.

Re-audit before building (Wave 0, on main 33df0710)

  • Stale on the board: the D-WFL claim "every slot is words_for(n_rows)". Since quack: HAVING over the K sinks, with a NULL-preserving grouped SUM #1266, scratch is tile-sized (at most 8 words).
  • Still true, and the reason for this PR: the fold arm still wrote derived bits on every tile. It ran mask_set_range + mask_and_assign, then read the result back with popcount_batch_u64. It also walked every tile, however narrow the range.

Changes (crates/lance-graph-mask-risc)

  • Program::fused_terminal() recognises exactly [Pred{Range, under: None | Plane}] followed by Count or Any on that slot.
  • Program::requires_scratch() is derived from fused_terminal(), not a separate flag.
  • touched_words(lo, hi) is the single definition of the touched word span, used by both the executor and the tests.
  • execute_into dispatches fused programs before its scratch checks. Validation still runs in full; it tracks its one declared slot in a local word instead of the caller's scratch.
  • Scratch::for_program / over_for_program carve zero slots when requires_scratch() is false.

No second evaluator, no new IR, and no ndarray change.

D-WFL-T1-FUSED′ is refuted for this shape. A range's interior words are all ones, so the existing popcount_batch_u64 / mask_any over the borrowed plane slice are enough. This was a lowering rule (as D-WFL-FUSE predicted), not a missing primitive. The general plane-∩-plane count still has no buffer-free primitive; see the board entry.

Measurements

cargo run --release -p lance-graph-mask-risc --example range_fused_probe. The two arms are asserted equal on every case. Latencies are medians in ns.

N range plane materialized fused derived words written (mat / fused) plane words read (mat / fused)
4,096 tiny dense 653 54 128 / 0 64 / 1
65,536 25% dense 9,002 119 2,048 / 0 1,024 / 257
1,048,576 tiny dense 138,620 56 32,768 / 0 16,384 / 1
1,048,576 25% dense 141,688 1,072 32,768 / 0 16,384 / 4,097
1,048,576 whole dense 148,396 4,435 32,768 / 0 16,384 / 16,384

The word columns come from the executor's contract, not from instrumentation. The fused arm's zero writes is the value the gate test below enforces.

Tests (tests/fused_terminal.rs, 7 tests, disable-verified)

  • Differential: fused result == Keep → popcount/any == scalar oracle.
    • Row counts: 37, 64, 1024, 1317.
    • Plane shapes: all-zero, all-one, clustered, scattered.
    • Ranges: [65,65), [0,0), single row, aligned/unaligned lo and hi, inside one word, across one word, across tiles, a tail shorter than 64 rows, whole population.
    • Disable (skip counting the tail edge word) → red.
  • No materialization: the caller's scratch buffer is filled with u64::MAX, and must still be all u64::MAX after the fold runs. A twin check proves the same probe does see Keep's carve.
    • Disable (always carve the declared slots) → red.
  • No allocation: Scratch::for_program allocates 0 bytes for a fold, counted per thread. A twin check proves the counter sees Keep's allocation.
  • Exact-shape guard: a scratch gate, a second op, or an All terminal is not fused.
  • Touched-word law: lo == hi → 0 words, otherwise floor((hi-1)/64) - floor(lo/64) + 1.
  • Validation still refuses an out-of-bounds range and an absent plane on the fused path.
  • Disabling fused_terminal entirely turns 4 of the 7 tests red.

cargo test -p lance-graph-mask-risc -p lance-graph-quack passes, including the DuckDB differential. clippy -D warnings and fmt are clean.

Board

  • New entry: .claude/board/entries/2026-09-23-terminal-elects-materialization-range-plane.md. It also records a working-model classification of the remaining ops.
  • Status cells updated only for D-WFL-W2b‴ (arm A shipped, arm B's burden still open) and D-WFL-T1-FUSED′ (refuted for this shape).
  • The append-only gate passes and there is no new citation decay.

Not done here

  • W2b-B's downstream-reuse burden (naming the consumer that justifies keeping the bitmap).
  • The execution-extent entry point (PR C).
  • A fused count over plane ∩ plane or over lane comparisons.
  • Any scheduler work; OQ-5 is untouched.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GXUahz73MZxtxWcfpHp9dG


Generated by Claude Code

…→ Count/Any

A mask expression denotes membership; whether it becomes bits is the
terminal's decision. Program::fused_terminal() recognises
Pred::Range{lo,hi} optionally gated by a RESIDENT plane and folded by
Count or Any, and execute_into evaluates it over the plane's touched words
plus two register-masked edge words: no scratch slot carved, no derived
membership word written, no row id. Keep is never fused — it is the
explicit election of a bitmap.

- ir.rs: FusedTerminal / FusedFold, Program::fused_terminal, derived
  Program::requires_scratch, touched_words (the one span spelling)
- exec.rs: fused dispatch after the slot-ceiling check, validation kept
  total via a local bookkeeping word; Scratch::for_program /
  over_for_program carve zero slots when !requires_scratch
- tests/fused_terminal.rs: fold vs Keep vs scalar oracle over 4 row
  counts x 4 plane shapes x the named edge ranges; poisoned-arena gate
  (fold writes nothing, Keep's carve is visible); allocation twin;
  exact-shape guard; touched-word law; validation still refuses

No ndarray primitive needed: the range's interior mask is all ones, so
existing popcount_batch_u64 / mask_any over the borrowed plane suffice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GXUahz73MZxtxWcfpHp9dG
The harness runs tests on parallel threads; a process-wide counter picked
up 120 stray bytes from a sibling test inside the sizing window.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GXUahz73MZxtxWcfpHp9dG
…Count

Reports latency (median), derived membership words written and plane
words read separately; asserts both arms agree on every case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GXUahz73MZxtxWcfpHp9dG
…uted for Range ∩ plane

New entry records the Wave 0 re-audit (tiled scratch made the
population-sized claims stale; derived tile writes remained), the fused
lowering, measurements, disable-verified falsifiers, and the WORKING-MODEL
classification of the remaining ops. Status cells updated for
D-WFL-W2b‴ and D-WFL-T1-FUSED′ only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GXUahz73MZxtxWcfpHp9dG
@cursor

cursor Bot commented Sep 23, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_45fdeef7-4598-4086-b4b1-8ff211f93fb5)

@AdaWorldAPI
AdaWorldAPI marked this pull request as ready for review September 23, 2026 13:01
@AdaWorldAPI
AdaWorldAPI merged commit 0724d00 into main Sep 23, 2026
10 checks passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: daa1585682

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +769 to +770
let mut written = [0u64; 1];
validate(program, planes, foreign, out_shape(&out), &mut written)?;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Size the fused validation bitmap for the destination slot

When a valid fused program writes to scratch slot 64 or higher, this one-word bitmap cannot record the destination: validate silently fails to mark the out-of-range bit and then rejects the terminal read with ScratchReadBeforeWrite. For example, a single Pred::Range { dst: 64 } followed by Count { mask: Scratch(64) } is accepted by Program::new and succeeds on the reference path, but now fails in execute_into; size the temporary bitmap for the addressed slot or validate the fused shape without scratch bookkeeping.

Useful? React with 👍 / 👎.

AdaWorldAPI pushed a commit that referenced this pull request Sep 23, 2026
…ithout rebasing

execute_into is now execute_extent over 0..n_rows. A partial extent is an
outer restriction in the same absolute row coordinates as Planes: only the
words it touches are visited (extent_tiles), a cut word is restricted in a
register at the terminal, lanes and masks are never copied or rebased, and
the #1268 fused fold intersects the program's range with the extent.
Partial extents accept Count/Any/All/MaskedSum/Min/Max (shipped merge laws)
and Keep (writes only in-extent bits of its absolute Out::Mask); anything
else is ExtentUnsupported. Foreign planes are key-addressed and never sliced.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GXUahz73MZxtxWcfpHp9dG
AdaWorldAPI pushed a commit that referenced this pull request Sep 23, 2026
Nine STATUS_BOARD rows still read "In PR" or "Queued" for work that has
merged. Each flip was checked against origin/main before editing:

- D-WFL-FUSE      Shipped: single-op #1270, multi-op chains ≤3 planes #1272
- D-WFL-W2b‴      Shipped W2b-A #1268; W2b-B reuse burden stays OPEN
- D-WFL-EXTENT    Shipped #1269 (`execute_extent`)
- D-WFL-T1-FUSED, T1-FUSED′  Shipped (ndarray #322 + #1270)
- D-WFL-L0        Shipped #1251
- D-WFL-W2a       Shipped #1268: un-gated Pred::Range, Count = hi−lo,
                  Any = lo<hi, zero scratch (`run_fused`)
- D-WFL-W2a′      Shipped #1268: `requires_scratch()` derived from the
                  fused lowerings, never a caller flag
- D-WFL-2         Partially shipped: range Count/Any landed as a lowering,
                  the `BoundedMask` window is not built

Status cells only; every row's description is unchanged, and the old
status is kept after "was:" where it carried detail. The row count stays
at 2203 lines.

Also corrects one exec.rs comment that still described the Boolean fold
as "a single 2/3-input op"; since #1272 it collapses whole chains.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GXUahz73MZxtxWcfpHp9dG
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants