Skip to content

mask-risc: absolute execution extent — execute_extent over [lo, hi) without rebasing - #1269

Merged
AdaWorldAPI merged 5 commits into
mainfrom
claude/fold-distillation-pr-wave-s57uj7
Sep 23, 2026
Merged

AdaWorldAPI merged 5 commits into
mainfrom
claude/fold-distillation-pr-wave-s57uj7

Conversation

@AdaWorldAPI

@AdaWorldAPI AdaWorldAPI commented Sep 23, 2026 •

Copy link
Copy Markdown
Owner

Adds execute_extent, which runs a mask-risc program over a slice of rows [lo, hi) in the same absolute row coordinates as Planes. Nothing is copied or renumbered. This is the unit a future scheduler could split work on; no scheduler, threads, queues or rayon are added, and OQ-5 is not touched.

API shape

  • New entry point: execute_extent(program, planes, foreign, scratch, out, lo..hi).
  • Existing callers are unchanged: execute_into is now execute_extent(.., 0..n_rows), and the whole-population extent still accepts every terminal.
  • Tile plan is crate-private: extent_tiles is executor detail (tile width, edge representation), not API. The in-crate test exec::extent_tile_tests pins that it covers exactly touched_words(lo, hi) with edge masks only on cut words.
  • Keep extents compose sequentially, not concurrently: partial Keep sinks compose in any sequential order. Concurrent execution requires word-disjoint sink ownership (boundaries on multiples of 64), separate partial sinks plus a merge, or another explicitly synchronized strategy — an unaligned split shares one physical u64 and the edge merge is a read-modify-write.
  • Refused before any work: lo > hi or hi > n_rows returns ExtentOutOfRange, and a terminal with no merge law returns ExtentUnsupported.

Absolute-coordinate rule

The extent narrows which rows run; it never shifts row numbers. For example, program Range [1000, 2000) over extent [1500, 1700) means [1000, 2000) ∩ [1500, 1700), which is 200 rows. Lane element r is row r, and tile word w is word w of every stored mask, whatever the extent.

How it is done:

  • Only the words the extent touches are visited.
  • A word that the extent cuts (an unaligned boundary) runs as its own one-word tile.
  • On that tile, the terminal's mask is limited to in-extent bits in a one-word register. All sets the outside bits instead, since set bits are neutral for it.
  • No lane offset, copy or rebased index is needed.

The fused Range ∩ plane → Count/Any path from #1268 intersects the program's range with the extent inside the same code. There is no second evaluator.

What execution now skips / what still writes a bitmap

  • Skipped: every tile outside the extent. A 1-row extent over 1M rows visits one word.
  • Still writes, deliberately: Keep writes only the in-extent bits of its population-sized Out::Mask. Every other bit stays as the caller left it, so disjoint extents can fill one buffer in any order.
  • Refused on a partial extent, because there is no merge law yet: BlendI32, the two scatter terminals, CountKeyRunsU32, GroupSumI32/GroupSumViaI32, GroupReduce. The _sym merge law stays recorded only; nothing is built for it.
  • Foreign planes are never sliced: Gather looks rows up by key, and a test pins that.

Tests (tests/extent.rs, 8 tests)

  • Edge matrix, 1,620 cases, checked against a scalar oracle:
    • Extents: empty, whole, one row, unaligned start and/or end, single word, crossing a word, crossing a tile, tail with N % 64 ≠ 0.
    • Program range vs extent: disjoint, partial overlap, one containing the other.
    • Planes: all-zero, all-one, scattered.
    • Shape and terminal: fused Count/Any; tiled Count/Any/All/Keep.
  • 1M rows: tiny extents read the correct absolute rows.
  • Split/merge: running the whole population equals merging the results of any partition, in forward, reverse and rotated order:
    • two-way splits at k ∈ {0, 1, 63, 64, 65, 127, 128, 129, N/2, N−1, N}, plus 40 random three-way splits;
    • N ∈ {1317, 4133};
    • terminals: Count (+), Any (∨), All (∧), MaskedSumI32 (+), MaskedMin/Max.
    • Keep partial results written into one buffer equal the whole-population Keep.
  • Non-rebasing check: lane values differ at rows 63–65 and 127–129. Extents starting at 65, 129, 63 and 127 must sum the absolute rows, and the test asserts that a worker-local reading gives a different answer, so a rebasing executor cannot pass. A second test pins Range[1000,2000) over [1500,1700): Count is 200, and Keep sets exactly bits 1500..1700.
  • Structural gate: a 1-row extent over 1M rows is one tile covering one word. The whole-population tile plan is identical to the old 8-word tiling. For every extent, the tiles are contiguous and disjoint and cover exactly the touched words.

Disable runs (committed first, then each file restored) — each breaks one mechanism and turns tests red:

disable tests red
tiles walk from row 0 7 / 8, incl. the structural gate
edge word not restricted 6
lanes read worker-local 5, incl. the absolute-rows check
fused path ignores the extent 4
Keep overwrites the whole edge word 3

Benchmarks (examples/extent_probe.rs, N = 1,048,576, results checked against an oracle)

shape extent median ns population words read lane elements read derived words written
fused Range∩plane→Count 1 row 62 1 0 0
fused 25% 1,172 4,097 0 0
fused whole 4,102 16,354 0 0
tiled Range→And→Count 1 row 161 1 0 2
tiled whole 158,051 16,384 0 32,768
lane EqU32→MaskedSumI32 1 row 172 1 128 1
lane whole 1,828,878 16,384 2,097,152 16,384

Cost follows the extent's width, not the population size. The word counts come from the extent's semantic span (touched_words), which the in-crate tile-plan test pins the executor to cover exactly; not from instrumentation.

What remains before scheduling

  • OQ-5 is still open.
  • Merge laws for the group terminals and the _sym SUM are not built; those terminals are refused on partial extents.
  • The next useful step is more folds that go straight from an expression to a terminal without writing a bitmap: plane ∩ plane → Count, lane predicate → Count, ternlog → Count. With those, what gets split across workers becomes small algebraic programs rather than tile-writing bitmap machines.

Board

Verification

  • cargo test -p lance-graph-mask-risc: all suites green.
  • cargo test -p lance-graph-quack: green, including the DuckDB differential.
  • cargo clippy -p lance-graph-mask-risc -p lance-graph-quack --all-targets -D warnings: clean.
  • cargo fmt: clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GXUahz73MZxtxWcfpHp9dG


Generated by Claude Code

Summary by CodeRabbit

  • New Features
    • Added execution over a selected range of absolute row positions, without changing lane or row coordinates. Full-population execution continues to work as before.
    • Supported partial-range operations include counts, boolean checks, masked sums and extrema, and selective mask updates. Unsupported operations and invalid ranges return errors.
    • Partial mask updates preserve bits outside the selected range; concurrent updates that share a word require care.
  • Documentation
    • Added guidance on range execution, supported operations, and combining results across partitions.

…ithout rebasing

execute_into is now execute_extent over 0..n_rows. A partial extent is an
outer restriction in the same absolute row coordinates as Planes: only the
words it touches are visited (extent_tiles), a cut word is restricted in a
register at the terminal, lanes and masks are never copied or rebased, and
the #1268 fused fold intersects the program's range with the extent.
Partial extents accept Count/Any/All/MaskedSum/Min/Max (shipped merge laws)
and Keep (writes only in-extent bits of its absolute Out::Mask); anything
else is ExtentUnsupported. Foreign planes are key-addressed and never sliced.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GXUahz73MZxtxWcfpHp9dG
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GXUahz73MZxtxWcfpHp9dG
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GXUahz73MZxtxWcfpHp9dG
…ution extent

Adds the N=1M dispatch-locality probe (fused / tiled / lane shapes over
1 row .. whole extents, oracle-checked), the D-WFL-EXTENT status row, and
an append-only correction of #1267's 'rebasing every mask and lane'
wording: the required property is correct absolute execution over an
unaligned extent, met with word bounds and register edge masks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GXUahz73MZxtxWcfpHp9dG
@cursor

cursor Bot commented Sep 23, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_9507e83a-da1e-4826-abe2-87827191010e)

@coderabbitai

coderabbitai Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 67.74% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 31 functions across 6 files. (3 skipped: … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: adding absolute-coordinate execution through execute_extent over [lo, hi) without rebasing.
Full details: Docstring Coverage

Explanation

Docstring coverage is 67.74% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 31 functions across 6 files. (3 skipped: 3 unsupported.)

  • Fix all pre-merge checks with AI

Comment @coderabbitai help to get the list of available commands.

@AdaWorldAPI
AdaWorldAPI marked this pull request as ready for review September 23, 2026 14:33
…-only

extent_tiles/ExtentTiles are executor detail (tile width, edge representation),
so they drop out of the public API; the structural tile-plan test moves in-crate
(exec::extent_tile_tests) and the probe reports touched words from the semantic
span. execute_extent's doc and the board entry now state that partial Keep sinks
compose in any SEQUENTIAL order only: an unaligned split shares a physical u64
and the edge merge is read-modify-write, so concurrent writers need word-disjoint
ownership, separate partial sinks plus merge, or explicit synchronization.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GXUahz73MZxtxWcfpHp9dG
@AdaWorldAPI
AdaWorldAPI merged commit bcec6b3 into main Sep 23, 2026
11 checks passed
AdaWorldAPI pushed a commit that referenced this pull request Sep 23, 2026
Nine STATUS_BOARD rows still read "In PR" or "Queued" for work that has
merged. Each flip was checked against origin/main before editing:

- D-WFL-FUSE      Shipped: single-op #1270, multi-op chains ≤3 planes #1272
- D-WFL-W2b‴      Shipped W2b-A #1268; W2b-B reuse burden stays OPEN
- D-WFL-EXTENT    Shipped #1269 (`execute_extent`)
- D-WFL-T1-FUSED, T1-FUSED′  Shipped (ndarray #322 + #1270)
- D-WFL-L0        Shipped #1251
- D-WFL-W2a       Shipped #1268: un-gated Pred::Range, Count = hi−lo,
                  Any = lo<hi, zero scratch (`run_fused`)
- D-WFL-W2a′      Shipped #1268: `requires_scratch()` derived from the
                  fused lowerings, never a caller flag
- D-WFL-2         Partially shipped: range Count/Any landed as a lowering,
                  the `BoundedMask` window is not built

Status cells only; every row's description is unchanged, and the old
status is kept after "was:" where it carried detail. The row count stays
at 2203 lines.

Also corrects one exec.rs comment that still described the Boolean fold
as "a single 2/3-input op"; since #1272 it collapses whole chains.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GXUahz73MZxtxWcfpHp9dG
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants