Skip to content

hdr: run Cascade on ndarray's rolling floor and σ-lattice - #1278

Merged
AdaWorldAPI merged 3 commits into
mainfrom
claude/llvm-codegen-polyfill-gni3cw
Sep 24, 2026
Merged

AdaWorldAPI merged 3 commits into
mainfrom
claude/llvm-codegen-polyfill-gni3cw

Conversation

@AdaWorldAPI

@AdaWorldAPI AdaWorldAPI commented Sep 24, 2026 •

Copy link
Copy Markdown
Owner

What changes

graph/blasgraph/hdr.rs now consumes ndarray::hpc::rolling_floor (AdaWorldAPI/ndarray#328):

  • Moments and drift: exact MomentsU32 moments, the reservoir and drift detection come from ndarray. ShiftAlert and ReservoirSample are re-exports.
  • Threshold derivation: band cuts (3σ / 2σ / 1σ / μ) and cascade cuts (1σ to 3σ in quarter-σ steps) are SigmaLevel points. The floor's current shape and running coordinates answer them per call, and nothing is cached.
  • Unchanged API: the public API stays the same: band, expose, thresholds, cascade_thresholds, cascade_at, query, observe, observe_batch and recalibrate.

Semantic deltas

  • Running coordinates: thresholds now follow the running (μ, σ). Before, they were frozen at the last recalibration.
  • Empirical band ranks: these now use the Gaussian tail of each σ level (1587, 228 and 13 per 10000) instead of the coarse 0.159, 0.023 and 0.001. Across reservoir lengths 1..=1000, the chosen sample index changes at:
    • 150 lengths for 1σ (159→158 at len 1000);
    • 98 lengths for 2σ (23→22);
    • 230 lengths for 3σ (1→1 at len 1000).
  • Empirical cascade ranks: unchanged at every length.
  • cascade_at(k): now uses the same shape-aware lookup. Before, it applied the anchor's σ arithmetic even when the empirical shape was active.

Tests

All 191 blasgraph tests pass with no re-pins. Clippy -D warnings and fmt are clean.

CI stays red until ndarray#328 merges, because CI builds against ndarray master.

Scope fence

The diff contains 0 matches for ZGamma, zspace, Fisher, hyperbolic, Palette256, FisherZTable, bgz or Mexican.

🤖 Generated with Claude Code

https://claude.ai/code/session_019HnekoM1EidTwQLS3oFVFm

hdr::Cascade's adaptive half (running statistics, reservoir, shape
evaluation, sigma/empirical floor switch, drift, recalibration) now
delegates to ndarray::hpc::rolling_floor::RollingFloor, which was
harvested from this file with its constants, cadence, drift rule and
reset semantics unchanged. The approximate integer Welford is replaced by
ndarray's exact MomentsU32; no floor decision changed in the reference
tests, and ndarray pins the checkpoint (mu, sigma) against the legacy
Welford as an oracle.

Kept here: Band and its classification, RankedHit, the three-stage
sampled query, stage levels, cascade_at, and the public API
(isqrt, ReservoirSample, ShiftAlert, Cascade::{for_width, calibrate,
observe, recalibrate, ...}). isqrt, ReservoirSample and ShiftAlert now
re-export the ndarray items. New: Cascade::observe_batch and
Cascade::floor.

The adaptive path still has no production caller; this file is the
integration/reference consumer for the upcoming popcount consumers
(Raumgewinn/influence, infight). Requires AdaWorldAPI/ndarray#328.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019HnekoM1EidTwQLS3oFVFm
@coderabbitai

coderabbitai Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Essentials

Run ID: caf6a62c-4319-43f4-be23-bfe84ee35aa9

📥 Commits

Reviewing files that changed from the base of the PR and between 26eb55c and 10671d2.

📒 Files selected for processing (1)
  • crates/lance-graph/src/graph/blasgraph/hdr.rs
 ____________________________________
< LLM (Looking for Leaks in Memory). >
 ------------------------------------
  \
   \   \
        \ /\
        ( )
      .( o ).
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

@cursor

cursor Bot commented Sep 24, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_acf98731-9b8d-4fbd-8240-37dfbcafa812)

Copy link
Copy Markdown
Owner Author

member-tests fails with unresolved import ndarray::hpc::rolling_floor (4 errors, all in hdr.rs).

CI builds against ndarray master, and hpc::rolling_floor does not exist there until AdaWorldAPI/ndarray#328 merges. This is the ordering dependency stated in the description, not a defect in this diff. Built against the #328 branch, cargo test -p lance-graph --lib -- graph::blasgraph passes 191/191 locally, and clippy -D warnings is clean.

No fix is needed on this branch. Once #328 is on ndarray master, a CI re-run here should pass.


Generated by Claude Code

Band (3σ/2σ/1σ/μ) and cascade (1σ…3σ in quarter-σ steps) thresholds
are SigmaLevel points answered on demand by the rolling floor's current
shape and coordinates. cascade_at(k) is the same lookup. Nothing is
cached; the public band/cascade API is unchanged.

Empirical band ranks now use the Gaussian tail of each σ level
(1587/228/13 per 10000) instead of the coarse 0.159/0.023/0.001.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019HnekoM1EidTwQLS3oFVFm
@AdaWorldAPI AdaWorldAPI changed the title hdr: run the Cascade rolling floor on ndarray's RollingFloor hdr: run Cascade on ndarray's rolling floor and σ-lattice Sep 24, 2026
@cursor

cursor Bot commented Sep 24, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_b2ab1720-f414-4c87-acb1-dbaf73dd4cd4)

@AdaWorldAPI
AdaWorldAPI marked this pull request as ready for review September 24, 2026 20:43
@AdaWorldAPI
AdaWorldAPI merged commit 5360283 into main Sep 24, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants