perf(storage): bound Usage page queries and avoid redundant counts

[liuxiaocs7]

Summary

Usage activity pages currently scan and project the remaining time range before selecting 50 rows. Seek on each source's existing composite index, filter and take 51 matches per source, then merge at most 153 candidates. Keep the existing ordering, cursors and revision transaction. Filtered counts select only search/status fields; unfiltered totals reuse complete statistics, and zero matches skip the activity query.

With 50,000 tool records, local second-page median/p95 falls from 53.530/60.211 ms to 0.803/0.958 ms on Node, and 59.392/76.300 ms to 0.853/1.128 ms on Electron. The performance evidence below compares explicit commits through the real Storage facade. Full statistics, exact filtered counts and sparse searches can still scan history. Statistics caching remains follow-up work; this complements #5410's frontend work.

Fixes #5515
Refs #5038

Verification

• Passed full build, typecheck, lint, and Knip for Desktop/UI; tracked-file formatting and ASF header audit passed. Whole-directory format/header commands flag preexisting untracked research artifacts excluded from this PR.
• Storage: 1,409 passed, 8 skipped, with NODE_OPTIONS=--disable-warning=ExperimentalWarning; Usage: 27/27 on both Node and Electron; Host Usage: 6/6. The two search-work regressions fail on the baseline. The mixed-source oracle covers all pages, exact counts, ties, Unicode, inclusive ranges and accounting variants.
• Same-process alternating A/B benchmarks: 3 warmups, 15 samples, 1k/10k/50k tools and 50k mixed tied records; second/middle/tail pages, filtered screens and no-match screens. SQL plans and search callback counts are collected outside timing.
• Full npm test was attempted and is not green: a Storage child-stderr assertion rejects Node's SQLite experimental warning (also reproduced on the baseline); unrelated Runtime Host/Eval lifecycle cases time out. The Host child-patch timeout also reproduces with the baseline Host test; the other failing cases pass focused reruns. The baseline worktree shares installed dependencies, so this is not a fully isolated environment comparison. Eval's Python suite was not reached after its JavaScript failure.

Performance evidence

Query details, reproducible benchmarks and regression coverage

Query details

Each source applies its cursor and filters before selecting up to 51 matching rows. For equal timestamps, sources below the cursor source include that timestamp, sources above it exclude it, and the same source seeks by (timestamp, stableStorageIdentity). The validated cursor replaces the redundant range upper bound so SQLite can seek on both composite-index columns. The global (ts DESC, source DESC, identity DESC) ordering, 50-row response, cursor validation and revision transaction remain unchanged.

The unfiltered activity total is the model-request count plus tool calls from the complete range statistics. Filtered exact counts select only search/status fields. A zero count skips the activity query inside that same snapshot.

Reproduce

Build Core and Storage at both revisions. From the optimized checkout, run:

MAKA_PERF_OUTPUT=/tmp/usage-pages-node node scripts/perf/usage-pages.mjs /path/to/built-baseline-checkout

On macOS, also exercise the shipped SQLite runtime:

ELECTRON_RUN_AS_NODE=1 MAKA_PERF_OUTPUT=/tmp/usage-pages-electron node_modules/electron/dist/Electron.app/Contents/MacOS/Electron scripts/perf/usage-pages.mjs /path/to/built-baseline-checkout

The harness calls the actual readUsageScreen facade at both revisions against
the same temporary database. It alternates before/after calls in one process,
performs three warm-ups and 15 measurements, and saves raw samples, median/p95,
environment, Git revisions, SQL and query plans. Instrumented search callback
counts run outside the timing loop. First-screen timings use reopened reader
connections, not a cold OS page cache. Fixtures are persisted directly into
the production schema; fixture insertion is not timed.

Deep-page requests use known fixture positions as valid cursors. This measures
the query after a cursor is available, not the UI's sequential deep-page walk.
Host projection repair, wire serialization, IPC/network and React are excluded.

Results

Measured on 2026-09-20 against baseline
0117d76c5688475e7467ee6db83057bf65edbabd and optimized commit
e39d80b11a636e57d5fc7eabffc53bcdcf8d4ec2, with a clean tracked working tree.
Timings are milliseconds; each cell is
median / p95. These are synthetic local measurements, not production latency
guarantees.

Runtime and fixture	Operation	Before	After
Node 24.14.0 / SQLite 3.51.2, 1,000 tools	Second page	1.541 / 2.050	0.755 / 0.920
Node, 10,000 tools	Second page	10.871 / 12.588	0.693 / 0.900
Node, 50,000 tools	Second page	53.530 / 60.211	0.803 / 0.958
Node, 50,000 mixed sources, equal timestamps	Second page	65.755 / 68.261	1.066 / 1.198
Node, 50,000 tools	Complete screen	180.570 / 197.739	79.035 / 86.963
Node, 50,000 mixed sources, equal timestamps	Complete screen	308.340 / 363.471	180.352 / 194.375
Electron 43.4.1 / SQLite 3.53.1, 50,000 tools	Second page	59.392 / 76.300	0.853 / 1.128
Electron, 50,000 mixed sources, equal timestamps	Second page	70.243 / 73.507	1.153 / 1.284
Electron, 50,000 tools	Complete screen	195.282 / 203.461	88.037 / 100.974
Electron, 50,000 mixed sources, equal timestamps	Complete screen	314.361 / 345.219	187.128 / 207.477

On Node, matching second-page searches over 50,000 tools go from 99,900 to 51
JavaScript lowercase calls; the mixed-source fixture goes from 99,900 to 153.
No-match tool searches go from 300,000 to 150,000 calls because count zero
avoids the second scan. These count function evaluations, not physical row
visits. Exact counts and sparse substring searches still depend on history size.

The continuation plan now uses the source's composite index as a seek:

SEARCH usage_tool_invocations USING INDEX usage_tool_invocations_screen
  (ts>? AND (ts,storage_key)<(?,?))

The outer sort may still use a temporary B-tree; its input is at most 153
matching candidates. Per-source filtering can inspect more than 51 index
entries. Full aggregates remain range scans, explaining why a complete screen
still costs substantially more than a continuation.

Regression coverage

The Storage facade tests verify exact totals and all pages against an independent
fixture oracle across canonical, legacy and tool sources, equal timestamps,
Unicode identities, duplicate display IDs, inclusive range boundaries,
success/error/aborted status, Unicode lowercase expansion, literal %/_, and
unreadable/free/unpriced canonical rows. Existing WAL, rollback, revision and
restore tests still exercise the production reader.

Two deterministic work regressions fail on the baseline: a matching continuation
over 512 tied records performs 924 search folds, and a no-match screen over 120
tools performs 720. Their bounded-work assertions avoid hardware-sensitive
millisecond thresholds. Node and Electron both pass the optimized Usage suite.

AI use

Select exactly one:

• No generative tool made a substantive contribution
• Generative tooling made a substantive contribution

Tool(s) and scope: Codex implemented the SQL optimization, regression tests, benchmark and PR notes, and performed automated reviews. AI-authored commits include Generated-by: Codex; retain the trailer when squashing.

Checklist

• Tests cover the change and fail without it
• Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

• Yes — described under Summary above
• No