Skip to content

perf(prover): pull-cursor CPU scheduler instead of fixed chunks, plus… - #901

Open
diegokingston wants to merge 1 commit into
fix/cpu-prover-schedulerfrom
fix/cpu-scheduler-no-chunk-barrier
Open

perf(prover): pull-cursor CPU scheduler instead of fixed chunks, plus…#901
diegokingston wants to merge 1 commit into
fix/cpu-prover-schedulerfrom
fix/cpu-scheduler-no-chunk-barrier

Conversation

@diegokingston

Copy link
Copy Markdown
Collaborator

… tests

Two follow-ups to the CPU-aware scheduler.

Keep the pull cursor. The CPU run_admitted batched order into chunks(workers) and ran each chunk to completion, so every table waited for the slowest member of its chunk — the behaviour the doc comment directly above run_admitted says heaviest_first plus the cursor were introduced to remove.

Sorting heaviest-first hides this while the weight estimate is accurate: chunks then hold similar-weight work. It bites when the estimate mispredicts, which is plausible here since the weight is estimate_table_vram_bytes — a device footprint used as a proxy for host time.

This keeps the original CUDA scheduler's shape (atomic cursor, heaviest-first, concurrency bounded by workers) but spawns the workers as Rayon tasks rather than OS threads. That preserves the fix this branch is for — nested per-table Rayon work stays inside the pool and work-steals — while a worker that finishes a small table pulls the next one immediately. TABLE_PARALLELISM keeps its meaning and the number of simultaneously live per-table working sets is unchanged. Blocking a pool worker is only acceptable because, unlike the CUDA path, there is no admission gate to wait on.

Add coverage. run_admitted is now three cfg-gated implementations with no tests. These are cfg-agnostic, so they exercise whichever one the build selects:

  • results land in the slot named by their own index, not by position in order (those differ once the order is heaviest-first);
  • a batch does not serialize when the estimate mispredicts — verified to FAIL at 565ms against the chunked implementation and pass at ~130ms here;
  • heaviest_first is a descending permutation.

Not addressed here, to keep the diff reviewable: the CPU staging block is a near-duplicate of the debug-checks one, and VramGate is still constructed and threaded as _gate on CPU rather than not built.

… tests

Two follow-ups to the CPU-aware scheduler.

**Keep the pull cursor.** The CPU `run_admitted` batched `order` into
`chunks(workers)` and ran each chunk to completion, so every table waited for
the slowest member of its chunk — the behaviour the doc comment directly above
`run_admitted` says `heaviest_first` plus the cursor were introduced to remove.

Sorting heaviest-first hides this while the weight estimate is accurate: chunks
then hold similar-weight work. It bites when the estimate mispredicts, which is
plausible here since the weight is `estimate_table_vram_bytes` — a device
footprint used as a proxy for host time.

This keeps the original CUDA scheduler's shape (atomic cursor, heaviest-first,
concurrency bounded by `workers`) but spawns the workers as Rayon tasks rather
than OS threads. That preserves the fix this branch is for — nested per-table
Rayon work stays inside the pool and work-steals — while a worker that finishes
a small table pulls the next one immediately. `TABLE_PARALLELISM` keeps its
meaning and the number of simultaneously live per-table working sets is
unchanged. Blocking a pool worker is only acceptable because, unlike the CUDA
path, there is no admission gate to wait on.

**Add coverage.** `run_admitted` is now three cfg-gated implementations with no
tests. These are cfg-agnostic, so they exercise whichever one the build selects:

  * results land in the slot named by their own index, not by position in
    `order` (those differ once the order is heaviest-first);
  * a batch does not serialize when the estimate mispredicts — verified to FAIL
    at 565ms against the chunked implementation and pass at ~130ms here;
  * `heaviest_first` is a descending permutation.

Not addressed here, to keep the diff reviewable: the CPU staging block is a
near-duplicate of the `debug-checks` one, and `VramGate` is still constructed
and threaded as `_gate` on CPU rather than not built.
@jotabulacios

Copy link
Copy Markdown
Collaborator

/bench

@yetanotherco yetanotherco deleted a comment from diegokingston Aug 5, 2026
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown

Benchmark — real block (ethrex_mainnet_25368371.bin) (median of 3)

continuations · epoch 2^22 · 12 epochs

Metric main PR Δ
Peak heap 47024 MB 50014 MB +2990 MB (+6.4%) 🔴
Prove time 157.521s 142.752s -14.769s (-9.4%) 🟢

-9.4% — beyond what 3 runs resolve. Use /bench-abba for a paired test of the same block (default 12 pairs, ~72 min, resolves ~1%).

Prove-time spread 0.5% (142.929s / 142.752s / 142.221s)

Commit: 0f2f293 · Baseline: cached · Runner: self-hosted bench

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants