Skip to content

Bench: plan and pool modes, xruns, latency, stress and repeats with spread - #194

Merged
linuxnow merged 7 commits into
mainfrom
feat/188-bench-modes
Oct 27, 2026
Merged

linuxnow merged 7 commits into
mainfrom
feat/188-bench-modes

Conversation

@linuxnow

@linuxnow linuxnow commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Closes #188. Part of #182. Also covers the xrun and latency fields asked for in #192.

Base: feat/bench-harness (PR #179 has not merged). Retarget this PR to main when #179 lands.

The grouping design note (docs/design/notes/2026-10-27-grouping-planner.md, "The benchmark") asks for the bench to run a strip as individual, chain, plan and pool, to report xruns and a stress profile, and to repeat each row three times with the median and the spread. This adds that, and a script that makes the grouped-vs-processes table from the documents.

What changes, in small commits:

  • Schema (still omx-bench/1, fields only): xruns, latency_frames, stress, repeats, spread, and the modes plan and pool. A spread has to hold its median, a stressed row has to report xruns, and the same scenario with and without stress is two rows.
  • xruns and latency_frames: a client on the run's private JACK server counts xrun callbacks over the measurement window (libjack through ctypes, no port, no process callback) and reads the capture latency at the playback port. If the server's own log names xruns the probe never heard, the row fails instead of reading 0.
  • Stress: one busy loop pinned to each core the host may use, plus a process that touches 256 MiB, for the whole window. The row carries busy<cores>+mem256. --stress none|on|both, both by default for the cvp and grp scenarios.
  • Repeats: --repeats (3 by default). Median per measure, a count takes the higher middle run, [min, max] under spread. A failed repeat fails the row and says which.
  • plan and pool: new family grouping-vs-processes (grp8, grp16, CLAP plugins, omx-drive pinned isolated). The plan and pool rows run with pins on and a verdict per plugin written into the daemon's own pin store, grouping on, and the package pins off. A daemon without the grouping setting gives an unsupported row; a plan of one segment, or a pool the daemon did not form from the members moved into it, gives a failed row.
  • bench/table.sh: figures with their spread, the individual-over-chain ratios, and xruns summed per mode with and without stress.

Tests (python3 -I bench/tests/test_bench.py, 51 tests, no JACK and no host needed) and the sabotage of each guard in bench/tests/sabotage.sh (34 new cases, each breaks one line in a copy and requires its test to go red). CI runs both.

Not done here, on purpose:

  • Nothing was run against JACK or the Pi. The ctypes calls to libjack are tested against a stand-in, and the plan and pool steps against a stand-in daemon, so the first real run on a desk or the Pi is the check of those two seams.
  • The plan, plan_get verbs and the grouping setting are those of the design note; the planner (L2 grouping: the planner and the plan, plan_get, plan_clear verbs #185) and the worker's xruns line (omx-clap-core#371) are not merged, so on today's daemon the plan and pool rows are unsupported.
  • The cross-chain pool is not measured: one strip is one chain, so the pool holds that strip's leftovers.

Base automatically changed from feat/bench-harness to main October 27, 2026 09:27
…s plan and pool (#188)

Fields only, so the document stays omx-bench/1. A spread has to hold its
median and name a field the row has, and a stressed row has to report its
xruns. The same scenario with and without stress is two rows.
A client on the run's private server counts the xrun callbacks over each
measurement window and reads the capture latency at the playback port. If the
server's own log names xruns the probe never heard, the row fails instead of
reading zero.
One busy loop pinned to each core the host may use, plus a process that keeps
touching 256 MiB. The row carries the profile name, so the same name on the
same machine is the same load. Scenarios that declare stress run both ways
unless --stress says otherwise.
--repeats (3 by default) runs a row again and writes the median of every
measure, a count taking the higher middle run, with [min, max] under spread. A
failed repeat fails the row and names which one; an unsupported row is not
repeated.
The grouping-vs-processes family (grp8, grp16) runs one strip four ways under
plugin-hostd: one process per plugin, one chain, plan, and plan with the
shareable members it left alone in a pool. The plan and pool rows run with pins
on, a verdict per plugin (isolated for the ones the strip names, in-process for
the rest) written into the daemon's own pin store, and grouping on. A plan of
one segment, a pool the daemon did not form, or a daemon without the grouping
setting is a row that says so, never a number.
The figures with their spread, the ratios of one process per plugin to one
chain, and the xruns summed per mode with and without stress, so the
grouped-vs-processes table is made from the documents and not typed.
@linuxnow
linuxnow force-pushed the feat/188-bench-modes branch from 22e51e5 to 0c55eb1 Compare October 27, 2026 09:33
@linuxnow
linuxnow merged commit 027f4e4 into main Oct 27, 2026
37 checks passed
@linuxnow
linuxnow deleted the feat/188-bench-modes branch October 27, 2026 10:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

L6 bench: plan and pool modes, xruns and stress fields, repeats with spread

1 participant