Repository navigation
Bench: plan and pool modes, xruns, latency, stress and repeats with spread - #194
Merged
Merged
Conversation
…s plan and pool (#188) Fields only, so the document stays omx-bench/1. A spread has to hold its median and name a field the row has, and a stressed row has to report its xruns. The same scenario with and without stress is two rows.
A client on the run's private server counts the xrun callbacks over each measurement window and reads the capture latency at the playback port. If the server's own log names xruns the probe never heard, the row fails instead of reading zero.
One busy loop pinned to each core the host may use, plus a process that keeps touching 256 MiB. The row carries the profile name, so the same name on the same machine is the same load. Scenarios that declare stress run both ways unless --stress says otherwise.
--repeats (3 by default) runs a row again and writes the median of every measure, a count taking the higher middle run, with [min, max] under spread. A failed repeat fails the row and names which one; an unsupported row is not repeated.
The grouping-vs-processes family (grp8, grp16) runs one strip four ways under plugin-hostd: one process per plugin, one chain, plan, and plan with the shareable members it left alone in a pool. The plan and pool rows run with pins on, a verdict per plugin (isolated for the ones the strip names, in-process for the rest) written into the daemon's own pin store, and grouping on. A plan of one segment, a pool the daemon did not form, or a daemon without the grouping setting is a row that says so, never a number.
The figures with their spread, the ratios of one process per plugin to one chain, and the xruns summed per mode with and without stress, so the grouped-vs-processes table is made from the documents and not typed.
linuxnow
force-pushed
the
feat/188-bench-modes
branch
from
October 27, 2026 09:33
22e51e5 to
0c55eb1
Compare
This was referenced Oct 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #188. Part of #182. Also covers the xrun and latency fields asked for in #192.
Base:
feat/bench-harness(PR #179 has not merged). Retarget this PR tomainwhen #179 lands.The grouping design note (docs/design/notes/2026-10-27-grouping-planner.md, "The benchmark") asks for the bench to run a strip as
individual,chain,planandpool, to report xruns and a stress profile, and to repeat each row three times with the median and the spread. This adds that, and a script that makes the grouped-vs-processes table from the documents.What changes, in small commits:
omx-bench/1, fields only):xruns,latency_frames,stress,repeats,spread, and the modesplanandpool. A spread has to hold its median, a stressed row has to report xruns, and the same scenario with and without stress is two rows.xrunsandlatency_frames: a client on the run's private JACK server counts xrun callbacks over the measurement window (libjack through ctypes, no port, no process callback) and reads the capture latency at the playback port. If the server's own log names xruns the probe never heard, the row fails instead of reading 0.busy<cores>+mem256.--stress none|on|both, both by default for the cvp and grp scenarios.--repeats(3 by default). Median per measure, a count takes the higher middle run,[min, max]underspread. A failed repeat fails the row and says which.planandpool: new familygrouping-vs-processes(grp8,grp16, CLAP plugins, omx-drive pinnedisolated). The plan and pool rows run with pins on and a verdict per plugin written into the daemon's own pin store,grouping on, and the package pins off. A daemon without the grouping setting gives anunsupportedrow; a plan of one segment, or a pool the daemon did not form from the members moved into it, gives afailedrow.bench/table.sh: figures with their spread, the individual-over-chain ratios, and xruns summed per mode with and without stress.Tests (
python3 -I bench/tests/test_bench.py, 51 tests, no JACK and no host needed) and the sabotage of each guard inbench/tests/sabotage.sh(34 new cases, each breaks one line in a copy and requires its test to go red). CI runs both.Not done here, on purpose:
plan,plan_getverbs and thegroupingsetting are those of the design note; the planner (L2 grouping: the planner and the plan, plan_get, plan_clear verbs #185) and the worker'sxrunsline (omx-clap-core#371) are not merged, so on today's daemon the plan and pool rows areunsupported.