Repository navigation
Conversation
One JSON document per run: the machine, the software, one row per scenario, host, mode, rate and frame count. validate.sh reads the schema and the rules across rows it cannot say (one row per key, a max not below its average, recover_ms only on an ok row) and names the file, the row and the rule of each problem.
run.sh starts a private JACK server per rate and frame count (jackd -n bench<pid> -d dummy), each host fresh per row on a port of its own (H1 omx-clap-host, H2 plugin-hostd with require_pins 0, H3 mod-host, H4 one jalv per plugin), wires a sine or jack_midiseq's chords through the plugins in series, and reads jack_cpu_load, RSS, PSS, CPU time and context switches of the host's processes over the window, the add time, and for H2 the time from kill -9 of the worker until its client is back and wired. One controller connection per host is held for the row: the daemon sees a worker's death while a controller is connected. The chains-vs-processes rows run the same N plugins as N workers, one chain, one shared process and N jalv processes. Every process carries the run's mark; the teardown stops them, puts the governor back, and the run exits 3 if a marked process is left. send.sh sends protocol lines by hand.
Five broken documents (an undeclared status, an ok row with no DSP load, an unsupported row with no reason, a max below its average, an undeclared field) each fail validate.sh naming file, row and rule. A runner whose host stop leaves omx-clap-host fails after the row; one that leaves jackd fails at the end, exit 3. The runner as it is ends green with nothing left.
The README maps hosts and modes, lists the options and the fields. The bench job, in debian:trixie with the channel's packages: the example against the schema (the harness's reader and python3-jsonschema), shellcheck, the sabotage script, and lv2-1 under every host the container has at 48 kHz / 256 frames, checking that a valid document comes out.
On Fedora 44 without lv2-calf-plugins every host answered the Calf add -101; the row now says the plugin is not installed. With it installed, all four hosts run lv2-1, chain3, clap16 and cvp3 there (mod-host 0.10.6.f14a230-11.1.fmx.fc44). The README names Fedora's packages; the sabotage script passes shellcheck.
debian:trixie amd64 in a container, private jackd dummy server, 48000 and 96000 Hz by 128 and 256 frames, 10 s windows, --qualify: 172 rows. mod-host is not in the Debian channel, so H3 rows are unsupported. At 48000/256: cvp16 runs 17 processes, 173.2 MB RSS, 7.25% CPU under H2 individual against 2, 40.3 MB, 4.77% as one chain; H2 kill1 recovers in 322.5, 778.1, 315.2 and 319.6 ms across the four settings. The DSP load of class ci shows the format, not a device.
A JACK server killed outright leaves its shared-memory registry in /dev/shm, and the next private server fails to start: in a container whose pid 1 reaps nothing (the CI job's), the sabotage step's green run after the jackd-left sabotage did. The last check now stops what it finds with SIGTERM, then SIGKILL after 3 s, and the leftover check after the sabotage counts running processes, not zombies. Green there with no init.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
bench/run.sh runs the same plugins under omx-clap-host alone (H1), plugin-hostd with omx-clap-host as its worker (H2), mod-host (H3) and one jalv per plugin (H4), on a private JACK server per rate and frame count, and writes one omx-bench/1 document per run.