Probe runner: pass --probe-notes/--hold to the CLAP prober; copy cost with voices, the LV2 note sweep and the note-onset latency

BUILDING.md

@@ -97,6 +97,12 @@ omx-clap-host does not take them yet (a tempo, FreeMixer/omx-clap-core#88; the n
 case prints `PENDING` with the issue and checks what the worker answers today. Without `OMX_CLAP_HOST_SRC` the target
 prints `skip test-lv2-adapter` and does nothing else.
 
+`make test-probe` runs the probe runner over `tests/probe-child` and `tests/lv2-probe-child`, stand-ins that print the
+lines of plugin-qualify's prober and `lv2_measure`. With `PLUGIN_QUALIFY_BUILD=<plugin-qualify checkout>/native/build`
+(`make -C native all` there) it also runs the real ones over plugin-qualify's instrument fixtures: a CLAP synth timed
+holding `--probe-notes` at every rate of omx-contract's `ORACLE_FLOOR_RATES`, and an LV2 synth that allocates on a note
+beside its clean twin. Without it those tests print `skip`.
+
 `make sabotage` (`tests/sabotage.py`) breaks the daemon on purpose, one guard at a time, and
 requires the named test to go red.
 

README.md

@@ -317,12 +317,20 @@ The runner never loads plugin code itself.
     plugin-hostd-probe run --out probe.json --log probe.stderr.log /usr/lib64/clap/plate.clap org.example.plate
     plugin-hostd-probe run --out probe.json lv2:http://example.org/plate
 
+A CLAP instrument is timed holding notes down: as many as the voice capacity it declares, or, when it declares none,
+the count `--probe-notes N` gives; `--hold N` holds that count whatever it declares. Both are passed to clap-qualify.
+Each rate's grid point carries the cost clap-qualify timed at that rate, in nanoseconds per sample, and for an instrument
+`voices`, the notes it held and why that many (`declared` or `probe`). A rate it did not time has no cost. An instrument has
+no input for the impulse, so its latency is measured from a note (`method: note-onset`), with `heard: false` at a rate
+where the note did not sound.
+
 An LV2 plugin is measured the way plugin-qualify's LV2 hosting pass measures it today, in two children under the same
 limits and the same wall clock. The first child (`lv2_measure --pin`) pins the bundle's files without instantiating the
 plugin. The second measures at the highest of `--rates`, in blocks of `--quantum`, with 30 instantiate and destroy cycles
 and 3 swept rt-safety runs. It runs under plugin-qualify's interposer (`rt_interpose.so` beside the child) and the noise
 stimulus. The document's binary is the plugin's library, with every pinned file of the bundle listed. The rate's grid
-point carries what run() did under the interposer, and `behaviour` carries the lifecycles, the threads and the features
+point carries what run() did under the interposer and, for a plugin with a MIDI input, its note sweep (what run() did
+while notes played), and `behaviour` carries the lifecycles, the threads and the features
 the plugin requires. A plugin no bundle holds exits 3 with no document. A plugin of which nothing could be pinned exits 2
 with no document, because the document has no binary to name. Its refusal is a code on stderr: `<plugin>: <code>: <why>`,
 `probe.unpinnable` for a bundle the pin cannot cover. Version 1 runs the quick measurement only: the deep lifecycle pass

doc/plugin-hostd-probe.1

@@ -41,6 +41,13 @@ Blocks to process per rate.
 .BI \-\-offer " EXT,..."
 Host extensions to offer on top of the child's own.
 .TP
+.BI \-\-probe\-notes " n"
+The notes a CLAP instrument that declares no voice capacity is timed holding. Passed to clap\-qualify; without it such
+an instrument has no cost.
+.TP
+.BI \-\-hold " n"
+The notes a CLAP instrument is timed holding, whatever voice capacity it declares. Passed to clap\-qualify.
+.TP
 .BI \-\-out " file"
 Where to write the document, atomically. Default standard output.
 .TP

include/plugin-hostd/protocol.h

@@ -116,7 +116,7 @@
 
 #define PHD_PROBE_FORMAT "plugin-hostd.probe-output"
 #define PHD_PROBE_FORMAT_MAJOR 1
-#define PHD_PROBE_FORMAT_MINOR 0
+#define PHD_PROBE_FORMAT_MINOR 1
 #define PHD_PROBE_PROTOCOL_NAME "plugin-hostd.probe-protocol"
 #define PHD_PROBE_PROTOCOL_VERSION 1
 /* version 1 of the probe protocol runs an LV2 plugin as plugin-qualify's LV2 hosting pass does today

protocol/probe-output.schema.in

@@ -745,8 +745,13 @@
             "method": {
               "enum": [
                 "impulse-onset",
-                "declared-port"
+                "declared-port",
+                "note-onset"
               ]
+            },
+            "heard": {
+              "description": "A note-onset latency's: whether the note sounded, so an onset that never came is not read as one.",
+              "type": "boolean"
             }
           }
         },
@@ -820,6 +825,31 @@
             },
             "samples": {
               "$ref": "#/$defs/fileRef"
+            },
+            "voices": {
+              "description": "An instrument's: the notes it held down while it was timed, and why that many.",
+              "type": "object",
+              "required": [
+                "held",
+                "basis"
+              ],
+              "additionalProperties": false,
+              "properties": {
+                "held": {
+                  "type": "integer",
+                  "minimum": 1
+                },
+                "basis": {
+                  "enum": [
+                    "declared",
+                    "probe"
+                  ]
+                },
+                "capacity": {
+                  "type": "integer",
+                  "minimum": 0
+                }
+              }
             }
           }
         },

protocol/probe-output.schema.json

@@ -745,8 +745,13 @@
             "method": {
               "enum": [
                 "impulse-onset",
-                "declared-port"
+                "declared-port",
+                "note-onset"
               ]
+            },
+            "heard": {
+              "description": "A note-onset latency's: whether the note sounded, so an onset that never came is not read as one.",
+              "type": "boolean"
             }
           }
         },
@@ -820,6 +825,31 @@
             },
             "samples": {
               "$ref": "#/$defs/fileRef"
+            },
+            "voices": {
+              "description": "An instrument's: the notes it held down while it was timed, and why that many.",
+              "type": "object",
+              "required": [
+                "held",
+                "basis"
+              ],
+              "additionalProperties": false,
+              "properties": {
+                "held": {
+                  "type": "integer",
+                  "minimum": 1
+                },
+                "basis": {
+                  "enum": [
+                    "declared",
+                    "probe"
+                  ]
+                },
+                "capacity": {
+                  "type": "integer",
+                  "minimum": 0
+                }
+              }
             }
           }
         },

src/probe.c

@@ -414,6 +414,8 @@ typedef struct
     const char *offer;
     const char *set;
     unsigned quantum, blocks;
+    /* the notes a CLAP instrument is timed holding: when it declares no voice capacity, and whatever it declares */
+    unsigned probe_notes, hold;
     long wall_s, cpu_s, files;
     long long data_bytes, file_bytes;
     long grace_ms;
@@ -1072,7 +1074,7 @@ static void take_step_records(run_t *r)
     r->steps.s[r->steps.len] = '\0';
 }
 
-/* clap-qualify's arguments: the rates, the block, the offer, the file and the id */
+/* clap-qualify's arguments: the rates, the block, the offer, the notes an instrument holds, the file and the id */
 static void clap_argv(const options_t *o, char **argv)
 {
     int argc = 0;
@@ -1098,6 +1100,22 @@ static void clap_argv(const options_t *o, char **argv)
         argv[argc++] = "--offer";
         argv[argc++] = (char *)o->offer;
     }
+    {
+        static char notes[32], hold[32];
+
+        if (o->probe_notes)
+        {
+            snprintf(notes, sizeof(notes), "%u", o->probe_notes);
+            argv[argc++] = "--probe-notes";
+            argv[argc++] = notes;
+        }
+        if (o->hold)
+        {
+            snprintf(hold, sizeof(hold), "%u", o->hold);
+            argv[argc++] = "--hold";
+            argv[argc++] = hold;
+        }
+    }
     argv[argc++] = (char *)o->plugin;
     if (o->id)
         argv[argc++] = (char *)o->id;
@@ -1454,8 +1472,25 @@ static void put_count(sb_t *b, const jnode_t *o, const char *from, const char *t
         sb_printf(b, ", \"%s\": %.0f", to, v);
 }
 
+/* a prober's note sweep, as a point carries it: whether it applied, the dialect it played in, and what process() or run()
+ * did while it played when the interposer counted */
+static void put_note_sweep(sb_t *b, const jnode_t *ns)
+{
+    sb_printf(b, ", \"noteSweep\": {\"applicable\": %s", jbool(ns, "applicable", 0) ? "true" : "false");
+    if (jstr(ns, "dialect"))
+    {
+        sb_printf(b, ", \"dialect\": ");
+        sb_str(b, jstr(ns, "dialect"));
+    }
+    put_count(b, ns, "allocationsInRun", "allocations");
+    put_count(b, ns, "locksInRun", "locks");
+    put_count(b, ns, "syscallsInRun", "syscalls");
+    sb_printf(b, "}");
+}
+
 /* lv2_measure's measurement line: the grid point of its rate and block, with what run() did under the interposer, and
- * the plugin's behaviour over the pass (lifecycles, threads, the features it requires); a translation, no judgement */
+ * its note sweep, and the plugin's behaviour over the pass (lifecycles, threads, the features it requires); a
+ * translation, no judgement */
 static void put_lv2_measure(sb_t *b, const jnode_t *m, long rate, unsigned quantum)
 {
     const jnode_t *rt = jget(m, "rtSafety"), *ev = jget(rt, "events"), *life = jget(m, "lifecycle"), *th = jget(m, "threads"),
@@ -1487,6 +1522,8 @@ static void put_lv2_measure(sb_t *b, const jnode_t *m, long rate, unsigned quant
             sb_printf(b, "]");
         }
         sb_printf(b, "}");
+        if (jget(rt, "noteSweep"))
+            put_note_sweep(b, jget(rt, "noteSweep"));
     }
     sb_printf(b, "}\n  ],\n  \"behaviour\": {");
     if (life)
@@ -1517,6 +1554,32 @@ static void put_lv2_measure(sb_t *b, const jnode_t *m, long rate, unsigned quant
     sb_printf(b, "]}}");
 }
 
+/* the cost arm's figures at one rate, as that rate's point carries them: nanoseconds per sample, and for an instrument
+ * the notes it held; a rate the arm did not time (an unmeasuredReason in place of the figures) gets no cost, never a
+ * neighbour's */
+static void put_cost(sb_t *b, const jnode_t *cost, const char *rate)
+{
+    const jnode_t *at = jget(jget(cost, "perRate"), rate), *voices = jget(at, "voices");
+    double median, p95, min, max, blocks, warmup, percentile, held, capacity;
+
+    if (!jnum(at, "nsPerSampleMedian", &median) || !jnum(at, "nsPerSampleP95", &p95) ||
+        !jnum(at, "nsPerSampleMin", &min) || !jnum(at, "nsPerSampleMax", &max) || !jnum(at, "blocks", &blocks) ||
+        !jnum(cost, "warmupBlocks", &warmup) || !jnum(cost, "percentile", &percentile))
+        return;
+    sb_printf(b, ", \"cost\": {\"unit\": \"ns-per-sample\", \"median\": %.9g, \"percentile\": {\"p\": %.9g, \"value\": %.9g}, "
+                 "\"min\": %.9g, \"max\": %.9g, \"blocks\": %.0f, \"warmupBlocks\": %.0f",
+              median, percentile, p95, min, max, blocks, warmup);
+    if (jnum(voices, "held", &held) && jstr(voices, "basis"))
+    {
+        sb_printf(b, ", \"voices\": {\"held\": %.0f, \"basis\": ", held);
+        sb_str(b, jstr(voices, "basis"));
+        if (jnum(voices, "capacity", &capacity))
+            sb_printf(b, ", \"capacity\": %.0f", capacity);
+        sb_printf(b, "}");
+    }
+    sb_printf(b, "}");
+}
+
 /* the rates a run measures at when it is given none: omx-contract's ORACLE_FLOOR_RATES, every kernel oracle's floor */
 static const unsigned g_floor_rates[] = OMX_ORACLE_FLOOR_RATES_INIT;
 
@@ -1568,6 +1631,8 @@ int main(int argc, char **argv)
                         "  --rates R,R,...      sample rates (default %s)\n"
                         "  --quantum N          block size in frames (default 512)     --blocks N   blocks per rate\n"
                         "  --offer EXT,...      host extensions to offer on top of the child's own\n"
+                        "  --probe-notes N      notes a CLAP instrument declaring no voice capacity is timed holding\n"
+                        "  --hold N             notes a CLAP instrument is timed holding, whatever it declares\n"
                         "  --out FILE           the document, written atomically (default stdout)\n"
                         "  --log FILE           the child's stderr (default discarded)\n"
                         "  --reason WHY         add, rescan, binary-changed, on-request, auto-measure, reference, ci\n"
@@ -1594,6 +1659,19 @@ int main(int argc, char **argv)
         else if (!strcmp(k, "--quantum")) o.quantum = (unsigned)atoi(v);
         else if (!strcmp(k, "--blocks")) o.blocks = (unsigned)atoi(v);
         else if (!strcmp(k, "--offer")) o.offer = v;
+        else if (!strcmp(k, "--probe-notes") || !strcmp(k, "--hold"))
+        {
+            char *end;
+            long n = strtol(v, &end, 10);
+
+            /* how many the prober can hold is the prober's to say: a count it refuses ends its run as any refusal does */
+            if (*end || n < 1 || n > INT_MAX)
+            {
+                fprintf(stderr, RUNNER_NAME ": %s takes a positive count of notes\n", k);
+                return 2;
+            }
+            *(k[2] == 'p' ? &o.probe_notes : &o.hold) = (unsigned)n;
+        }
         else if (!strcmp(k, "--out")) o.out = v;
         else if (!strcmp(k, "--log")) o.log = v;
         else if (!strcmp(k, "--reason")) o.reason = v;
@@ -1758,7 +1836,8 @@ int main(int argc, char **argv)
     {
         const char *plugin_id = line ? jstr(line, "id") : NULL;
         const jnode_t *clap = jget(line, "clap"), *integ = jget(line, "integrity"), *rt = jget(line, "rtSafety"),
-                      *notes = jget(line, "noteSweep"), *latency = jget(clap, "latency"), *ext = jget(line, "hostExtensions");
+                      *notes = jget(line, "noteSweep"), *latency = jget(clap, "latency"), *ext = jget(line, "hostExtensions"),
+                      *cost = jget(line, "cost");
         const char *layout_sha = NULL, *unmeasured = NULL;
         int firstpoint = 1;
         size_t k;
@@ -1925,19 +2004,26 @@ int main(int argc, char **argv)
             put_step(&doc, &g_steps[i]);
         }
         sb_printf(&doc, "\n  ]");
-        /* the grid: one point per rate the child reported a latency for */
+        /* the grid: one point per rate the child reported a latency for, with the cost the child timed at that rate */
         if (latency && latency->n)
         {
             sb_printf(&doc, ",\n  \"grid\": [");
             for (k = 0; k < latency->n && k < MAX_POINTS; k++)
             {
                 double decl = 0, meas = 0;
+                int from_note;
 
                 jnum(latency->kid[k], "declaredFrames", &decl);
                 jnum(latency->kid[k], "measuredFrames", &meas);
+                /* an instrument has no input for the impulse: its onset is measured from a note, which may not sound */
+                from_note = jstr(latency->kid[k], "from") && !strcmp(jstr(latency->kid[k], "from"), "note");
                 sb_printf(&doc, "%s\n    {\"rate\": %ld, \"quantum\": %u, \"outcome\": \"%s\", \"latency\": {\"declaredFrames\": %.0f, "
-                                "\"measuredFrames\": %.0f, \"method\": \"impulse-onset\"}",
-                          k ? "," : "", atol(latency->key[k]), o.quantum, OUT_OK, decl, meas);
+                                "\"measuredFrames\": %.0f, \"method\": \"%s\"",
+                          k ? "," : "", atol(latency->key[k]), o.quantum, OUT_OK, decl, meas, from_note ? "note-onset" : "impulse-onset");
+                if (from_note)
+                    sb_printf(&doc, ", \"heard\": %s", jbool(latency->kid[k], "heard", 1) ? "true" : "false");
+                sb_printf(&doc, "}");
+                put_cost(&doc, cost, latency->key[k]);
                 if (firstpoint && rt)
                 {
                     double v = 0;
@@ -1975,15 +2061,7 @@ int main(int argc, char **argv)
                     }
                     sb_printf(&doc, "}");
                     if (notes)
-                    {
-                        sb_printf(&doc, ", \"noteSweep\": {\"applicable\": %s", jbool(notes, "applicable", 0) ? "true" : "false");
-                        if (jstr(notes, "dialect"))
-                        {
-                            sb_printf(&doc, ", \"dialect\": ");
-                            sb_str(&doc, jstr(notes, "dialect"));
-                        }
-                        sb_printf(&doc, "}");
-                    }
+                        put_note_sweep(&doc, notes);
                     firstpoint = 0;
                 }
                 sb_printf(&doc, "}");