Make telemetry actually reach PostHog, and cut audit --help down to what you need to read

CHANGELOG.md

@@ -1,5 +1,17 @@
 # Changelog
 
+## 1.0.1-beta.1 — 2026-08-16
+
+### Fixes
+
+- **Stop the dashboard server's telemetry from stranding its own events, and stop it printing `Error while flushing PostHog` while doing it.** Four options on the `posthog-node` client each disabled a different part of the library's delivery machinery, and together they turned a slow network into lost events plus a stack trace in the user's terminal — the one `failproofai audit` starts, where `launch()`'s log filter only strips the Server Action skew block. The injected `resilientFetch` was the root of it: it retried five times over ~40s and then returned a synthetic `200` so the library would never log a network error, but posthog-node does not merely hand its abort signal to an injected fetch, it **races that fetch against its own `requestTimeout`** (`Promise.race([fetchPromise, deadline])`) precisely because an injected fetch may ignore the signal — which ours did, by stripping it. A ~40s budget racing a 5s deadline can never return in time, so the synthetic `200` was unreachable code, the `console.error` it existed to prevent fired anyway at 5s, and the retries ran on detached from a client that had already given up. Worse, that `200` was the wrong answer even when it did land: posthog-node deliberately does NOT dequeue a batch that failed with a network error, so reporting success is what would have made it discard events that never arrived. The wrapper is gone; plain global fetch is what the library expects. `fetchRetryCount` was `0`, leaving that wrapper as the only thing retrying, at the wrong layer — the library retries inside a single flush, knows which errors are retryable, and keeps its queue coherent while doing it. `requestTimeout` was `5000`, half the library's own default, so every attempt had half the room. And `flushInterval` was `0`, which is falsy and therefore disables the flush timer outright — that is the one that actually stranded events, because the batch posthog-node retains after a network error then had nothing scheduled to resend it and sat in an in-memory queue (`PostHogMemoryStorage`, so nothing survives the process) until some unrelated later event happened to trigger a flush. `flushAt: 1` is unchanged and deliberate: volume is a handful of events per process, batching buys nothing, and sending immediately is the best defense a memory-only queue has against the process dying. Measured against a server that answers correctly but takes 6s — a slow network, not an outage — the old options delivered the event **four times** and logged two flush errors, because the wrapper re-POSTed the same batch on each of its own retries while the library still held its retained copy; the new ones deliver it **once**, with nothing logged. The exit drain is now idempotent, since `beforeExit` re-fires every time a handler schedules async work and an unguarded one started a fresh 30s `shutdown()` on each pass. **No event, trigger or property changed** — all 73 call sites across the three dispatchers fire exactly as before. (#701)
+
+- Cover telemetry delivery against the real `posthog-node` and a real socket, in `__tests__/lib/telemetry-delivery.test.ts`. The existing suite mocks the library wholesale, and that mock is what let the above live in the tree: the constructor was called with the right *shape*, so it passed while events were being stranded. The new tests assert on bytes that arrived over a socket — including gunzipping the batch body, without which a green test means nothing, since posthog-node gzips it and a raw read silently parses as "no events delivered". They pin that a captured event reaches `/batch/` with its properties intact, that a transient 500 is retried and still delivered, that a successful flush logs no error, and — for the hook dispatcher carrying the other 37 call sites — that `trackHookEvent` reaches `/capture/` and that `flushHookTelemetry` lands events the caller never awaited. Two facts they nail down rather than fix: posthog-node **overwrites `$lib`** with its own name on the server path, so `trackEvent`'s `"failproofai"` never lands and `product` is the attribution that actually survives (the raw-fetch hook dispatcher has no SDK to overwrite it, so its `"failproofai-hooks"` does); and the opt-out still sends nothing. (#701)
+
+- Cut `failproofai audit --help` down to what somebody actually needs to read: two lines saying what the command is, then one aligned USAGE block listing the five things a person can type. It had grown to four sections and forty lines — a `USAGE` block that also carried the headless entry point, a separate `SCHEDULING` block, a `WHAT IT DOES` block re-describing the same scan, and a paragraph about which config file the flags write, which is an implementation detail of the CLI agreeing with the dashboard rather than something to tell a reader at the moment they are looking for a flag. Every command keeps its own usage — `--schedule` still names its optional day count, its 1–90 range, and `--email` — and the local-only promise survives, moved into the second line where it reads as part of what the command IS. `--scheduled` is deliberately dropped from the listing: it is not a flag anybody types but the second entry point the daemon spawns, and advertising a machine-facing flag one letter away from `--schedule` is how somebody starts a full scan meaning to configure one. It still works and still refuses every argument it always refused. Command names are brand teal through the same `colorOn()` gate as the rest of the CLI, so piped output stays plain, and every line fits 80 columns. No behaviour changed — all five commands dispatch exactly as before, verified against a temp home. (#701)
+
+- Correct a comment in `hook-telemetry.ts` claiming `isTelemetryEnabled()` is memoised. `lib/telemetry-enabled.ts` documents at length that it is resolved fresh on every call, deliberately — an opt-out a long-lived process ignores until restart is not an opt-out — so the comment described the exact optimisation that file rejects. (#701)
+
 ## 1.0.1-beta.0 — 2026-08-14
 
 ### Features

__tests__/audit/audit-cli-help.test.ts

@@ -0,0 +1,92 @@
+// @vitest-environment node
+/**
+ * `failproofai audit --help`.
+ *
+ * Two things worth pinning. The obvious one is that every command a person can
+ * type is listed — a scheduling flag that exists and is undiscoverable is the
+ * same to a user as one that does not exist.
+ *
+ * The subtle one is the ALIGNMENT. The description column is produced by
+ * padding against the raw command string, because `c()` wraps it in ANSI escape
+ * bytes that occupy no terminal columns — pad against the coloured string and
+ * every row shifts left by the width of an escape sequence, but only when
+ * colour is on, which is never how the output is read in CI. So the width
+ * assertions below run in BOTH modes.
+ */
+import { describe, it, expect, afterEach } from "vitest";
+import { helpText } from "../../src/audit/cli";
+
+/** Strip ANSI so a rendered line can be measured in terminal columns. */
+const plain = (s: string): string => s.replace(/\[[0-9;]*m/g, "");
+
+describe("audit --help", () => {
+  const originalEnv = { ...process.env };
+
+  afterEach(() => {
+    process.env = { ...originalEnv };
+  });
+
+  function render(color: boolean): string {
+    if (color) {
+      process.env.FORCE_COLOR = "1";
+      delete process.env.NO_COLOR;
+    } else {
+      process.env.NO_COLOR = "1";
+      delete process.env.FORCE_COLOR;
+    }
+    return helpText();
+  }
+
+  it("lists every command a person can type", () => {
+    const text = plain(render(false));
+    for (const command of [
+      "failproofai audit",
+      "failproofai audit --schedule [days]",
+      "failproofai audit --no-schedule",
+      "failproofai audit --status",
+      "failproofai audit -h, --help",
+    ]) {
+      expect(text).toContain(command);
+    }
+    // --email modifies --schedule rather than standing alone, so it is named in
+    // that entry rather than given a row of its own.
+    expect(text).toContain("--email <address>");
+  });
+
+  it("omits --scheduled, which the daemon spawns and nobody types", () => {
+    // One letter from `--schedule` and it starts a full scan instead of
+    // configuring one. It still works; it is just not advertised beside it.
+    expect(plain(render(false))).not.toContain("--scheduled");
+  });
+
+  it("keeps the local-only promise the docs also make", () => {
+    expect(plain(render(false))).toMatch(/runs on this machine/i);
+  });
+
+  it.each([true, false])("aligns and fits 80 columns with color=%s", (color) => {
+    const lines = plain(render(color)).split("\n");
+
+    for (const line of lines) {
+      expect(line.length).toBeLessThanOrEqual(80);
+    }
+
+    // Every command row and every continuation line shares one description
+    // column. Derive it from the first row rather than restating the constant,
+    // so this fails on drift instead of being updated to match it.
+    const first = lines.find((l) => l.includes("failproofai audit  "));
+    expect(first).toBeDefined();
+    const descCol = first!.indexOf("Scan your session history");
+    expect(descCol).toBeGreaterThan(0);
+
+    const continuations = lines.filter(
+      (l) => l.startsWith(" ".repeat(descCol)) && l.trim().length > 0,
+    );
+    // The rows carry six continuation lines between them. A floor rather than
+    // an exact count, so reworded copy does not fail this — but a regression in
+    // the padding math moves them off `descCol` entirely and drops it to zero.
+    expect(continuations.length).toBeGreaterThanOrEqual(6);
+    for (const line of continuations) {
+      expect(line[descCol]).not.toBe(" ");
+    }
+  });
+});