fix: serialize Hermes inbox doorbells with typed commands

[dnth]

Intent

Fix the fork-specific Hermes steering race deferred from PR #68: an ordinary durable-inbox doorbell currently reaches Hermes terminal input before the shared .hermes-delivery.lock, so it can interleave with a concurrent typed slash command.

Serialize Hermes doorbells at the single shared ring boundary with the same delivery lock used by typed commands, preserving the durable inbox as the ordinary-text data plane and leaving all non-Hermes harness behavior unchanged. Avoid a second lock owner, a polling workaround, timing sleeps, or a compatibility path.

Accepted requirements:

• AC1: Every Hermes terminal write caused by an inbox doorbell and every typed Hermes command participates in one canonical .hermes-delivery.lock critical section before input reaches the terminal; lock acquisition and release are correct on success, refusal, timeout, and signal paths.
• AC2: A deterministic concurrency regression forces a typed slash command and an ordinary inbox doorbell to overlap and proves their bytes cannot interleave, while the durable ordinary-text record remains intact and acknowledgement semantics are unchanged.
• AC3: Busy/idle classification, turn-start acknowledgement for typed commands, ordinary inbox delivery, decision closing, and non-Hermes harness behavior remain unchanged through executable behavioral tests.
• AC4: Lock ordering cannot deadlock with the inbox, metadata, or backend locks; tests cover concurrent ring attempts, a failed/swallowed ring, and recovery after the lock holder exits.
• AC5: Focused Hermes/inbox/backend tests, the applicable behavior suite, bin/fm-lint.sh, and any affected harness-compatibility checks pass.
• AC6: The final diff follows the single-owner contract and adds no unrelated PR feat: add durable local steering inbox #68 changes.

Deliberate design decisions a reviewer reading only the diff would not know:

1. The lock is taken inside fm_task_inbox_ring rather than at each call site on purpose. That function is the ONE boundary where every doorbell-caused Hermes terminal write happens: both production callers (bin/fm-send.sh's first ring after enqueue, and bin/fm-watch.sh's re-ring) go through it. Putting the critical section there serializes both callers without creating a second lock owner, which the task explicitly forbids.

2. The wait is BOUNDED and a contended ring is a skip, not a block. It reuses the library's existing fm_task_inbox_lock_acquire (bounded by FM_TASK_INBOX_LOCK_WAIT_SECS, default 5) rather than the unbounded fm_lock_acquire_wait the typed plane uses. This is deliberate: bin/fm-watch.sh is a long-lived singleton watcher, and an unbounded wait there would park the whole watcher behind a long Hermes turn. A refusal returns the new code 5, the durable record is untouched, and the existing re-ring ladder retries - which is exactly the pre-existing semantics for the other non-delivery outcomes (1 composer-pending, 2 send failed). This is not a polling workaround; it is the ring's already-established best-effort contract.

3. Return code 5 is new rather than reusing 1. Code 1 means "the composer provenly holds pending text" and bin/fm-send.sh prints that exact reason to the operator. Reusing it would have made fm-send report a false cause, so a distinct code with its own message keeps the reported reason honest.

4. fm_task_inbox_hermes_delivery_lock_path was added so the lock path has exactly one owner. bin/fm-send.sh previously composed the path inline; it now calls the helper. The ring derives the state dir and task id from the record path using the same string derivation the OMP branch of that function already uses for its ready marker, rather than adding new parameters to a public function with two production callers and existing tests.

5. The subshell exists specifically so the lock is released on signal paths (EXIT HUP INT TERM) without requiring either caller to cooperate via its own trap. The trap is installed only AFTER a proven acquisition; installing it before would make a refused acquisition release a lock owned by someone else.

Deadlock analysis (AC4): ordering is one-directional and cannot cycle. The inbox sequence lock and the metadata lock are both released before the ring is called, so neither is ever held across it. The watcher singleton lock is only ever acquired before this lock, never after. fm_lock_try_acquire already reclaims a lock from a holder that exited, so a crashed holder cannot wedge the boundary permanently.

Verification performed: tests/fm-task-inbox.test.sh (14 cases), tests/fm-hermes-harness.test.sh (22 cases), tests/fm-send-inbox.test.sh, tests/fm-omp-task-inbox-doorbell.test.sh, tests/fm-backend.test.sh, bin/fm-lint.sh (exit 0) and bin/fm-doc-audience-check.sh all pass. Both new regressions were deliberately checked for vacuity: with the serialization removed they fail ("expected exit 5, got 0") and with it restored they pass. An earlier end-to-end formulation that only observed the fake backend was discarded precisely because it passed with the fix removed.

No maintained prose surface changed, and no per-harness verification fact changed: the fix adds no vendor-emitted signal, only a filesystem lock, so no live-harness proof was required.

Firstmate-Validation-Generation: 4fad98529b0c29941f211a7b7151c2a4

What Changed

• Added a canonical Hermes delivery-lock path and wrapped inbox doorbells in a bounded, signal-safe critical section with explicit refusal handling.
• Updated fm-send.sh to use the shared lock path for typed Hermes delivery while keeping ordinary inbox records durable and retries best-effort.
• Added regression coverage for concurrent delivery, lock contention/recovery/signals, and unchanged non-Hermes behavior.

Risk Assessment

✅ Low: The change centralizes Hermes doorbell locking on the existing shared lock, preserves non-Hermes behavior, and correctly releases then exits on signal paths without introducing a separate owner or compatibility path.

Testing

Ran the focused inbox, Hermes harness, fm-send inbox, OMP doorbell, and backend behavior suites against target commit a3c05ca. The lock refusal, holder recovery, concurrent serialization, and signal-release-without-delivery cases all passed, with a transcript captured as evidence. Linters were not run because this assigned test phase explicitly forbids lint/static-analysis commands.

Evidence: Hermes inbox behavioral test transcript

ok - inbox: a steer is written durably and round-trips byte-exact with a self-describing doorbell
ok - inbox: the handled mv is the idempotent ack and sequences are never reissued
ok - inbox: concurrent writers serialize on the sequence lock and lose nothing
ok - inbox: ladder bookkeeping ignores a concurrently removed inbox
ok - inbox: the re-ring ladder paces by grace, escalates once, and resets on ack
ok - inbox: a held Hermes delivery lock refuses the doorbell without touching the terminal
ok - inbox: a Hermes doorbell reclaims the delivery lock after its holder exits
ok - inbox: concurrent Hermes doorbells serialize into whole terminal lines
ok - inbox: a signaled Hermes ring releases its lock and aborts before delivery
ok - watcher: an unhandled aged message on an idle pane re-rings without waking firstmate, and the ack silences it
ok - watcher: a busy pane just waits - the record is durable and no doorbell is typed
ok - watcher: a healthy or empty inbox stays completely silent
ok - watcher: acknowledgement silences an unwritable ladder without a stale wake
ok - watcher: unwritable ladder bookkeeping surfaces a stale wake after the doorbell
ok - watcher: a spent ring budget emits exactly one ordinary stale wake for recovery
exit=0

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 1 issue found → auto-fixed ✅

• 🚨 bin/fm-task-inbox-lib.sh:297 - AC1 requires lock release to be correct on signal paths, but the HUP/INT/TERM trap only releases the lock and does not exit. A signal after acquisition and before/while fm_task_inbox_ring_deliver can therefore release the lock and then continue into the terminal write without holding it, reintroducing the Hermes interleaving race. The signal handler must terminate or otherwise prevent delivery after releasing the lock.

🔧 Fix: Abort signaled Hermes delivery after releasing lock
✅ Re-checked - no issues remain.

✅ **Test** - passed

✅ No issues found.

• bash tests/fm-task-inbox.test.sh
• bash tests/fm-hermes-harness.test.sh
• bash tests/fm-send-inbox.test.sh
• bash tests/fm-omp-task-inbox-doorbell.test.sh
• bash tests/fm-backend.test.sh
• Verified worktree remained clean after testing.

✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.