feat(telemetry): trace the end-of-turn wait and the turn handoff

[davidzhao]

Port of livekit/agents#7130.

Description

eou_detection only covered the turn-detector inference, so the endpointing delay itself (often seconds) showed up as an empty gap between user_turn and agent_turn. This makes the wait, the user hook, and the speech queue visible.

eou_wait span, one per user turn, child of user_turn:

• back-dated to the last speaking time, so the bar starts where the user stopped talking;
• ends on the turn decision with lk.eou.outcome (committed / user_resumed / dropped), lk.eou.wait_duration, lk.eou.rearm_count, the final lk.eou.endpointing_delay, and the last lk.eou.source;
• a later trigger for the same turn (late STT final, another VAD end of speech) re-arms the wait with a rearmed event instead of starting a new span;
• a rejected commit counts in lk.eou.not_committed_count and keeps waiting; resumed speech ends the span at the speech start (user_resumed event) and lk.eou.resume_count on user_turn counts those; teardown drops it. A running eou_detection is closed with a superseded event so it stays inside the wait. A turn-detection mode change to or from manual drops the pending wait.

eou_detection keeps its name and now nests under eou_wait.

on_user_turn_completed span around the user hook that gates the reply, under the user_turn it completes: recognition hands the open user_turn span to the activity, which ends it after the hook. Exceptions are recorded redaction-aware (session and job flags); StopResponse is an event.

Per-turn breakdown on the reply's agent_turn: lk.speech.queue_wait (scheduling to first generation authorization), plus lk.end_of_turn_delay, lk.transcription_delay, lk.on_user_turn_completed_delay next to lk.e2e_latency.

user_turn
├─ user_speaking
└─ eou_wait                 ← new, rearmed ×1
   └─ eou_detection         ← now nested under the wait
on_user_turn_completed      ← new
agent_turn  (lk.speech.queue_wait=0.01)

Changes Made

• voice/audio_recognition.ts: eou_wait state machine (ensureEouWaitSpan / endEouWaitSpan / releaseEouDetectionSpan / stampUserTurnResumes), EndOfTurnInfo.userTurnSpan / userTurnSpanAdopted, lazily-created user_turn starts at the earliest known anchor, updateOptions now cancels the pending decision on a manual mode switch.
• voice/agent_activity.ts: on_user_turn_completed span, adopted user_turn ended after the hook, recordQueueWait, recordUserTurnStages.
• voice/speech_handle.ts: scheduled / authorized timestamps and the queue-wait accessor.
• telemetry/trace_types.ts: seven new attributes (none PII); unit docs on the existing delay attributes.
• voice/amd.ts: AMD duration attributes converted to seconds.

Adaptations from the Python source

• lk.agent_label uses agent.id (JS agents have no label), like every other JS site.
• A non-StopResponse hook error keeps the existing JS behavior (log, record on the span, continue to the reply); Python drops the turn. Left unchanged to keep this PR to telemetry.
• Every duration attribute on a span is now in seconds, as in Python. lk.eou.endpointing_delay (eou_detection, eou_wait), lk.eou.detection_delay, lk.transcription_delay / lk.end_of_turn_delay (user_turn), and lk.amd.speech_duration / lk.amd.delay were stamped in milliseconds before, which put them at odds with the new attributes and with the Python SDK on shared dashboards. Called out in the changeset.

Testing

• New voice/eou_wait_span.test.ts (10 tests): commit, re-arm, resumed speech, teardown drop, not-committed, nested and superseded eou_detection, manual mode switch, and a full fake session asserting the wait, the hook span, the queue wait and session-only redaction of a hook exception.
• audio_recognition_span.test.ts parenting assertions updated (eou_detection → eou_wait → user_turn).
• Full agents suite green; build, typecheck, lint, API report updated.

🤖 Generated with Claude Code

[changeset-bot]

🦋 Changeset detected

Latest commit: 4c929a4

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 39 packages

Name	Type
@livekit/agents	Minor
@livekit/agents-plugin-anam	Minor
@livekit/agents-plugin-anthropic	Minor
@livekit/agents-plugin-assemblyai	Minor
@livekit/agents-plugin-azure	Minor
@livekit/agents-plugin-baseten	Minor
@livekit/agents-plugin-bey	Minor
@livekit/agents-plugin-cartesia	Minor
@livekit/agents-plugin-cerebras	Minor
@livekit/agents-plugin-deepgram	Minor
@livekit/agents-plugin-did	Minor
@livekit/agents-plugin-elevenlabs	Minor
@livekit/agents-plugin-fishaudio	Minor
@livekit/agents-plugin-google	Minor
@livekit/agents-plugin-hume	Minor
@livekit/agents-plugin-inworld	Minor
@livekit/agents-plugin-krisp	Minor
@livekit/agents-plugin-lemonslice	Minor
@livekit/agents-plugin-liveavatar	Minor
@livekit/agents-plugin-livekit	Minor
@livekit/agents-plugin-meta	Minor
@livekit/agents-plugin-minimax	Minor
@livekit/agents-plugin-mistral	Minor
@livekit/agents-plugin-mistralai	Minor
@livekit/agents-plugin-neuphonic	Minor
@livekit/agents-plugin-openai	Minor
@livekit/agents-plugin-perplexity	Minor
@livekit/agents-plugin-phonic	Minor
@livekit/agents-plugin-protoface	Minor
@livekit/agents-plugin-resemble	Minor
@livekit/agents-plugin-rime	Minor
@livekit/agents-plugin-runway	Minor
@livekit/agents-plugin-sarvam	Minor
@livekit/agents-plugin-silero	Minor
@livekit/agents-plugin-soniox	Minor
@livekit/agents-plugin-tavus	Minor
@livekit/agents-plugins-test	Minor
@livekit/agents-plugin-trugen	Minor
@livekit/agents-plugin-xai	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR