Skip to content

feat(telemetry): one agent_turn span per speech handle - #2500

Draft
davidzhao wants to merge 1 commit into
dz/telemetry-coveragefrom
dz/telemetry-agent-turn
Draft

davidzhao wants to merge 1 commit into
dz/telemetry-coveragefrom
dz/telemetry-agent-turn

Conversation

@davidzhao

@davidzhao davidzhao commented Sep 15, 2026

Copy link
Copy Markdown
Member

Port of livekit/agents#7143. Stacked on #2499.

Description

A reply that calls a tool runs two generations (LLM steps) in two tasks. They were two agent_turn spans linked only by lk.parent_generation_id, so one response rendered as two turns. A speech handle is now exactly one agent_turn.

  • The speech handle owns the span. The first reply task (pipeline, realtime, or say) opens agent_turn under agent_session; the follow-up generation after a tool call continues the open span instead of opening a second one. It ends with the speech in SpeechHandle._markDone, recording the speech's error (redaction-aware) and the gen_ai.invoke_agent.duration metric for the whole turn.
  • Each generation is a generation event on the span with lk.generation_id and lk.parent_generation_id; the span's lk.generation_id names the latest generation and the new lk.generation_count how many there were. lk.speech_id is set at creation.
  • Every step's llm_node, function_tool, tts_node, realtime_inference and agent_speaking nest under the one turn.
  • A preemptive generation discarded for a successor answering the same user turn hands its open agent_turn to that successor (preemptive_generation_discarded event; lk.speech_id follows the speech that answered). An attempt cancelled with no successor still ends as its own turn. The queue-wait and interruption helpers tolerate a span already ended with the speech.
agent_turn  (lk.speech_id, lk.generation_count=2; events: generation ×2)
├─ llm_node          generation 1: the tool call
├─ function_tool
├─ llm_node          generation 2: the reply
├─ tts_node
└─ agent_speaking

Changes Made

  • voice/speech_handle.ts: span ownership, _generationId / _parentGenerationId, _takeAgentTurn / _continueAgentTurn, end in _markDone.
  • voice/agent_activity.ts: withAgentTurn and continueDiscardedTurn wrap the pipeline, realtime and say tasks; per-step span creation removed.
  • telemetry/otel_metrics.ts: recordInvokeAgentDuration (gen_ai.invoke_agent.duration, unit s, job attribution); telemetry/trace_types.ts: lk.generation_count, the metric name.

Adaptations from the Python source

  • say previously had no agent_turn span in JS; it now gets one like Python's TTS task.
  • lk.generation_id / lk.parent_generation_id were never stamped on agent_turn in JS before (the constants existed unused); they are now, as <speech_id>_<step>.

Testing

  • New voice/agent_turn_span.test.ts (6 tests): a tool-calling reply is one agent_turn with two generation events and both llm_nodes, the tool, tts_node and agent_speaking inside it; a plain reply is one generation; the discarded preemptive hand-off; an LLM failure fails the turn; the duration metric when sampled out.
  • Full agents suite green; build, typecheck, lint, API report updated.

🤖 Generated with Claude Code

@davidzhao
davidzhao added this pull request to stack #2502 September 15, 2026 06:32
@davidzhao
davidzhao force-pushed the dz/telemetry-agent-turn branch from 9a563af to 91cea4d Compare September 15, 2026 06:46
@changeset-bot

changeset-bot Bot commented Sep 15, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 241cd41

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 39 packages
Name Type
@livekit/agents Minor
@livekit/agents-plugin-anam Minor
@livekit/agents-plugin-anthropic Minor
@livekit/agents-plugin-assemblyai Minor
@livekit/agents-plugin-azure Minor
@livekit/agents-plugin-baseten Minor
@livekit/agents-plugin-bey Minor
@livekit/agents-plugin-cartesia Minor
@livekit/agents-plugin-cerebras Minor
@livekit/agents-plugin-deepgram Minor
@livekit/agents-plugin-did Minor
@livekit/agents-plugin-elevenlabs Minor
@livekit/agents-plugin-fishaudio Minor
@livekit/agents-plugin-google Minor
@livekit/agents-plugin-hume Minor
@livekit/agents-plugin-inworld Minor
@livekit/agents-plugin-krisp Minor
@livekit/agents-plugin-lemonslice Minor
@livekit/agents-plugin-liveavatar Minor
@livekit/agents-plugin-livekit Minor
@livekit/agents-plugin-meta Minor
@livekit/agents-plugin-minimax Minor
@livekit/agents-plugin-mistral Minor
@livekit/agents-plugin-mistralai Minor
@livekit/agents-plugin-neuphonic Minor
@livekit/agents-plugin-openai Minor
@livekit/agents-plugin-perplexity Minor
@livekit/agents-plugin-phonic Minor
@livekit/agents-plugin-protoface Minor
@livekit/agents-plugin-resemble Minor
@livekit/agents-plugin-rime Minor
@livekit/agents-plugin-runway Minor
@livekit/agents-plugin-sarvam Minor
@livekit/agents-plugin-silero Minor
@livekit/agents-plugin-soniox Minor
@livekit/agents-plugin-tavus Minor
@livekit/agents-plugins-test Minor
@livekit/agents-plugin-trugen Minor
@livekit/agents-plugin-xai Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@davidzhao
davidzhao force-pushed the dz/telemetry-agent-turn branch from 91cea4d to dd4e334 Compare September 16, 2026 04:28
Port of livekit/agents#7143. A reply that calls a tool runs two
generations (LLM steps) in two tasks; they were two agent_turn spans
linked only by lk.parent_generation_id, so one response rendered as two
turns. A speech handle is now exactly one agent_turn.

- The speech handle owns the span: the first reply task (pipeline,
  realtime, or say) opens agent_turn under agent_session; the follow-up
  generation after a tool call continues the open span. It ends with
  the speech in SpeechHandle._markDone, recording the speech's error
  redaction-aware and the gen_ai.invoke_agent.duration histogram for
  the whole turn (new in JS: otel_metrics.recordInvokeAgentDuration and
  trace_types.METRIC_GEN_AI_INVOKE_AGENT_DURATION; the Python metric
  already existed).
- Each generation is a `generation` event with lk.generation_id and
  lk.parent_generation_id (SpeechHandle._generationId /
  _parentGenerationId, `<speech id>_<step>` like Python); the span
  carries the latest generation id and the new lk.generation_count.
- A preemptive generation discarded for a successor answering the same
  user turn hands its open agent_turn over (preemptive_generation_discarded
  event, lk.speech_id follows the speech that answered), both on a newer
  attempt and on the real reply after onUserTurnCompleted invalidated it.
  The queue-wait and interruption helpers tolerate an ended span.
- `say` gets an agent_turn too, as in Python's _tts_task; JS had none.

Tests: agent_turn_span.test.ts (tool call is one turn with two
generation events and every step nested; plain reply is one generation;
discarded preemptive hand-off; LLM failure fails the turn; duration
metric when sampled out; sampled-out hand-off). The preemptive-guard
stand-in handle gained _takeAgentTurn; the PII key test skips METRIC_*
names, which are not attribute keys.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@davidzhao
davidzhao force-pushed the dz/telemetry-agent-turn branch from dd4e334 to 241cd41 Compare September 16, 2026 04:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant