fix(cli): prebuffer console playback so realtime-paced audio doesn't underrun

[u9g]

Problem

Running a duplex model (e.g. GPTLiveModel) under python agent.py console produces audible clicks and stutters mid-word.

A duplex model streams its voice at real-time pace: one 100 ms frame every ~100 ms. The console speaker callback also pulls 100 ms per tick, so the buffer runs with a single block of margin. Whenever the gap between two arriving frames exceeds ~100 ms, the callback finds the buffer empty and zero-fills the block. That inserted silence is the click.

TTS pipelines never hit this because TTS delivers audio much faster than real time and the buffer stays deep.

Measured

Instrumented _sd_output_callback and let the agent speak its greeting with no mic input. Frames arrived every ~100 ms with occasional 150–180 ms gaps; each gap produced a buffer depth of zero mid-utterance.

Before (samples queued per 100 ms callback, one utterance):

2400 2400 2400 2400 2400 2400 2400 2400 2400 2400 2400 2400 2400 2400 2400 0 4800 4800 ... 4800 2400 0

After:

2400 4800 7200 7200 7200 7200 ... 7200 7200 4800 2400 0

Fix

ConsoleAudioOutput holds the head of each segment until 300 ms is queued, or until the segment is flushed (so short utterances are not delayed past their end). A mid-segment underrun does not re-prime, so the failure mode stays a single short gap rather than a longer one.

Commits:

1. Pure move: the drain logic leaves AgentsConsole._sd_output_callback and becomes ConsoleAudioOutput.read_into, so it can be unit tested.
2. The prebuffer, with tests in tests/test_cli_console_output.py.

Follow-up

lk agent console (Go, pkg/console/pipeline.go) pads its playback ring with silence the same way and likely needs the same margin. Not measured yet.

[devin-ai-integration]

Devin Review found 1 potential issue.

[devin-ai-integration]

🟡 Primed interruption invents played audio

An interruption during _priming reports elapsed buffering time as played audio, although every callback emitted silence. forward_generation then truncates realtime history at that false position.

Learn more

_capture_start begins when the first frame arrives, but the new priming period deliberately outputs only silence. If clear_buffer() interrupts the segment before priming completes, _wait_for_playout() subtracts that capture timestamp from the interruption time and reports up to 300 ms as played. The generation path accepts any positive playback position as proof that audio reached the speakers and uses it when truncating realtime-model history in forward_generation. The remote model can therefore retain assistant audio that the user never heard.

Example: A 100 ms frame arrives, then the user interrupts 200 ms later before the 300 ms threshold. Every speaker callback produced zeros, but the completion event reports about 200 ms played and realtime history is truncated there instead of removing the unheard response.

Recommended fix: Track actual samples removed from _output_buf, or start interruption timing when _maybe_mark_playback_started() first drains audio. Report zero when interruption occurs before playback starts, and add a test that interrupts during priming and asserts both no playback-start event and a zero playback position.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[u9g]

Before/after recording of the same scripted call in python agent.py console with GPTLiveModel. The caller's lines were synthesized with cue-cli's inference TTS client and fed into the console's mic path on a fixed script; the speaker callback output was captured to WAV, so what you hear is exactly what the sounddevice callback played.

Before (livekit-agents 1.8.1). Zero-filled blocks land inside agent speech at 0:18.0 and 0:38.3, the clicks you hear mid-word:

console_crackle_before.mp4

After (this PR). Same script, no zero-filled blocks inside speech; agent speech starts 300 ms later:

console_crackle_after.mp4

Counted offline on the captured agent track: a 100 ms block of exact digital zeros bracketed by speech within 300 ms on both sides. GPT-Live's own pauses carry low-level noise, so exact zeros only come from the callback's zero fill.

[davidzhao]

lg, though worth mentioning this will be deprecated eventually