Skip to content

openai: keep GPT-Live's input clock running when no audio is pushed - #7239

Open
u9g wants to merge 3 commits into
mainfrom
jason/gpt-live-silence-feed
Open

openai: keep GPT-Live's input clock running when no audio is pushed#7239
u9g wants to merge 3 commits into
mainfrom
jason/gpt-live-silence-feed

Conversation

@u9g

@u9g u9g commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Problem

GPT-Live speaks only while its input clock runs, and that clock is the audio the client appends. A session with no microphone never appends anything, so every ask times out after 10 seconds with:

failed to generate a reply: the model did not start speaking when asked

That is what happens under a text simulation, in lk agent console --text, and whenever input audio is disabled. #7236 papered over the first case by refusing to start; the other two still hang.

Measured against the live service with the adapter's generate_reply: with nothing pushed the ask times out; with 100 ms silence frames pushed at 10 Hz the reply opens in 1.6 s with transcript and audio. session.input_audio.mute alone does not do it.

Change

  • GPTLiveSession appends 100 ms of silence whenever nothing has been pushed for 200 ms, once the session has started. A live microphone is never padded: the feeder only fills gaps.
  • The fix: refuse to start a DuplexModel under a text simulation #7236 guard is removed. A duplex model now runs under a text simulation like any other model.
  • Pure refactor first: push_audio split into the timestamp and a shared _append_audio.

Tests

tests/test_gpt_live_model.py: idle session appends silence; a live microphone is not padded. The fake sockets ignore silent appends by default so the existing sequence assertions stay exact; the two tests that assert on silent input opt back in. The two #7236 tests are removed with the guard. 99 tests pass across the GPT-Live and duplex adapter files.

End to end: a GPT-Live agent under lk agent simulate (text mode) now holds full conversations on this branch. Same run on main fails every scenario on turn one.

What text mode still cannot do well

GPT-Live has no typed user-input channel; the plugin delivers typed turns as session.commentary.append with an instruction to reply. In the end-to-end run the voice model delegated to the backend (and so reached the tools) in one of three scenarios, versus three of three in audio mode. Text mode is now usable for exercising conversation flow, not for asserting tool behavior. That limit is the protocol's, not this change's.

agents-js counterpart: livekit/agents-js#2484.

u9g added 3 commits September 11, 2026 15:24
GPT-Live speaks only while its input clock runs, and that clock is the audio the client appends. A session with no microphone (a text simulation, a text-mode console, muted input) never appended anything, so every ask timed out with "the model did not start speaking when asked". Verified against the live service: the same ask that times out with no input opens a reply in under two seconds once silence flows.

GPTLiveSession now appends 100 ms of silence whenever nothing has been pushed for 200 ms, once the session has started. A live microphone is never padded.
The guard from #7236 treated the missing audio as fatal. The GPT-Live plugin now supplies the input clock itself, so a text simulation runs a duplex model like any other.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant