Blaze Livekit plugin integration by HoangPN711 · Pull Request #5050 · livekit/agents

HoangPN711 · 2026-03-09T15:01:57Z

Summary

Add Blaze plugin support for LiveKit Agents.

Changes

add Blaze STT plugin
add Blaze TTS plugin
add Blaze LLM plugin
add Blaze plugin packaging/config helpers
add Blaze plugin README

Motivation

Enable Blaze voice AI services to be used through the existing LiveKit Agents plugin architecture.

Notes

The implementation is isolated under livekit-plugins/livekit-plugins-blaze and follows the existing provider plugin pattern.

CLAassistant · 2026-03-09T15:02:08Z

All committers have signed the CLA.

HoangPN711 · 2026-05-05T23:09:30Z

Hi @tinalenguyen,

This PR adds the Blaze provider integration for LiveKit Agents and keeps the implementation scoped under livekit-plugins/livekit-plugins-blaze.

It follows the existing STT/TTS/LLM plugin structure used by other provider integrations. I’ve also addressed the outstanding review comments on the PR.

Would appreciate a review when you have time. Thanks!

tinalenguyen · 2026-05-13T00:43:59Z

hi @HoangPN711! thank you for the contribution, could you bump the version and add the plugin to this pyproject file

HoangPN711 · 2026-05-13T23:17:54Z

Hi @tinalenguyen , thanks for reviewing this PR!

Updated as requested:

Bumped livekit-plugins-blaze to 1.5.9 (aligned with the current livekit-agents release)
Added blaze = ["livekit-plugins-blaze>=1.5.9"] to livekit-agents/pyproject.toml

Please let me know if any further changes are needed.

- Track audio_emitted separately from stream_initialized so a WS drop after started-byte-stream but before first PCM stays private-reconnect and framework-retryable. - Key STT empty-segment PCM buffer by (task, STT instance id) so FallbackAdapter with multiple Blaze STTs cannot mix pending audio.

trunghlt · 2026-08-10T16:53:44Z

Addressed latest Devin findings in fd5540c61:

TTS pre-audio drop abandoned the turn — reconnect/retry now uses audio_emitted (real push) instead of stream_initialized (can flip true on started-byte-stream alone).
STT pending PCM shared across instances — pending buffer is keyed by (task, STT instance id) so FallbackAdapter with multiple Blaze STTs cannot mix audio.

Unit tests: tests/test_plugin_blaze.py (58 passed locally).

- Race audio reader against input drain / reconnect resend so failed-request, close, or idle timeout abort the text pump immediately. - Restrict plaintext ws:// to true loopback only (drop .local exception) so bearer tokens never cross the LAN in cleartext; handle [IPv6] hosts.

trunghlt · 2026-08-10T17:08:39Z

Addressed latest Devin findings in d59ab2bc2:

TTS mid-turn reader failures delayed until end of text — race reader_task against input drain / reconnect resend (FIRST_COMPLETED) so errors abort the pump immediately.
Bearer over plaintext ws:// for .local hosts — only true loopback keeps ws://; mDNS/LAN .local now forces wss://.

Unit tests: tests/test_plugin_blaze.py (58 passed).

Homogeneous Task[Any] cast for asyncio.wait and wrap drain in create_task so type-check (3.10/3.13) passes after mid-turn race.

- Key task-local STT pending PCM with WeakKeyDictionary so GC'd instances cannot leak audio via recycled id(self). - Parse timeout env vars with empty/invalid fallback (no crash). - Move TTS query text and STT transcript content to DEBUG; keep length/metrics at INFO.

trunghlt · 2026-08-10T17:28:15Z

Addressed latest Devin findings in 2d4cfc250:

STT pending keyed by id(self) → task-local WeakKeyDictionary on the instance (no GC/id-reuse leak)
Blank/invalid timeout env crashes startup → _env_float with safe fallback
User speech/transcript at INFO → content at DEBUG; metrics remain INFO

Unit tests: 60 passed (tests/test_plugin_blaze.py).

Devin found that JSON lines that decode to str/int/list (e.g. keepalive) raise AttributeError in _extract_tool_calls and abort a partially delivered reply as a non-retryable APIConnectionError. Skip non-dict payloads with a warning; add a regression test.

Devin: effective_connect_timeout only matched DEFAULT_API_CONNECT_OPTIONS by identity, so stream-adapter/voice copies kept the short framework timeout and ignored BLAZE_*_TIMEOUT. Compare timeout values instead. Also strip URL userinfo in ws_base_url so a misconfigured BLAZE_API_URL cannot redirect the first-frame bearer token via embedded credentials.

Devin: empty-segment pending PCM was prepended without comparing the stored sample_rate/num_channels to the new segment, so a format change garbled the WAV header and transcription. Discard pending (and reset empty_count) on mismatch; compute pending_duration with capture-time format. Regression test included.

Devin: SpeechStream._run set closing_ws when the input channel drained but never closed the WebSocket, so async-for on recv hung until task cancel. After end_input, wait a short grace for trailing finals then close the WS so _run completes cleanly on intentional shutdown.

Devin: empty pcm path returned alternatives=[], while all other paths return one SpeechData — callers that index alternatives[0] crash. Return a single empty SpeechData (text="", confidence=0.0) for shape parity. Update unit test.

The gateway protocol (and agents-js) does not define a speech-start acknowledgement. Unconditional recv after speech-start either stalled until idle timeout (60s) when the server waits for a query, or dropped the first status/audio frame when pipelined. Let the normal reader loop handle whatever follows speech-start.

- Emit END_OF_SPEECH on SpeechStream teardown if speaking and no final arrived (grace close / peer drop left the user turn open). - Normalize api_url (config + STT/TTS/LLM constructors) so trailing slashes never produce //v1/... HTTP paths.

trunghlt · 2026-08-10T20:28:07Z

Hi @tinalenguyen — friendly re-review nudge when you have a moment.

Status on 6ae1f7da7:

CI green: ruff, unit-tests, type-check (3.10/3.13), blockguard, release gate, CLA
Devin Review SUCCESS (latest findings addressed; remaining E501 note is a false positive — E501 ignored project-wide and line is exactly 100 chars)
Prior review points from your 2026-07-24 pass remain addressed (blaze.vn URLs, authors, py.typed, PCM→WAV via rtc.AudioFrame.to_wav_bytes(), framework SentenceTokenizer, tests)
MERGEABLE; only blocked on review approval

Happy to address any further feedback. Thanks!

trunghlt · 2026-08-12T21:15:29Z

Hi @tinalenguyen — friendly re-review nudge when you have a moment.

Status on 6ae1f7da7:

CI green: ruff, unit-tests, type-check (3.10/3.13), blockguard, release gate, CLA
Devin Review SUCCESS (latest findings addressed; remaining E501 note is a false positive — E501 ignored project-wide and ruff is green)
Branch is MERGEABLE with main
Prior review feedback (README/blaze.vn, authors, py.typed, PCM→WAV, tokenizer) already addressed in earlier commits

Happy to rebase or take any further feedback. Thanks!

Keep main's plugin optional-deps pins at 1.6.10 and re-add blaze entry. Bump livekit-plugins-blaze to 1.6.10 to match workspace pins.

trunghlt · 2026-08-13T05:19:06Z

Automated fix: resolved merge conflicts with origin/main (1.6.10).

Kept main’s optional-deps pins at >=1.6.10 / browser >=0.2.10
Re-added blaze = ["livekit-plugins-blaze>=1.6.10"]
Bumped livekit-plugins-blaze to 1.6.10 to match workspace pins

Head: eb3f6b1af

devin-ai-integration

Devin Review found 1 new potential issue.

View 9 additional findings in Devin Review.

devin-ai-integration · 2026-08-13T05:26:46Z

+                        finally:
+                            # Always cancel and await reader_task to prevent
+                            # "Task destroyed but pending" / exception-not-retrieved.
+                            await utils.aio.gracefully_cancel(reader_task)


🟡 Audio-reader failures can be swallowed and surface later as stray asyncio error logs

The background audio reader is only cancelled and waited on (utils.aio.gracefully_cancel(reader_task) at livekit-plugins/livekit-plugins-blaze/livekit/plugins/blaze/tts.py:938) without ever collecting the error it already failed with, so a failed speech turn can later print an unrelated "exception was never retrieved" error in the logs.
Impact: Operators see confusing stray error tracebacks that are not tied to the request that actually failed.

Why cancel-only does not consume a completed task's exception

utils.aio.gracefully_cancel is an alias of cancel_and_wait (livekit-agents/livekit/agents/utils/aio/utils.py:1-24): it attaches a done-callback, calls fut.cancel() and awaits an internal waiter. It never calls task.result()/task.exception(), so for a task that had already completed with an exception the exception stays unretrieved and asyncio logs it when the task is garbage collected.

Reachable path: input drains successfully, then await ws.send(json.dumps({"event": "speech-end"})) (livekit-plugins/livekit-plugins-blaze/livekit/plugins/blaze/tts.py:928) raises ConnectionClosed before await reader_task on line 930 is reached. Control jumps to the finally on lines 935-938, which only cancels the reader — but _read_audio has typically already terminated with its own ConnectionClosed/APITimeoutError. That exception is never retrieved.

A safe fix is to await the reader inside a try/except Exception: pass (or check reader_task.done() and read reader_task.exception()) before/after cancelling.

Suggested change

finally:

# Always cancel and await reader_task to prevent

# "Task destroyed but pending" / exception-not-retrieved.

await utils.aio.gracefully_cancel(reader_task)

finally:

# Always cancel and await reader_task to prevent

# "Task destroyed but pending" / exception-not-retrieved.

await utils.aio.gracefully_cancel(reader_task)

if reader_task.done() and not reader_task.cancelled():

# Consume any pending reader exception so asyncio

# does not log "exception was never retrieved".

reader_task.exception()

Was this helpful? React with 👍 or 👎 to provide feedback.