Repository navigation
feat(dictation): transcribe on the GPU, in pieces, with the best installed model - #13
Merged
Merged
Conversation
…alled model - Backend "auto" instead of "cpu": the model runs on Vulkan when there is a GPU. ggml-vulkan's coopmat path returns empty text on RDNA4 (RX 9070 XT), so GGML_VK_DISABLE_COOPMAT is set through the C runtime (koffi, _putenv_s) before the backend starts. T3CODE_DICTATION_BACKEND=cpu opts out. - The final transcript runs in pieces of at most 25 s, cut at the quietest moment: one run over minutes of audio drifts (Parakeet) or stops at the token limit (Cohere, Canary). - Model ranking knows Cohere Transcribe 03-2026 and Canary 1B v2 and looks in <T3 home>/dictation/models too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
github-actions
Bot
added
vouch:trusted
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Dictation ran Parakeet on the CPU because Vulkan returned empty text. On an RX 9070 XT (RDNA4) the cause is ggml-vulkan's cooperative-matrix path; with
GGML_VK_DISABLE_COOPMAT=1the GPU transcribes correctly and about five times faster than the CPU. Long recordings were also one model run, which drifts (Parakeet) or truncates (encoder-decoder models) after a minute or two.auto(Vulkan when there is a GPU). The coopmat switch is set through the C runtime (_putenv_svia koffi) becauseprocess.envdoes not reach ggml'sgetenvon Windows.T3CODE_DICTATION_BACKEND=cpuopts out.<T3 home>/dictation/models.Measured on 60 German Multilingual LibriSpeech clips (15 min), RX 9070 XT:
Before chunking, one run over 2 min gave 40–70% WER for every model.
Tests:
vp test run src/dictation/(48 passed), server typecheck, lint and fmt on the touched files.Claude Opus 5.5 in T3 Code (Claude Code harness).
🤖 Generated with Claude Code