Skip to content

feat(speech): add Vulkan whisper-tiny variants - #1398

Closed
msluszniak wants to merge 1 commit into
rne-rewritefrom
@ms/whisper-vulkan
Closed

feat(speech): add Vulkan whisper-tiny variants#1398
msluszniak wants to merge 1 commit into
rne-rewritefrom
@ms/whisper-vulkan

Conversation

@msluszniak

Copy link
Copy Markdown
Member

Description

Registers the fp16 and int8 Vulkan exports of multilingual whisper-tiny as WHISPER.TINY.VULKAN_FP16 / WHISPER.TINY.VULKAN_INT8, adds vulkan to the speechToText entry in the download-libs.js feature map so the backend lib is fetched, and offers both in the speech demo on Android.

Encode runs on Vulkan (mel preprocessor falls back to XNNPACK); decode stays on XNNPACK, where it measured faster. On an Adreno 840 the encode is about 1.6x the XNNPACK time and decode is 4.08 ms, down from 11.96 ms.

The exports require the ExecuTorch 1.4.1 Vulkan runtime published in v0.10.0-libs-1.4.1. Older Vulkan libs lack the gelu tanh clamp, the reduce-shader barrier fix and the Adreno local-work-group fix, and produce NaNs or a device loss.

Introduces a breaking change?

  • Yes
  • No

Type of change

  • Bug fix (change which fixes an issue)
  • New feature (change which adds functionality)
  • Documentation update (improves or adds clarity to existing documentation)
  • Other (chores, tests, code style improvements etc.)

Tested on

  • iOS
  • Android

Testing instructions

  1. Run apps/speech on an Android device.
  2. Open Transcribe Audio File, pick Tiny Multilingual (Vulkan fp16), transcribe the sample file, and check the transcript carries no <|0.00|> timestamp tokens.
  3. Repeat with Tiny Multilingual (Vulkan int8).
  4. Repeat both on Live Transcription (Mic).

Screenshots

Related issues

Checklist

  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have updated the documentation accordingly
  • My changes generate no new warnings

Additional notes

Verified on a Galaxy S26 Ultra against the exact libvulkan_executorch_backend.so published in v0.10.0-libs-1.4.1 (md5 3aba5c82..., matched between the installed APK and the release tarball).

The speechToText feature map lists xnnpack, coreml and now vulkan, but not mlx, even though the registry exposes MLX_BF16 / MLX_INT8 whisper variants. That looks like a pre-existing gap and is left alone here.

Registers the fp16 and int8 Vulkan exports of multilingual whisper-tiny and
lets the speech demo pick them on Android.

The Vulkan encode needs the ExecuTorch 1.4.1 runtime from v0.10.0-libs-1.4.1:
earlier Vulkan libs lack the gelu tanh clamp, the reduce-shader barrier fix and
the Adreno local-work-group fix.
@msluszniak msluszniak self-assigned this Aug 31, 2026
@msluszniak msluszniak added platform: android Issues and tasks related to Android performance Related to all issues and tasks focused on improving performance labels Aug 31, 2026
@msluszniak

Copy link
Copy Markdown
Member Author

Folded into #1392, which already carries the registry variant wiring.

@msluszniak msluszniak closed this Aug 31, 2026
@msluszniak
msluszniak deleted the @ms/whisper-vulkan branch August 31, 2026 11:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Related to all issues and tasks focused on improving performance platform: android Issues and tasks related to Android

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant