Skip to content

Vulkan: size texture-vs-buffer choice by the bound, not the trace hint - #22401

Open
msluszniak wants to merge 1 commit into
pytorch:mainfrom
msluszniak:ms/vulkan-symbolic-texture-limits
Open

Vulkan: size texture-vs-buffer choice by the bound, not the trace hint#22401
msluszniak wants to merge 1 commit into
pytorch:mainfrom
msluszniak:ms/vulkan-symbolic-texture-limits

Conversation

@msluszniak

@msluszniak msluszniak commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Summary

filter_invalid_reprs() decides whether a tensor can live in a texture by comparing required_image_extents(tensor_val.shape) against the device limits. For a symbolic dimension that comparison resolves at the dim's hint -- the size of the example the model happened to be traced with -- rather than the maximum its exported range allows.

A model traced with a small example therefore picks a texture that is only valid at that size, and silently produces wrong results once it runs larger. Nothing raises.

This matters in practice because exporters conventionally trace with a small example input while declaring a large dynamic max, so the failing combination is the common one.

Reproduction

The Supertonic TTS text encoder (Supertone/supertonic-3), dynamic T in [8, 512], always executed at T=512 on a Galaxy S26 Ultra (Adreno 840). Only the trace-time example length differs, and the edge graphs are identical -- 552 call_function nodes, same ops, same literal args, same symbolic shapes, zero differing nodes:

traced at cosine vs CPU emitted .pte
T=64 0.537966 18147970 bytes
T=128 0.537966 18147970 bytes (identical file)
T=192 0.999414 18149890 bytes
T=512 0.999414 18149890 bytes (identical file)

The emitted binaries cluster exactly on the correctness boundary, which is what identifies this as a lowering decision rather than a bad kernel. With this change the T=64 export emits the 18149890-byte artifact and scores 0.999414.

Evaluating the bound also handles the unbounded case: a dim with no finite upper bound cannot be shown to fit any texture, so it falls back to buffer storage.

Test plan

Measured on Supertonic sub-models, against a CPU reference that the XNNPACK delegate reproduces at cosine 1.000000:

sub-model before after
text_encoder 0.406303 0.999414
vector_estimator 0.994432 0.999994
vocoder 0.016757 0.999977

Partitioner/lowering-side only, so no runtime rebuild is needed to reproduce the failure or the fix.

Note: the vocoder and vector_estimator numbers above also require #22399 (clamp with a symbolic bound), which is an instance of the same underlying theme -- a value derived from a dynamic dimension being read once at build time. This PR is independent of that one and touches a different file.

cc @SS-JIA @manuelcandales @digantdesai @cbilgin

filter_invalid_reprs() decides whether a tensor can live in a texture by
comparing required_image_extents(tensor_val.shape) against the device limits.
For a symbolic dimension that comparison resolves at the dim's *hint* -- the
size of the example the model happened to be traced with -- rather than the
maximum its exported range allows.

So a model traced with a small example picks a texture that is only valid at
that size, and silently produces wrong results once it runs larger. Nothing
raises.

Repro: the Supertonic TTS text encoder, dynamic T in [8, 512], always executed
at T=512 on a Galaxy S26 Ultra (Adreno 840). Only the trace-time example length
differs; the edge graphs are identical (552 nodes, same ops, same literal args,
same symbolic shapes):

  traced at T= 64  cosine 0.537966   .pte 18147970 bytes
  traced at T=128  cosine 0.537966   .pte 18147970 bytes (identical file)
  traced at T=192  cosine 0.999414   .pte 18149890 bytes
  traced at T=512  cosine 0.999414   .pte 18149890 bytes (identical file)

The emitted binaries cluster exactly on the correctness boundary, which is what
identifies this as a lowering decision rather than a bad kernel. With this
change the T=64 export emits the 18149890-byte artifact and scores 0.999414.

Evaluating the bound instead also covers the unbounded case: a dim with no
finite upper bound cannot be shown to fit any texture, so it falls back to
buffer storage.

Measured on Supertonic (CPU reference reproduced by the XNNPACK delegate at
cosine 1.000000):

  text_encoder     0.406303 -> 0.999414
  vector_estimator 0.994432 -> 0.999994
  vocoder          0.016757 -> 0.999977
@msluszniak
msluszniak requested a review from SS-JIA as a code owner September 1, 2026 09:16
@pytorch-bot pytorch-bot Bot added the module: vulkan Issues related to the Vulkan delegate and code under backends/vulkan/ label Sep 1, 2026
@pytorch-bot

pytorch-bot Bot commented Sep 1, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22401

Note: Links to docs will display an error until the docs builds have been completed.

⚠️ 15 Awaiting Approval

As of commit 8255516 with merge base 5428092 (image):

AWAITING APPROVAL - The following workflows need approval before CI can run:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 1, 2026
msluszniak added a commit to software-mansion-labs/executorch that referenced this pull request Sep 1, 2026
Backport of pytorch/executorch#22401.

filter_invalid_reprs() compares required_image_extents(tensor_val.shape) against
the device texture limits. For a symbolic dim that resolves at the dim's HINT
(the example the model was traced with), not the maximum its range allows, so a
model traced with a small example picks a texture only valid at that size and
silently computes garbage when run larger.

Supertonic text encoder, dynamic T in [8,512], always run at T=512, identical
edge graphs (552 nodes, zero differing args):

  traced T=64/128  cosine 0.537966
  traced T=192+    cosine 0.999414

With the fix the T=64 export matches T=192+ exactly. Combined with the clamp
guard, on S26 Ultra vs a CPU reference XNNPACK reproduces at 1.000000:

  text_encoder     0.406303 -> 0.999414
  vector_estimator 0.994432 -> 0.999994
  vocoder          0.016757 -> 0.999977
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. module: vulkan Issues related to the Vulkan delegate and code under backends/vulkan/

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants