Skip to content

Vulkan: do not partition ops whose symbolic arguments the runtime reads at build time - #22399

Open
msluszniak wants to merge 2 commits into
pytorch:mainfrom
msluszniak:ms/vulkan-partitioner-guards
Open

Vulkan: do not partition ops whose symbolic arguments the runtime reads at build time#22399
msluszniak wants to merge 2 commits into
pytorch:mainfrom
msluszniak:ms/vulkan-partitioner-guards

Conversation

@msluszniak

@msluszniak msluszniak commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Several Vulkan op implementations read a scalar or list argument with extract_scalar() / get_int_list() when the node is built, and bake the result into the dispatch. When that argument is symbolic (derived from a dynamic dimension) it is never refreshed. Depending on the op this either aborts at execute time or, worse, silently computes against a stale value.

This PR declines the affected nodes via are_node_inputs_supported_fn (following the existing _check_copy_is_noop precedent) so they fall back instead. Making these ops genuinely support symbolic arguments means plumbing the values through as symints and refreshing them on resize, which is left for a follow-up.

1. clamp with a symbolic bound — silent wrong results. get_val_or_inf() in UnaryOp.cpp:

float get_val_or_inf(ComputeGraph& graph, const ValueRef& val, bool max) {
  if (!graph.val_is_none(val)) {
    return graph.extract_scalar<float>(val);   // read once, at build time

Nothing raises. This is most damaging when clamp is applied to index tensors, where a wrong limit silently reorders or drops data downstream. hardtanh, hardshrink and leaky_relu use the same path and take the same guard.

Minimal repro — clamp(idx, 0, L-1) feeding index_select, exported with L dynamic and executed at L's upper bound, so the runtime shapes are identical to the static export:

result
static export cosine 1.000000 vs CPU
dynamic export all 2048 output values are zero
with this fix cosine 1.000000

2. constant_pad_nd with a symbolic pad — abort. A symbolic pad list is serialized as a VALUELIST, and Pad.cpp reads it with get_int_list():

Exception raised from toIntList at .../graph/containers/Value.h:272:
(isIntList()) is false! Expected value to have type IntList, got VALUELIST instead.

Most size-taking ops (View.cpp, Expand.cpp, Repeat.cpp, Squeeze.cpp, ...) already use the symint-aware extract_int_or_symint_list(). Switching Pad.cpp to it would not be sufficient on its own: add_constant_pad_nd_node() also bakes the amounts into a params buffer at build time, so the op would trade an abort for a wrong result.

3. _native_batch_norm_legit_no_training on a non-4d input — abort.

Exception raised from add_native_batch_norm_node at .../ops/impl/BatchNorm.cpp:65:
(in_sizes.size() == 4) is false! BatchNorm only support 4d tensor

The registration carries no rank constraint, so any conv1d model (rank-3 activations) partitions a batch norm the runtime refuses. This affects 1-D audio models generally.

Test plan

Reproduced with the Supertonic TTS model (Supertone/supertonic-3) on a Galaxy S26 Ultra (Adreno 840), lowering with VulkanPartitioner and running executor_runner. Reference outputs are from CPU and are reproduced by the XNNPACK delegate at cosine 1.000000, confirming the inputs and references are sound.

sub-model before after
vocoder SIGABRT (BatchNorm only support 4d), then cosine 0.016757 0.999977
vector_estimator cosine 0.994432 0.999994
text_encoder SIGABRT (VALUELIST) runs

These changes are partitioner-side only, so no runtime rebuild is needed to reproduce either the failures or the fixes.

Note: text_encoder still returns wrong output after these fixes (cosine 0.538, improved from 0.406). Its statistics match the reference closely (matching value multiset, std ratio 1.009) so it looks like a further placement/layout issue rather than bad arithmetic. I have ruled out slice_copy, arange and index.Tensor as the cause by op-exclusion bisection and am still narrowing it; that will be reported separately.

Two op registrations advertise support the runtime does not have, so a graph
containing either lowers cleanly and then aborts at execute time.

constant_pad_nd: a symbolic pad list is serialized as a VALUELIST, and Pad.cpp
reads it with get_int_list(), raising "Expected value to have type IntList, got
VALUELIST instead". add_constant_pad_nd_node() also bakes the pad amounts into
a params buffer at build time, so a pad derived from a dynamic dim would be
stale even if the list were read symbolically; decline both cases rather than
trade an abort for a wrong result.

_native_batch_norm_legit_no_training: add_native_batch_norm_node() asserts
in_sizes.size() == 4, so any conv1d model (rank-3 activations) aborts with
"BatchNorm only support 4d tensor".

Both now fall back instead of aborting. Found with the Supertonic TTS model,
whose text encoder hits the first (VITS relative-attention pads are derived
from the sequence length) and whose vocoder hits the second.
@msluszniak
msluszniak requested a review from SS-JIA as a code owner September 1, 2026 08:25
@pytorch-bot pytorch-bot Bot added the module: vulkan Issues related to the Vulkan delegate and code under backends/vulkan/ label Sep 1, 2026
@pytorch-bot

pytorch-bot Bot commented Sep 1, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22399

Note: Links to docs will display an error until the docs builds have been completed.

⚠️ 16 Awaiting Approval

As of commit 6da6af3 with merge base 5428092 (image):

AWAITING APPROVAL - The following workflows need approval before CI can run:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 1, 2026
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

msluszniak added a commit to software-mansion-labs/executorch that referenced this pull request Sep 1, 2026
Backport of pytorch/executorch#22399.

Two op registrations advertise support the runtime does not have, so a graph
containing either lowers cleanly and then aborts at execute time.

constant_pad_nd: a symbolic pad list is serialized as a VALUELIST, and Pad.cpp
reads it with get_int_list(), raising "Expected value to have type IntList, got
VALUELIST instead". add_constant_pad_nd_node() also bakes the pad amounts into
a params buffer at build time, so a pad derived from a dynamic dim would be
stale even if the list were read symbolically; decline both cases rather than
trade an abort for a wrong result.

_native_batch_norm_legit_no_training: add_native_batch_norm_node() asserts
in_sizes.size() == 4, so any conv1d model (rank-3 activations) aborts with
"BatchNorm only support 4d tensor".

Both now fall back instead of aborting. Found with the Supertonic TTS model,
whose text encoder hits the first (VITS relative-attention pads are derived
from the sequence length) and whose vocoder hits the second.
get_val_or_inf() in UnaryOp.cpp reads each clamp bound with
extract_scalar<float>() when the node is BUILT and bakes the result into the
dispatch. A bound derived from a dynamic dimension is never refreshed, so the
op silently computes against a stale limit. Nothing raises.

This is worst when clamp is applied to index tensors: a wrong limit silently
reorders or drops data downstream rather than perturbing it.

Minimal repro -- clamp(idx, 0, L-1) feeding index_select, exported with L
dynamic and executed at L's upper bound, so the runtime shapes are identical to
the static export:

  static export : cosine 1.000000 vs CPU
  dynamic export: all 2048 output values are zero
  with this fix : cosine 1.000000

On the Supertonic TTS model (S26 Ultra, Adreno 840), against a CPU reference
that the XNNPACK delegate reproduces at cosine 1.000000:

  vocoder          0.016757 -> 0.999977
  vector_estimator 0.994432 -> 0.999994

hardtanh, hardshrink and leaky_relu read their bounds through the same
build-time get_val_or_inf() path, so they take the same guard.
@msluszniak msluszniak changed the title Vulkan: do not partition ops the runtime will reject Vulkan: do not partition ops whose symbolic arguments the runtime reads at build time Sep 1, 2026
msluszniak added a commit to software-mansion-labs/executorch that referenced this pull request Sep 1, 2026
Backport of pytorch/executorch#22399.

get_val_or_inf() in UnaryOp.cpp reads each clamp bound with
extract_scalar<float>() when the node is BUILT and bakes the result into the
dispatch, so a bound derived from a dynamic dim is never refreshed and the op
silently computes against a stale limit. Worst when clamp is applied to index
tensors, where a wrong limit reorders or drops data downstream.

Supertonic on S26 Ultra (Adreno 840), vs a CPU reference that XNNPACK
reproduces at cosine 1.000000:

  vocoder          0.016757 -> 0.999977
  vector_estimator 0.994432 -> 0.999994

hardtanh, hardshrink and leaky_relu share the same build-time path.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. module: vulkan Issues related to the Vulkan delegate and code under backends/vulkan/

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants