Vulkan: do not partition ops whose symbolic arguments the runtime reads at build time - #22399
Open
msluszniak wants to merge 2 commits into
Open
Vulkan: do not partition ops whose symbolic arguments the runtime reads at build time#22399msluszniak wants to merge 2 commits into
msluszniak wants to merge 2 commits into
Conversation
Two op registrations advertise support the runtime does not have, so a graph containing either lowers cleanly and then aborts at execute time. constant_pad_nd: a symbolic pad list is serialized as a VALUELIST, and Pad.cpp reads it with get_int_list(), raising "Expected value to have type IntList, got VALUELIST instead". add_constant_pad_nd_node() also bakes the pad amounts into a params buffer at build time, so a pad derived from a dynamic dim would be stale even if the list were read symbolically; decline both cases rather than trade an abort for a wrong result. _native_batch_norm_legit_no_training: add_native_batch_norm_node() asserts in_sizes.size() == 4, so any conv1d model (rank-3 activations) aborts with "BatchNorm only support 4d tensor". Both now fall back instead of aborting. Found with the Supertonic TTS model, whose text encoder hits the first (VITS relative-attention pads are derived from the sequence length) and whose vocoder hits the second.
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22399
Note: Links to docs will display an error until the docs builds have been completed.
|
This PR needs a
|
msluszniak
added a commit
to software-mansion-labs/executorch
that referenced
this pull request
Sep 1, 2026
Backport of pytorch/executorch#22399. Two op registrations advertise support the runtime does not have, so a graph containing either lowers cleanly and then aborts at execute time. constant_pad_nd: a symbolic pad list is serialized as a VALUELIST, and Pad.cpp reads it with get_int_list(), raising "Expected value to have type IntList, got VALUELIST instead". add_constant_pad_nd_node() also bakes the pad amounts into a params buffer at build time, so a pad derived from a dynamic dim would be stale even if the list were read symbolically; decline both cases rather than trade an abort for a wrong result. _native_batch_norm_legit_no_training: add_native_batch_norm_node() asserts in_sizes.size() == 4, so any conv1d model (rank-3 activations) aborts with "BatchNorm only support 4d tensor". Both now fall back instead of aborting. Found with the Supertonic TTS model, whose text encoder hits the first (VITS relative-attention pads are derived from the sequence length) and whose vocoder hits the second.
get_val_or_inf() in UnaryOp.cpp reads each clamp bound with extract_scalar<float>() when the node is BUILT and bakes the result into the dispatch. A bound derived from a dynamic dimension is never refreshed, so the op silently computes against a stale limit. Nothing raises. This is worst when clamp is applied to index tensors: a wrong limit silently reorders or drops data downstream rather than perturbing it. Minimal repro -- clamp(idx, 0, L-1) feeding index_select, exported with L dynamic and executed at L's upper bound, so the runtime shapes are identical to the static export: static export : cosine 1.000000 vs CPU dynamic export: all 2048 output values are zero with this fix : cosine 1.000000 On the Supertonic TTS model (S26 Ultra, Adreno 840), against a CPU reference that the XNNPACK delegate reproduces at cosine 1.000000: vocoder 0.016757 -> 0.999977 vector_estimator 0.994432 -> 0.999994 hardtanh, hardshrink and leaky_relu read their bounds through the same build-time get_val_or_inf() path, so they take the same guard.
msluszniak
added a commit
to software-mansion-labs/executorch
that referenced
this pull request
Sep 1, 2026
Backport of pytorch/executorch#22399. get_val_or_inf() in UnaryOp.cpp reads each clamp bound with extract_scalar<float>() when the node is BUILT and bakes the result into the dispatch, so a bound derived from a dynamic dim is never refreshed and the op silently computes against a stale limit. Worst when clamp is applied to index tensors, where a wrong limit reorders or drops data downstream. Supertonic on S26 Ultra (Adreno 840), vs a CPU reference that XNNPACK reproduces at cosine 1.000000: vocoder 0.016757 -> 0.999977 vector_estimator 0.994432 -> 0.999994 hardtanh, hardshrink and leaky_relu share the same build-time path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Several Vulkan op implementations read a scalar or list argument with
extract_scalar()/get_int_list()when the node is built, and bake the result into the dispatch. When that argument is symbolic (derived from a dynamic dimension) it is never refreshed. Depending on the op this either aborts at execute time or, worse, silently computes against a stale value.This PR declines the affected nodes via
are_node_inputs_supported_fn(following the existing_check_copy_is_noopprecedent) so they fall back instead. Making these ops genuinely support symbolic arguments means plumbing the values through as symints and refreshing them on resize, which is left for a follow-up.1.
clampwith a symbolic bound — silent wrong results.get_val_or_inf()inUnaryOp.cpp:Nothing raises. This is most damaging when clamp is applied to index tensors, where a wrong limit silently reorders or drops data downstream.
hardtanh,hardshrinkandleaky_reluuse the same path and take the same guard.Minimal repro —
clamp(idx, 0, L-1)feedingindex_select, exported withLdynamic and executed atL's upper bound, so the runtime shapes are identical to the static export:2.
constant_pad_ndwith a symbolic pad — abort. A symbolic pad list is serialized as aVALUELIST, andPad.cppreads it withget_int_list():Most size-taking ops (
View.cpp,Expand.cpp,Repeat.cpp,Squeeze.cpp, ...) already use the symint-awareextract_int_or_symint_list(). SwitchingPad.cppto it would not be sufficient on its own:add_constant_pad_nd_node()also bakes the amounts into a params buffer at build time, so the op would trade an abort for a wrong result.3.
_native_batch_norm_legit_no_trainingon a non-4d input — abort.The registration carries no rank constraint, so any conv1d model (rank-3 activations) partitions a batch norm the runtime refuses. This affects 1-D audio models generally.
Test plan
Reproduced with the Supertonic TTS model (
Supertone/supertonic-3) on a Galaxy S26 Ultra (Adreno 840), lowering withVulkanPartitionerand runningexecutor_runner. Reference outputs are from CPU and are reproduced by the XNNPACK delegate at cosine 1.000000, confirming the inputs and references are sound.BatchNorm only support 4d), then cosine 0.016757VALUELIST)These changes are partitioner-side only, so no runtime rebuild is needed to reproduce either the failures or the fixes.
Note:
text_encoderstill returns wrong output after these fixes (cosine 0.538, improved from 0.406). Its statistics match the reference closely (matching value multiset, std ratio 1.009) so it looks like a further placement/layout issue rather than bad arithmetic. I have ruled outslice_copy,arangeandindex.Tensoras the cause by op-exclusion bisection and am still narrowing it; that will be reported separately.