fix(genai): prefer device-native EPs over cross-device accelerators#1138
Merged
Conversation
DingmaomaoBJTU
force-pushed
the
dingmaomaobjtu-fix-ov-ep-override-test
branch
from
July 20, 2026 02:55
a2117c8 to
0924ed1
Compare
On OpenVINO hosts, resolve_genai_ep('cpu') incorrectly resolved to
'openvino' because OpenVINO advertises (npu, gpu, cpu) support and
appears before CPUExecutionProvider in the priority list.
Generic fix: prefer EPs whose *primary* device (first entry in
EP_SUPPORTED_DEVICES) matches the resolved device. Multi-device EPs
like OpenVINO target npu primarily — when the user says --device cpu
or --device gpu they expect the native EP, not a cross-device
accelerator that also happens to support it.
Examples:
--device cpu → CPUExecutionProvider (not OpenVINO)
--device gpu → DmlExecutionProvider (not OpenVINO/QNN)
--device npu → QNN/OpenVINO/VitisAI (unchanged, all primary=npu)
Falls back to eps[0] when no native EP is available.
Fixes the born-broken test_device_cpu_overrides_device_and_ep on OV
hosts (PR #1046).
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
DingmaomaoBJTU
changed the title
xieofxie
approved these changes
Jul 20, 2026
tezheng
added a commit
that referenced
this pull request
Jul 21, 2026
Resolve conflicts preferring branch features, with three deliberate exceptions where main had the stronger change: - check_ops.py / op_utils.py: drop the dead `sys_info` writer parameter (main's cleanup; it was collected but never persisted per its own comment). - _perf_genai.py: adopt main's device-native EP preference (#1138) but return via the branch's `short_ep_name`/`EPNameOrAlias` idiom; cast str->EPName for the EP_SUPPORTED_DEVICES lookup. - check_patterns.py: keep the branch's single-source-of-truth re-export of EP checkers from check_ops (required by test_ep_checker_dedup) plus main's RuntimeCheckerRulesPrefilter body; include main's additive NvTensorRTRTX choice (a GPU EP, outside the branch's NPU carve-out). Also fix a pre-existing branch bug (fails identically on the pre-merge tip): the winml-genai perf path passed the raw EpAtSourceParamType `(ep, source)` tuple straight into GenaiPerfConfig.ep instead of the EP name. Unpack it like the ONNX path does. Restores test_explicit_ep_overrides_device / test_explicit_ep_without_device. mypy (405 files) and ruff (src/ tests/) are green. This merge introduces zero new test failures; the 20 remaining failures in analyze/build/perf CLI tests are pre-existing and identical on the pre-merge tip (stale EP/device-resolution mocks + one `tuple.device` bug in the build EP path).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
On OpenVINO hosts,
resolve_genai_ep("cpu")incorrectly resolved to"openvino"instead of"cpu", causingtest_device_cpu_overrides_device_and_epto fail (born-broken since #1046).Root Cause
resolve_eps("cpu")returns EPs in_DEVICE_EP_MAPpriority order derived fromEP_SUPPORTED_DEVICESdeclaration order. SinceOpenVINOExecutionProviderdeclares("npu", "gpu", "cpu")and appears beforeCPUExecutionProvider, the old code blindly tookeps[0]→"openvino".Fix (generic)
Prefer EPs whose primary device (first entry in
EP_SUPPORTED_DEVICES) matches the resolved device. Multi-device EPs like OpenVINO targetnpuprimarily — when the user says--device cpuor--device gputhey expect the native EP, not a cross-device accelerator.--devicecpuopenvino❌cpu✅gpuopenvino(if OV first)dml/cuda✅npuqnn/openvinoFalls back to
eps[0]when no device-native EP is available.Verification
test_device_cpu_overrides_device_and_eppasses on OV host