FIX: select the native extension by interpreter architecture on Windows - #727
Open
om singhal (Om-singhaI) wants to merge 2 commits into
Open
FIX: select the native extension by interpreter architecture on Windows#727om singhal (Om-singhaI) wants to merge 2 commits into
om singhal (Om-singhaI) wants to merge 2 commits into
Conversation
The loader in mssql_python/ddbc_bindings.py derived the architecture token of the compiled module from platform.machine(). On Windows that call reports the host CPU, not the architecture the interpreter was built for. An x64 CPython running on a Windows ARM64 machine, which is what the default python.org installer gives you, therefore looked for the arm64 .pyd while the installed win_amd64 wheel only ships the amd64 .pyd. The lookup missed on every import, a warning was printed to stdout, and the module was picked by directory order through the fallback branch. On Windows the architecture now comes from sysconfig.get_platform(), which is derived from the interpreter build and matches the wheel tag. Its platform string is reduced to the amd64, arm64 or win32 token and fed through the existing normalize_architecture() mapping, so the resulting file names are unchanged for native x64 and ARM64 installs. macOS and Linux keep using platform.machine() as before. The module search moved into find_module_path() and the fallback notice is now a RuntimeWarning instead of a print() call, so scripts whose stdout is parsed by other tools no longer receive a stray warning line. The fallback to the first matching file is kept as it was. tests/test_000_dependencies.py built its expected file name from platform.machine() as well, which made test_python_extension_exists fail on the same configuration. It now uses the loader's helper. New tests cover the x64 interpreter on an ARM64 host, the native ARM64 and x64 interpreters, the 32 bit interpreter, the non Windows passthrough, and the exact match, fallback warning and no match branches of find_module_path(). Refs microsoft#726
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes Windows native-extension selection in mssql_python/ddbc_bindings.py by deriving architecture from the running interpreter (via sysconfig.get_platform()) instead of the host CPU (platform.machine()), preventing unnecessary fallback loading (and stdout noise) when x64 Python runs on Windows ARM64.
Changes:
- Add loader helpers (
get_interpreter_architecture,get_module_architecture,find_module_path) and move module file resolution intofind_module_path(). - Replace the stdout
print()fallback notice with aRuntimeWarningviawarnings.warn(). - Update
tests/test_000_dependencies.pyto align expected extension naming with the loader and add targeted tests for Windows interpreter-vs-host architecture behavior.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
mssql_python/ddbc_bindings.py |
Uses interpreter-derived Windows architecture and refactors module discovery into find_module_path() with warning-based fallback reporting. |
tests/test_000_dependencies.py |
Aligns dependency expectations with the loader and adds regression tests covering Windows ARM64 host + x64 interpreter and other architecture cases. |
CHANGELOG.md |
Documents the GH-726 loader fix and warning behavior change under Unreleased/Fixed. |
Suppressed comments (1)
mssql_python/ddbc_bindings.py:164
- The fallback warning is emitted from inside the loader, so without a
stacklevelit will point atddbc_bindings.pyrather than the caller’s import site. Settingstacklevel=2makes the warning actionable for users (it will report the line that imported/triggered the loader).
warnings.warn(
f"Using fallback module file {module_files[0]} instead of {expected_module}",
RuntimeWarning,
)
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…ension The build names the 32 bit Windows artifact with the win32 token (pybind/build.bat and pybind/CMakeLists.txt both map x86 to win32), while the loader normalized the interpreter architecture to x86 and so looked for a file that is never produced. get_module_architecture now renames x86 to win32 the same way it already renames x64 to amd64, and the test for a 32 bit interpreter expects the token the build actually writes. The fallback notice is emitted with stacklevel 2 so the warning points at the import site rather than at the loader itself. Raised in review.
Contributor
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
Code Coverage Report
Files needing attentionmssql_python.pybind.logger_bridge.cpp: 59.2% mssql_python.pybind.ddbc_bindings.h: 61.5% mssql_python.pybind.logger_bridge.hpp: 70.8% mssql_python.pybind.ddbc_bindings.cpp: 75.5% mssql_python.__init__.py: 77.6% mssql_python.row.py: 77.6% mssql_python.pybind.connection.connection_pool.cpp: 81.4% mssql_python.pybind.connection.connection.cpp: 84.3% mssql_python.logging.py: 85.5% mssql_python.connection.py: 85.7% |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR title: FIX: select the native extension by interpreter architecture on Windows
Branch: fix/loader-interpreter-architecture (local only, not pushed)
Work Item / Issue Reference
Summary
mssql_python/ddbc_bindings.pychose which compiled extension to load fromplatform.machine(). On Windows that reports the host CPU, not the architecture the running interpreter was built for. With an x64 CPython on a Windows ARM64 machine (the default python.org installer is x64 and pip installs thewin_amd64wheel), the loader looked forddbc_bindings.cp314-arm64.pyd, which the wheel never shipped. The lookup missed on every import, aWarning: Using fallback module file ...line was printed to stdout, and the module was picked by directory order through the fallback branch.Fix
sysconfig.get_platform(), which is derived from the interpreter build and matches the wheel tag (win-amd64toamd64,win-arm64toarm64,win32towin32). The token then goes through the existingnormalize_architecture()mapping, so the file names are unchanged for native x64 and native ARM64 installs. macOS and Linux keep usingplatform.machine()as before (macOS is universal2 anyway).find_module_path(), and the fallback notice is now emitted as aRuntimeWarningthroughwarnings.warn()instead ofprint(), so tools that parse a script's stdout no longer receive a stray warning line. The fallback to the first matching file is kept as it was.tests/test_000_dependencies.pybuilt its expected file name fromplatform.machine()as well, sotest_python_extension_existsfailed on the same configuration. It now derives the raw architecture from the loader'sget_interpreter_architecture().Changes
mssql_python/ddbc_bindings.py: newget_interpreter_architecture(),get_module_architecture()andfind_module_path()helpers; module level code now calls them. No change to the public driver API.tests/test_000_dependencies.py:DependencyTesteruses the loader helper; the two "conceptual" tests that re simulated theprint()and theImportErrorstring now exercisefind_module_path()against a temp directory; new tests monkeypatchsysconfig.get_platformandplatform.machineto cover an x64 interpreter on an ARM64 host, a native x64 interpreter, a native ARM64 interpreter, a 32 bit interpreter, and the Linux and macOS passthrough.CHANGELOG.md: entry under Unreleased / Fixed.Before (loader source executed with
platform.system()=Windows,platform.machine()=ARM64,sysconfig.get_platform()=win-amd64, directory containing onlyddbc_bindings.cp314-amd64.pyd):After (same inputs):
Validation
python -m pytest tests/test_000_dependencies.py: 26 passed, 3 skipped on main; 33 passed, 3 skipped with this change (macOS, Python 3.14, prebuilt 1.13.0 universal2 binary placed next to the source tree). The 3 skips are the Windows only dependency tests and the Linux only distribution test.ImportError: cannot import name 'find_module_path'), and the reproduction above prints the stdout warning again.black --check --line-length=100 mssql_python/ tests/: clean.flake8on the two changed files reports only findings that already exist on main (one pre existing E501 went away with the replaced line).sysconfig.get_platformandplatform.machine, and by executing the loader source with those values patched. A run on a Windows ARM64 machine with an x64 interpreter would be the definitive check:python -c "import mssql_python"should print nothing.Possible follow ups (not in this PR)
module_files[0]when nothing matches exactly. Raising instead of guessing would be stricter, but it changes behaviour for anyone relying on the fallback today, so it is left for a separate decision.CMakeLists.txtandbuild.batname the x86 binary with awin32token, while the loader normalizedwin32tox86and looked for a file the build never produces. The second commit aligns the loader with the build (x86 becomeswin32, the same way x64 becomesamd64) and the 32 bit test case expects thewin32token. No x86 wheels are published today, so shipped packages are unaffected. The fallback notice also getsstacklevel=2so it points at the import site.