Skip to content

FIX: select the native extension by interpreter architecture on Windows - #727

Open
om singhal (Om-singhaI) wants to merge 2 commits into
microsoft:mainfrom
Om-singhaI:fix/loader-interpreter-architecture
Open

FIX: select the native extension by interpreter architecture on Windows#727
om singhal (Om-singhaI) wants to merge 2 commits into
microsoft:mainfrom
Om-singhaI:fix/loader-interpreter-architecture

Conversation

@Om-singhaI

@Om-singhaI om singhal (Om-singhaI) commented Aug 22, 2026

Copy link
Copy Markdown

PR title: FIX: select the native extension by interpreter architecture on Windows

Branch: fix/loader-interpreter-architecture (local only, not pushed)


Work Item / Issue Reference

GitHub Issue: #726


Summary

mssql_python/ddbc_bindings.py chose which compiled extension to load from platform.machine(). On Windows that reports the host CPU, not the architecture the running interpreter was built for. With an x64 CPython on a Windows ARM64 machine (the default python.org installer is x64 and pip installs the win_amd64 wheel), the loader looked for ddbc_bindings.cp314-arm64.pyd, which the wheel never shipped. The lookup missed on every import, a Warning: Using fallback module file ... line was printed to stdout, and the module was picked by directory order through the fallback branch.

Fix

  • On Windows the architecture now comes from sysconfig.get_platform(), which is derived from the interpreter build and matches the wheel tag (win-amd64 to amd64, win-arm64 to arm64, win32 to win32). The token then goes through the existing normalize_architecture() mapping, so the file names are unchanged for native x64 and native ARM64 installs. macOS and Linux keep using platform.machine() as before (macOS is universal2 anyway).
  • The file search moved into find_module_path(), and the fallback notice is now emitted as a RuntimeWarning through warnings.warn() instead of print(), so tools that parse a script's stdout no longer receive a stray warning line. The fallback to the first matching file is kept as it was.
  • tests/test_000_dependencies.py built its expected file name from platform.machine() as well, so test_python_extension_exists failed on the same configuration. It now derives the raw architecture from the loader's get_interpreter_architecture().

Changes

  • mssql_python/ddbc_bindings.py: new get_interpreter_architecture(), get_module_architecture() and find_module_path() helpers; module level code now calls them. No change to the public driver API.
  • tests/test_000_dependencies.py: DependencyTester uses the loader helper; the two "conceptual" tests that re simulated the print() and the ImportError string now exercise find_module_path() against a temp directory; new tests monkeypatch sysconfig.get_platform and platform.machine to cover an x64 interpreter on an ARM64 host, a native x64 interpreter, a native ARM64 interpreter, a 32 bit interpreter, and the Linux and macOS passthrough.
  • CHANGELOG.md: entry under Unreleased / Fixed.

Before (loader source executed with platform.system() = Windows, platform.machine() = ARM64, sysconfig.get_platform() = win-amd64, directory containing only ddbc_bindings.cp314-amd64.pyd):

expected_module  : ddbc_bindings.cp314-arm64.pyd
module_path      : ddbc_bindings.cp314-amd64.pyd   (via fallback)
stdout captured  : 'Warning: Using fallback module file ddbc_bindings.cp314-amd64.pyd instead of ddbc_bindings.cp314-arm64.pyd\n'

After (same inputs):

architecture     : amd64
module_path      : ddbc_bindings.cp314-amd64.pyd   (exact match)
stdout captured  : ''
warnings raised  : []

Validation

  • python -m pytest tests/test_000_dependencies.py: 26 passed, 3 skipped on main; 33 passed, 3 skipped with this change (macOS, Python 3.14, prebuilt 1.13.0 universal2 binary placed next to the source tree). The 3 skips are the Windows only dependency tests and the Linux only distribution test.
  • With the source change stashed, the updated test module fails at collection (ImportError: cannot import name 'find_module_path'), and the reproduction above prints the stdout warning again.
  • black --check --line-length=100 mssql_python/ tests/: clean. flake8 on the two changed files reports only findings that already exist on main (one pre existing E501 went away with the replaced line).
  • Not verified on real Windows ARM64 hardware. The Windows behaviour is covered by monkeypatching sysconfig.get_platform and platform.machine, and by executing the loader source with those values patched. A run on a Windows ARM64 machine with an x64 interpreter would be the definitive check: python -c "import mssql_python" should print nothing.

Possible follow ups (not in this PR)

  • The fallback still loads module_files[0] when nothing matches exactly. Raising instead of guessing would be stricter, but it changes behaviour for anyone relying on the fallback today, so it is left for a separate decision.
  • The 32 bit naming was inconsistent on main: CMakeLists.txt and build.bat name the x86 binary with a win32 token, while the loader normalized win32 to x86 and looked for a file the build never produces. The second commit aligns the loader with the build (x86 becomes win32, the same way x64 becomes amd64) and the 32 bit test case expects the win32 token. No x86 wheels are published today, so shipped packages are unaffected. The fallback notice also gets stacklevel=2 so it points at the import site.

The loader in mssql_python/ddbc_bindings.py derived the architecture token
of the compiled module from platform.machine(). On Windows that call reports
the host CPU, not the architecture the interpreter was built for. An x64
CPython running on a Windows ARM64 machine, which is what the default
python.org installer gives you, therefore looked for the arm64 .pyd while
the installed win_amd64 wheel only ships the amd64 .pyd. The lookup missed
on every import, a warning was printed to stdout, and the module was picked
by directory order through the fallback branch.

On Windows the architecture now comes from sysconfig.get_platform(), which
is derived from the interpreter build and matches the wheel tag. Its
platform string is reduced to the amd64, arm64 or win32 token and fed
through the existing normalize_architecture() mapping, so the resulting
file names are unchanged for native x64 and ARM64 installs. macOS and Linux
keep using platform.machine() as before.

The module search moved into find_module_path() and the fallback notice is
now a RuntimeWarning instead of a print() call, so scripts whose stdout is
parsed by other tools no longer receive a stray warning line. The fallback
to the first matching file is kept as it was.

tests/test_000_dependencies.py built its expected file name from
platform.machine() as well, which made test_python_extension_exists fail
on the same configuration. It now uses the loader's helper. New tests
cover the x64 interpreter on an ARM64 host, the native ARM64 and x64
interpreters, the 32 bit interpreter, the non Windows passthrough, and the
exact match, fallback warning and no match branches of find_module_path().

Refs microsoft#726
Copilot AI lite review requested due to automatic review settings August 22, 2026 07:02
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes Windows native-extension selection in mssql_python/ddbc_bindings.py by deriving architecture from the running interpreter (via sysconfig.get_platform()) instead of the host CPU (platform.machine()), preventing unnecessary fallback loading (and stdout noise) when x64 Python runs on Windows ARM64.

Changes:

  • Add loader helpers (get_interpreter_architecture, get_module_architecture, find_module_path) and move module file resolution into find_module_path().
  • Replace the stdout print() fallback notice with a RuntimeWarning via warnings.warn().
  • Update tests/test_000_dependencies.py to align expected extension naming with the loader and add targeted tests for Windows interpreter-vs-host architecture behavior.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
mssql_python/ddbc_bindings.py Uses interpreter-derived Windows architecture and refactors module discovery into find_module_path() with warning-based fallback reporting.
tests/test_000_dependencies.py Aligns dependency expectations with the loader and adds regression tests covering Windows ARM64 host + x64 interpreter and other architecture cases.
CHANGELOG.md Documents the GH-726 loader fix and warning behavior change under Unreleased/Fixed.
Suppressed comments (1)

mssql_python/ddbc_bindings.py:164

  • The fallback warning is emitted from inside the loader, so without a stacklevel it will point at ddbc_bindings.py rather than the caller’s import site. Setting stacklevel=2 makes the warning actionable for users (it will report the line that imported/triggered the loader).
    warnings.warn(
        f"Using fallback module file {module_files[0]} instead of {expected_module}",
        RuntimeWarning,
    )

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread mssql_python/ddbc_bindings.py Outdated
Comment thread tests/test_000_dependencies.py Outdated
…ension

The build names the 32 bit Windows artifact with the win32 token
(pybind/build.bat and pybind/CMakeLists.txt both map x86 to win32), while
the loader normalized the interpreter architecture to x86 and so looked
for a file that is never produced. get_module_architecture now renames
x86 to win32 the same way it already renames x64 to amd64, and the test
for a 32 bit interpreter expects the token the build actually writes.

The fallback notice is emitted with stacklevel 2 so the warning points at
the import site rather than at the loader itself.

Raised in review.
@dlevy-msft-sql

Copy link
Copy Markdown
Contributor

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@github-actions

Copy link
Copy Markdown

Code Coverage Report

Diff coverage Overall coverage Lines covered
100% 82% 7777 of 9449

Files needing attention

mssql_python.pybind.logger_bridge.cpp: 59.2%
mssql_python.pybind.ddbc_bindings.h: 61.5%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.pybind.ddbc_bindings.cpp: 75.5%
mssql_python.__init__.py: 77.6%
mssql_python.row.py: 77.6%
mssql_python.pybind.connection.connection_pool.cpp: 81.4%
mssql_python.pybind.connection.connection.cpp: 84.3%
mssql_python.logging.py: 85.5%
mssql_python.connection.py: 85.7%

View Azure DevOps build

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants