FEAT: conda recipe for mssql-python (self-contained repackage of the signed wheel) - #734
FEAT: conda recipe for mssql-python (self-contained repackage of the signed wheel)#734Jahnvi Thakkar (jahnvi480) wants to merge 4 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a dormant conda-build recipe (conda/mssql-python/) that repackages the prebuilt, ESRP-signed mssql-python wheel into a self-contained conda package by co-locating the mssql_python wheel and the mssql-python-odbc payload wheel in the same site-packages, with a Linux-only patchelf step to ensure the vendored driver resolves conda-provided runtime libraries.
Changes:
- Introduces
meta.yamlwith repackaging-safe conda-build settings (no relocation/prefix scanning) plus runtime dependency declarations for the bundled ODBC Driver payload. - Adds
build.sh/bld.batto install or extract wheels fully offline fromWHEELS_DIR, including Linux RUNPATH stamping and a win-arm64 bulk-copy limitation workaround. - Adds
conda/.gitattributesto enforce LF line endings for*.shscripts.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| conda/mssql-python/meta.yaml | Conda recipe metadata, build safeguards for signed binaries, and runtime dependency declarations. |
| conda/mssql-python/build.sh | Linux/macOS repackaging script; installs/extracts wheels and applies Linux RUNPATH “$ORIGIN climb” fix. |
| conda/mssql-python/bld.bat | Windows repackaging script; installs/extracts wheels and removes mssql_py_core on win-arm64. |
| conda/.gitattributes | Forces LF EOL for shell scripts to keep conda-build working across platforms. |
Suppressed comments (1)
conda/mssql-python/bld.bat:53
- Same wildcard issue as above for
ODBC_WHL: if the wheel isn't present, theforloop will still assign the literal pattern, so the intended explicit error message won't fire. Also switch the failure exits toexit /b 1for consistent batch behavior.
set "ODBC_WHL="
for %%W in ("%WHEELS_DIR%\mssql_python_odbc-%MSSQL_ODBC_VERSION%-py3-none-win_*.whl") do set "ODBC_WHL=%%~fW"
if not defined ODBC_WHL (
echo ERROR: no mssql_python_odbc==%MSSQL_ODBC_VERSION% py3-none-win wheel in "%WHEELS_DIR%"
exit 1
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…signed wheel) Repackages the prebuilt, ESRP-signed mssql-python wheel into a conda package offline (no compile), vendoring the ODBC Driver 18 payload inside it (the v1.11.0 self-contained model -- no separate mssql-python-odbc conda package). - meta.yaml: python >=3.10 floor (host + run); repackage-safe conda-build flags (error_overlinking / error_overdepending off, binary_relocation off, skip pyc on the osx/win cross legs); Linux run deps openssl >=3,<4 / krb5 / libtool for driver reachability (#563); win vc14_runtime. - build.sh: Linux/macOS repackage; patchelf $ORIGIN climb (#563); the cross osx-arm64 leg extracts the universal2 wheel without Python, filtered on CONDA_PY so it can never grab another interpreter's cpXY wheel. - bld.bat: Windows repackage; the cross win-arm64 leg extracts without Python and strips the x64 mssql_py_core (no arm64 build yet -> bulk copy raises a clean "not available" error); guarded wheel globs; exit /b on failure. - .gitattributes: *.sh eol=lf, *.bat eol=crlf.
3e6d637 to
6cce9af
Compare
📊 Code Coverage Report
Diff CoverageDiff: main...HEAD, staged and unstaged changesNo lines with coverage information in this diff. 📋 Files Needing Attention📉 Files with overall lowest coverage (click to expand)mssql_python.pybind.logger_bridge.cpp: 59.2%
mssql_python.pybind.ddbc_bindings.h: 61.5%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.pybind.ddbc_bindings.cpp: 75.5%
mssql_python.__init__.py: 77.6%
mssql_python.row.py: 77.6%
mssql_python.pybind.connection.connection_pool.cpp: 81.4%
mssql_python.pybind.connection.connection.cpp: 83.7%
mssql_python.logging.py: 85.5%
mssql_python.connection.py: 85.9%🔗 Quick Links
|
…g tweaks - meta.yaml: pin azure-identity >=1.12.0 in run: -- conda does NOT inherit the wheel's install_requires, so a solve against conda-forge could otherwise pick <1.12.0. - build.sh: count libmsodbcsql (driver) and libodbcinst (driver manager) separately so a Linux payload missing EITHER fails loudly (a lone libodbcinst would ship no SQL driver). - Condense the verbose block comments in meta.yaml / build.sh / bld.bat to concise, meaningful lines while keeping the essential "why".
…cross legs
The osx-arm64 (built on Intel) and win-arm64 (built on x64) legs extract the wheel
without running the target Python, and the pipeline can't import-test them. A real
Apple-Silicon test found every osx-arm64 build (py3.11-3.14) shipped the cp310
ddbc_bindings, so only 3.10 could import. The cp${CONDA_PY} wheel filter already picks
the right wheel; this adds a static post-extract assert that the packaged
mssql_python/ddbc_bindings.cp<CONDA_PY>-* actually exists -- the python-tag twin of the
existing win-arm64 PE-arch assert -- so a wrong-Python binding fails the build instead
of the user's import.
… hardening Cross-built osx-arm64 (on Intel) extracts the code wheel without running the arm64 Python, via a first-match glob that grabbed the alphabetically-first cp310 universal2 wheel for EVERY python -- so py3.11-3.14 shipped the cp310 ddbc_bindings and only 3.10 could import (found on a real Apple-Silicon Mac; the pipeline can't catch it because it never runs the arm64 import). Filter the code wheel by cp<CONDA_PY> (mirrors bld.bat's win_arm64 filter) and add a static post-extract assert that the packaged mssql_python/ddbc_bindings.cp<CONDA_PY>-* exists on the un-runnable cross legs. Also brings the recipe byte-in-line with PR #734: comment trim, azure-identity >=1.12.0 floor (conda does not inherit the wheel's install_requires), per-type Linux driver counters, guarded wheel globs, exit /b, and *.bat CRLF.
…a-build) The `python >=3.10` floor added in review broke conda-build's per-Python variant: a version range in `host: python` overrides the `--python` matrix, so every `--python 3.10..3.14` build collapsed into ONE package pinned to python_abi cp<newest=3.14> (build string `py>=310`). Every leg (win-64/arm64, osx-64/arm64, linux-64/aarch64) then failed validation with: package mssql-python-1.14.0-py314_0 requires python_abi 3.14.* *_cp314 ... python =3.10 is not installable `host: python` must be bare so conda-build binds the per-Python variant. The >=3.10 floor is enforced implicitly -- only py310-py314 are built and each package's python_abi pins its exact minor. Verified with `conda-build --output`: --python 3.10 -> py310_0, --python 3.13 -> py313_0. azure-identity >=1.12.0 (a real run floor) is kept.
Gaurav Sharma (bewithgaurav)
left a comment
Gaurav Sharma (bewithgaurav)
left a comment
There was a problem hiding this comment.
blocker on macOS, will re-review once done
| # security-serviced deps are declared here). OpenSSL for TLS is dlopen'd, so | ||
| # overlinking can't see it; pinned <4 (Driver 18 supports the OpenSSL 1.1/3.0 ABI | ||
| # only). macOS uses Homebrew OpenSSL, Windows SChannel. | ||
| - openssl >=3,<4 # [linux] |
There was a problem hiding this comment.
just double checked this one - On macOS this package is not self-contained for TLS and still points to homebrew installation instead of conda env
concrete break scenario: end user on a conda-only Mac (no Homebrew) runs connect("...;Encrypt=yes"), the default.
The connect fails with SSL Provider: [OpenSSL library could not be loaded, make sure OpenSSL 1.0, 1.1, or 3.0 is installed].
repro'd on the shipped 1.14.0 osx-arm64 package, one real SQL Server, Encrypt=yes:
- Homebrew present: TLS completes (login-failed), and the driver binds
/opt/homebrew/Cellar/openssl@3/3.6.3/lib/libssl.3.dylib. - Homebrew hidden (
sandbox-execdenying reads of/opt/homebrew, which simulates a clean Mac without deleting anything): the same connect fails withOpenSSL library could not be loaded.
why the recipe misses it:
openssl is declared [linux] only, the macOS payload bundles no libssl/libcrypto, and the $ORIGIN climb in build.sh has no macOS twin
conda's own openssl does land in $PREFIX/lib (pulled in by python), but the driver never falls back to it, so the env copy is unreachable.
Why CI and manual runs look green:
- both have Homebrew OpenSSL, so the driver silently binds that copy
- the masking-immune audit only covers Linux, so nothing catches this on macOS.
direction: give the macOS driver a way to resolve the env's own openssl (the @loader_path twin of the Linux climb pointing at $PREFIX/lib, or bundle/symlink it into the payload), and add a macOS self-containment check on a Homebrew-free agent so it cannot be masked again.
| REM Extract the py3-none-win odbc wheel into the SAME site-packages so | ||
| REM mssql_python_odbc\libs\ sits beside mssql_python\ and the loader finds the driver. | ||
| set "ODBC_WHL=" | ||
| for %%W in ("%WHEELS_DIR%\mssql_python_odbc-%MSSQL_ODBC_VERSION%-py3-none-win_*.whl") do if exist "%%~fW" set "ODBC_WHL=%%~fW" |
There was a problem hiding this comment.
The code wheel is matched arch-specifically (cp%CONDA_PY%-*-win_arm64.whl) but the ODBC wheel here uses py3-none-win_*, which matches both win_amd64 and win_arm64. The py3-none tag only means "no Python bytecode" — the ODBC payload DLLs inside are still architecture-specific.
This currently works only because of the implicit "one wheel per target WHEELS_DIR" contract. If both arch wheels are ever staged in the same dir, the for loop picks a filesystem-order-dependent match (last one wins, no break, no arch assert) and could silently vendor an x64 driver into a win-arm64 package. Unlike the code wheel, there's no post-extract arch guard here, so a wrong-arch driver would import fine and only fail at the user's first connect().
Consider making this glob arch-specific to mirror the code-wheel handling (e.g. select -win_arm64.whl on the arm64 leg / -win_amd64.whl otherwise), and optionally fail if more than one candidate matches. The same py3-none-* asymmetry exists in build.sh (lower-risk there since macOS ships universal2 payloads).
Work Item / Issue Reference
Summary
First of a series splitting the large conda onboarding branch (#720) into smaller, reviewable PRs.
Adds the conda-build recipe that repackages the prebuilt, ESRP-signed
mssql-pythonwheel into a self-contained conda package (it vendors the ODBC Driver 18 payload — there is no separate companion package). The recipe compiles nothing; it repackages the already-signed wheel, fully offline.Files
conda/mssql-python/meta.yaml— recipe manifest (version fromMSSQL_PYTHON_VERSION; repackage-safe build flags so conda-build never rewrites/scans the signed binaries).conda/mssql-python/build.sh— Linux/macOS: install/extract the code + odbc wheels into site-packages, then stamp the relative$ORIGINRUNPATH climb (Publish mssql-python as a conda-forge package #563) onto the vendored Linux driver so it resolves the env's own krb5/openssl.conda/mssql-python/bld.bat— Windows: extract the code + odbc wheels; on win-arm64 it strips the x64mssql_py_core(no arm64 build exists yet, so bulk copy is a documented limitation on that platform).conda/.gitattributesSafety: this recipe is dormant on
main— nothing runs it until the standalone conda build pipeline (a follow-up PR) invokes it, so merging is safe and changes no existing behavior.Follow-up PRs in this split: (2) conda build pipeline + the gates/probes it runs, (3) conda release pipeline, (4) the GitHub conda-audit workflow.