Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions conda/.gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Keep shell build scripts LF so conda-build works on Linux/macOS agents,
# regardless of the checkout host's core.autocrlf setting.
*.sh text eol=lf

# Keep .bat scripts CRLF for Windows conda-build agents, regardless of the checkout
# host's core.autocrlf. cmd.exe tolerates LF here (no goto/labels), but pinning CRLF
# removes any doubt on odd checkouts.
*.bat text eol=crlf
53 changes: 53 additions & 0 deletions conda/mssql-python/bld.bat
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
@echo on
REM Repackage the prebuilt, signed mssql-python wheel into a conda package (offline) and
REM vendor the ODBC Driver 18 payload inside it. conda-build exports PKG_NAME /
REM PKG_VERSION / CONDA_PY; the pipeline exports WHEELS_DIR + MSSQL_ODBC_VERSION.
setlocal enabledelayedexpansion

set "SP=%PREFIX%\Lib\site-packages"
if not exist "%SP%" mkdir "%SP%"
set "PKG_UNDERSCORE=%PKG_NAME:-=_%"

REM Native leg: pip installs the matching wheel. Cross win-arm64 leg: the arm64 host
REM Python can't run on the x64 agent, so extract the cp%CONDA_PY% wheel (a zip) with tar.
"%PYTHON%" -c "import sys" >nul 2>&1
if errorlevel 1 (
echo Host Python "%PYTHON%" not executable here ^(non-emulated cross-build^); extracting cp%CONDA_PY% win_arm64 code wheel without Python.
set "CODE_WHL="
for %%W in ("%WHEELS_DIR%\!PKG_UNDERSCORE!-%PKG_VERSION%-cp%CONDA_PY%-*-win_arm64.whl") do if exist "%%~fW" set "CODE_WHL=%%~fW"
if not defined CODE_WHL (
echo ERROR: no %PKG_NAME%==%PKG_VERSION% cp%CONDA_PY% win_arm64 wheel in "%WHEELS_DIR%"
exit /b 1
)
echo Extracting "!CODE_WHL!" into "%SP%"
tar -xf "!CODE_WHL!" -C "%SP%"
if errorlevel 1 exit /b 1
REM win-arm64 cross can't run the arm64 Python, so statically prove the extracted
REM binding is for THIS interpreter (the osx-arm64 twin bug shipped a cp310 .so in
REM every build); a wrong-Python .pyd would only fail at the user's import.
if not exist "%SP%\mssql_python\ddbc_bindings.cp%CONDA_PY%-*.pyd" (
echo ERROR: extracted "!CODE_WHL!" has no mssql_python\ddbc_bindings.cp%CONDA_PY% pyd ^(wrong-Python binding^).
exit /b 1
)
REM The win-arm64 wheel bundles the x64 mssql_py_core (no arm64 build exists yet); an
REM x64 .pyd can't load on arm64 and fails the PE-arch assert, so strip it. Bulk copy
REM then raises a clean "not available" error (lazy import); the rest of the DBAPI works.
echo Removing x64 mssql_py_core from the win-arm64 package; bulk copy is unavailable on win-arm64 until an arm64 build ships.
if exist "%SP%\mssql_py_core" rmdir /s /q "%SP%\mssql_py_core"
if exist "%SP%\mssql_py_core.libs" rmdir /s /q "%SP%\mssql_py_core.libs"
) else (
"%PYTHON%" -m pip install --no-deps --no-index --find-links "%WHEELS_DIR%" %PKG_NAME%==%PKG_VERSION% -vv
if errorlevel 1 exit /b 1
)
Comment thread
Copilot marked this conversation as resolved.

REM Extract the py3-none-win odbc wheel into the SAME site-packages so
REM mssql_python_odbc\libs\ sits beside mssql_python\ and the loader finds the driver.
set "ODBC_WHL="
for %%W in ("%WHEELS_DIR%\mssql_python_odbc-%MSSQL_ODBC_VERSION%-py3-none-win_*.whl") do if exist "%%~fW" set "ODBC_WHL=%%~fW"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code wheel is matched arch-specifically (cp%CONDA_PY%-*-win_arm64.whl) but the ODBC wheel here uses py3-none-win_*, which matches both win_amd64 and win_arm64. The py3-none tag only means "no Python bytecode" — the ODBC payload DLLs inside are still architecture-specific.

This currently works only because of the implicit "one wheel per target WHEELS_DIR" contract. If both arch wheels are ever staged in the same dir, the for loop picks a filesystem-order-dependent match (last one wins, no break, no arch assert) and could silently vendor an x64 driver into a win-arm64 package. Unlike the code wheel, there's no post-extract arch guard here, so a wrong-arch driver would import fine and only fail at the user's first connect().

Consider making this glob arch-specific to mirror the code-wheel handling (e.g. select -win_arm64.whl on the arm64 leg / -win_amd64.whl otherwise), and optionally fail if more than one candidate matches. The same py3-none-* asymmetry exists in build.sh (lower-risk there since macOS ships universal2 payloads).

if not defined ODBC_WHL (
echo ERROR: no mssql_python_odbc==%MSSQL_ODBC_VERSION% py3-none-win wheel in "%WHEELS_DIR%"
exit /b 1
)
echo Extracting "!ODBC_WHL!" into "%SP%"
tar -xf "!ODBC_WHL!" -C "%SP%"
if errorlevel 1 exit /b 1
104 changes: 104 additions & 0 deletions conda/mssql-python/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
#!/bin/bash
# Repackage the prebuilt, signed mssql-python wheel into a conda package (offline) and
# vendor the ODBC Driver 18 payload inside it -- no separate mssql-python-odbc conda
# package. Both the code wheel and the py3-none-<plat> odbc wheel land in the SAME
# site-packages, so mssql_python_odbc/libs/ sits beside mssql_python/ and the C++
# loader finds the driver. conda-build exports PKG_NAME / PKG_VERSION / CONDA_PY;
# WHEELS_DIR (one wheel per target) + MSSQL_ODBC_VERSION come from the pipeline.
set -euo pipefail
# Emulated aarch64 leg: $PYTHON is the aarch64 interpreter under qemu-user; point it at
# the aarch64 glibc loader so it doesn't abort. Harmless no-op on every other leg.
[ -d /usr/aarch64-linux-gnu ] && export QEMU_LD_PREFIX="${QEMU_LD_PREFIX:-/usr/aarch64-linux-gnu}"

odbc_ver="${MSSQL_ODBC_VERSION:?MSSQL_ODBC_VERSION not set}"

# Native / QEMU-emulated legs: the host Python runs, so pip installs both wheels. Cross
# osx-arm64 (built on Intel): the arm64 Python can't execute, so extract both wheels
# (zips) with unzip instead -- same approach as the Windows bld.bat.
if "$PYTHON" -c "import sys" >/dev/null 2>&1; then
"$PYTHON" -m pip install --no-deps --no-index --find-links "$WHEELS_DIR" "$PKG_NAME==$PKG_VERSION" -vv
"$PYTHON" -m pip install --no-deps --no-index --find-links "$WHEELS_DIR" "mssql-python-odbc==$odbc_ver" -vv
else
echo "Host Python '$PYTHON' is not executable on this agent (non-emulated cross-build);"
echo "extracting both wheels into \$SP_DIR without running Python."
mkdir -p "$SP_DIR"
pkg_underscore="${PKG_NAME//-/_}"
# universal2 wheels are cpXY-specific (compiled ddbc_bindings), so filter on the
# target CONDA_PY to never grab another interpreter's wheel (mirrors bld.bat).
code_whl=""
for w in "$WHEELS_DIR/${pkg_underscore}-${PKG_VERSION}-cp${CONDA_PY}-"*.whl; do
[ -e "$w" ] && { code_whl="$w"; break; }
done
[ -n "$code_whl" ] || { echo "ERROR: no ${PKG_NAME}==${PKG_VERSION} cp${CONDA_PY} wheel in '$WHEELS_DIR'" >&2; exit 1; }
odbc_whl=""
for w in "$WHEELS_DIR"/mssql_python_odbc-"$odbc_ver"-py3-none-*.whl; do
[ -e "$w" ] && { odbc_whl="$w"; break; }
done
[ -n "$odbc_whl" ] || { echo "ERROR: no mssql_python_odbc==$odbc_ver py3-none wheel in '$WHEELS_DIR'" >&2; exit 1; }
echo "Extracting '$code_whl' -> '$SP_DIR'"
unzip -oq "$code_whl" -d "$SP_DIR"
# osx-arm64 cross can't run the arm64 Python, so statically prove the extracted
# binding is for THIS interpreter -- a cpXY ddbc_bindings for another Python (the bug
# where every osx-arm64 build shipped the cp310 .so) would only fail at the user's
# import. The python-tag twin of the win-arm64 PE-arch assert.
ls "$SP_DIR"/mssql_python/ddbc_bindings.cp${CONDA_PY}-*.so >/dev/null 2>&1 || {
echo "ERROR: '$code_whl' has no mssql_python/ddbc_bindings.cp${CONDA_PY}-*.so (wrong-Python binding)." >&2
exit 1
}
echo "Extracting '$odbc_whl' -> '$SP_DIR'"
unzip -oq "$odbc_whl" -d "$SP_DIR"
fi

# ---------------------------------------------------------------------------
# Linux driver reachability (#563) -- the core fix.
# ---------------------------------------------------------------------------
# Declaring krb5/openssl/libltdl as conda deps drops them in $PREFIX/lib, but that is
# INERT: the vendored ODBC .so ship with a bare DT_RUNPATH=$ORIGIN (no climb), so the
# loader never looks in $PREFIX/lib and falls through to SYSTEM krb5 (#563 crash) or
# can't find libltdl.so.7. Fix: stamp a relative "$ORIGIN:$ORIGIN/<climb>" onto
# libmsodbcsql* + libodbcinst.so.2 so they resolve THIS env's $PREFIX/lib. Safe to
# patch -- the Linux .so are malware-scanned, not code-signed (only Windows .dll /
# macOS .dylib are, and those are never touched). Linux-only: the glob is a no-op on
# macOS. audit_bundled_binaries.py asserts the same exact climb.
prefix_lib="$PREFIX/lib"
shopt -s nullglob
have_linux_payload=0
[ -d "$SP_DIR/mssql_python_odbc/libs/linux" ] && have_linux_payload=1
# Count the driver (libmsodbcsql) and the driver manager (libodbcinst) separately so a
# payload missing EITHER fails loudly -- a lone libodbcinst would ship no SQL driver.
msodbc_seen=0
odbcinst_seen=0
for libdir in "$SP_DIR"/mssql_python_odbc/libs/linux/*/*/lib; do
# Exact climb from this driver dir up to $PREFIX/lib (from the real layout, never a
# hard-coded ../ count).
climb="$("$PYTHON" -c 'import os,sys; print(os.path.relpath(sys.argv[1], sys.argv[2]))' "$prefix_lib" "$libdir")"
want="\$ORIGIN:\$ORIGIN/$climb"
for so in "$libdir"/libmsodbcsql-*.so.* "$libdir"/libodbcinst.so.2; do
[ -e "$so" ] || continue
case "$(basename "$so")" in
libmsodbcsql-*.so.*) msodbc_seen=$((msodbc_seen + 1)) ;;
libodbcinst.so.2) odbcinst_seen=$((odbcinst_seen + 1)) ;;
esac
got="$(patchelf --print-rpath "$so" 2>/dev/null || true)"
if [ "$got" = "$want" ]; then
echo "RPATH-OK (already baked) $(basename "$so") -> $got"
continue
fi
patchelf --set-rpath "$want" "$so"
got="$(patchelf --print-rpath "$so")"
# Assert the EXACT intended RUNPATH, not just "no absolute entry".
if [ "$got" != "$want" ]; then
echo "ERROR: patch did not yield the exact expected RUNPATH ('$got' != '$want')." >&2
exit 1
fi
echo "RPATH-PATCHED $(basename "$so") -> $got"
done
done
shopt -u nullglob
# A Linux payload missing the driver OR the driver manager is a bypass hole (a bare
# `conda build` skipping the orchestrator audit would ship un-asserted binaries).
if [ "$have_linux_payload" = "1" ] && { [ "$msodbc_seen" = "0" ] || [ "$odbcinst_seen" = "0" ]; }; then
echo "ERROR: Linux payload present but incomplete (libmsodbcsql=$msodbc_seen, libodbcinst.so.2=$odbcinst_seen)." >&2
exit 1
fi
[ "$msodbc_seen" -gt 0 ] && echo "LINUX_RPATH_CLIMB_OK" || true
88 changes: 88 additions & 0 deletions conda/mssql-python/meta.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
{% set version = environ.get('MSSQL_PYTHON_VERSION', '1.14.0') %}

package:
name: mssql-python
version: "{{ version }}"

build:
number: 0
# This recipe REPACKAGES a prebuilt, signed wheel -- it compiles nothing -- so
# conda-build's from-source checks mis-fire on the vendored binaries. Downgrade
# overlinking/overdepending to warnings, and disable the relocation / prefix scans
# that would corrupt the signature or fail the pure-repackage packaging step.
error_overlinking: false
error_overdepending: false
binary_relocation: false
detect_binary_files_with_prefix: false
# .pyc byte-compilation runs the TARGET Python, which can't execute on the cross legs
# (osx-arm64 built on Intel, win-arm64 on x64). Skip pyc there; Python regenerates it.
skip_compile_pyc:
- "**/*.py" # [osx or win]
# Passed into the isolated conda-build env so build.sh / bld.bat can install the
# prebuilt wheel from --find-links offline (WHEELS_DIR) and locate the odbc wheel to
# vendor in (MSSQL_ODBC_VERSION).
script_env:
- WHEELS_DIR
- MSSQL_ODBC_VERSION

requirements:
build:
# patchelf stamps the $ORIGIN RPATH climb (#563) so the declared conda
# krb5/openssl/libltdl below are actually reachable. Linux-only, build-time only.
- patchelf # [linux]
host:
# Bare `python` (NO version range): conda-build binds the per-Python variant from the
# --python matrix here, so each build emits the matching py3XX package. A range such
# as `python >=3.10` collapses ALL --python builds into ONE cp<newest> package (build
# string "py>=310") whose python_abi is pinned to 3.14, so `conda create python=3.10
# mssql-python` can't solve. The >=3.10 floor is enforced implicitly: only py310-py314
# are built and each package's python_abi pins its exact minor.
- python
- pip
run:
# Bare `python` for the same reason as host (the floor is the per-build python_abi,
# NOT a range on python -- a range breaks the per-Python solve).
- python
# conda does NOT inherit the wheel's install_requires, so pin azure-identity here.
- azure-identity >=1.12.0
# ODBC Driver 18 payload deps (the driver ships inside this package, so its
# security-serviced deps are declared here). OpenSSL for TLS is dlopen'd, so
# overlinking can't see it; pinned <4 (Driver 18 supports the OpenSSL 1.1/3.0 ABI
# only). macOS uses Homebrew OpenSSL, Windows SChannel.
- openssl >=3,<4 # [linux]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just double checked this one - On macOS this package is not self-contained for TLS and still points to homebrew installation instead of conda env

concrete break scenario: end user on a conda-only Mac (no Homebrew) runs connect("...;Encrypt=yes"), the default.
The connect fails with SSL Provider: [OpenSSL library could not be loaded, make sure OpenSSL 1.0, 1.1, or 3.0 is installed].

repro'd on the shipped 1.14.0 osx-arm64 package, one real SQL Server, Encrypt=yes:

  • Homebrew present: TLS completes (login-failed), and the driver binds /opt/homebrew/Cellar/openssl@3/3.6.3/lib/libssl.3.dylib.
  • Homebrew hidden (sandbox-exec denying reads of /opt/homebrew, which simulates a clean Mac without deleting anything): the same connect fails with OpenSSL library could not be loaded.

why the recipe misses it:

openssl is declared [linux] only, the macOS payload bundles no libssl/libcrypto, and the $ORIGIN climb in build.sh has no macOS twin

conda's own openssl does land in $PREFIX/lib (pulled in by python), but the driver never falls back to it, so the env copy is unreachable.

Why CI and manual runs look green:

  • both have Homebrew OpenSSL, so the driver silently binds that copy
  • the masking-immune audit only covers Linux, so nothing catches this on macOS.

direction: give the macOS driver a way to resolve the env's own openssl (the @loader_path twin of the Linux climb pointing at $PREFIX/lib, or bundle/symlink it into the payload), and add a macOS self-containment check on a Homebrew-free agent so it cannot be masked again.

# libmsodbcsql NEEDs libkrb5 / libgssapi_krb5 on Linux (macOS framework, Windows SSPI).
- krb5 # [linux]
# libodbcinst.so.2 NEEDs libltdl.so.7 (not bundled); libtool provides it and the
# #563 climb makes the driver resolve THIS env's copy.
- libtool # [linux]
# msodbcsql18.dll imports VCRUNTIME140.dll but the vendored vcredist ships only
# msvcp140.dll; declare the serviced conda runtime.
- vc14_runtime # [win]
# Re-assert the wheel's platform floor (conda drops the wheel tag); never stricter.
- __glibc >=2.28 # [linux]
- __osx >=15.0 # [osx]

test:
imports:
- mssql_python

about:
home: https://github.com/microsoft/mssql-python
# Ships MIT code AND the proprietary ODBC Driver 18 payload, so the license is both.
license: MIT AND LicenseRef-Microsoft-Proprietary
license_file:
- ../../LICENSE
# The proprietary EULAs live with the ODBC payload; reference them there rather than
# duplicating. conda-build embeds them in info/licenses/.
- ../../mssql_python_odbc/licenses/MICROSOFT_ODBC_DRIVER_FOR_SQL_SERVER_LICENSE.txt
- ../../mssql_python_odbc/licenses/MICROSOFT_VISUAL_STUDIO_LICENSE.txt
summary: Microsoft driver for Python to interact with SQL Server and Azure SQL.
description: |
mssql-python is a DB API 2.0 (PEP 249) compliant driver for SQL Server,
Azure SQL, and Azure Synapse. This conda package is self-contained: the
proprietary Microsoft ODBC Driver 18 payload ships inside it (the same model as
the v1.11.0 wheel), so no separate driver package is required.

extra:
recipe-maintainers:
- jahnvithakkar
Loading