Skip to content

GH-51614: [C++][FlightRPC] Universal macOS PKG Installer #51765

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
alinaliBQ wants to merge 3 commits into
base: main
Choose a base branch
from
+107 −4
Open

GH-51614: [C++][FlightRPC] Universal macOS PKG Installer #51765

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e955807
Universal macOS PKG Installer
alinaliBQ Oct 5, 2026
63100ca
Address Helder's comments
alinaliBQ Oct 9, 2026
851d248
Address Bryce's comments
alinaliBQ Oct 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 107 additions & 4 deletions .github/workflows/cpp_extra.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -567,10 +567,10 @@ jobs:
fail-fast: false
matrix:
include:
- { architecture: AMD64, macos-version: "15-intel", build-type : debug }
- { architecture: AMD64, macos-version: "15-intel", build-type : release }
- { architecture: ARM64, macos-version: "14", build-type : debug }
- { architecture: ARM64, macos-version: "14", build-type : release }
- { architecture: AMD64, macos-version: "26-intel", build-type : debug }
- { architecture: AMD64, macos-version: "26-intel", build-type : release }
- { architecture: ARM64, macos-version: "26", build-type : debug }
- { architecture: ARM64, macos-version: "26", build-type : release }
env:
ARROW_BUILD_TESTS: ON
ARROW_BUILD_TYPE: ${{ matrix.build-type }}
Expand Down Expand Up @@ -672,6 +672,13 @@ jobs:
name: flight-sql-odbc-pkg-installer-${{ matrix.architecture }}
path: build/cpp/ArrowFlightSQLODBC-*.pkg
if-no-files-found: error
- name: Upload AMD64 ODBC dylib to the job
if: matrix.build-type == 'release' && matrix.architecture == 'AMD64'
uses: actions/upload-artifact@v7
with:
name: flight-sql-odbc-dylib-${{ matrix.architecture }}
path: build/cpp/${{ matrix.build-type }}/libarrow_flight_sql_odbc.dylib
if-no-files-found: error
- name: Register Flight SQL ODBC Driver
run: |
sudo cpp/src/arrow/flight/sql/odbc/install/unix/install_odbc.sh $(pwd)/build/cpp/${{ matrix.build-type }}/libarrow_flight_sql_odbc.dylib
Expand All @@ -686,6 +693,100 @@ jobs:
sudo chown -R $USER /Library/ODBC
ci/scripts/cpp_test.sh $(pwd) $(pwd)/build

odbc-macos-universal:
needs: odbc-macos
name: Create ODBC universal macOS installer
runs-on: macos-26
timeout-minutes: 30
steps:
- name: Download ARM64 PKG
uses: actions/download-artifact@v8
with:
name: flight-sql-odbc-pkg-installer-ARM64
path: arm64-pkg
- name: Download AMD64 dylib
uses: actions/download-artifact@v8
with:
name: flight-sql-odbc-dylib-AMD64
path: amd64-dylib
- name: Build universal macOS installer
run: |
set -euo pipefail

pkg_path="$(ls arm64-pkg/ArrowFlightSQLODBC-*.pkg)"
pkg_name="$(basename "${pkg_path}" .pkg)"

# Use the ARM64 .pkg and swap in a universal ODBC dylib
# that works on ARM64 and AMD64.
pkgutil --expand-full "${pkg_path}" expanded

# The unversioned name libarrow_flight_sql_odbc.dylib is a symlink
# to the versioned dylib that the
# install name (@rpath/libarrow_flight_sql_odbc.<SOVERSION>.dylib)
# resolves to. Run lipo on the real file so the symlinks stay intact.
symlink_path="$(find expanded -name libarrow_flight_sql_odbc.dylib)"
lib_dir="$(dirname "${symlink_path}")"
dylib_path="$(realpath "${symlink_path}")"
echo "Merging into ${dylib_path} (via ${symlink_path})"
lipo -create "${dylib_path}" amd64-dylib/libarrow_flight_sql_odbc.dylib \
-output "${dylib_path}"
Comment thread
alinaliBQ marked this conversation as resolved.

# Verify every dylib (including the symlinks) is universal before re-packaging.
for lib in "${lib_dir}"/libarrow_flight_sql_odbc*.dylib; do
echo "Checking ${lib}"
lipo "${lib}" -verify_arch arm64 x86_64
done
Comment on lines +735 to +738

@amoeba amoeba Oct 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be simplified? A comment wouldn't be a bad call either.

Suggested change
for lib in "${lib_dir}"/libarrow_flight_sql_odbc*.dylib; do
dylib_archs="$(lipo -archs "${lib}")"
echo "${lib}: ${dylib_archs}"
for arch in arm64 x86_64; do
if ! grep -qw "${arch}" <<< "${dylib_archs}"; then
echo "${lib} is missing '${arch}' (got: ${dylib_archs})"
exit 1
fi
done
done
for lib in "${lib_dir}"/libarrow_flight_sql_odbc*.dylib; do
echo "Checking ${lib}"
lipo -verify_arch arm64 x86_64 "${lib}"
done

@alinaliBQ alinaliBQ Oct 13, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added suggestion and a comment 🙂

@amoeba amoeba Oct 14, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made a mistake here, I guess that's not how you call lipo. This fails,

$ lipo $LIB -verify_arch arm64 x86_86
lipo: -verify_arch requires exactly one input file

so I think what you had is fine.

@amoeba amoeba Oct 14, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hrm... But this works fine on CI (older CLT). Let me see if I can figure out what's up.

@amoeba amoeba Oct 14, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on flutter/flutter#188625 (comment), looks like a regression in the v27 XCode/CLT based. Since we don't want this job to break when we upgrade runners to macos-27, I think it's safer just to use what you had before (testing one arch at a time) and add a comment.


pkgutil --flatten-full expanded "${pkg_name}.pkg"
- name: Upload universal ODBC PKG
uses: actions/upload-artifact@v7
with:
name: flight-sql-odbc-pkg-installer-universal
path: ArrowFlightSQLODBC-*.pkg
if-no-files-found: error

odbc-macos-universal-verify:
needs: odbc-macos-universal
name: ODBC universal installer verify ${{ matrix.architecture }} macOS ${{ matrix.macos-version }}
runs-on: macos-${{ matrix.macos-version }}
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
include:
- architecture: AMD64
macos-version: "26-intel"
- architecture: ARM64
macos-version: "26"
steps:
- name: Download universal PKG
uses: actions/download-artifact@v8
with:
name: flight-sql-odbc-pkg-installer-universal
- name: Install ODBC PKG
run: |
set -euo pipefail

pkg_path="$(ls ArrowFlightSQLODBC-*.pkg)"
sudo installer -pkg "${pkg_path}" -target / -verbose
- name: Check ODBC dylib installation
run: |
set -euo pipefail

dylib="/Library/ODBC/arrow-odbc/lib/libarrow_flight_sql_odbc.dylib"

if [ ! -f "${dylib}" ]; then
echo "ODBC dylib not found at ${dylib}"
exit 1
fi
echo "Found ODBC dylib: ${dylib}"

# Verify every installed dylib (including the symlinks) is universal.
for lib in /Library/ODBC/arrow-odbc/lib/libarrow_flight_sql_odbc*.dylib; do
echo "Checking ${lib}"
lipo "${lib}" -verify_arch arm64 x86_64
done
Comment on lines +785 to +788

@amoeba amoeba Oct 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing that the .pkg installs on both platforms is good, the remainder of the check is basically re-asserting what we already asserted in the previous step. At the least we could simplify this similar to my comment above (just check that the dylib supports both arches).

@alinaliBQ alinaliBQ Oct 13, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

simplified the step for verifying archs.


odbc-msvc:
needs: check-enabled
if: needs.check-enabled.outputs.is_enabled == 'true'
Expand Down Expand Up @@ -856,6 +957,8 @@ jobs:
- msvc-arm64
- odbc-linux
- odbc-macos
- odbc-macos-universal
- odbc-macos-universal-verify
- odbc-msvc
- odbc-nightly
uses: ./.github/workflows/report_ci.yml
Expand Down
Loading