Repository navigation
fix: restore GCC 13.3 ARM64 Release builds and macOS regression coverage #538
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
5528f2f
8b8909a
2211d06
6401537
5dce69c
2c1162d
e76798b
38707ae
5b34e5f
30cdff8
b362dbd
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| # Licensed to the Apache Software Foundation (ASF) under one | ||
| # or more contributor license agreements. See the NOTICE file | ||
| # distributed with this work for additional information | ||
| # regarding copyright ownership. The ASF licenses this file | ||
| # to you under the Apache License, Version 2.0 (the | ||
| # "License"); you may not use this file except in compliance | ||
| # with the License. You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| name: Prepare release assets | ||
| description: Cache and verify pinned GitHub release archives before building | ||
| runs: | ||
| using: composite | ||
| steps: | ||
| - uses: actions/cache@v4 | ||
| with: | ||
| path: ${{ runner.temp }}/paimon-release-assets | ||
| key: release-assets-v1-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('third_party/versions.txt', 'ci/scripts/setup_rust.sh') }} | ||
| - shell: bash | ||
| run: | | ||
| set -eu | ||
| source third_party/versions.txt | ||
| cache_dir="${RUNNER_TEMP}/paimon-release-assets" | ||
| for entry in "${DEPENDENCIES[@]}"; do | ||
| read -r var archive url <<< "${entry}" | ||
| case "${var}" in | ||
| PAIMON_ZLIB_URL|PAIMON_ZSTD_URL|PAIMON_PROTOBUF_URL|PAIMON_ARROW_URL) | ||
| checksum_var="${var%_URL}_BUILD_SHA256_CHECKSUM" | ||
| bash ci/scripts/download_release_asset.sh "${url}" "${cache_dir}/${archive}" "${!checksum_var}" | ||
| echo "${var}=${cache_dir}/${archive}" >> "${GITHUB_ENV}" | ||
| ;; | ||
| esac | ||
| done | ||
| echo "PAIMON_RELEASE_ASSET_DIR=${cache_dir}" >> "${GITHUB_ENV}" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -43,13 +43,96 @@ jobs: | |
| - name: Run CMake module tests | ||
| shell: bash | ||
| run: ci/scripts/test_cmake_modules.sh | ||
| - name: Test release asset downloads | ||
| shell: bash | ||
| run: bash ci/scripts/test_download_release_asset.sh | ||
| - name: Run packaging argument tests | ||
| shell: bash | ||
| run: ci/scripts/test_packaging_args.sh | ||
| - name: Run asan_symbolize tests | ||
| shell: bash | ||
| run: ci/scripts/test_asan_symbolize.sh | ||
|
|
||
| macos-build-and-test: | ||
| name: ${{ matrix.name }} | ||
| runs-on: macos-15 | ||
| timeout-minutes: 180 | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| include: | ||
| - name: appleclang-release-arm64 | ||
| cc: /usr/bin/clang | ||
| cxx: /usr/bin/clang++ | ||
| build_type: Release | ||
| - name: appleclang-debug-arm64 | ||
| cc: /usr/bin/clang | ||
| cxx: /usr/bin/clang++ | ||
| build_type: Debug | ||
| - name: gcc15-release-macos-arm64 | ||
| cc: /opt/homebrew/bin/gcc-15 | ||
| cxx: /opt/homebrew/bin/g++-15 | ||
| build_type: Release | ||
| gcc_formula: gcc@15 | ||
| env: | ||
| CC: ${{ matrix.cc }} | ||
| CXX: ${{ matrix.cxx }} | ||
| PAIMON_BUILD_JOBS: '3' | ||
| CARGO_BUILD_JOBS: '3' | ||
| steps: | ||
| - name: Checkout paimon-cpp | ||
| uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 | ||
| with: | ||
| lfs: true | ||
| - name: Install build dependencies | ||
| shell: bash | ||
| run: | | ||
| brew install cmake ninja protobuf openssl@3 curl | ||
| echo "$(brew --prefix protobuf)/bin" >> "$GITHUB_PATH" | ||
| echo "CMAKE_PREFIX_PATH=$(brew --prefix openssl@3):$(brew --prefix curl)" >> "$GITHUB_ENV" | ||
| - name: Install GCC | ||
| if: matrix.gcc_formula | ||
| shell: bash | ||
| env: | ||
| GCC_FORMULA: ${{ matrix.gcc_formula }} | ||
| run: brew install "${GCC_FORMULA}" | ||
| - name: Show compiler versions | ||
| shell: bash | ||
| run: | | ||
| "${CC}" --version | ||
| "${CXX}" --version | ||
| - name: Setup ccache | ||
| uses: ./.github/actions/setup-ccache | ||
| with: | ||
| cache-key-prefix: ccache-${{ matrix.name }} | ||
| - name: Prepare release assets | ||
| uses: ./.github/actions/setup-release-assets | ||
| - name: Install Rust and protobuf tools | ||
| shell: bash | ||
| run: ci/scripts/setup_rust.sh | ||
| - name: Install flatc and libclang (Vortex) | ||
| shell: bash | ||
| run: ci/scripts/setup_flatc.sh | ||
| - name: Build, test and verify installation | ||
| shell: bash | ||
| run: | | ||
| ci/scripts/build_paimon.sh \ | ||
| --source_dir "$(pwd)" \ | ||
| --build_type "${{ matrix.build_type }}" \ | ||
| --install_smoke | ||
|
Comment on lines
+119
to
+122
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. All three new macOS jobs fail before reaching CTest or the installation smoke test. In the current run, compilation of |
||
| - name: Upload test logs | ||
| if: failure() | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: ${{ matrix.name }}-test-logs | ||
| path: | | ||
| build/build/test-logs/ | ||
| build/Testing/Temporary/ | ||
| build/CMakeFiles/CMakeConfigureLog.yaml | ||
| - name: Show ccache statistics | ||
| if: always() | ||
| run: ccache -s | ||
|
|
||
| build-and-test: | ||
| name: ${{ matrix.name }} | ||
| runs-on: ${{ matrix.runner || 'ubuntu-24.04' }} | ||
|
|
@@ -88,6 +171,12 @@ jobs: | |
| cc: gcc-14 | ||
| cxx: g++-14 | ||
| build_args: --build_type Release | ||
| - name: gcc13-release-aarch64 | ||
| runner: ubuntu-24.04-arm | ||
| cc: gcc-13 | ||
| cxx: g++-13 | ||
| gcc_version: '13.3.0' | ||
| build_args: --build_type Release | ||
|
Comment on lines
+174
to
+179
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The GCC 13.3 ARM64 regression job still cannot compile the current head. Its build log fails in |
||
| - name: clang-debug-aarch64 | ||
| runner: ubuntu-24.04-arm | ||
| - name: asan-ubsan-aarch64 | ||
|
|
@@ -109,13 +198,28 @@ jobs: | |
| uses: ./.github/actions/setup-ccache | ||
| with: | ||
| cache-key-prefix: ccache-${{ matrix.name }} | ||
| - name: Prepare release assets | ||
| uses: ./.github/actions/setup-release-assets | ||
| - name: Install Rust and protobuf tools | ||
| shell: bash | ||
| run: ci/scripts/setup_rust.sh | ||
| - name: Install build and debugging dependencies | ||
| run: | | ||
| sudo apt-get update | ||
| sudo apt-get install -y gdb libcurl4-openssl-dev libssl-dev | ||
| - name: Install and verify regression compiler | ||
| if: matrix.gcc_version | ||
| shell: bash | ||
| env: | ||
| CC: ${{ matrix.cc }} | ||
| CXX: ${{ matrix.cxx }} | ||
| EXPECTED_GCC_VERSION: ${{ matrix.gcc_version }} | ||
| run: | | ||
| sudo apt-get install -y "${CC}" "${CXX}" | ||
| "${CC}" --version | ||
| "${CXX}" --version | ||
| test "$("${CC}" -dumpfullversion)" = "${EXPECTED_GCC_VERSION}" | ||
| test "$("${CXX}" -dumpfullversion)" = "${EXPECTED_GCC_VERSION}" | ||
| - name: Install flatc and libclang (Vortex) | ||
| shell: bash | ||
| run: ci/scripts/setup_flatc.sh | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| #!/usr/bin/env bash | ||
| # Licensed to the Apache Software Foundation (ASF) under one | ||
| # or more contributor license agreements. See the NOTICE file | ||
| # distributed with this work for additional information | ||
| # regarding copyright ownership. The ASF licenses this file | ||
| # to you under the Apache License, Version 2.0 (the | ||
| # "License"); you may not use this file except in compliance | ||
| # with the License. You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
| # | ||
| # Fetch a pinned GitHub release asset. Use the release API if the browser | ||
| # download endpoint times out, and publish a file only after verification. | ||
| set -euo pipefail | ||
|
|
||
| url=$1 | ||
| output=$2 | ||
| sha256=$3 | ||
|
|
||
| verify() { | ||
| if command -v sha256sum >/dev/null; then | ||
| echo "${sha256} $1" | sha256sum -c - | ||
| else | ||
| echo "${sha256} $1" | shasum -a 256 -c - | ||
| fi | ||
| } | ||
|
|
||
| if [[ -f "${output}" ]] && verify "${output}"; then | ||
| exit 0 | ||
| fi | ||
|
|
||
| mkdir -p "$(dirname "${output}")" | ||
| tmp=$(mktemp "${output}.XXXXXX") | ||
| trap 'rm -f "${tmp}"' EXIT | ||
| curl_args=(--proto '=https' --tlsv1.2 -fsSL --retry 3 --connect-timeout 15 --max-time 180) | ||
| if ! curl "${curl_args[@]}" "${url}" -o "${tmp}" || ! verify "${tmp}"; then | ||
| release=${url#https://github.com/} | ||
| repo=${release%%/releases/download/*} | ||
| asset=${release##*/} | ||
| tag=${release#*/releases/download/} | ||
| tag=${tag%/*} | ||
| asset_url=$(curl "${curl_args[@]}" "https://api.github.com/repos/${repo}/releases/tags/${tag}" | | ||
| python3 -c 'import json, sys | ||
| print(next(a["url"] for a in json.load(sys.stdin)["assets"] if a["name"] == sys.argv[1]))' "${asset}") | ||
| curl "${curl_args[@]}" -H 'Accept: application/octet-stream' "${asset_url}" -o "${tmp}" | ||
| verify "${tmp}" | ||
| fi | ||
| mv -f "${tmp}" "${output}" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The GCC macOS entry also needs C++ dependencies built against GCC's standard library. Dependency resolution remains
AUTO, and this job's configure log already selects Homebrew's system fmt. That library uses libc++, whereasg++-15uses libstdc++. After supplying the include path, a minimalfmt::format("value={}", 42)program reproduces a link failure forfmt::v12::vformat[abi:cxx11]with GCC; the same program links with AppleClang. Thus fixing the missing header alone still leaves this entry unable to link. Please explicitly build fmt, and any other selected C++ binary dependencies, with the selected GCC toolchain rather than consuming incompatible Homebrew bottles.