Skip to content

fix: restore GCC 13.3 ARM64 Release builds and macOS regression coverage - #538

Closed
lucasfang wants to merge 11 commits into
apache:mainfrom
lucasfang:codex/fix-gcc13-arm64-release
Closed

lucasfang wants to merge 11 commits into
apache:mainfrom
lucasfang:codex/fix-gcc13-arm64-release

Conversation

@lucasfang

@lucasfang lucasfang commented Nov 9, 2026 •

Copy link
Copy Markdown
Member

Purpose

Linked issue: closes #537

GCC 13.3 ARM64 Release compilation fails with -Wmaybe-uninitialized promoted to an error in the realtime store tests. Default-construct the affected optionals before assigning through the status macros, including the same pattern found in deletion-vector and file-index code, while retaining -Werror.

Complete macOS regression exposed additional portability failures. Use fixed-width integer literals and expectations in tests, normalize temporary paths, convert early dates with calendar arithmetic, fix Darwin symbolizer text I/O, and make the REST mock server stop reliably when macOS does not wake accept() on shutdown. Add date, symbolizer and idle-server shutdown regression coverage.

Correct the macOS optional-module build and runtime setup: Apple-native C/Rust dependency toolchains and frameworks, Vortex Darwin libraries, TBB deprecation handling, Lucene's CMake 4 script-mode compatibility, and Jindo's installed dylib alias.

Incorporate and extend the changes from #536: add macOS 15 ARM64 AppleClang Release/Debug and Homebrew GCC 15 Release jobs with CTest, installed external CMake consumer verification and failure artifacts; support Homebrew ccache/libclang and macOS CPU detection. Give each compiler/configuration its own cache and artifact names. Add a Linux ARM64 GCC 13.3 Release regression job that verifies both C and C++ compiler versions are exactly 13.3.0. Explicitly enable Avro and ORC rather than relying on their defaults. Keep all available optional modules enabled and leave external network tests disabled. Lumina 0.4.0 includes Linux x86_64 and Linux AArch64 artifacts: enable both in CMake and Linux CI, preserve the existing GCC 8 exclusion, and disable Lumina on macOS and unsupported architectures.

Tests

Validated locally on macOS ARM64 with GCC 13.3.0, Release and strict project warnings, using a compatible macOS SDK. Apple-specific native dependencies use Apple's toolchain where required; Paimon's C++ code uses GCC.

  • Original realtime-store translation-unit compilation reproduced both reported diagnostics; the fixed source compiled successfully with -O3 -DNDEBUG -Wall -Werror.
  • Baseline Release build completed; ctest --test-dir build-gcc13-release --output-on-failure -j 3 --timeout 900: 31/31 targets passed (7,678 GoogleTest cases passed, 14 skipped).
  • Full-feature Release build completed; ctest --test-dir build-gcc13-full --output-on-failure -j 3 --timeout 900: 39/39 targets passed (8,651 GoogleTest cases passed, 35 skipped).
  • Full-feature modules: Avro, ORC, Mosaic, Lance, Vortex, Jindo, OSS, S3, Lucene, full-text and REST. Lumina and external network tests were disabled.
  • After incorporating ci: add macOS ARM64 Release build and test coverage #536, rebuilt the full-feature configuration and reran ctest --test-dir build-gcc13-full -R '^paimon-vortex-format-test$' --output-on-failure --timeout 180: passed.
  • bash -n ci/scripts/build_paimon.sh ci/scripts/setup_flatc.sh
  • bash ci/scripts/test_cmake_modules.sh: all 25 checks passed.
  • bash ci/scripts/test_packaging_args.sh: all 28 checks passed.
  • bash ci/scripts/test_asan_symbolize.sh: passed, including the Darwin subprocess text-I/O regression.
  • All configured pre-commit hooks passed for changed files; git diff --check passed.
  • Isolated mocked-command checks passed for Linux x86_64/AArch64 and Darwin x86_64/ARM64 feature selection, plus Darwin libclang setup and existing flatc detection.
  • Lumina correction: verified the pinned 0.4.0 archive's SHA256 and its AArch64 ELF binary; exercised the actual Lumina CMake macro's Linux AArch64 configure and artifact installation. Seven mocked CI cases cover Linux x86_64/AArch64/arm64, Darwin, unsupported architectures and the GCC 8 exclusion. This does not execute Linux binaries.
  • CI expansion: checked the GCC 13.3 ARM64 matrix entry and exercised compiler-version acceptance/rejection with mocked commands, including a mismatched C++ compiler. Mocked all three macOS matrix invocations to verify build types, Avro/ORC flags, Lumina exclusion and installation smoke invocation. All workflow shell blocks pass Bash syntax checks; changed-file pre-commit checks pass.
  • Documentation: all changed-file pre-commit hooks, including Sphinx lint, pass. Built the build and integration guides together in an isolated Sphinx HTML project using sphinx-build -W --keep-going -b html; no warnings or unresolved internal references. The full API/documentation site is left to documentation CI.

Linux ARM64, sanitizers, and the new macOS matrix jobs/installed consumers were not run locally. The expanded CI matrix will provide the Linux GCC 13.3 and macOS compiler/configuration results; the existing Linux jobs remain enabled.

API and Format

No public API, storage format or protocol changes. Date conversion now validates the Gregorian calendar directly, including year zero, instead of relying on platform-specific timegm behavior.

Documentation

Update the README, contributor setup notes and build guide to describe macOS ARM64 CI coverage, the Linux GCC 13.3 regression job, explicit Avro/ORC flags, compiler/platform exclusions and validation limits. Correct Lumina's platform requirements to include Linux AArch64. Distinguish macOS Intel architecture detection from ARM64 CI coverage, and clarify that cross compilation is not supported end to end.

Generative AI tooling

Generated-by: OpenAI Codex (GPT-6)

@SteNicholas SteNicholas left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lucasfang, I left some comments for this pull request. PTAL.

Comment on lines +119 to +122
ci/scripts/build_paimon.sh \
--source_dir "$(pwd)" \
--build_type "${{ matrix.build_type }}" \
--install_smoke

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All three new macOS jobs fail before reaching CTest or the installation smoke test. In the current run, compilation of paimon_file_index_objlib fails with fmt/format.h not found. Configuration selects system fmt, but that object target does not receive its include directory: add_paimon_lib() applies STATIC_LINK_LIBS to the final libraries, and the internal paimon_shared dependency does not propagate compile usage requirements to this object target. Please wire fmt's usage requirements into the object-library compilation, or explicitly select bundled fmt for these jobs, and verify that the actual macOS matrix reaches its tests.

Comment on lines +72 to +74
- name: gcc15-release-macos-arm64
cc: /opt/homebrew/bin/gcc-15
cxx: /opt/homebrew/bin/g++-15

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The GCC macOS entry also needs C++ dependencies built against GCC's standard library. Dependency resolution remains AUTO, and this job's configure log already selects Homebrew's system fmt. That library uses libc++, whereas g++-15 uses libstdc++. After supplying the include path, a minimal fmt::format("value={}", 42) program reproduces a link failure for fmt::v12::vformat[abi:cxx11] with GCC; the same program links with AppleClang. Thus fixing the missing header alone still leaves this entry unable to link. Please explicitly build fmt, and any other selected C++ binary dependencies, with the selected GCC toolchain rather than consuming incompatible Homebrew bottles.

Comment on lines +174 to +179
- name: gcc13-release-aarch64
runner: ubuntu-24.04-arm
cc: gcc-13
cxx: g++-13
gcc_version: '13.3.0'
build_args: --build_type Release

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The GCC 13.3 ARM64 regression job still cannot compile the current head. Its build log fails in src/paimon/rest/rest_token_file_system_test.cpp:638 at std::optional<std::shared_ptr<FileSystem>> cached = cache->GetIfPresent(token);, with the same -Werror=maybe-uninitialized diagnostic on the shared-pointer payload. This configuration enables REST, so the optional-initialization fixes in the realtime-store and deletion-vector code leave another build blocker before CTest can run. Please include this remaining initialization in the workaround and rerun the actual GCC 13.3.0 ARM64 Release job with strict warnings retained.

@lucasfang
lucasfang marked this pull request as draft November 10, 2026 01:18
@lucasfang lucasfang closed this Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] GCC 13.3 ARM64 Release build fails with maybe-uninitialized errors

2 participants