Repository navigation
Conversation
Fix optional assignment diagnostics and portability failures found by the macOS ARM64 full-feature regression. Incorporate the macOS CI coverage from apache#536. Closes apache#537
There was a problem hiding this comment.
@lucasfang, I left some comments for this pull request. PTAL.
| ci/scripts/build_paimon.sh \ | ||
| --source_dir "$(pwd)" \ | ||
| --build_type "${{ matrix.build_type }}" \ | ||
| --install_smoke |
There was a problem hiding this comment.
All three new macOS jobs fail before reaching CTest or the installation smoke test. In the current run, compilation of paimon_file_index_objlib fails with fmt/format.h not found. Configuration selects system fmt, but that object target does not receive its include directory: add_paimon_lib() applies STATIC_LINK_LIBS to the final libraries, and the internal paimon_shared dependency does not propagate compile usage requirements to this object target. Please wire fmt's usage requirements into the object-library compilation, or explicitly select bundled fmt for these jobs, and verify that the actual macOS matrix reaches its tests.
| - name: gcc15-release-macos-arm64 | ||
| cc: /opt/homebrew/bin/gcc-15 | ||
| cxx: /opt/homebrew/bin/g++-15 |
There was a problem hiding this comment.
The GCC macOS entry also needs C++ dependencies built against GCC's standard library. Dependency resolution remains AUTO, and this job's configure log already selects Homebrew's system fmt. That library uses libc++, whereas g++-15 uses libstdc++. After supplying the include path, a minimal fmt::format("value={}", 42) program reproduces a link failure for fmt::v12::vformat[abi:cxx11] with GCC; the same program links with AppleClang. Thus fixing the missing header alone still leaves this entry unable to link. Please explicitly build fmt, and any other selected C++ binary dependencies, with the selected GCC toolchain rather than consuming incompatible Homebrew bottles.
| - name: gcc13-release-aarch64 | ||
| runner: ubuntu-24.04-arm | ||
| cc: gcc-13 | ||
| cxx: g++-13 | ||
| gcc_version: '13.3.0' | ||
| build_args: --build_type Release |
There was a problem hiding this comment.
The GCC 13.3 ARM64 regression job still cannot compile the current head. Its build log fails in src/paimon/rest/rest_token_file_system_test.cpp:638 at std::optional<std::shared_ptr<FileSystem>> cached = cache->GetIfPresent(token);, with the same -Werror=maybe-uninitialized diagnostic on the shared-pointer payload. This configuration enables REST, so the optional-initialization fixes in the realtime-store and deletion-vector code leave another build blocker before CTest can run. Please include this remaining initialization in the workaround and rerun the actual GCC 13.3.0 ARM64 Release job with strict warnings retained.
Purpose
Linked issue: closes #537
GCC 13.3 ARM64 Release compilation fails with
-Wmaybe-uninitializedpromoted to an error in the realtime store tests. Default-construct the affected optionals before assigning through the status macros, including the same pattern found in deletion-vector and file-index code, while retaining-Werror.Complete macOS regression exposed additional portability failures. Use fixed-width integer literals and expectations in tests, normalize temporary paths, convert early dates with calendar arithmetic, fix Darwin symbolizer text I/O, and make the REST mock server stop reliably when macOS does not wake
accept()on shutdown. Add date, symbolizer and idle-server shutdown regression coverage.Correct the macOS optional-module build and runtime setup: Apple-native C/Rust dependency toolchains and frameworks, Vortex Darwin libraries, TBB deprecation handling, Lucene's CMake 4 script-mode compatibility, and Jindo's installed dylib alias.
Incorporate and extend the changes from #536: add macOS 15 ARM64 AppleClang Release/Debug and Homebrew GCC 15 Release jobs with CTest, installed external CMake consumer verification and failure artifacts; support Homebrew ccache/libclang and macOS CPU detection. Give each compiler/configuration its own cache and artifact names. Add a Linux ARM64 GCC 13.3 Release regression job that verifies both C and C++ compiler versions are exactly 13.3.0. Explicitly enable Avro and ORC rather than relying on their defaults. Keep all available optional modules enabled and leave external network tests disabled. Lumina 0.4.0 includes Linux x86_64 and Linux AArch64 artifacts: enable both in CMake and Linux CI, preserve the existing GCC 8 exclusion, and disable Lumina on macOS and unsupported architectures.
Tests
Validated locally on macOS ARM64 with GCC 13.3.0, Release and strict project warnings, using a compatible macOS SDK. Apple-specific native dependencies use Apple's toolchain where required; Paimon's C++ code uses GCC.
-O3 -DNDEBUG -Wall -Werror.ctest --test-dir build-gcc13-release --output-on-failure -j 3 --timeout 900: 31/31 targets passed (7,678 GoogleTest cases passed, 14 skipped).ctest --test-dir build-gcc13-full --output-on-failure -j 3 --timeout 900: 39/39 targets passed (8,651 GoogleTest cases passed, 35 skipped).ctest --test-dir build-gcc13-full -R '^paimon-vortex-format-test$' --output-on-failure --timeout 180: passed.bash -n ci/scripts/build_paimon.sh ci/scripts/setup_flatc.shbash ci/scripts/test_cmake_modules.sh: all 25 checks passed.bash ci/scripts/test_packaging_args.sh: all 28 checks passed.bash ci/scripts/test_asan_symbolize.sh: passed, including the Darwin subprocess text-I/O regression.git diff --checkpassed.sphinx-build -W --keep-going -b html; no warnings or unresolved internal references. The full API/documentation site is left to documentation CI.Linux ARM64, sanitizers, and the new macOS matrix jobs/installed consumers were not run locally. The expanded CI matrix will provide the Linux GCC 13.3 and macOS compiler/configuration results; the existing Linux jobs remain enabled.
API and Format
No public API, storage format or protocol changes. Date conversion now validates the Gregorian calendar directly, including year zero, instead of relying on platform-specific
timegmbehavior.Documentation
Update the README, contributor setup notes and build guide to describe macOS ARM64 CI coverage, the Linux GCC 13.3 regression job, explicit Avro/ORC flags, compiler/platform exclusions and validation limits. Correct Lumina's platform requirements to include Linux AArch64. Distinguish macOS Intel architecture detection from ARM64 CI coverage, and clarify that cross compilation is not supported end to end.
Generative AI tooling
Generated-by: OpenAI Codex (GPT-6)