Repository navigation
build(deps): validate shared third-party patch state - #570
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Linked issue: close #569
A dependency version or patch change can currently reuse sources and installed libraries marked successful by an older build. Introduce shared validation of source versions, archive checksums, ordered patches, and registered build inputs. Reject changed or incomplete state with explicit cleanup paths; preserve incremental builds when inputs are unchanged and never delete files automatically.
Migrate Arrow, ORC, Jieba, OSS SDK v2, and Vortex to the shared mechanism. Align ORC and Vortex build directories with the other dependencies and remove
PAIMON_VORTEX_SOURCE_DIR. Uselsdifffrom patchutils to enumerate patched files and removePAIMON_OPTIONAL_INSTALLso installation requires built artifacts.Tests
The patch lifecycle test script is kept locally and is not included in this PR. Local checks with Unix Makefiles and Ninja cover patch content changes with preserved timestamps; patch additions, removal, and ordering; source and build input changes; partial patch failures and process interruption; build/install retry; inconsistent completion records; legacy artifacts and cleanup paths; source damage and direct-step/install guards; directory migration; and unchanged-build reuse.
Additional cases cover file additions/deletions, unsafe patch paths, subproject registration, cleanup followed by a clean rebuild, and the Vortex adapter's Cargo working directory and output layout using simulated Cargo. Installation checks validate completion state rather than individual installed files.
Regression cases also cover multi-file diffs with unmatched brackets and header-like hunk content; missing or failing
lsdiff; local directory URLs and archive checksums; truncated verification manifests; and rejecting clean-first or source URL changes before overwriting patched sources.Manual verification with the real ORC patch
Built and installed ORC 2.1.1 using
cmake_modules/orc.diffin an isolated build directory. Each build used the same command:After the first successful build, changed an added comment in
orc.diffthat is applied toMemoryPool.hh. The next build failed during CMake regeneration, before changing the existing library or installed header. Actual error output (call stack omitted):Kept the modified patch, deleted exactly the seven reported paths, then reconfigured and rebuilt successfully. The top-level CMake cache and downloaded source archive were retained. The rebuild ended with:
Verified that both the extracted source and installed
MemoryPool.hhcontained the changed comment, and that patch and installation completion records matched the new identity. Another unchanged build ran only the guard and preserved source and artifact contents and timestamps. Restoredorc.diffto its original contents after the experiment.API and Format
No public C++ API, storage format, or protocol changes.
PAIMON_VORTEX_SOURCE_DIRandPAIMON_OPTIONAL_INSTALLare removed; patched dependencies use checksum-verified source archives and single-configuration CMake generators. Existing dependency builds may require the cleanup reported by the new checks.Documentation
Update the build guide with the patchutils requirement and dependency reuse, cleanup, and retry instructions. Install patchutils explicitly in the CI build matrix, GCC 8 and release-candidate jobs, and in both Ubuntu and CentOS development containers.
Generative AI tooling
Generated-by: OpenAI Codex (codex-cli 0.167.0)