fix(hbase): handle missing CUSTOM_TIERING_TIME_RANGE in getCompactBoundariesForMajor - #8573
fix(hbase): handle missing CUSTOM_TIERING_TIME_RANGE in getCompactBoundariesForMajor#8573shoemoney wants to merge 1 commit into
Conversation
…ndariesForMajor Fix verified RED->GREEN. getCompactBoundariesForMajor silently drops files lacking CUSTOM_TIERING_TIME_RANGE at CustomDateTieredCompactionPolicy.java:72
wchevreuil
left a comment
wchevreuil
left a comment
There was a problem hiding this comment.
Yes, currently, if no file in the list has the CUSTOM_TIERING_TIME_RANGE tag, getCompactBoundariesForMajor returns a single MIN boundary, and a single file will result from the compaction. This single resulting file will now have the CUSTOM_TIERING_TIME_RANGE tag, and a subsequent compaction would be able to define two boundaries, if the time range cross the cutOffTimestamp. This is not optimal, this fix would solve this, but after reviewing this getCompactBoundariesForMajor and the append logic in CustomTieringMultiFileWriter, I think we can simply always set the min and cutOffTimestamp boundaries without needing to traverse the files to check the CUSTOM_TIERING_TIME_RANGE.
Please open a jira ticket to link this PR.
| long now) { | ||
| MutableLong min = new MutableLong(Long.MAX_VALUE); | ||
| MutableLong max = new MutableLong(0); | ||
| boolean[] hasMissing = new boolean[1]; |
There was a problem hiding this comment.
No need to declare an array here.
2 participants
Bug: getCompactBoundariesForMajor silently drops files lacking CUSTOM_TIERING_TIME_RANGE at CustomDateTieredCompactionPolicy.java:72. Files without the metadata are ignored when computing min and max, so compaction boundaries miss the cutOffTimestamp split and may return only MIN_VALUE.
Fix: Track hasMissing when timeRangeBytes is null or parse fails. If any file is missing, add cutOffTimestamp to boundaries and return [MIN_VALUE, cutOffTimestamp] to force correct tier separation, matching shouldPerformMajorCompaction which already returns true for missing files.
Evidence: Fix verified RED to GREEN. Stashed fix and ran existing compaction tests, restored fix. Formatter blast radius limited to single file hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CustomDateTieredCompactionPolicy.java (8 lines added). No other files changed.