Skip to content

[flink][spark] Fix create_tag ignoring the table's default retention - #11294

Open
danzhewuju wants to merge 1 commit into
apache:masterfrom
danzhewuju:bugfix/fix-ignore-table-default-retention-error
Open

danzhewuju wants to merge 1 commit into
apache:masterfrom
danzhewuju:bugfix/fix-ignore-table-default-retention-error

Conversation

@danzhewuju

Copy link
Copy Markdown
Contributor

Purpose

Fixes #11293.
Scheduled jobs commonly set tag.default-time-retained on the table and omit time_retained in CALL ...create_tag. The procedure forwarded a null duration, so the tag was written without a retention period. Regular TTL cleanup skips those tags, allowing historical snapshots and their data files to accumulate.
Use the existing Table.createTag overloads that apply the table default when time_retained is omitted. This covers both the latest snapshot and a specified snapshot in the shared Flink procedure, the Flink 1.18 procedure, and the shared Spark procedure. An explicit duration still overrides the table default.

Tests

  • Added a Flink SQL regression that verifies the persisted default retention for both snapshot forms and an explicit one-hour override. The regression fails on the baseline and passes with the fix.
  • All nine selected Flink tag tests pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] create_tag ignores tag.default-time-retained when time_retained is omitted

1 participant