Conversation
OverwriteAction::assert_requirements lets a caller supply arbitrary TableRequirements alongside the ones an action derives automatically: a supplied requirement of an auto-derived kind (UuidMatch, RefSnapshotIdMatch for the target ref) replaces the derived one, any other kind is added, and repeated calls accumulate. This makes it possible to assert a pinned base snapshot id that differs from whatever the in-memory Table currently reports -- needed so a retried commit can keep asserting the same base every attempt rather than silently rebasing onto whatever is current. Transaction::stage_commit runs every action's commit logic (including its storage side effects, e.g. writing manifest files) and returns the resulting identifier/requirements/updates as a new StagedCommit value, without ever touching the catalog. StagedCommit serializes to exactly the JSON body the Iceberg REST catalog protocol expects for a table or transaction commit, so it can be submitted later, elsewhere, including as one element of a multi-table transactions/commit call. Since Transaction::apply's local requirement-check is a sanity check against the in-memory Table's own metadata, it's a tautology for auto-derived requirements but actively wrong for caller-supplied ones that are meant to be validated by the catalog instead (they may deliberately diverge from what's currently loaded). ActionCommit now carries those separately via with_unchecked_requirements/ take_unchecked_requirements so only auto-derived requirements go through the local check; both still end up in the final commit. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
CI's check-public-api job flags any undeclared public API surface change; this snapshot was missed when the feature landed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two related additions to
Transaction, needed for a caller (raicode) that must assert a commit against an externally-pinned base snapshot -- possibly one that no longer matches whatever the in-memoryTablecurrently reports -- and, separately, that wants to compute a commit's requirements/updates without submitting them to a catalog at all (e.g. to submit later as one element of a multi-tabletransactions/commit).OverwriteAction::assert_requirements(requirements)-- supply arbitraryTableRequirements alongside the ones this action derives automatically (currentlyUuidMatchand aRefSnapshotIdMatchfor the target branch). A supplied requirement of an auto-derived kind replaces the derived one; any other kind is added alongside; repeated calls accumulate.Transaction::stage_commit()-- runs every action's commit logic (including storage side effects, e.g. writing manifest files) and returns a newStagedCommit { identifier, requirements, updates }value instead of submitting anything to a catalog.StagedCommitserializes to exactly the JSON body the Iceberg REST catalog protocol expects for a table commit, or for one element of a multi-tabletransactions/commitcall.Why the ActionCommit change
Transaction::apply's existing local requirement check (requirement.check(Some(table.metadata()))?) validates every requirement against the in-memoryTableit's being applied to. For auto-derived requirements this is a tautology (they're read straight off that same table), but for a caller-supplied override it's actively wrong -- the whole reason to override is to assert something that may not match what's currently loaded (e.g. a retry that must keep asserting an old base even though a fresher load of the table has since moved past it, so the catalog -- not this local check -- can correctly reject it as a genuine conflict).ActionCommitnow carries caller-supplied requirements separately viawith_unchecked_requirements/take_unchecked_requirements, so only auto-derived requirements go through the local check; both still end up in the final commit.Test plan
cargo test -p iceberg --lib-- 1798 passed, 0 failedcargo fmt -p iceberg -- --checkcargo clippy -p iceberg --lib --all-targets-- cleanassert_requirementsoverriding an auto-derived requirement, adding a new requirement kind, accumulating across multiple calls;stage_commitnever touching the catalog, its output being independently submittable as a valid commit viaTableCommit::builder()+catalog.update_table(), respecting anassert_requirementsoverride end-to-end, and serializing to the exact REST catalog JSON shape.🤖 Generated with Claude Code