Skip to content

storage: let one persist sink batch span multiple timestamps - #38934

Draft
martykulma wants to merge 1 commit into
maz-sink-test-harnessfrom
maz-sink-multi-ts-batch
Draft

martykulma wants to merge 1 commit into
maz-sink-test-harnessfrom
maz-sink-multi-ts-batch

Conversation

@martykulma

@martykulma martykulma commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

BatchBuilderAndMetadata pinned a batch to one timestamp, which blocks any
grouping across timestamps.

Now it tracks the largest timestamp staged. That's what append_batches needs
after an UpperMismatch to tell a batch entirely below a raised lower, which
gets deleted, from one that straddles it, which is re-appended under the
narrowed description with persist filtering the rest on read.

No behavior change, write_batches still opens a builder per timestamp so the
largest staged is the only one.

a_straddling_batch_is_usable_under_a_raised_lower covers it end to end.

🤖 Generated with Claude Code

BatchBuilderAndMetadata no longer pins a batch to a single timestamp. It
tracks the largest timestamp staged instead, which is what append_batches
needs to tell a batch entirely below a raised append lower from one that
straddles it. The straddling batch is re-appended under the narrowed
description and persist filters the out-of-bounds updates on read.

No behavior change: write_batches still opens a builder per timestamp, so
the largest staged timestamp is the only one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@martykulma
martykulma added this pull request to stack #38937 September 19, 2026 00:30
@martykulma martykulma changed the title maz sink multi ts batch storage: let one persist sink batch span multiple timestamps Sep 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant