Skip to content

[fix][txn] Prevent queued snapshot writes from stalling - #26565

Open
lhotari wants to merge 3 commits into
apache:masterfrom
lhotari:lh-fix-snapshot-worker-handoff
Open

[fix][txn] Prevent queued snapshot writes from stalling#26565
lhotari wants to merge 3 commits into
apache:masterfrom
lhotari:lh-fix-snapshot-worker-handoff

Conversation

@lhotari

@lhotari lhotari commented Sep 12, 2026

Copy link
Copy Markdown
Member

Depends on #26567. Its interceptor fixture class-loading fix is included so the broker test suite can run successfully.

Fixes #19650

Motivation

SegmentAbortedTxnProcessorTest.testClearSnapshotSegments can observe only one of the two expected snapshot segments because the persistent worker schedules its next task before releasing its operating state. If that task runs immediately, it cannot acquire the state and returns, leaving the next write queued until another operation triggers processing.

A related race occurs when another caller drains the queue between the initial empty check and acquiring the operating state: the second empty check returns without releasing that state.

Modifications

Release the operating state before scheduling the next successful task. When the second queue check finds no work, release the state and recheck the queue to cover an append that observed the worker as busy.

Add deterministic tests for immediate executor handoff and for queue drain/append interleavings, retaining assertions that queued writes run serially and eventually complete.

Verifying this change

  • Make sure that the change passes the CI checks.

  • The immediate-executor test fails on the original implementation because the second write never starts.

  • The queue-interleaving test fails without the empty-queue fix because an appended write never completes.

  • Both regression tests passed 10 invocations each with retries disabled; temporary invocation counts were removed.

  • All 6 tests in SegmentAbortedTxnProcessorTest passed, including testClearSnapshotSegments.

  • ./gradlew spotlessCheck checkstyleMain checkstyleTest passed.

Does this pull request potentially affect one of the following parts:

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Delegate the context loader mocks to an empty URLClassLoader with the test loader as parent, so logging service discovery receives valid resource enumerations.

Assisted-by: Codex
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky-test: SegmentAbortedTxnProcessorTest.testClearSnapshotSegments

1 participant