Skip to content

[c++] Add write callbacks with batch completion dispatch - #4328

Open
naivedogger wants to merge 8 commits into
apache:mainfrom
naivedogger:feature/rust-batch-write-callbacks
Open

naivedogger wants to merge 8 commits into
apache:mainfrom
naivedogger:feature/rust-batch-write-callbacks

Conversation

@naivedogger

Copy link
Copy Markdown
Contributor

Purpose

Linked issue: close #4276

Add C++ write callbacks so applications can handle write completion without maintaining a separate WriteResult waiting queue.

Brief change log

  • Add callback overloads for Append, AppendArrowBatch, Upsert, and Delete.
  • Register callbacks directly on Rust write batches and dispatch them through shared workers, avoiding per-record async waiting tasks.
  • Add coverage for registration races, error propagation, result ordering, and callback lifetime and exceptions.

Tests

  • Rust core: 794 tests passed; Rust C++ bindings: 9 tests passed.
  • Existing C++ test binary: 4 non-cluster tests passed.
  • cargo fmt, cargo clippy, and git diff --check passed.
  • No live-cluster tests or full Maven verification for this revision.

API and Format

  • Add C++ callback overloads while preserving existing Wait/await result semantics and retry behavior. Shared completion allocation and notification scheduling change.
  • Return submission status synchronously and report completion through callbacks. Submission errors do not register callbacks; multi-bucket ArrowBatch writes remain non-atomic.
  • Callbacks may execute concurrently and out of order. The completion queue is unbounded, and Flush/destruction do not drain callbacks.
  • The existing Rust shutdown-retry issue, delivery deadlines, and durable recovery are outside this PR's scope.
  • No wire or storage format changes.

Documentation

Update the C++ API reference with callback examples, compatibility notes, partial-failure semantics, and application responsibilities for memory limits and shutdown.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] C++ client: provide async/callback API for write result instead of blocking Wait()

1 participant