Fix Kafka change feed continuation after partition splits

[Tomas Varon (tvaron3)]

Summary

• Project stale parent change feed continuations onto each post-split child range.
• Preserve record-level LSN overrides only for the token range that produced them.
• Apply the same projection during runtime split handling and connector restart/rebalance.
• Fail loudly when split resolution returns no overlapping ranges instead of silently dropping the task.
• Add Core, Kafka state, restart, and live source-task regression coverage.

Why

A parent continuation can contain divergent child LSNs after a physical partition split. Kafka previously collapsed that state to one scalar token and applied it to every child, which could advance a lower-LSN child past unread records. A live repro confirmed permanent marker loss. This is related to the Spark bounded change feed issue in #49883, but Kafka manifests the shared continuation-projection mistake as data loss rather than a bounded-read hang.

[azure-pipelines]

Azure Pipelines:
Successfully started running 2 pipeline(s).
32 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[azure-pipelines]

Azure Pipelines:
Successfully started running 2 pipeline(s).
32 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Pull request overview

Fixes Kafka Cosmos DB change feed continuation handling after physical partition splits by projecting stale parent continuations onto the effective child feed ranges, preventing one child’s higher LSN from incorrectly advancing a sibling range (data loss risk). This mirrors the underlying “composite continuation must not be collapsed” lesson from the related Spark fix (#49883), but applies it to the Kafka connector’s runtime split handling and restart/rebalance paths.

Changes:

• Add continuation projection utilities in CosmosChangeFeedRequestOptions to scope composite continuation tokens (and item-level LSN overrides) to the correct effective feed range.
• Update Kafka connector state handling to project/partition continuations on restart/rebalance and when resolving splits/merges; fail fast when split resolution returns no overlaps.
• Add unit + connector regression tests covering divergent child continuations, state extraction behavior, and a live source-task scenario.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/models/CosmosChangeFeedRequestOptions.java	Implements continuation projection to an effective FeedRange and exposes helper to extract range-scoped continuation strings.
sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/implementation/ImplementationBridgeHelpers.java	Extends the internal accessor to allow extracting a feed-range-scoped continuation.
sdk/cosmos/azure-cosmos/CHANGELOG.md	Adds release note for the Kafka continuation projection fix.
sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/implementation/KafkaChangeFeedContinuationProjectionTest.java	Adds unit tests for projecting parent continuation state onto child ranges and scoping item-LSN overrides.
sdk/cosmos/azure-cosmos-kafka-connect/src/test/java/com/azure/cosmos/kafka/connect/implementation/source/KafkaCosmosChangeFeedStateTest.java	Adds tests ensuring KafkaCosmosChangeFeedState extraction scopes continuation correctly and preserves/drops item LSN as intended.
sdk/cosmos/azure-cosmos-kafka-connect/src/test/java/com/azure/cosmos/kafka/connect/implementation/source/CosmosSourceTaskTest.java	Adds a regression test that exercises polling with divergent child continuations to ensure no missed marker after projection.
sdk/cosmos/azure-cosmos-kafka-connect/src/test/java/com/azure/cosmos/kafka/connect/CosmosSourceConnectorTest.java	Updates split/restart test setup to use child-scoped continuation tokens and to project state per feed range when building task units.
sdk/cosmos/azure-cosmos-kafka-connect/src/main/java/com/azure/cosmos/kafka/connect/implementation/source/KafkaCosmosChangeFeedState.java	Adds extraction method that projects continuation to a target feed range and scopes item-LSN override to the owning range; improves null itemLsn deserialization.
sdk/cosmos/azure-cosmos-kafka-connect/src/main/java/com/azure/cosmos/kafka/connect/implementation/source/CosmosSourceTask.java	Uses projected child continuation state for new ranges and fails loudly when no overlaps are found.
sdk/cosmos/azure-cosmos-kafka-connect/src/main/java/com/azure/cosmos/kafka/connect/CosmosSourceConnector.java	Projects offset continuation state onto current container feed ranges for restart/rebalance correctness.
sdk/cosmos/azure-cosmos-kafka-connect/pom.xml	Exports routing internals needed by new tests/utilities under JPMS test runs.
sdk/cosmos/azure-cosmos-kafka-connect/CHANGELOG.md	Adds release note for the connector-side data loss fix after splits.

[Tomas Varon (tvaron3)]

/azp run java - cosmos - kafka

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).