Skip to content

[FLINK-40690][pipeline-connector/fluss] Preserve nested row field positions - #4549

Open
fxbing wants to merge 1 commit into
apache:masterfrom
fxbing:fix/20260917-fluss-nested-row-mapping
Open

fxbing wants to merge 1 commit into
apache:masterfrom
fxbing:fix/20260917-fluss-nested-row-mapping

Conversation

@fxbing

@fxbing fxbing commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

What is the purpose of this pull request?

Fix incorrect nested ROW values in the Fluss pipeline sink when the parent column mapping reorders columns or does not cover every child position. For example, mapping [nested, id] to [id, nested] must preserve (7, 8) inside the nested row. See FLINK-40690.

Brief change log

  • Wrap nested records with their own identity mapping using the existing factory, instead of reusing the parent mapping.
  • Add regressions for reordered and sparse parent mappings, wider children, null fields, multiple ROW levels, and BinaryRecordData.
  • Add a pipeline regression that writes to a pre-existing Fluss table with reordered top-level columns and reads back the complete result, asserting exactly Row.of(42, Row.of(7, 8)) without a LIMIT.

Verifying this change

  • All six new regression tests fail with assertion failures before the fix.
  • The new pipeline regression also fails against the old implementation after a real write/read round trip, returning Row.of(42, Row.of(8, 7)).
  • Clean reactor builds passed all 28 selected tests with both Java 11 / Flink 1.20.3 and Java 17 / Flink 2.2.0: six CdcAsFlussRowTest cases, 19 FlussConversionsTest cases, two FlussEventSerializationSchemaTest cases, and the new FlussSinkPipelineITCase#testNestedRowWithReorderedColumns. No failures, errors, or skips.
  • Spotless apply/check and git diff --check passed.
  • Tests used an isolated local Maven repository with a consistent set of cached Apache Fluss 20260901.094454 SNAPSHOT artifacts, avoiding incompatible locally built SNAPSHOT jars. No project dependency or POM changes were required.

Documentation

  • Does this pull request introduce a new feature? No.
  • If yes, how is the feature documented? Not applicable.
Was generative AI tooling used to co-author this PR?
  • Yes — Codex CLI; Claude and GLM assisted with analysis, implementation, and review.

Use a child-local identity mapping when adapting nested ROW values.
Add six adapter regressions and a pipeline write/read regression for reordered columns.

Generated-by: Codex CLI 0.154.0

@loserwang1024 loserwang1024 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants