Skip to content

Fix IcebergIO rejecting timestamp strings with a space separator - #40861

Open
mananmangal wants to merge 1 commit into
apache:masterfrom
mananmangal:iceberg-timestamp-space-separator
Open

mananmangal wants to merge 1 commit into
apache:masterfrom
mananmangal:iceberg-timestamp-space-separator

Conversation

@mananmangal

@mananmangal mananmangal commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #40860

IcebergIO only accepted a T between date and time when writing a STRING value to a timestamp or timestamptz column. A space, which RFC 3339 section 5.6 allows and Python's str(datetime) produces, threw DateTimeParseException.

When the character after the date is a space, the value is now parsed with a formatter that expects a space instead of T. All other values go through the existing OffsetDateTime.parse / LocalDateTime.parse calls unchanged.

Testing: added cases to IcebergUtilsTest for both column types, including invalid separators. :sdks:java:io:iceberg:test passes.

@github-actions github-actions Bot added java io iceberg Iceberg IO connector size/s S: 10-49 lines changed labels Oct 28, 2026
@mananmangal
mananmangal force-pushed the iceberg-timestamp-space-separator branch from 1e88658 to 6a4ea25 Compare October 28, 2026 05:23

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📋 Review Summary

This PR updates IcebergUtils.getIcebergTimestampValue to accept space-separated RFC 3339 timestamp strings when converting Beam STRING values to Iceberg timestamp and timestamptz columns, avoiding exception-based fallback overhead by checking the separator character up front. The implementation and unit test coverage are clean and well-targeted.

🔍 General Feedback

  • One minor consistency detail to watch out for: DateTimeFormatterBuilder.toFormatter() defaults to ResolverStyle.SMART, whereas LocalDateTime.parse (ISO_LOCAL_DATE_TIME) and OffsetDateTime.parse (ISO_OFFSET_DATE_TIME) use ResolverStyle.STRICT.

💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.

Accept a space between date and time when writing a STRING value to a
timestamp or timestamptz column, as allowed by RFC 3339 section 5.6.

Fixes apache#40860
@mananmangal
mananmangal force-pushed the iceberg-timestamp-space-separator branch from 6a4ea25 to f7586e9 Compare October 28, 2026 05:34
@github-actions github-actions Bot added size/m M: 50-249 lines changed and removed size/s S: 10-49 lines changed labels Oct 28, 2026
@mananmangal

Copy link
Copy Markdown
Contributor Author

/gemini review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📋 Review Summary

This PR updates IcebergUtils to support space-separated ISO-8601/RFC-3339 timestamp strings when writing STRING fields to Iceberg timestamp and timestamptz columns. The previous review feedback regarding strict date resolution (ResolverStyle.STRICT and IsoChronology.INSTANCE) has been properly addressed, and unit tests cover both valid and invalid date/separator cases.

🔍 General Feedback

  • The up-front hasSpaceSeparator check avoids exception-handling overhead on the hot path while preserving strict parsing semantics matching DateTimeFormatter.ISO_LOCAL_DATE_TIME and DateTimeFormatter.ISO_OFFSET_DATE_TIME.
  • Test coverage in IcebergUtilsTest thoroughly validates space-separated timestamps, invalid separators, and invalid calendar dates across both timestamp and timestamptz types.

💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.

@github-actions

Copy link
Copy Markdown
Contributor

Assigning reviewers:

R: @ahmedabu98 for label iceberg.
Backup reviewer: chamikaramj (if the main reviewer is unresponsive, you can escalate to them by tagging their username)

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

@codecov

codecov Bot commented Oct 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 0.00%. Comparing base (2213e54) to head (f7586e9).
⚠️ Report is 45 commits behind head on master.

Additional details and impacted files
@@       Coverage Diff       @@
##   master   #40861   +/-   ##
===============================
===============================

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

iceberg Iceberg IO connector io java Next Action: Reviewers size/m M: 50-249 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: IcebergIO rejects timestamp strings with a space between date and time

1 participant