Repository navigation
fix: keep timezone-aware datetimes aware and UTC-normalise naive coercion - #76
Conversation
…cion DatabendDateTime.result_processor unconditionally did value.replace(tzinfo=None), which has two problems: - It strips tzinfo even for timezone=True columns, so a DateTime(timezone=True) / TIMESTAMP WITH TIME ZONE column loses its awareness. - It drops tzinfo without converting to UTC first, so a tz-aware value carrying a non-UTC offset yields the wrong naive wall-clock. Only coerce to naive for timezone=False columns, and convert to UTC before dropping tzinfo so the wall-clock is correct regardless of the driver's offset; timezone=True columns are returned unchanged. None and string parsing are preserved. Adds a server-less unit test.
|
CI failure is unrelated to this PR — it's a Databend nightly server flake. This PR only changes DatabendDateTime.result_processor (tz handling) plus a server-less Both failures are in the compliance suite's SequenceTest, on a server-side error: The test creates a sequence, then nextval() reports it as unknown — the server can't find a Root cause: CI is pinned to databend:nightly. Last green main was 2026-02-28; these runs are on Suggest a re-run (may pass on retry) and/or pinning CI to a known-good Databend tag. Happy to file |
hantmac
left a comment
There was a problem hiding this comment.
Reviewed result processing and SQLAlchemy DateTime/TIMESTAMP dialect adaptation. No blocking findings. For datetime inputs, timezone=False now preserves the instant by normalizing to UTC before dropping tzinfo, while timezone=True retains the driver's aware value. None, naive datetime, and existing string parsing behavior are unchanged.
Validation on Python 3.11 with databend-driver 0.34.2:
- The added regression test passes on SQLAlchemy 1.4.54 and 2.0.55.
- Extra checks pass on both versions for DateTime and TIMESTAMP, timezone=True/False, six offsets (including half/quarter-hour offsets and date rollover), plus None/naive/string values.
- Combined with #82: 20 unit tests pass; the same two failures (test_view_names and test_columns) reproduce on unchanged main. No new unit-test failures.
CI caveat: the historical failed run's logs return HTTP 410, and GitHub refuses reruns older than a month, so I cannot independently confirm the author's sequence-flake diagnosis. Fresh sibling CI on the same unchanged workflow currently fails even earlier: MinIO download returns 410 and Databend fails to start, before pytest (https://github.com/databendlabs/databend-sqlalchemy/actions/runs/39038774957). No live-server datetime integration run was completed in this review. These CI/bootstrap issues need separate maintenance.
Non-blocking existing limitation: timezone=True string results still follow the old naive parser, and this PR intentionally does not change datetime bind serialization. The approval applies to the scoped datetime-object result-processing fix, not a claim of end-to-end timezone support for every input path. Approving and merging.
Problem
DatabendDateTime.result_processorcurrently coerces every non-string value with an unconditionalvalue.replace(tzinfo=None):The driver returns tz-aware (UTC) datetimes, so this does fix the naive-column case — but it has two side effects:
timezone=Truecolumns. ADateTime(timezone=True)/TIMESTAMP WITH TIME ZONEcolumn loses its tzinfo, even thoughself.timezoneis available (inherited fromsqltypes.DATETIME) to distinguish the two cases.replace(tzinfo=None)truncates the offset instead of converting it — yielding the wrong naive wall-clock (e.g. a+02:00value comes back two hours off).Fix
timezone=Falsecolumns still yield naive values (so the compliance suite's naive round-trips hold), but the value is normalised to UTC before tzinfo is dropped, so the wall-clock is correct regardless of the driver's offset.timezone=Truecolumns are returned unchanged, keeping their awareness.Noneand string inputs are handled exactly as before.Tests
tests/unit/test_databend_dialect.py— server-less, callsresult_processordirectly:timezone=Falsecolumn;+02:00) input is normalised to UTC, not merely truncated;Noneis preserved, strings still parse;timezone=Truecolumn keeps its tzinfo.