Skip to content

Fix Timestamp.from_datetime() precision for far-future datetimes#710

Open
gaoflow wants to merge 1 commit into
msgpack:mainfrom
gaoflow:fix-from-datetime-far-future-precision
Open

Fix Timestamp.from_datetime() precision for far-future datetimes#710
gaoflow wants to merge 1 commit into
msgpack:mainfrom
gaoflow:fix-from-datetime-far-future-precision

Conversation

@gaoflow

@gaoflow gaoflow commented Jul 3, 2026

Copy link
Copy Markdown

Timestamp.from_datetime() computes the whole-second part from the float datetime.timestamp(). A float64 cannot hold microsecond precision for datetimes far from the epoch, so timestamp() rounds the seconds up while the exact microsecond is still used for the nanoseconds. The result is a Timestamp one second in the future, and near datetime.max it raises OverflowError:

>>> import datetime as dt
>>> from msgpack.ext import Timestamp
>>> d = dt.datetime(3000, 1, 1, 0, 0, 0, 999999, tzinfo=dt.timezone.utc)
>>> Timestamp.from_datetime(d).to_datetime()
datetime.datetime(3000, 1, 1, 0, 0, 1, 999999, tzinfo=datetime.timezone.utc)   # +1 second
>>> Timestamp.from_datetime(dt.datetime(9999, 12, 31, 23, 59, 59, 999999, tzinfo=dt.timezone.utc))
OverflowError: date value out of range

The Cython packer already uses integer timedelta arithmetic and is correct. This makes the pure-Python from_datetime() do the same, so the two paths agree and the round-trip invariant from_datetime(d).to_datetime() == d holds for far-future datetimes. Naive datetimes keep their existing local-time interpretation (matching datetime.timestamp()).

Follow-up to #662, which fixed the pre-epoch rounding direction in the same method.

Existing tests are unchanged; I added a regression test covering the far-future round-trip, the exact seconds/nanoseconds, agreement with packing the datetime directly, and the former OverflowError case. Full suite is green on both the C-extension and pure-Python paths.

from_datetime() derived the whole-second part from the float
datetime.timestamp(), which cannot hold microsecond precision far from
the epoch. It rounded the seconds up while still taking the exact
microsecond for the nanoseconds, so the result was one second in the
future (and raised OverflowError near datetime.max).

Use integer timedelta arithmetic, matching the Cython packer, so the
pure-Python path agrees with the reference implementation.
Comment thread msgpack/ext.py
Comment on lines 161 to 162
utc = datetime.timezone.utc
return datetime.datetime.fromtimestamp(0, utc) + datetime.timedelta(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please use _EPOCH here too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants