Skip to content

fix(core): sort an Ion's out-of-order rows as its siblings do - #535

Merged
blalterman merged 2 commits into
masterfrom
fix/ion-time-order
Nov 5, 2026
Merged

blalterman merged 2 commits into
masterfrom
fix/ion-time-order

Conversation

@blalterman

Copy link
Copy Markdown
Owner

Closes #507.

An Ion built directly from rows out of time order kept them unsorted and logged only that the index was "not monotonically increasing". Plasma, Spacecraft, Vector and Tensor all sort such rows, with one warning that counts and locates them. Ion.set_data now does the same, through the _put_in_time_order_unless_checked helper Vector and Tensor use. An Ion built by a Plasma inherits already-checked times and is unchanged.

After this, Base._verify_datetimeindex's order warning is reached only by an index whose values cannot be compared (such as [1, "a", 2.0]), which has no order and is kept. Its docstring, and _put_in_time_order_unless_checked's, now say so.

This is the last live follow-up in #507. The double warning it describes no longer occurs on master: the order decision already lives in one place, Core._time_disorder. Its other item, the untested SIDCLoader.download_data, is untested by design (network). The accepted mutation survivors it lists stay recorded in the issue.

Tests (2122452a, then 8e9eae37)

  • test_ion_sorts_out_of_order_rows_with_one_warning: rows given in time order 2, 0, 1 come back sorted, values attached to their times, with one warning reading "1 of 3 rows are earlier than the row before them".
  • The out-of-order case of test_ion_warns_about_exactly_the_index_problems_present now expects neither of the old warnings.
  • Both landed as xfail(strict=True) and xfailed; with the fix and markers in place both were XPASS(strict); markers removed in the fix commit.
  • Full suite 6649 passed, core doctests 16 passed, ruff, import-linter (2 kept, 0 broken).

🤖 Generated with Claude Code

blalterman and others added 2 commits November 5, 2026 16:05
Target tests, xfail(strict=True) until Ion sorts its data: Plasma,
Spacecraft, Vector and Tensor sort rows out of time order with one warning
that counts and locates them, while an Ion built directly keeps them and
logs the vague 'not monotonically increasing' warning. Refs #507.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ion.set_data now puts its data in time order through
_put_in_time_order_unless_checked, as Vector and Tensor do and as Plasma
and Spacecraft do through _put_in_time_order: one warning counts and
locates the rows out of order, and the rows are sorted. Before, an Ion
built directly kept them and Base's _verify_datetimeindex logged only
'not monotonically increasing'. That warning is now reached only by an
index whose values cannot be compared. An Ion built by a Plasma inherits
checked times and is unchanged.

Before: both target tests xfailed. With the fix and markers in place: both
XPASS(strict). Markers removed: full suite 6649 passed.

Closes #507.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@claude

claude Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Claude finished @blalterman's task in 25s —— View job


Review of PR #535

I reviewed the diff only. I did not run the tests or read the helper's source, so the points about behaviour I haven't checked are marked as such.

Summary: This is a small, focused change. Ion.set_data now calls _put_in_time_order_unless_checked before super().set_data(data). The local data is rebound, so the later data.columns.names check and the rest of the method use the sorted frame. That matches how Vector and Tensor work, and it closes the last live item in #507.

What looks good

  • Test-first process. The commit history follows CLAUDE.md: the xfail(strict) tests land first, then the fix that removes the markers.
  • New test. test_ion_sorts_out_of_order_rows_with_one_warning checks the sorted index, that each value stays attached to its time, and the exact warning text. It builds on the real _ion_data helper and has no pinned numbers without a source.
  • Updated parametrized case. decreasing-datetimes now expects set(). The docstring explains why, so the case still guards against the old "not monotonically increasing" message.
  • Docs. The Base._verify_datetimeindex and _put_in_time_order_unless_checked docstrings, the Ion.set_data docstring (which now lists the NaT ValueError), and the CHANGELOG entry are consistent with each other and with the behaviour.

Minor suggestions (non-blocking)

  1. NaT behaviour is new for Ion but untested. Ion.set_data now raises ValueError for missing timestamps, and the docstring says so. A small pytest.raises(ValueError) test would pin it. Vector and Tensor may already have an equivalent test that you could copy. By the repo's "second copy" rule, a parametrized test over Ion, Vector and Tensor may be better than a third copy.
  2. The "already-checked" path has no test. The PR description says an Ion built by a Plasma is unchanged, and the full suite probably covers that indirectly. A test asserting that no extra warning is logged when a Plasma builds an Ion from unsorted input would show that the helper's skip works, and not only that the warning count is one.
  3. The sorted index could be pinned more tightly. The test uses ion.data.index.equals(times.sort_values()). ion.data.index.is_monotonic_increasing would pin the intent without repeating the expected answer. This is a taste point.
  4. Caller-visible side effect, assuming the helper returns a new frame. If _put_in_time_order returns a new sorted frame, the caller's input isn't mutated, which is the right behaviour. Confirm that the sort doesn't add a deep copy on the already-sorted path. CLAUDE.md asks that load and compute paths avoid .copy(deep=True).

Risks

  • Behaviour change. Anyone relying on an Ion keeping its rows unsorted will now see them reordered. The CHANGELOG notes this, and it's the intended fix.
  • No performance or security concerns. The sort is skipped when the times are already checked, and otherwise costs one monotonicity check plus a sort on out-of-order input.

Overall this looks good to merge once you've considered suggestion 1.

@blalterman
blalterman merged commit 1edff2e into master Nov 5, 2026
12 checks passed
@blalterman
blalterman deleted the fix/ion-time-order branch November 5, 2026 21:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remaining mutation-test survivors after the test-quality fix program

1 participant