OpenEphysBinaryRawIO: detect timestamp gaps and add gap_tolerance_ms#1878
Open
h-mayorquin wants to merge 2 commits into
Open
OpenEphysBinaryRawIO: detect timestamp gaps and add gap_tolerance_ms#1878h-mayorquin wants to merge 2 commits into
h-mayorquin wants to merge 2 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
OpenEphysBinaryRawIOpreviously had no gap detection (there was a literal# TODO for later : gap checking), so a within-recording discontinuity, for example from a dropped USB packet, was silently treated as contiguous and every sample after the gap got the wrong time. This adds gap detection using the per-sample sample indices the format already stores insample_numbers.npy(v0.6+) ortimestamps.npy(pre-v0.6): any non-unit increment is a gap. Detection is exact, since these are integer sample counters with no jitter.The interface matches the cross-reader gap API from #1773 that is already in Blackrock and Neuralynx.
gap_tolerance_ms=None(the default) raises aValueErrorwith a per-gap report, so the reader never hands back a silently mis-timed recording; a float segments the recording at gaps larger than the threshold and absorbs smaller ones (0.0segments on every gap); andignore_integrity_checks=Truebypasses the check for fast loads or corrupt-file recovery. It also adds_get_openephysbinary_timestampsfor direct access to the raw per-sample indices, matching_get_blackrock_timestampsand_get_neuralynx_timestamps.One intended behavior change: loading a recording that contains a gap now errors by default instead of returning a single silently-concatenated segment. Clean recordings are unaffected; code that was unknowingly reading a gapped recording will need to pass
gap_tolerance_msorignore_integrity_checks=True. The tests synthesize gaps from a real clean fixture (openephysbinary/v0.6.x_neuropixels_with_sync), since no public OpenEphys recording with a real gap exists.@alejoe91