Skip to content

iterator gap fix#11

Open
kylmcgr wants to merge 3 commits into
devfrom
10-nwbslicer-should-deal-with-timestamp-gaps
Open

iterator gap fix#11
kylmcgr wants to merge 3 commits into
devfrom
10-nwbslicer-should-deal-with-timestamp-gaps

Conversation

@kylmcgr

@kylmcgr kylmcgr commented Jun 9, 2026

Copy link
Copy Markdown
  • NWBAxisArrayIterator now splits chunks that contain a gap.
  • NWBClockDrivenProducer now returns a CoordinateAxis with the true per-sample timestamps instead of a LinearAxis.

@kylmcgr kylmcgr requested review from cboulay and Copilot June 9, 2026 21:36
@kylmcgr kylmcgr self-assigned this Jun 9, 2026
@kylmcgr kylmcgr linked an issue Jun 9, 2026 that may be closed by this pull request

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves correctness when NWB streams have explicit timestamps that include gaps by preventing time axes from implicitly “flattening” over missing time. It updates both chunked iteration and clock-driven playback to preserve true per-sample time semantics across gaps.

Changes:

  • Add gap detection (find_gaps) and use it to split iterator chunks that span timestamp gaps.
  • Update NWBSlicer.read_by_time (and NWBClockDrivenProducer) to emit a CoordinateAxis with true per-sample timestamps when a requested window spans a gap.
  • Add comprehensive tests + new gappy/irregular NWB fixtures to validate behavior, including the gap_tol control.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/ezmsg/nwb/slicer.py Adds gap detection helper and switches read_by_time to CoordinateAxis when windows span gaps.
src/ezmsg/nwb/iterator.py Splits chunk messages at detected timestamp gaps; plumbs gap_tol through iterator settings and prefetch worker.
src/ezmsg/nwb/clockdriven.py Forwards gap_tol into slicer time-window reads so clock-driven playback preserves true timestamps across gaps.
tests/conftest.py Adds session-scoped fixtures that generate synthetic gappy and irregular timestamped NWB files for testing.
tests/test_slicer.py Adds slicer-level tests covering gap-free windows, gap-spanning windows, inside-gap empty windows, and irregular streams.
tests/test_iterator.py Adds iterator-level tests ensuring chunks are split at gaps without dropping/reordering samples; tests gap_tol behavior.
tests/test_clockdriven.py Adds clock-driven tests verifying empty chunks inside gaps, CoordinateAxis for gap-spanning windows, and multi-producer sync across gaps.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/ezmsg/nwb/slicer.py
Comment on lines 485 to +489
out_data = info.dset[start_idx:stop_idx]
time_axis = template.axes["time"]
ts_window = np.asarray(ts_arr[start_idx:stop_idx])
has_gain = hasattr(time_axis, "gain")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NWBSlicer should deal with timestamp gaps

2 participants