Do not read a reordered or invalid timestamp as a counter overflow

[marknolan]

The problem

Shimmer firmware stamps a packet when its sample tick starts it and does not write out a
packet it never stamped, so a timestamp field of exactly zero marks an invalid record
rather than the counter reaching its origin. LogAndStream v1.00.x–v1.01.003 could produce
one under SD write back-pressure.

unwrap() reads every backward step as an overflow, so one such record adds a whole
modulo — 512 seconds on the 3-byte counter — to every later sample:

>>> x = np.array([7406116, 7406506, 0, 7406571])   # from an affected recording
>>> r = unwrap(x.copy(), 2**24)
>>> r[-1] - r[0]
16777671        # 455 ticks + one modulo

Those four samples span 455 ticks, about 14 ms. A 9 minute 30 second trial containing four
such records was reported as 43 minutes 38.

An out-of-order or duplicated record costs exactly the same, which is the wider half of
the problem — nothing about the current rule distinguishes a reordered packet from an
overflow.

The change

Each sample is classified by its modular forward distance from the one before it:

a repeat of the previous value	holds the timeline
no further back than the reorder window	a reordered packet, placed where it was taken
3-byte counter, exactly zero, predecessor mid-range	invalid; flagged, not unwrapped
everything else	forward motion — an overflow when the raw value fell

Forward is the default, which is what keeps an overflow preceded by a long dropout
classified as an overflow.

reorder_window_ticks(period_ticks, modulo) sizes the window. Everything new is
keyword-optional, so existing callers and any HardwareRevision subclass keep working
unchanged; ShimmerReader passes the header divider down, since it is the tick count
between samples directly and the reader is the only thing that knows it.

Three things that are easy to get wrong

Compare modular distances, not unwrapped values. A packet arriving late from before
an overflow boundary looks like forward motion of nearly a whole modulo, so it is accepted
— and the next real sample is then read as a second overflow. Two modulos from one
out-of-order packet.

Size the window in sample periods, not as a fraction of the modulo. A reorder swaps
adjacent packets; a dropout spanning the overflow point is most of a modulo. At
modulo / 8 on the 2-byte counter, every dropout between 1.75 s and 2.0 s reads as a
reorder and the overflow is silently lost — and a 1.75 s gap is ordinary. Eight periods
shrinks that band to about 16 ms.

An unknown period means no window, not an infinite one. An infinite window reads every
backward step as a reorder and loses every overflow, which is worse than no reorder
detection at all.

Two behaviour notes

• unwrap() no longer modifies its input in place. It also casts to int64 first, so
  a 32-bit input array can no longer overflow after enough wraps.
• A negative difference has one modulo added rather than being reduced outright, so a
  value at or above the modulo — not something a real counter emits, but something a
  caller can pass, and something test_unwrap_device_timestamps does — stays plain
  forward motion.

find_invalid_timestamps() judges a zero against the last non-zero sample, so a run of
consecutive invalid records is measured from the last record that carried a timestamp
rather than from the zero before it.

Tests

83 pass, including the reader suite against the real binary fixtures — so the change is
exercised end to end on actual recordings, not only on synthetic arrays.

test/resources/timestamp_unwrap.json holds 26 conformance vectors and 11 window
derivations, covering every case above. They are copied byte-identically from the Shimmer
firmware repository, where the rule is specified and a reference implementation regenerates
and re-checks them in CI. The Java, C# and TypeScript Shimmer APIs run the same file —
four implementations of one wire format drifted apart once already, the same defect in all
four, so if they disagree now one of them is wrong.

Five mutations were checked rather than assumed, each caught: ignoring the reorder window,
sizing it as modulo / 8, letting an unknown period become infinite, judging a zero
against the element before it rather than the last non-zero, and dropping the period on the
reader call. The last of those found a genuine gap — nothing had covered the reader
actually threading the period through — and a test was added for it.

black clean.

Happy to adjust naming, placement, or the vector file location to suit the project.

[lumagi]

Wow, thank you for the PR. It will probably take a bit of time to review this, but I will try to get on it asap.

[marknolan]

@lumagi we're in the middle of reviewing the same update to our other APIs (e.g., below) so if it might be best to hold off on reviewing until they are in and I will update this PR should anything change.:

• Shimmer-Java-Android-API
• Shimmer-C-API PR
• SwiftAPI