DEV-1023: size the timestamp reorder window from the sampling rate, and run the shared vectors

[marknolan]

Follow-on to #147, bringing this SDK onto the merged DEV-1023 rule. The spec and the vectors are
ShimmerResearch/log-and-stream-common#136.

What was still wrong after #147

#147 stopped an unstamped 0x000000 record being read as a roll-over. It left
the wider half of the problem in place: a reordered or duplicated packet costs
exactly the same 512 s, and the threshold that was supposed to tell the two
apart was sized as a fraction of the counter's range.

Those two quantities are unrelated. A reorder swaps packets that are adjacent in
time — a handful of sample periods. A dropout that happens to span the wrap point
is most of a modulo. Sizing the window by the modulo puts the boundary between
them in the middle of ordinary dropout territory:

counter	old window	misread as a reorder	typical?
16-bit (pre-LogAndStream 0.5.4)	0.25 s	every dropout of 1.75–2.0 s	a bad afternoon on a BT link
24-bit	64 s	every dropout of 448–512 s	rare

A misread there loses the roll-over for the rest of the session.

The rule now

forward   = (raw − lastRaw) mod modulo
backwards = modulo − forward
forward == 0                     → hold (a duplicate)
backwards ≤ W                    → placed where it was taken (reordered)
24-bit, raw == 0, lastRaw mid-range → invalid; flagged, timeline untouched
otherwise                        → forward motion, a wrap when the raw value fell
W = min(8 × 32768 / rateHz, modulo / 8)

Three things in there are easy to get wrong, and each has a vector:

Forward is the default. A roll-over preceded by a long dropout is still a
roll-over, however much was lost. A rule that defaults the other way — "a
backward step is corrupt unless it clears some threshold" — fails exactly there.

Decide on the modular distance, not on unwrapped values. A packet arriving
late from just before a wrap boundary has a candidate above its predecessor,
so a comparison accepts it as forward motion of nearly a whole modulo and then
reads the next real sample as a second wrap. [2²⁴ − 10, 5, 2²⁴ − 10, 70] lands
at 33554502 — two modulos out, from one out-of-order packet.

An unknown rate is not an infinite window. 8 × 32768 / 0 is Infinity in
several of these languages, and an infinite window reads every backward step as
a reorder and loses every roll-over — the original bug, restored, with no symptom
until a recording comes out short.

API

• reorderWindowTicks(hz, modulo), exported and pure.
• StreamTimelineOptions gains samplingRateHz and reorderWindowTicks;
  setSamplingRateHz(hz | null) and setReorderWindowTicks(ticks | null).
• TimelineState gains reorderWindowTicks, so a host can see that its rate
  reached the timeline.

Both clients pass the rate at _prepareStreamTimeline(). That is one line each,
and nothing else would notice it going missing — so there is a test per client
asserting the window is 5120 ticks and not the 2097152-tick fallback.

Nothing existing changes behaviour. A timeline nobody has told the rate to
still uses an eighth of the modulo. The other four APIs disable the branch
outright when the rate is unknown; they import files, where there is no host
clock to appeal to, whereas this SDK has the missed-wrap recovery as a second
witness. Follow-up: drop the fallback once every client always knows its rate.

One thing the fallback does not get. A reorder is judged before the
invalid-zero test, which is right for a window of a few sample periods — a zero
that close to an origin is genuinely ambiguous and choosing wrong costs 16 ms.
At 64 s wide the same order would read an unstamped record as a packet 64 s
late, place it 64 s early and mark it valid — worse than either answer the
rule is choosing between. So the fallback window classifies reorders but never
overrules the zero test.

Tests

tests/fixtures/timestamp_unwrap.json — 27 vectors and 11 window derivations,
copied byte-identically from log-and-stream-common, where the rule is
specified and a reference implementation regenerates and re-checks it in CI. Git
blob de91de25accc7c74c0422f7e279a535da92579d9 in both repositories. The Java, C#,
Python and Swift APIs run the same file — five implementations of one wire format
drifted apart once already, with the same defect in all five, and reviewing them
against each other by hand is what let that happen.

1944 tests pass, typecheck, lint and prettier clean.

Nine mutations checked rather than assumed, each caught:

mutation	caught by
size the window as modulo / 8, ignoring the rate	7 tests
let an unusable rate become an infinite window	3
drop the clamp against the modulo	3
test the zero before the reorder	3
let the fallback window overrule the zero test	2
widen the reorder branch to any backward step	14
stop recomputing the window when the width changes	1
Shimmer3: stop telling the timeline the rate	1
Shimmer3R: stop telling the timeline the rate	1

The first harness I wrote reported nothing caught, because --reporter=basic
is not a vitest 5 reporter and every run was erroring out before it reached a
test. Worth saying out loud: a mutation run that catches nothing is a broken
harness until proven otherwise.

Since the first push — two review findings, both fixed here

An explicit reorder window was not clamped. reorderWindowTicks() caps at
modulo / 8 so a backward step large enough to be a roll-over always exists;
setReorderWindowTicks() and the constructor option did not, so a caller could
express the one thing the clamp exists to prevent:

new StreamTimeline({ timestampBits: 16, reorderWindowTicks: 100000 })
stamp(65000); stamp(100)   →  unwrapped = 100, wraps = 0

A plain 2-byte roll-over, silently lost. Clamped in _recomputeReorderWindow so
it follows a later setTimestampBits rather than only the moment the setter ran.
Nothing passes an override today except the vectors, all at or below the cap, so
no existing behaviour changes.

"No sample yet" was not distinguishable from a sample — in three of the five
APIs. Java, C# and Swift keep (lastUnwrapped, cycle) and encoded the reset
state as (0, 0); a reorder landing exactly on the counter's origin reaches that
state mid stream. This timeline keeps lastRaw with a null sentinel and was
already right, but the divergence was real and unwatched:

	[520, 0, 2²⁴ − 16]
web SDK, pyshimmer	[520, 0, -16]
Java, C#, Swift (before)	[520, 0, 16777200]

512 seconds apart, and no vector covered it. It is now
reorder-onto-origin-then-earlier-packet-24bit in the shared file, the other
three are fixed, and reverting either fix reproduces it as a test failure. It is
also the first vector whose final cycle is negative.

Also corrected a doc comment: setSamplingRateHz claimed a client calls it
whenever the rate changes. Both clients call it once per stream. Calling it mid
stream does work and is now covered.

Related

• Spec + vectors: Specify the host timestamp-unwrap rule, and give it conformance vectors log-and-stream-common#136
• Java driver: DEV-1023: one timestamp-unwrap rule — invalid zeros, reorders, duplicates and wraps Shimmer-Java-Android-API#300
• C#: DEV-1023/DEV-1024: one timestamp-unwrap rule, and a Shimmer3 SD file that parses Shimmer-C-API#203
• Python: Do not read a reordered or invalid timestamp as a counter overflow seemoo-lab/pyshimmer#44
• Swift: DEV-1023: unwrap the sample counter by the shared rule, and run the shared vectors SwiftAPI#17

Reorder and duplicate detection came out of @MAzalya's parallel fix for this
defect on the Java driver (DEV-1030). The polarity and the window sizing here
differ, but the observation that a backward step is not always a roll-over is
hers.

Release: patch bump via cut-release after merge; the vendor sync into
verisense-device-console and webBLEDemos is a release step, not part of this.

🤖 Generated with Claude Code

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

_{Tip: disable this comment in your organization's Code Review settings.}