Specify the host timestamp-unwrap rule, and give it conformance vectors

[marknolan]

Docs and test material only — no firmware source is touched, and nothing here
compiles into an image. The firmware does not unwrap its own counter; hosts do.

Why

Four host APIs unwrap this counter — the Java driver behind Consensys, the C# API,
pyshimmer and the TypeScript web SDK — and all four got it wrong the same way: a record
the firmware never stamped, read as a roll-over, put every later sample 512 s late.

Fixing that in four places showed the spec was part of the problem. §2.1 recommended
comparing against half the modulo and cross-checking long gaps against host elapsed
time, and then said in the same breath that at 2^16 "the host cross-check is not
optional". An SD file has no host clock. The recommendation could not be followed by
the case that needed it most, and each implementation improvised something different.

What the rule is now

Classify each sample by its modular forward distance from the last one, with forward
motion as the default:

  if forward == 0:            hold                            // duplicate
  elif backwards <= W:        lastUnwrapped - backwards       // 1. reordered packet
  elif modulo == 2^24 and raw == 0 and lastRaw < modulo - 32768:
                              reject, state untouched         // 2. never stamped
  else:                       lastUnwrapped + forward         // 3. forward; a wrap iff raw < lastRaw

  W = 0 when the rate is unknown; else min(8 * 32768 / rateHz, modulo / 8)

Four things the prose now calls out, each of which a real implementation got wrong:

Compare modular distances, not unwrapped values	A packet arriving late from before a boundary looks like forward motion of nearly a modulo, so it is accepted and the next real sample is read as a second wrap. 16777206, 5, 16777206, 70 is the smallest case
Forward motion is the default	A roll-over preceded by a long dropout is still a roll-over. Rules that treat an unexplained backward step as corruption unless it clears a large threshold lose the wrap
Size the window in sample periods	modulo / 8 reads every 1.75–2.0 s dropout on the 2-byte counter as a reorder and silently loses the wrap — an ordinary Bluetooth gap. Eight periods shrinks that band to ~16 ms
An unknown rate means no window	32768 / 0 is infinity in most languages, which classifies every backward step as a reorder and loses every wrap — a silent return to worse-than-naive behaviour

The tick domain is called out explicitly as the 32768 Hz real-time clock the packet
counter runs on, never a TCXO sampling clock (312500 / 255765.625 Hz on the boards that
have one) — that one is easy to get wrong and silently widens the window ~9.5×.

Two limits are stated rather than hidden: a packet more than eight sample periods late
is indistinguishable from a roll-over, and a gap longer than a whole modulo cannot be
recovered from the counter at all.

The vectors

Test/conformance/timestamp_unwrap.json — 26 vectors and 11 window derivations,
covering every case named above. Three of them are the ones that separate this rule from
the ones it replaces:

vector	what it catches
wrap-spanning-dropout-1p8s-16bit	fails under a modulo / 8 window (placed 1.8 s early)
reorder-across-wrap-boundary-24bit	fails when the comparison is on unwrapped values (→ a second wrap)
wrap-after-heavy-loss-24bit / -16bit	fails when "corrupt" rather than "forward" is the default

Test/host/crosscheck_timestamp_unwrap.py is the reference implementation. It generates
the file, and make timestamp-unwrap re-checks that every expectation in it still
reproduces — so the prose, the vectors and the implementations that consume them
cannot drift apart at the source. A hand-edited expectation, a vector added without
regenerating, or a rule change the file was not updated for all fail it. Verified by
tampering with one expectation: exit 1, with the differing field named.

It also has --emit csharp, because one of the consuming test projects has no way to
load a data file and has to transcribe.

Where it sits

Rebased onto the new Test/host suite, so it is a cross-check in that Makefile rather
than a CI job of its own — the workflow stays driven entirely by the Makefile, as its
header asks. Like crosscheck_host_constants.py it needs no compiler and no binary;
unlike every other check there, its subject is outside this repository, because the
firmware does not unwrap its own counter.

The vectors go in Test/conformance/ rather than docs/: docs/ is prose-only and
docs-in-step.yml matches ^docs/.+\.md$, so a JSON there would be both clutter and
invisible to that check. One file per topic, so crc.json and friends have somewhere
obvious to go later.

Verification

make -C Test/host timestamp-unwrap

→ 26 vectors, 11 derivation cases, revision 1 - all expectations reproduce.

Every expectation in the file was computed by the reference implementation, not typed;
the nine most load-bearing were independently hand-derived first and matched.

🤖 Generated with Claude Code

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

_{Tip: disable this comment in your organization's Code Review settings.}

[marknolan]

Added a 27th vector, and a fifth entry in the list of things §2.1 says are easy
to get wrong.

The rule here keeps lastRaw. Three of the five host APIs keep lastUnwrapped
and a cycle count instead — that is the state they already had — and derive
lastRaw back out of it, so they need some other way to say no sample yet.
(0, 0) is the obvious encoding, and it is wrong: a reorder that lands exactly
on the counter's origin produces lastUnwrapped = 0, cycle = 0 in the middle of
a stream. The next packet is then treated as the first one and passed through, so
a packet from just before the origin is placed a whole modulo late instead of
sixteen ticks behind.

Found by running the two formulations against each other rather than by reading
them:

	[520, 0, 16777200]
previous-raw form (this reference, web SDK, pyshimmer)	[520, 0, -16]
(0, 0) as the reset state (Java, C#, Swift)	[520, 0, 16777200]

A 512 second disagreement between host APIs that are meant to be identical, and
nothing in the vector set could see it. It can now: the three affected APIs each
have a fix in flight, and reverting any of them reproduces this as a test
failure rather than as an argument.

reorder-onto-origin-then-earlier-packet-24bit is also the first vector whose
final cycle is negative. That is a real state, not an error — the raw value is
derived back out of it on the next sample — and worth a host proving it handles.

Revision stays 1. This branch has not merged, so there is no published
revision 1 for the set to differ from, and bumping now would leave a revision
that never existed in main. Every consumer copy is updated in the same round;
the four that load the file are byte-identical by git blob
(de91de25accc7c74c0422f7e279a535da92579d9) and the two that transcribe it are
regenerated by --emit.