Split the SD audit gap histogram by what caused each gap - #137
Merged
Merged
Conversation
The pooled histogram hid the most useful thing in the data. Block-boundary gaps and mid-block gaps come from different mechanisms, and their size distributions are the evidence for which mechanism is at work, but combined into one list neither shape is visible. It showed immediately on this morning's soak. The two populations look nothing alike: boundary gaps averaged 8.9 samples with a maximum of 35, mid-block gaps averaged 1.6 with a hard ceiling of 4 across 176 of them. That ceiling is the old sampling fail-safe giving up after a fixed three sample periods, and its absence in a later firmware is the fail-safe having become a wall-clock timeout. Reading that off a combined histogram was not possible; side by side it takes a second. Each population now gets a summary row and its own histogram. The summary carries the maximum explicitly, because that is the number the ceiling shows up in and the histogram below it is truncated at twelve sizes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.
Tip: disable this comment in your organization's Code Review settings.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The audit tool already attributed each gap to a cause, block boundary or
mid-block, but then pooled both into one size histogram. That threw away the
part worth having: the two mechanisms produce differently shaped distributions,
and the shape is the evidence for which one is at work.
This morning's soak is the case in point. The two populations look nothing
alike:
The mid-block ceiling of 4 samples, across 176 gaps with not one above it, is
the old sampling fail-safe giving up after a fixed three sample periods. In the
firmware built from current main that ceiling is gone and the tail runs to 38,
which is the same fail-safe having become a wall-clock timeout of about 100 ms.
That comparison answered a question I had otherwise expected to need a debugger
and another soak for. Neither number is visible in a combined histogram.
Splitting also surfaced something I had not noticed at all: every
block-boundary gap is a multiple of 5 samples - 5, 10, 15, 25, 30, 35 - and
the same holds in an unrelated recording from a different device and card. At
504 Hz five samples is about one SD write, which is what a single-buffer design
predicts: everything arriving during a write is lost, so the loss quantises in
units of one write. It was invisible while the two populations were mixed.
Each population now gets a summary row and its own histogram. The summary
carries the maximum explicitly, because that is where a ceiling shows up and
the histogram below is truncated at twelve distinct sizes.
Checked against four files: the two soak recordings, the recording the original
fault was traced from, and a Shimmer3R file that the tool correctly refuses.
Covers the empty-population case (the fixed build has no boundary gaps at all)
and the truncation path.
🤖 Generated with Claude Code