Skip to content

Split the SD audit gap histogram by what caused each gap - #137

Merged
marknolan merged 1 commit into
mainfrom
chore/sd-audit-gap-attribution
Sep 18, 2026
Merged

marknolan merged 1 commit into
mainfrom
chore/sd-audit-gap-attribution

Conversation

@marknolan

Copy link
Copy Markdown
Member

The audit tool already attributed each gap to a cause, block boundary or
mid-block, but then pooled both into one size histogram. That threw away the
part worth having: the two mechanisms produce differently shaped distributions,
and the shape is the evidence for which one is at work.

This morning's soak is the case in point. The two populations look nothing
alike:

  gap profile by cause:
                        gaps  samples    mean  median    max   per min
      block boundary      50      445     8.9     5.0     35      1.17
      mid-block          176      274     1.6     1.0      4      4.12

The mid-block ceiling of 4 samples, across 176 gaps with not one above it, is
the old sampling fail-safe giving up after a fixed three sample periods. In the
firmware built from current main that ceiling is gone and the tail runs to 38,
which is the same fail-safe having become a wall-clock timeout of about 100 ms.
That comparison answered a question I had otherwise expected to need a debugger
and another soak for. Neither number is visible in a combined histogram.

Splitting also surfaced something I had not noticed at all: every
block-boundary gap is a multiple of 5 samples
- 5, 10, 15, 25, 30, 35 - and
the same holds in an unrelated recording from a different device and card. At
504 Hz five samples is about one SD write, which is what a single-buffer design
predicts: everything arriving during a write is lost, so the loss quantises in
units of one write. It was invisible while the two populations were mixed.

Each population now gets a summary row and its own histogram. The summary
carries the maximum explicitly, because that is where a ceiling shows up and
the histogram below is truncated at twelve distinct sizes.

Checked against four files: the two soak recordings, the recording the original
fault was traced from, and a Shimmer3R file that the tool correctly refuses.
Covers the empty-population case (the fixed build has no boundary gaps at all)
and the truncation path.

🤖 Generated with Claude Code

The pooled histogram hid the most useful thing in the data. Block-boundary
gaps and mid-block gaps come from different mechanisms, and their size
distributions are the evidence for which mechanism is at work, but combined
into one list neither shape is visible.

It showed immediately on this morning's soak. The two populations look nothing
alike: boundary gaps averaged 8.9 samples with a maximum of 35, mid-block gaps
averaged 1.6 with a hard ceiling of 4 across 176 of them. That ceiling is the
old sampling fail-safe giving up after a fixed three sample periods, and its
absence in a later firmware is the fail-safe having become a wall-clock
timeout. Reading that off a combined histogram was not possible; side by side
it takes a second.

Each population now gets a summary row and its own histogram. The summary
carries the maximum explicitly, because that is the number the ceiling shows up
in and the histogram below it is truncated at twelve sizes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

Tip: disable this comment in your organization's Code Review settings.

@marknolan
marknolan merged commit 25272f1 into main Sep 18, 2026
4 checks passed
@marknolan
marknolan deleted the chore/sd-audit-gap-attribution branch September 18, 2026 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant