Skip to content

Refuse a torn export.pdb instead of converting it (G1d) - #4

Merged
jrgutier merged 1 commit into
mainfrom
fix/pdb-torn-file-gate
Jul 31, 2026
Merged

Refuse a torn export.pdb instead of converting it (G1d)#4
jrgutier merged 1 commit into
mainfrom
fix/pdb-torn-file-gate

Conversation

@jrgutier

Copy link
Copy Markdown
Owner

Implements W1b from .omc/plans/playlist-phantom-entry-root-cause.md — and this is the actual root-cause fix for the phantom playlist entry, now that forensics on the real drive have identified it.

What happened

A conversion published two playlist entries the settled export.pdb does not contain — track 444 spliced second-to-last into Organic House - Top 100, track 2984 mid-chain into walker-royce. convert exited 0; only a later verify caught it.

Evidence recovered from the drive:

Event UTC
rekordbox last wrote export.pdb 12:09:38
convert started 12:09:52 — 14 s later
convert finished ~12:12:21
verify ran 12:12:24

export.pdb has not been modified since 12:09:38, so convert and verify read the same file. That rules out the stale-source explanation previously assumed. Parsed today, the unchanged file yields exactly verify's expected lists, with 444 and 2984 in no playlist at all — deterministically.

So the read at 12:09:52 saw a page image that no longer exists, on removable media where mtime reflects a metadata flush rather than the final byte landing. A stale slot read as present parses cleanly, lands in a coherent chain, and downstream nothing can distinguish it from a real entry.

The fix

The page header already carried the contradiction. Each page declares num_rows alongside num_row_offsets — and the walker decoded that field and threw it away. Now:

  • present-bit count must equal the declared num_rows
  • every row offset must point into the heap between the page header and the backward-growing row index (previously bounded only by page size, so stale index bytes could parse as a row)

Validated against real hardware before enforcing

Check Result
Data pages where popcount == num_rows 997 / 997
Present row bodies outside the heap 0
Tombstoned slots still parsing as deletions 104
Real file after the gate identical — 3,673 tracks, 45 playlists, 3,872 entries

Scope — stated plainly

This catches the demonstrated signature class, not torn reads in general. An image whose header was written before the pages it describes satisfies both checks and still passes. That residue is why the plan also carries an independent pre-publish oracle (W1d).

Note this also means W1's source fingerprint alone would not have caught this: convert would have hashed the torn bytes and verify the settled ones, reporting "stale source, re-run convert" — laundering a real fault into benign user activity.

Verification

699 tests (3 new), 88% branch coverage, ruff + mypy clean. Both gate tests confirmed red first.

🤖 Generated with Claude Code

https://claude.ai/code/session_01V3aF4y4GdsuJta2bfYUZ8w

A conversion published two playlist entries that the settled export.pdb does not
contain — tracks 444 and 2984, spliced into "Organic House - Top 100" and
"walker-royce". convert exited 0; only a later verify found them.

What the drive shows
--------------------
rekordbox last wrote export.pdb at 12:09:38 UTC on 29 Jul. convert started 14
seconds later at 12:09:52 and finished 12:12:21; verify ran at 12:12:24. The pdb
has not been modified since 12:09:38, so convert and verify read the same file
and the source did not change between them — which rules out the stale-source
explanation this was previously attributed to. Parsed today, that unchanged file
yields exactly what verify expected: 65 entries in "Organic House - Top 100",
and tracks 444 and 2984 in no playlist at all. The parse is deterministic. So
the read at 12:09:52 saw a page image that no longer exists, 14 seconds after
rekordbox's last recorded write to removable media where mtime reflects a
metadata flush rather than the final byte landing.

A stale slot read as present is the mechanism: it parses cleanly, lands in a
coherent chain, and downstream nothing can tell it from a real entry.

The header already knew
-----------------------
Each page declares num_rows alongside num_row_offsets, and the walker decoded
that field and discarded it. It is now enforced against the count of present
bits, so a page offering more rows than it claims to hold is refused rather
than converted. Row offsets are also bounded to the heap between the page
header and the backward-growing index, instead of merely to the page.

Both invariants were validated against a real 3,673-track export before being
enforced: 997/997 data pages agree, every present row body sits inside the heap,
and the 104 tombstoned slots in that file still parse as the deletions they are.
The same file parses byte-identically with the gate active — 3,673 tracks, 45
playlists, 3,872 entries.

Coverage is honest, not total: a torn image whose header was written before the
pages it describes satisfies both checks and still passes. That residue is why
the plan also carries an independent pre-publish oracle.

699 tests (3 new), 88% branch coverage, ruff + mypy clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3aF4y4GdsuJta2bfYUZ8w
@jrgutier
jrgutier merged commit 8cfb9d3 into main Jul 31, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant