Refuse a torn export.pdb instead of converting it (G1d) - #4
Merged
Conversation
A conversion published two playlist entries that the settled export.pdb does not contain — tracks 444 and 2984, spliced into "Organic House - Top 100" and "walker-royce". convert exited 0; only a later verify found them. What the drive shows -------------------- rekordbox last wrote export.pdb at 12:09:38 UTC on 29 Jul. convert started 14 seconds later at 12:09:52 and finished 12:12:21; verify ran at 12:12:24. The pdb has not been modified since 12:09:38, so convert and verify read the same file and the source did not change between them — which rules out the stale-source explanation this was previously attributed to. Parsed today, that unchanged file yields exactly what verify expected: 65 entries in "Organic House - Top 100", and tracks 444 and 2984 in no playlist at all. The parse is deterministic. So the read at 12:09:52 saw a page image that no longer exists, 14 seconds after rekordbox's last recorded write to removable media where mtime reflects a metadata flush rather than the final byte landing. A stale slot read as present is the mechanism: it parses cleanly, lands in a coherent chain, and downstream nothing can tell it from a real entry. The header already knew ----------------------- Each page declares num_rows alongside num_row_offsets, and the walker decoded that field and discarded it. It is now enforced against the count of present bits, so a page offering more rows than it claims to hold is refused rather than converted. Row offsets are also bounded to the heap between the page header and the backward-growing index, instead of merely to the page. Both invariants were validated against a real 3,673-track export before being enforced: 997/997 data pages agree, every present row body sits inside the heap, and the 104 tombstoned slots in that file still parse as the deletions they are. The same file parses byte-identically with the gate active — 3,673 tracks, 45 playlists, 3,872 entries. Coverage is honest, not total: a torn image whose header was written before the pages it describes satisfies both checks and still passes. That residue is why the plan also carries an independent pre-publish oracle. 699 tests (3 new), 88% branch coverage, ruff + mypy clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V3aF4y4GdsuJta2bfYUZ8w
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements W1b from
.omc/plans/playlist-phantom-entry-root-cause.md— and this is the actual root-cause fix for the phantom playlist entry, now that forensics on the real drive have identified it.What happened
A conversion published two playlist entries the settled
export.pdbdoes not contain — track 444 spliced second-to-last intoOrganic House - Top 100, track 2984 mid-chain intowalker-royce.convertexited 0; only a laterverifycaught it.Evidence recovered from the drive:
export.pdbconvertstartedconvertfinishedverifyranexport.pdbhas not been modified since 12:09:38, so convert and verify read the same file. That rules out the stale-source explanation previously assumed. Parsed today, the unchanged file yields exactly verify's expected lists, with 444 and 2984 in no playlist at all — deterministically.So the read at 12:09:52 saw a page image that no longer exists, on removable media where mtime reflects a metadata flush rather than the final byte landing. A stale slot read as present parses cleanly, lands in a coherent chain, and downstream nothing can distinguish it from a real entry.
The fix
The page header already carried the contradiction. Each page declares
num_rowsalongsidenum_row_offsets— and the walker decoded that field and threw it away. Now:num_rowsValidated against real hardware before enforcing
num_rowsScope — stated plainly
This catches the demonstrated signature class, not torn reads in general. An image whose header was written before the pages it describes satisfies both checks and still passes. That residue is why the plan also carries an independent pre-publish oracle (W1d).
Note this also means W1's source fingerprint alone would not have caught this: convert would have hashed the torn bytes and verify the settled ones, reporting "stale source, re-run convert" — laundering a real fault into benign user activity.
Verification
699 tests (3 new), 88% branch coverage, ruff + mypy clean. Both gate tests confirmed red first.
🤖 Generated with Claude Code
https://claude.ai/code/session_01V3aF4y4GdsuJta2bfYUZ8w