From 8e95eba93867f3b8b418a6e1c8c386f40fe2e9d3 Mon Sep 17 00:00:00 2001 From: jrgutier Date: Fri, 31 Jul 2026 23:12:51 -0500 Subject: [PATCH] Record why the 13-bit row count is right, measured on a real export MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The reader takes a page's slot count from the 13-bit field packed at +24. crate-digger models the same bytes differently — an 8-bit count at +24 with a u16 "num_rows_large" at +34 taking over past 255 rows — and if that were right this reader would mis-parse every dense page, silently, with verify unable to notice because both sides share the reader. Fixtures cannot settle it: they are packed by the same assumption the parser makes. A real 3,673-track export can, and does. Across its 997 data pages, 14 carry 284 rows: word & 0x1FFF = 284 on those pages, and they parse correctly — the library converts to 3,872 playlist entries and verifies with 0 discrepancies. the byte at +24 reads 28, which is just 284 & 0xFF. Alone it under-reports. the u16 at +34 is not a row count at all: 283, 84, 14 and 20 across pages that all hold 284 rows, and 0x1FFF on 271 others. This repo names that field transaction_row_index, which fits — 283 is the last index of 284 rows. on the 983 pages holding <=255 rows the two readings agree, which is why the divergence stays invisible on small libraries. Comment only, no behaviour change. Written down so the next reader does not re-open the question from the ksy alone and "fix" a correct parser. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01V3aF4y4GdsuJta2bfYUZ8w --- src/rb2engine/reader/pdb.py | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/src/rb2engine/reader/pdb.py b/src/rb2engine/reader/pdb.py index 9862871..c05065d 100644 --- a/src/rb2engine/reader/pdb.py +++ b/src/rb2engine/reader/pdb.py @@ -200,7 +200,27 @@ def _parse_file_header(data: bytes) -> Any: def _decode_page_counts(page: bytes) -> tuple[int, int, int]: - """Return (num_row_offsets, num_rows, page_flags) from the packed u32 at +24.""" + """Return (num_row_offsets, num_rows, page_flags) from the packed u32 at +24. + + The 13-bit field is the whole slot count, and that is deliberate. crate-digger + models these bytes differently — an 8-bit ``num_rows_small`` at +24 with a + ``num_rows_large`` u16 at +34 taking over past 255 rows — which would mean this + reader mis-parses any dense page. Measured against a real 3,673-track export + (997 data pages, 14 of them carrying 284 rows): + + * ``word & 0x1FFF`` = 284 on those pages, and they parse correctly — the + library converts to 3,872 playlist entries and verifies with 0 discrepancies. + * the byte at +24 is 28, which is just ``284 & 0xFF``; alone it under-reports. + * the u16 at +34 is **not** a row count. It reads 283, 84, 14, 20 across pages + that all hold 284 rows, and 0x1FFF on 271 others. This repo calls that field + ``transaction_row_index``, which fits — 283 is the last index of 284 rows. + * on the 983 pages holding <=255 rows, the 13-bit field and the byte agree, + which is why the divergence is invisible on small libraries. + + Do not "fix" this to the crate-digger shape without re-running that measurement + on a real export; fixtures cannot settle it, since they are packed by the same + assumption the parser makes. + """ word = struct.unpack_from("> 13) & 0x7FF