From b0005be56f8a2f1aaac37bfad32432961daee88c Mon Sep 17 00:00:00 2001 From: jrgutier Date: Fri, 31 Jul 2026 22:50:12 -0500 Subject: [PATCH] Document what Engine DJ does to a converted library MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two findings from measuring a real 3,673-track stick, both previously unwritten down and both easy to mistake for defects. Opening Engine DJ can merge its desktop library onto the stick -------------------------------------------------------------- A convert followed by a clean verify, then Engine DJ opened, then verify again: playlist_count expected=45 actual=48, naming playlists in no rekordbox playlist. Those playlists are real and Engine added them on purpose. Controlled repeat with Engine's desktop database cleared, same conversion both times: 48 playlists with the desktop populated, 45 with it cleared, and in the cleared run not one row changed. So the extras were Engine syncing its own desktop library, not corruption and not something rb2engine wrote — their lastEditTime values predated the conversion, consistent with rows that already existed on the desktop side. Troubleshooting now covers this, including how to tell Engine's rows from ours: we pin lastEditTime to the epoch on every playlist we write, so anything with a real timestamp came from Engine. That discriminator survives Engine reassigning ids, which the id watermark does not. Engine populates currentPlayedIndiciator itself ----------------------------------------------- Diffing a pristine conversion against the same database after Engine had opened and closed it: Information.currentPlayedIndiciator 0 -> 1698144667125441751 was the ONLY change. No schema objects, no pragma changes, and Track, PerformanceData, AlbumArt, Playlist and PlaylistEntity all byte-identical. Recorded next to the pin so the next reader does not have to re-derive it, and so the question "should we write that field too?" has a documented answer: no. Engine sets it on first open, and writing an opaque constant would forfeit byte-identical rebuilds for nothing. Docs and comments only — no behaviour change. 704 tests, ruff + mypy clean. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01V3aF4y4GdsuJta2bfYUZ8w --- docs/TROUBLESHOOTING.md | 50 +++++++++++++++++++++++++++++++++++ src/rb2engine/writer/build.py | 9 +++++++ 2 files changed, 59 insertions(+) diff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.md index 551fd6c..1da5319 100644 --- a/docs/TROUBLESHOOTING.md +++ b/docs/TROUBLESHOOTING.md @@ -122,6 +122,56 @@ titles were adjusted. --- +## `verify` reports extra playlists after you opened Engine DJ + +**Symptom:** `convert` succeeds and `verify` is clean, then a later `verify` +reports something like: + +``` +track library: playlist_count: expected=45 actual=48 +``` + +and names playlists that exist in Engine but in no rekordbox playlist. + +**Cause:** opening Engine DJ with the drive attached can **merge Engine's own +desktop library onto the stick**. Those playlists are real and were added by +Engine on purpose — they are not corruption, and `rb2engine` did not write them. +`verify` is doing its job: it compares the database against the rekordbox +source, and after a desktop merge the database legitimately holds more than the +source describes. + +Confirmed by experiment on a 3,673-track library. Identical conversion both +times; the only variable was Engine's desktop database: + +| Engine desktop library | Playlists after opening Engine | +|---|---| +| Populated | 48 — three added, playlist ids running past our allocation | +| Cleared | 45 — none added, no row altered | + +**How to tell them apart from a real defect.** `rb2engine` pins `lastEditTime` +to `1970-01-01 00:00:00` on every playlist row it writes, so anything with a +real timestamp came from Engine: + +```bash +sqlite3 "/Volumes/MY_USB/Engine Library/Database2/m.db" \ + "SELECT id, title, lastEditTime FROM Playlist + WHERE lastEditTime <> '1970-01-01 00:00:00';" +``` + +Rows listed there were written by Engine, not by this tool. A genuine +conversion defect would carry the pinned epoch. + +**Fix:** nothing is broken. **Run `verify` immediately after `convert`, before +launching Engine DJ** — that is the only moment the database is guaranteed to +contain exactly what the conversion produced. If you want the stick to match +rekordbox exactly, re-run `convert`; it rebuilds the database from the source. + +Note that Engine also rewrites `m.db` harmlessly on open (the file's timestamp +and size change by a page or two) even when it merges nothing. A changed +timestamp alone does not mean your library was modified. + +--- + ## A track appears once when it was in a playlist twice **Symptom:** a track that was listed twice in one rekordbox playlist appears diff --git a/src/rb2engine/writer/build.py b/src/rb2engine/writer/build.py index 5f11d80..d0d9a39 100644 --- a/src/rb2engine/writer/build.py +++ b/src/rb2engine/writer/build.py @@ -43,6 +43,15 @@ # Determinism pin for columns that Engine DDL triggers stamp with strftime('%s') # (Track.lastEditTime on PerformanceData UPDATE) and for opaque Information # fields that create_m_db may mint randomly. Fixed epoch 0 — not wall-clock. +# +# Leaving currentPlayedIndiciator at 0 is deliberate and costs nothing: Engine +# DJ populates it itself the first time it opens the library. Measured on a +# 3,673-track stick by diffing a pristine conversion against the same database +# after Engine had opened and closed it — that single cell (0 → +# 1698144667125441751) was the *only* change Engine made. No schema objects, no +# pragma changes, and Track, PerformanceData, AlbumArt, Playlist and +# PlaylistEntity all byte-identical. Writing a value here would buy nothing and +# would forfeit byte-identical rebuilds. _DETERMINISTIC_EPOCH = 0 _DETERMINISTIC_PLAYED_INDICATOR = 0