Skip to content

Migrate jooq to sqldelight - #15

Open
jsarabia wants to merge 5 commits into
mainfrom
migrate-jooq-to-sqldelight
Open

jsarabia wants to merge 5 commits into
mainfrom
migrate-jooq-to-sqldelight

Conversation

@jsarabia

Copy link
Copy Markdown
Contributor

No description provided.

jsarabia and others added 4 commits September 2, 2026 09:28
Replace the jOOQ + prebuilt otter-db backend with SQLDelight across :shared,
moving the query layer fully into KMP commonMain and cutting Android database
init roughly in half. jOOQ is retained (not deleted) as a switchable fallback,
the characterization oracle, and the init-benchmark baseline.
Backend
- 19 .sq tables reproducing CreateAppDb.sql (schema-parity test); every DAO
  method as typed .sq queries under the sqlite-3.18 dialect (Android-7 safe:
  hand-rolled upserts, no ON CONFLICT DO UPDATE).
- 15 clean DAO interfaces (…/database/dao) + SQLDelight impls (…/database/
  sqldelight) behind a jOOQ-free DaoProvider; INTEGER AS Int/Boolean adapters
  keep the entity types.
- SqlDelightDatabaseMigrator: raw-SQL port of the 14-step DatabaseMigrator,
  proven byte-identical to jOOQ's migrator by a differential test over
  hand-authored v10/v12 legacy fixtures (version + columns-by-name + row data).
- Repositories moved off IAppDatabase onto DaoProvider + ambient transactions;
  the 3 raw-jOOQ repos (Collection/Resource/ResourceContainer) rewritten onto
  typed queries — no org.jooq imports remain under repositories/.
Platform wiring
- Driver construction hoisted out of Koin into DatabaseDriverFactory +
  SqlDelightDatabaseProvider (+ platform Jdbc/Android factories); db filename
  centralized to DATABASE_FILE_NAME. DI carries a commented jOOQ↔SQLDelight switch.
- Desktop: JdbcSqliteDriver, open-or-migrate an existing tr.sqlite (no wipe),
  Mac App Store sandbox path-rewrite preserved.
- Android: AndroidSqliteDriver in "dumb" mode (installed_entity migrator owns
  versioning; framework SQLite, no bundled engine).
Proof
- Differential characterization suite: the same 104 tests pass against BOTH the
  jOOQ and SQLDelight backends (+ schema parity, migrator differential).
- Integration tests (real ULB import, exact row counts) green on SQLDelight.
- On-device API-24 / SQLite 3.9.2 instrumented tests pass, incl. the v12→14
  take-rebuild migration.
- Cold-init benchmark (API 24, n=10): jOOQ 314ms vs SQLDelight 154ms (~2×).
AndroidDirectoryProvider (bug fix): app data dir and user data dir were
pointing to the same path, so InitializeProjects would find the ULB
that InitializeUlb had just extracted and run ExistingSourceImporter on
it — a full 26 s updateSource + mergeMedia + mergeText pass that was
all redundant on a fresh install, and would repeat on every cold start
until the user created a translation project. Fixed: app/ and user/ are
now separate subdirectories under filesDir.

DaoProvider.withBulkLoad / SqlDelightAppDatabase: wraps the first-
install seed when content_entity is empty. Sets journal_mode=WAL (via
executeQuery — Android's driver rejects execute() on row-returning
statements), synchronous=NORMAL, and foreign_keys=OFF for the duration,
then re-enables FKs and asserts PRAGMA foreign_key_check. The content
index is intentionally left in place: the importer's linkVerseResources
/ linkChapterResources self-joins depend on it for every chapter, and
dropping it turned those queries into full table scans — a 2× slowdown.

SqlDelightContentDao: chunked multi-row INSERT for content_entity
(90 rows / chunk, ~344 statements vs ~31 k individual inserts).

SqlDelightLanguageDao: chunked INSERT OR IGNORE for language_entity
(166 rows / chunk, ~42 statements vs ~7 k individual inserts). slug has
a UNIQUE constraint so OR IGNORE is safe without ON CONFLICT DO UPDATE.

InitializeApp: per-step timing logs (INIT <name> took N ms) so the
initialization profile is visible in logcat without a profiler.

Tests: SqlDelightBulkLoadTest (desktop, file-backed) and
SqlDelightAndroidBulkLoadTest (instrumented, API 24) pin the
withBulkLoad behaviour — including the Android-specific WAL-via-
executeQuery fix — and confirm the index is kept, not rebuilt.
Imported narration projects showed no chapter/book completion — and
export greyed out — until each book was opened, which also affected
what could be exported.

Two upstream-inherited layers were at fault (both predate the SQLDelight
migration, from b74855b):

- calculateNarrationProgress gated the whole book on
  projectFilesAccessor.isInitialized(), but that per-book resource
  container is only written when a book is first opened
  (InitializeProjectFiles). A freshly imported project failed the gate
  and read 0% for every book. Now it also computes when the book has
  recorded work — any chapter with a selected take, which import
  restores — so imported projects show progress without being opened.
  Un-worked books still short-circuit to 0 with no per-chapter file I/O.

- Chapter completion came only from active_verses.json, which import
  does not populate (and imported takes carry no DB markers). Completed
  chapters therefore read 0% until opening reconstructed that file from
  the compiled chapter take. ProjectCompletionStatus now falls back to
  the selected chapter take's embedded cue markers, read via
  OratureAudioFile (format-agnostic) and matched to every verse, so a
  take missing markers is not counted complete.

Covers NARRATION and DIALECT (shared progress path); TRANSLATION was
unaffected (DB-backed).
list() returned bare child names instead of paths from the archive root, so a
caller that lists a directory and then streams each result, as importSources
does for a backup's carried sources, never found the entries on Android.

normalizePath now also resolves `.`, `..`, doubled and trailing slashes the way
the zip filesystem on the other platforms does, without letting a path climb
above the archive root.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@mXaln
mXaln force-pushed the migrate-jooq-to-sqldelight branch from 736eb63 to decf172 Compare September 15, 2026 12:36
…eader-fix

Fix AndroidZipFileReader.list to return archive-root paths
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants