Skip to content

Commit ff9881d

Browse files
committed
docs(driver-sql): name the json storage-format record in words, and qualify the changeset's byte-for-byte claim
The docblock of unencoded-json-text.ts cited an issue number that no longer resolves; it now names the live record (the backfill's doc block and its roundtrip test) instead. The changeset says the rewrite is byte-for-byte only where the engine reads the stored bytes back verbatim, names the cells that are left as stored instead, and lists every client SqlDriver treats as SQLite. Comment and changeset text only. Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
1 parent 8cb2d8e commit ff9881d

2 files changed

Lines changed: 10 additions & 6 deletions

File tree

‎.changeset/19912-json-backfill-depth-limit.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,8 @@
55

66
fix(driver-sql): the local SQLite `Field.json` storage backfill no longer turns a deeply nested array or object into a string on the next schema sync (#19912)
77

8-
The backfill that converges legacy json cells on their JSON-encoded form (#12380) ran one `UPDATE … set col = json_quote(col)` over every TEXT cell SQLite's `json_valid()` rejects. `json_valid()` answers 0 for JSON nested more than 1000 levels deep (SQLite's JSON depth limit in every build this repository bundles), while the driver reads such a cell with `JSON.parse` without trouble. So a deep array written correctly through the driver was quoted into a JSON string by the next `syncSchema` / `initObjects`, and read back as a string from then on — silently, with no error.
8+
The backfill that converges legacy json cells on their JSON-encoded form (it came with the change that made the SQLite write path JSON-encode every json value; the issue number that change cites no longer resolves, and its live record is the `SqlDriver.backfillCanonicalJsonEncoding` doc block and `sql-driver-12380-json-roundtrip.test.ts`) ran one `UPDATE … set col = json_quote(col)` over every TEXT cell SQLite's `json_valid()` rejects. `json_valid()` answers 0 for JSON nested more than 1000 levels deep (SQLite's JSON depth limit in every build this repository bundles), while the driver reads such a cell with `JSON.parse` without trouble. So a deep array written correctly through the driver was quoted into a JSON string by the next `syncSchema` / `initObjects`, and read back as a string from then on — silently, with no error.
99

10-
SQL now only pre-selects the candidate cells, a page at a time. The driver's own codec decides each one: a cell `JSON.parse` reads is left exactly as stored; a cell it cannot read is a legacy plain string and is rewritten to `JSON.stringify` of that string — byte-for-byte what the old statement wrote for it. Each rewrite is a compare-and-set on the text it was decided from, so a value written concurrently is never overwritten, and a re-run over a converged table still writes nothing. This covers every local SQLite face that inherits the backfill: `SqlDriver` on better-sqlite3, `SqliteWasmDriver`, and `TursoDriver` in local mode.
10+
SQL now only pre-selects the candidate cells, a page at a time. The driver's own codec decides each one: a cell `JSON.parse` reads is left exactly as stored; a cell it cannot read is a legacy plain string and is rewritten to `JSON.stringify` of that string, byte-for-byte what the old statement wrote for it wherever the engine reads the stored bytes back verbatim. A cell the engine does not read back verbatim is left as stored and keeps reading as it did, where the old statement rewrote it: on `SqliteWasmDriver`, a legacy text with a leading U+FEFF or an embedded NUL (sql.js drops both when it reads the text); on any engine, text holding invalid UTF-8. Each rewrite is a compare-and-set on the text it was decided from, so a value written concurrently is never overwritten, and a re-run over a converged table still writes nothing. This covers every local SQLite face that inherits the backfill: `SqlDriver` on every client it treats as SQLite (`better-sqlite3`, `sqlite3` and its alias `sqlite`), `SqliteWasmDriver`, and `TursoDriver` in local mode.
1111

1212
The decision rule is exported from `@objectstack/driver-sql` as `recoverUnencodedJsonText(stored)`, and `@objectstack/driver-turso`'s remote codec-residue backfill now imports it instead of carrying its own copy, so the local and remote backfills apply one rule. That is a new public export on `@objectstack/driver-sql`'s root entry, and the reason this package takes `minor`: the function returns `null` for text `JSON.parse` accepts and `JSON.stringify(stored)` for text it rejects, and it is exported so that `SqlDriver.backfillCanonicalJsonEncoding` and the remote backfill's `recoverResidueCell` decide each cell by one shared rule rather than by two copies that could drift apart. The remote backfill's behaviour is unchanged.

‎packages/drivers/driver-sql/src/unencoded-json-text.ts‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,11 @@
66
*
77
* Two faces hold a json value that reached disk WITHOUT its JSON encoding:
88
*
9-
* - local SQLite (this package's `SqlDriver.backfillCanonicalJsonEncoding`,
10-
* #12380): a string the pre-#12380 `formatInput` stored raw;
9+
* - local SQLite (this package's `SqlDriver.backfillCanonicalJsonEncoding`): a
10+
* string an older `formatInput` stored raw, before the SQLite write path
11+
* JSON-encoded every json value. The issue number that change cites no
12+
* longer resolves; its live record is that method's doc block and
13+
* `sql-driver-12380-json-roundtrip.test.ts`;
1114
* - Turso remote (`remote-codec-residue-backfill.ts` in
1215
* `@objectstack/driver-turso`, #19868): a scalar the pre-#19844 remote batch
1316
* door stored raw.
@@ -26,8 +29,9 @@
2629
* `s`.
2730
* - **`JSON.parse` accepts it** ⇒ it reads back as what it parses to, and it is
2831
* never rewritten. Either it already is the stored form of that value, or it
29-
* is one of the collisions the #12380 ruling accepts as unrecoverable (the
30-
* string `'{"a":1}'` and the object `{a:1}` were the same bytes).
32+
* is one of the collisions the ruling on that storage format accepts as
33+
* unrecoverable (the string `'{"a":1}'` and the object `{a:1}` were the same
34+
* bytes; recorded in the same doc block and test).
3135
*
3236
* ⛔ SQL may PRE-FILTER candidates (`json_valid(col) = 0`) but never decides.
3337
* SQLite's JSON parser and the driver's disagree, and the disagreement runs the

0 commit comments

Comments
 (0)