Skip to content

fix(driver-turso): converge remote date/json cells the pre-fix batch door stored unconverted - #19904

Merged
objectstack-fleet[bot] merged 2 commits into
mainfrom
claude/issue-19868-turso-remote-residue
Sep 24, 2026
Merged

objectstack-fleet[bot] merged 2 commits into
mainfrom
claude/issue-19868-turso-remote-residue

Conversation

@objectstack-fleet

@objectstack-fleet objectstack-fleet Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #19868

Clause-②: no

What this changes

Every remote Turso schema door ends in completeRemoteSchemaSync. It now runs a second backfill after the canonical temporal one. The new pass converges the date and json cells that the pre-#19844 syncSchemasBatch door stored without formatInput. It rewrites only the cells whose stored text already determines their value.

Disposition of the residue the card's title names (seat decision, after the dev's report): a stored true reading back as 1, a string '42' reading back as 42, and a string 'true' reading back as true have bytes identical to another writer's. That is the class the maintainer's ruling of 2026-08-26 on #12380 already settled for the local face (packages/drivers/driver-sql/CHANGELOG.md: "The class stops growing; it is not retroactively repaired.", "with that cost accepted explicitly"), for the same reason ("No migration can know which was written"). A new branch of a ruled class reuses that ruling with its reason, so it is the answer on the remote face too: this PR converges every exactly recoverable class, discloses the rest in its changeset, and closes the card. The first line was changed from Refs to Fixes by the seat accordingly.

Recoverability, measured (A1/A2)

The pre-fix arm was run on the libsql SQLite double: RemoteTransport.syncSchemasBatch alone, then create with no registration. The fixed driver then booted over the same database through syncSchemasBatch. The json column is TEXT affinity, so every scalar lands as TEXT and typeof separates nothing. The Turso-wire column comes from @libsql/client's hrana encoder, read in node_modules: a JS number is sent as float, and a boolean as integer 1n/0n. The double's better-sqlite3 binds a JS number (and the stub's true → 1) as REAL. Both shapes were measured into a TEXT column.

written (pre-fix) cell on the double cell on the Turso wire fixed read fixed door stores rewritten?
json 'hello' TEXT hello (json_valid 0) same 'hello' "hello" yes, read unchanged
json '' TEXT empty (json_valid 0) same '' "" yes, read unchanged
json true / false TEXT 1.0 / 0.0 TEXT 1 / 0 1 / 0 true / false no. 1.0 is also a pre-fix number 1. 1 is also a pre-fix string '1' and the fixed door's number 1
json 0 / 1 / 42 TEXT 0.0 / 1.0 / 42.0 same 0 / 1 / 42 (correct) 0 / 1 / 42 no (a pre-fix string '42.0' leaves the same bytes)
json '42' TEXT 42 same 42 "42" no. 42 is what the fixed door stores for the number 42
json 'true' TEXT true same true "true" no. true is what the fixed door stores for the boolean true
json null NULL same null NULL already canonical
date Date (UTC midnight) 2025-07-28T00:00:00.000Z same 2025-07-28 2025-07-28 yes
date Date = UTC+8 local midnight of 07-28 2025-07-27T16:00:00.000Z same 2025-07-27 2025-07-27 yes, to 2025-07-27 (see below)
date …Z / +08:00 / -05:00 / zone-naive (space or T) / leading or trailing space the text as sent same its leading YYYY-MM-DD its leading YYYY-MM-DD yes
date epoch number / non-date text / bare day same bytes from both doors same as stored as stored not residue

On the date question in A2: a full timestamp maps to exactly one day under the codec. SqlDriver.toDateOnly takes the LEADING day of a string (so 2025-07-28T23:30:00-05:00 is 2025-07-28, not the UTC 07-29) and the UTC day of a Date. A Date built at local midnight east of UTC therefore already reads, and is already written by the fixed door, as the previous day. Measured with new Date(2025, 6, 28) under TZ=Asia/Shanghai: stored 2025-07-27T16:00:00.000Z, read 2025-07-27, and the fixed door stores 2025-07-27. The writer's zone is not in the cell, so the day the caller meant is not recoverable by any door. This PR writes the day the codec already answers and does not try to recover the caller's.

The invariant the change keeps: it rewrites a cell only when the fixed READ already answers a determined value, and it writes the fixed WRITE door's form of that value. No converted cell reads differently afterwards; that is pinned. What changes is what a filter compares. Before the boot, { d: '2025-07-28' } matched only the bare-day row. After it, the filter also matches all 8 full-timestamp rows whose day is 07-28, and { d: '2025-07-27' } matches the UTC+8 row, which it missed before.

Landing site and design

  • packages/drivers/driver-turso/src/remote-codec-residue-backfill.ts (new): the pass.
    • SQL only pre-filters, cheaply enough for every boot. For json, a candidate is TEXT with json_valid() = 0. For date, it is TEXT longer than 10 characters whose left-trimmed first 10 characters glob as YYYY-MM-DD; the trim uses the exact String.prototype.trim set, bound as an argument.
    • The driver's own codec decides and computes the new value, handed over and not copied. For json that is JSON.parse / JSON.stringify, the formatOutput / formatInput json arms. For date it is SqlDriver.toDateOnly.
    • Each write is compare-and-set on the text it was computed from.
    • The pass is batched by a rowid cursor and reuses the temporal backfill's batchSize and maxBatches defaults, 500 × 20. It is resumable: a budget-stopped column is unmarked and resumes at the next sync. A second boot writes nothing.
    • It is quiet: nothing throws out of it, and failures are logged at warn.
    • The probe is one SELECT per table, and all tables go out in one round-trip.
  • turso-driver.ts: completeRemoteSchemaSync calls backfillRemoteCodecResidueQuietly() after the temporal backfill, which calls the private backfillRemoteCodecResidue(). Per-process marks live in remoteCodecResidueConverged. No public surface was added: the module is not exported from index.ts, and the method is private.
  • Deliberately excluded:
    • Single-value media columns. Their quoted-or-bare id form is an ADR-0104 deployment fact the remote doors never ask about, and both forms read the same.
    • JSON nested past SQLite's JSON depth limit. Measured on SQLite 3.53.4, json_valid() answers 0 at 1001 levels and JSON.parse reads it. It is withheld, because re-quoting it would turn an array into a string. The codec, not SQLite, decides.
  • packages/drivers/driver-sql is not touched. No helper was shared: the local face's backfillCanonicalJsonEncoding is a Knex UPDATE, and this pass needs a rowid walk plus a JS codec.

A4: the local face

Measured through TursoDriver local (:memory:) syncSchemasBatch and create with the same inputs. Every date lands as the 10-character day and every json scalar as its JSON encoding. The local door registers field types before any write, so it has no residue from this door, and nothing here runs locally. Full-timestamp date rows written before formatInput gained its date arm would be a different producer. They were not measured here (shallow history), and they are not fixed here.

Tests

packages/drivers/driver-turso/src/turso-remote-codec-residue-backfill.test.ts, 14 cases, over real SQLite cells:

  • the before-state, including the card's filter miss on the fixed read path;
  • every recoverable class, with its cell before, its cell after, and its read-back, plus the day filter and a $lte bare-day bound;
  • no read changes, across every row;
  • the ambiguous classes (both the double and the wire shapes), the canonical dates, null, a media column and deep JSON, all left byte-for-byte;
  • idempotence: a second boot issues no UPDATE and one probe statement, and a second sync in-process issues no probe;
  • a failing probe leaves the boot up and the columns unmarked;
  • paging, budget and resume; compare-and-set;
  • the date pre-filter equals toDateOnly over a corpus that includes every trim character, and JS_TRIM_CHARS equals the engine's own trim over the BMP;
  • the json pre-filter is a superset whose only extra is deep JSON.

Package runs on 257ac60607 (patch round 1; round 1 on 4afdb8c170 read the same counts):

  • pnpm --filter @objectstack/driver-turso test: Test Files 59 passed (59) · Tests 1331 passed (1331).
  • pnpm --filter @objectstack/driver-turso typecheck: tsc --noEmit, exit 0. The package tsconfig includes src/**/*, so the new test file is compiled.

Ablation (on 4afdb8c170; patch round 1 changed spelling and prose only, and the runtime JS_TRIM_CHARS string is byte-identical), run once through scripts/ablation-replace.mjs, which deleted only the wiring line await this.backfillRemoteCodecResidueQuietly();. The anchor went 1 → 0, and the blob went a686f4985bc4 → ae43cfb30ecb. Four pins went red: the date rewrite and filter, the json re-encode, no-read-change (whose non-vacuity leg fails), and idempotence (the probe count). Tests 4 failed | 10 passed (14). The restore was proven: blob a686f4985bc4 equals HEAD, git diff HEAD is empty, and git status is clean. The source resolves through a relative import, so no dist/ is involved.

Gates

On 257ac60607 (patch round 1): --commands derived the same 61 commands (diff empty), and after a cached whole-package build all 61 exited 0 on the first pass; --ran: 61 run, 0 NOT-MEASURED. pnpm check:nul-bytes, node scripts/check-issue-citations.mjs and node scripts/check-changeset-no-major.mjs --base origin/main: exit 0.

Round 1: node scripts/pm/dispatch-gates.mjs --commands on 4afdb8c170 derived 61 commands, each run with its exit captured before any pipe. Three first exited 3 (PREREQUISITE NOT MET: dual-build-cjs-loads, lean-entry-closure, type-check-debt). After building the objectql closure and all packages (turbo run build --filter='./packages/*' --filter='./packages/*/*'), all three exited 0. --ran: 61 derived famil(ies) accounted for — 61 run, 0 NOT-MEASURED. Also by hand:

  • node scripts/check-issue-citations.mjs (the live verdict): exit 0.
  • pnpm check:driver-conformance: exit 0.
  • pnpm check:nul-bytes: exit 0.
  • The control-byte self-scan over the four files: no match.

Acceptance notes

  • Out of scope, class (a), for the seat to file. The LOCAL [finding] driver-sql's SQLite json codec is not round-trip faithful for a string whose content is valid JSON — '123' reads back as a number, and Postgres does not agree #12380 json backfill has the same depth hole this PR avoids. SqlDriver.backfillCanonicalJsonEncoding selects json_valid(??) = 0, so a json value nested past 1000 levels is json_quoted into a string on the next syncSchema. Reproduced on TursoDriver local :memory:: before the re-sync, Array.isArray(row.j) is true; after it, false (typeof string), and the cell starts with a quote character.
  • Steady-state cost: remote boots now pay one more read round-trip. That is a probe scanning each managed table that has date or json columns, once per process. Turso bills rows read.
  • The libsql double does not bind like the Turso wire: better-sqlite3 binds every JS number, and the stub's true → 1, as REAL, while hrana sends a boolean as INTEGER. The tests pin both shapes.
  • The remote schema doors never call resolveFileColumnsMoved, so an engine-supplied ADR-0104 resolver is never asked in remote mode. Whether a remote deployment can reach the moved state was not measured.

Generated by Claude Code

…door stored unconverted

A remote boot now rewrites a `date` stored as a full timestamp to the
calendar day the driver already reads it as, and a json string stored
bare (text that does not parse) to its JSON encoding. Both are exactly
what the fixed write door stores for the same value, so no converted
cell reads differently; the day filter now matches. Every json cell
whose text parses (a stored `true` reading as 1, a string '42' reading
as 42) is left as stored: its bytes collide with another writer's.

Claude-Session: https://claude.ai/code/session_01TEhopqrWQYBycZzyJHpAZr
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions github-actions Bot added size/l documentation Improvements or additions to documentation tests tooling labels Sep 23, 2026
@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/driver-turso, touching 25 documentable anchor(s).

2 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:

  • content/docs/data-modeling/drivers.mdx (via TursoDriver (symbol, a top-level class))
  • content/docs/plugins/packages.mdx (via TursoDriver (symbol, a top-level class))
What this run could not see
  • 5 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 60 of 215 client-bound route-ledger rows — the other 155 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 155: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 100 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 6 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json d1ca8741dda9ff8f72d1a1d3cbef5c0fb3dc687e → packageMentionDocs.

Which tree this was computed on

This run read content/docs from e4e89671f9a482f80fc2f772a43627fdc62e005e — the merge of head 257ac606073144ff8594440f459356f62d8a396b into base d1ca8741dda9ff8f72d1a1d3cbef5c0fb3dc687e, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin e4e89671f9a482f80fc2f772a43627fdc62e005e && git checkout e4e89671f9a482f80fc2f772a43627fdc62e005e
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin d1ca8741dda9ff8f72d1a1d3cbef5c0fb3dc687e 257ac606073144ff8594440f459356f62d8a396b && git checkout -B drift-repro d1ca8741dda9ff8f72d1a1d3cbef5c0fb3dc687e && git merge --no-ff 257ac606073144ff8594440f459356f62d8a396b

node scripts/docs-audit/affected-docs.mjs --json d1ca8741dda9ff8f72d1a1d3cbef5c0fb3dc687e

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs d1ca8741dda9ff8f72d1a1d3cbef5c0fb3dc687e → pass the list as
args.docs, on the commit named under Which tree this was computed on.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Contract review

Served-tier: CONTRACT_REVIEW_TIER
Head-sha: 4afdb8c1709755136c19733e3a3068656338e1a3

Head = PR head; 4 files vs merge-base 1f0b3419ff, none governed; every latest check run green (Vercel is an external commit status). Measured read-only on SQLite 3.53.4 and on @libsql/hrana-client's valueToProto; nothing was written in any checkout.

① Derived judgments

(a) Exact recoverability HOLDS for both rewritten classes.

  • json: the fixed door JSON.stringifys every non-null json value, so a fixed-door string is stored quoted and is json_valid = 1 (lone-surrogate and \u0000 escapes included). No non-string pre-fix writer produces json_valid = 0 text: hrana sends a boolean as integer, a number as float, and throws on non-finite. So unparseable text came from a string, and the rewrite leaves the driver read identical. JSON past the depth limit is withheld (measured, pinned).
  • date: the pre-filter and SqlDriver.toDateOnly agree for Z, offsets, zone-naive, and leading/trailing whitespace including U+FEFF and CR LF. The rewritten value is the LEADING day, exactly what the read answers and what the fixed door stores. One safe-direction divergence: a date text with an embedded NUL after the day is not selected.

(b) No converted cell reads differently: PINNED and HOLDS (the fixed driver's formatOutput over the raw pre-backfill rows equals find after the boot, with non-vacuity asserted).

(c) Mechanics TRUE and PINNED: compare-and-set by rowid on the page's own text (1 change, then 0 on a stale retry; a racing write survives); paging by a rowid cursor, with resume needing no checkpoint; idempotence (second boot 0 UPDATE, 1 probe); quietness (warn, the boot stays up, columns stay unmarked); cost is one read round-trip per backfill.

(d) TWO FALSE SENTENCES:

  1. Changeset: "No read of a date field ever returned it." FALSE for the affected population. Before driver-turso remote syncSchemasBatch — the engine boot door — skips read-coercion registration, managed-object recording and the canonical backfill: a remote boolean can read back as 1, JSON as a string #19844 the batch door registered no field types, so formatOutput returned the stored full timestamp, as the pending driver-turso remote syncSchemasBatch — the engine boot door — skips read-coercion registration, managed-object recording and the canonical backfill: a remote boolean can read back as 1, JSON as a string #19844 changeset in the same release says. An upgrader reads this as "nothing observable is lost" just before the bytes are dropped.
  2. The JS_TRIM_CHARS docblock says "Written as escapes and BOUND as an argument". FALSE: only the five control characters are escapes; the other twenty are literal invisible characters in the source.
  3. Nit: "a superset of the cells that could be rewritten" does not hold for the embedded-NUL date text (safe direction). Optional.

(e) The disposition under the #12380 ruling CARRIES: the remote ambiguous class is the same on-disk byte collision for the same reason, and the PR body's quotations match packages/drivers/driver-sql/CHANGELOG.md verbatim and frame the remote extension as a seat decision.

② Semver level

@objectstack/driver-turso: patch, Clause-②: no, CONFIRMED. No codec, DDL or read changes, and no public surface is added. The precedent for an at-rest convergence at boot without a codec change is #5770 and #19844 (no BREAKING banner, no adr-0087 marker); #12380 carried BREAKING because it changed the bytes of new writes, and this PR changes none. The one loss visible out of band, the time of day in a date column, is disclosed.

③ Boundary flags

Implemented-by: claude/issue-19868-turso-remote-residue
Reviewed-by: session_01TEhopqrWQYBycZzyJHpAZr

VERDICT: FAIL: ① (d) does not hold. The changeset sentence "No read of a date field ever returned it." is false for the population it addresses, and the JS_TRIM_CHARS docblock misdescribes the source. The rewrite logic corrupts no cell. A patch round correcting both texts is in flight.


Generated by Claude Code

… fix; spell trim chars as escapes

The changeset said no read of a `date` field ever returned the dropped
time of day. Before the boot-door read fix (which ships in the same
release) a boot-synced object read such a cell back exactly as stored,
so every read claim now says what it is relative to. `JS_TRIM_CHARS`
held twenty of its twenty-five characters as literal invisible
characters although its docblock said escapes; all twenty-five are now
`\uXXXX` escapes and the runtime string is byte-identical. The
pre-filter's superset wording names its one miss: a date text with an
embedded NUL after the day, which SQLite's length() undercounts.

Claude-Session: https://claude.ai/code/session_01TEhopqrWQYBycZzyJHpAZr
Co-authored-by: Claude <noreply@anthropic.com>
@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Contract review

Served-tier: CONTRACT_REVIEW_TIER
Head-sha: 257ac606073144ff8594440f459356f62d8a396b

Narrow re-review of the patch round over record 5799532789 (head 4afdb8c170, FAIL on two sentences). 4afdb8c170 is an ancestor of 257ac60607 (one commit, fast-forward). Delta = 2 files, +14/−8: .changeset/19868-turso-remote-date-json-residue.md and packages/drivers/driver-turso/src/remote-codec-residue-backfill.ts. The PR's file list vs merge-base 1f0b3419ff is unchanged at 4 files, none governed.

① Derived judgments

(a) The executable delta is the spelling of JS_TRIM_CHARS, nothing else: a comment-stripped diff of the module at the two heads differs only in the two initializer lines. Both initializers evaluated: 25 code points (0009 000A 000B 000C 000D 0020 00A0 1680 2000–200A 2028 2029 202F 205F 3000 FEFF), runtime sha256 3655dd5354dc40c90551fa4b2adea0ad2e02b6f674f368fc25000dab61e08548 on both sides, IDENTICAL. The head initializer is 25 \uXXXX escapes with 0 literal non-ASCII or non-printable characters; the set equals every BMP code point String.prototype.trim removes. It is bound, not interpolated (ltrim(col, ?) with args: [JS_TRIM_CHARS], bound in placeholder order by convergeColumn and buildTableProbe). The docblock sentence "Written as escapes and BOUND as an argument, never inlined into SQL." is now TRUE.

(b) The four rewritten changeset sentences are TRUE, each checked against .changeset/19844-turso-remote-boot-read-coercion.md and the code at head. The read-side and write-side date arms both apply SqlDriver.toDateOnly, and the backfill writes toDateOnly(stored) through the driver's own codec. toDateOnly is idempotent on its own answer, so the rewrite changes no read. #19844 landed as 1f89ba0d70 (#19863), an ancestor of the merge-base, and its changeset is still unconsumed on origin/main d1ca8741dd, so both ship in the same changeset version. The pre-fix halves quote #19844's own bullets and match the code: the old syncSchemasBatch registered no field types, so a boot-synced date read back as stored and a json field as its text. The backfill never touches a cell that parses. No sentence a CHANGELOG reader would act on wrongly remains.

(c) NUL exception reproduced on SQLite 3.51.2 (node:sqlite): the text 2025-07-28, then NUL, then T10:00:00Z is typeof text, length() 10, 21 blob bytes, and the head pre-filter does not select it; a full timestamp and a U+FEFF-padded zone-naive text are selected; a bare day is not. Both reworded docblocks say exactly that. Precision, not a defect: the escape is only a NUL as the 11th character of an unpadded text. A later NUL is selected and rewritten to the day the read already answers, so no cell is wrongly rewritten under either reading.

(d) PR body: first line Fixes #19868; the seat's #12380 disposition paragraph is intact; neither flagged sentence appears (0 hits each). The refreshed readings cite 257ac60607 for the package runs and gates, and keep the ablation on 4afdb8c170 with the byte-identity justification (a) confirms.

(e) CI at 257ac60607: latest per name over 34 names: 29 success, 5 skipped, 0 failure, 0 in progress; all seven required contexts success. Vercel is an external commit status.

(f) Everything record 5799532789 judged true and the delta does not touch is CARRIED: exact recoverability of both classes, the no-read-change pin, compare-and-set, paging, idempotence and quietness, the #12380 disposition, the 14-case test file and the turso-driver.ts wiring (unchanged in the delta).

② Semver level

@objectstack/driver-turso: patch, Clause-②: no: CONFIRMED and carried. The delta changes one constant's spelling (runtime bytes identical) and prose; no surface, codec, DDL or read change.

③ Boundary flags

Implemented-by: claude/issue-19868-turso-remote-residue
Reviewed-by: session_01TEhopqrWQYBycZzyJHpAZr

VERDICT: PASS


Generated by Claude Code

@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review September 23, 2026 17:59
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Sep 23, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Sep 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor

⛔ merge queue 构建失败 — 先分诊,再决定要不要重排

队列构建 35899547571 红了。队列跑的是全量套件(PR 侧 CI 只跑 affected 子集),
所以失败的测试可能在本 PR 没碰过的包里 —— 那不是重排能修的。每次盲目重排都会让排在后面的所有 PR 重建一轮。

失败的 job(日志抽取,best effort):

  • Test Core (2/6) — 失败步骤: Run this shard's tests

    @objectstack/runtime:test:  FAIL   local  src/package-uninstall-org-scope.integration.test.ts > #7705 — org-scoped uninstall must not orphan env-wide sys_metadata rows > does NOT sweep up another orga
      ↳ 失败原因: @objectstack/runtime:test: Error: Test timed out in 5000ms.
    

↳ 失败原因 是判读的关键:超时(Test timed out in … / Hook timed out in …)多半是负载/时序,不是本 PR 的回归;
断言(AssertionError: …)才指向真实的行为改变。两者的 FAIL 行长得一模一样,只有这一行能区分。

⚠️ 断言这一侧有一类例外,判据是断言在测什么,不是它是不是 AssertionError。 断言的对象是产品行为(一个值、一个形状、一次拒收)⇒ 照上面读:真实的行为改变,去查,⛔ 不要重排掉;
断言的对象是这次实验自身的有效性前提(跑完的耗时、负载下的先后、任何只在时间预算内才成立的条件)⇒ 它跟超时是同一类,同样对负载敏感,重排一次是合法的判别手段。
识别是机械的:断言的消息或它比较的值本身点名了一段时长、一个时间戳、一个耗时计数。实测过的一对 —— AssertionError: SecurityPlugin.init() ran: expected false to be true 测的是产品行为(真回归);
AssertionError: this run took over a second, so second-precision stamps could have differed too: expected 1006 to be less than 1000 测的是实验前提:它守护的那条不变式当时是绿的,同一个 head 原样重排一次即成功。
穿着 AssertionError 外衣的时间测量,仍然是时间测量。(⛔ 这只改「怎么读一次红」,不改「哪些测试可以重排」——后者由别处管。)

跨 PR 相同签名(24h,按失败测试文件聚合):

  • src/package-uninstall-org-scope.integration.test.ts — 24h 窗口内只有本 PR 撞到过,暂不汇总(再有一个不同 PR 撞到就会自动开汇总 issue)。
  • ⚠️ 24h 评论账本没读完(超过 5 页仍未读到窗口尽头),所以上面的「不同 PR 数」是下界,不是全量。

历史信号:

  • 本 PR 过去 24h 无队列失败记录(首次)。
  • 过去 24h 队列共有 0 个失败构建(不含本次)。

分诊清单:

  1. 失败测试在本 PR 改动的包里 → 真回归,修 PR。
  2. 失败测试与本 PR 无关 → 看上面的「跨 PR 相同签名」;已有汇总 issue ⇒ flaky/环境问题实锤,去那张 issue 上谈,修好前重排只会再烧一轮全队列。
  3. 两者都不是 → 可能与同组 PR 语义冲突;等前面的 PR 落地或失败出队后再重排一次即可,不要连续重排。

Generated by Claude Code · merge-queue-triage workflow (#4859)

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Merge-queue ejection — new signature, not re-queued

domain:engine#1, session_01TEhopqrWQYBycZzyJHpAZr, written 2026-09-23T18:18Z.

Signature (queue build 35899547571, merge_group on gh-readonly-queue/main/pr-19904-8490127962834a125239ae85d9e6a88c0386896c, the PR alone on main 8490127962):

  • job Test Core (2/6), step "Run this shard's tests";
  • @objectstack/runtime · src/package-uninstall-org-scope.integration.test.ts › #7705 — org-scoped uninstall must not orphan env-wide sys_metadata rows › does NOT sweep up another organization’s rows;
  • first error line: Error: Test timed out in 5000ms. (a timeout, not an assertion).

Ledger reading: no queue-flake-anchor issue exists for src/package-uninstall-org-scope.integration.test.ts, open or closed. The triage comment (5800378376) shows this PR as the only hit in its 24h window, and that window's comment ledger was not read to its end. So this is a new signature, and under the seat's landing rules a new signature is ⛔ not re-queued.

Initial judgment: not this PR's regression.

  • The PR changes four paths, all under packages/drivers/driver-turso/ plus its changeset. The failing file imports @objectstack/objectql, @objectstack/driver-sql and @objectstack/metadata-protocol and runs a real SqlDriver on on-disk better-sqlite3. It does not import driver-turso, and nothing it runs is in this diff.
  • The same base passed: main 8490127962 has every Test Core shard success, Test Core (2/6) included, and the test file is unchanged since spec: inline view arms of the runtime write door require the object binding #8089.
  • The failure mode is a 5000 ms timeout in an eight-case file that builds a fresh on-disk database per case: load-sensitive, the class the triage comment reads as "负载/时序".

What unblocks it: a human upgrade of the ledger (this file's timeout recorded as known flaky), or an instruction to re-queue once. In either case the seat re-arms auto-merge on the unchanged head 257ac60607. A second ejection on the same signature stops re-queuing for good and is diagnosed afresh. #19868 stays pm:dispatched meanwhile. The two same-file cards queued behind it (#19893 p1, #19894 p2) stay undispatched until this lands.


Generated by Claude Code

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Re-queue on the maintainer's instruction — once, on the unchanged head

domain:engine#1, session_01TEhopqrWQYBycZzyJHpAZr, written 2026-09-24T01:28Z.

Whose instruction, verbatim, where: the maintainer (huangyiirene), in this seat's session session_01TEhopqrWQYBycZzyJHpAZr (chat), answering the seat's ejection report: 「重投 #19904」.

Signature re-queued against (recorded in 5800424463): queue build 35899547571, Test Core (2/6), @objectstack/runtime src/package-uninstall-org-scope.integration.test.ts › does NOT sweep up another organization’s rows, first error Error: Test timed out in 5000ms.; no queue-flake-anchor for the file. Ledger basis: none — this re-queue rests on the maintainer's instruction, not on a ledger entry.

Head: unchanged, 257ac60607; every check on it green, mergeable_state: clean. Auto-merge is re-armed now. ⛔ One re-queue only: a second ejection on the same signature stops re-queuing, and the item is handed to the next seat with a fresh diagnosis.


Generated by Claude Code

@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Sep 24, 2026
Merged via the queue into main with commit 467fa76 Sep 24, 2026
43 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-19868-turso-remote-residue branch September 24, 2026 01:46
akarma-synetal pushed a commit to akarma-synetal/framework that referenced this pull request Sep 28, 2026
…ON nested past SQLite's depth limit is no longer quoted into a string (objectstack-ai#19972)

Fixes objectstack-ai#19912

Clause-②: no

## What was wrong

`SqlDriver.backfillCanonicalJsonEncoding` (the local SQLite `Field.json`
storage backfill from objectstack-ai#12380) ran one statement per json column on every
`syncSchema` / `initObjects` over an existing table:

```sql
update ?? set ?? = json_quote(??) where typeof(??) = 'text' and json_valid(??) = 0
```

SQLite's `json_valid()` answers 0 for JSON nested more than 1000 levels
deep, while the driver reads such a cell with `JSON.parse` without
trouble. So a deep array the current write door stored correctly was
quoted into a JSON string on the next schema sync and read back as a
string from then on. No error, no log line: a correct cell silently
became a wrong one.

## The fix

SQL now only pre-filters. The driver's codec decides.

- **One rule, shared by both faces.** New
`packages/drivers/driver-sql/src/unencoded-json-text.ts` exports
`recoverUnencodedJsonText(stored)`: when `JSON.parse` accepts the text
it returns `null` and the cell is left as stored; when `JSON.parse`
rejects it, the cell is a legacy plain string and the function returns
`JSON.stringify(stored)`, which is what the write path stores. It is
exported from `@objectstack/driver-sql`, and the Turso remote residue
backfill (`remote-codec-residue-backfill.ts`, from PR objectstack-ai#19904) now
imports it. That remote change is an import plus one line in place of
its inline copy, and the remote behaviour does not change.
- **The local pass** (`convergeJsonColumn`) reads the `typeof(col) =
'text' and json_valid(col) = 0` candidates 500 at a time in `rowid`
order, applies the rule to each, and rewrites only the cells the rule
rejects. Each rewrite is a compare-and-set, `where rowid = ? and
typeof(col) = 'text' and col = ?` bound to the text the decision read,
so a value written concurrently is never overwritten. One transaction
covers each page's writes. The pass walks every page, so a column still
converges in a single sync, and memory stays bounded to one page.
- **Idempotence holds.** A converted cell is outside the pre-filter. A
withheld cell is re-read on every sync but never written. A re-run over
a converged table changes zero rows, and that is pinned with
`total_changes()`.
- **Logging.** The failure posture is unchanged: `warn`, and the rows
stay legacy. The warn now also reports how many rows the committed pages
converted. Withheld cells get one `info` line per column.

## Measured before acting (the dispatch's H1 to H5)

- **H1, confirmed** at `2c1011b`: the method and the statement are
exactly as quoted above, guarded by `if (!this.isSqlite || !fields ||
fields.length === 0) return;` and `if (!tableExisted) return;`.
- **H2, confirmed. It is a build-time constant, and every bundled build
has 1000.** `json_valid` of an N-level array, measured on each bundled
engine:

  | engine | SQLite | N=1000 | N=1001 | N=2000 | N=2001 |
  |---|---|---|---|---|---|
  | better-sqlite3 13.0.3 | 3.53.4 | 1 | 0 | 0 | 0 |
  | sql.js 1.14.1 | 3.49.1 | 1 | 0 | 0 | 0 |
  | `@libsql/client` 0.17.4 (local) | 3.45.1 | 1 | 0 | 0 | 0 |

`JSON.parse` (Node v22.22.2) reads 1001, 1999, 2000 and 2001 levels. The
bundled `sqlite3.c` defines `JSON_MAX_DEPTH 1000` unless
`SQLITE_JSON_MAX_DEPTH` is set, and notes the limit "was 2000 prior to
version 3.42.0". A Turso server build, or any custom or older build, can
therefore differ. The fix does not depend on the value.
- **H3, confirmed.** No override of the method exists outside
`sql-driver.ts`. `SqliteWasmDriver` forces `isSqlite` and inherits
`syncSchema` / `initObjects` unchanged. In local mode,
`TursoDriver.syncSchema` goes to `super.syncSchema`, then `initObjects`,
then `super.initObjects`; `initObjects` goes to `super`;
`syncSchemasBatch` calls `super.syncSchema` once per schema. All three
doors reach the backfill, and the Turso local face runs on
better-sqlite3 (`toKnexConfig`). The Turso doors are pinned in a test.
The wasm face was measured with a scratch probe but is not pinned (see
Acceptance notes): through both `syncSchema` and `initObjects`, the
1001-level array read as an array after two backfills, and `bare` was
converted.
- **H4: the decision rule is shared, the paging machinery is not.** The
remote pass runs over libsql `execute`/`batch` (`RemoteBackfillClient`),
with a probe and a per-invocation budget. The local pass runs over knex.
Sharing the walker would take more than an import swap on the remote
side. The pre-filter SQL reads the same on both faces, but it only
narrows the scan and never decides.
- **H5, confirmed. The two are identical by construction.** The SQLite
json arm of `formatOutput` is `JSON.parse(text)` inside a try/catch. The
rule asks the same question, and a test pins the link through the
driver: the backfill rewrites a cell exactly when the driver's read of
that cell answers the stored text itself.

**Preservation, byte for byte.** Stored and then read on better-sqlite3,
`json_quote(col)` equals `JSON.stringify` of the text read back, for
every non-surrogate BMP code point plus astral characters and an
embedded NUL: 63,499 samples, 0 mismatches. Lone surrogates never reach
disk; better-sqlite3 stores them as U+FFFD. The compare-and-set only
writes when the stored bytes equal what was read, so every cell written
is written with the bytes the retired statement produced.

## Tests

New
`packages/drivers/driver-sql/src/sql-driver-json-backfill-depth-limit.test.ts`
(7 tests):

1. The card's reproduction: `bare`, a 1001-level array and `[1,2]`. Only
`bare` is quoted, and the array still reads as an array after a
**second** backfill.
2. Idempotence: `total_changes()` does not move over two re-runs,
including with a withheld cell present.
3. Preservation: 11 legacy bare texts (empty, control characters, an
embedded NUL, U+2028/2029, a leading U+FEFF, emoji and CJK) are each
converted to the exact bytes `select json_quote(?)` gives, and each
still reads as itself.
4. The rule is the read codec. The cases are 1000-, 1001- and 2001-level
arrays, a 1001-level object, parseable and unparseable texts, `1e400`
and an escaped lone surrogate. A cell is rewritten if and only if its
pre-backfill read answers its own text, and no read moves.
5. Compare-and-set: a second connection writes the deep array into a
candidate row between the page read and the write, and the write leaves
it alone.
6. Paging: 1,100 rows, with deep cells at rows 0, 499, 500, 999 and
1099. All 1,095 legacy rows converge in one pass, and the log payloads
report `rowsConverted: 1095` and `rowsWithheld: 5`.
7. A unit test of `recoverUnencodedJsonText`.

New
`packages/drivers/driver-turso/src/turso-local-json-backfill-depth-limit.test.ts`
(3 tests) runs the dev's reproduction on `new TursoDriver({ url:
':memory:' })`, once through each local door: `syncSchema`,
`initObjects` and `syncSchemasBatch`.

Suites run at `74ff944`:

- driver-sql: 181 files passed, 11 skipped; 2677 tests passed, 170
skipped. The skips are the live PG and MySQL cells; CI runs them in
`Temporal Conformance`.
- driver-turso: 60 files, 1334 tests passed. This includes the unchanged
`turso-remote-codec-residue-backfill.test.ts`, run against the rebuilt
driver-sql `dist`.
- driver-sqlite-wasm: 29 files, 518 tests passed.
- `tsc --noEmit` exit 0 for driver-sql and driver-turso. `--listFiles`
shows each new test file inside its package's program.

## Reverse verification

Both ablations went through `scripts/ablation-replace.mjs` (anchor hit 1
time, blob changed, and restore proven as blob == HEAD with an empty
`git diff HEAD`) from the committed fix at `74ff944`.

- **Ablation 1: the retired statement put back** in place of
`convergeJsonColumn`. Afterwards driver-sql was rebuilt.
`ablation-dist-preflight` found the marker `json_quote(??)` in
`dist/index.js` and `dist/index.mjs`; before the mutation it was absent
from both.
- driver-sql file: **4 red, 3 green**. Red: the reproduction, the
read-codec rule, paging, and compare-and-set. Compare-and-set is red
because the old code never reads candidates, so the injection never
fired (`expected +0 to be 1`); ablation 2 isolates it. Green:
idempotence, preservation, and the unit test, which is the expected
direction because the old statement is also idempotent and
byte-identical on bare text.
- Turso file: **3 of 3 red**, all with `expected false to be true` on
`Array.isArray`.
- Restore leg: rebuilt, the preflight with `--absent` found the marker
in none of the 6 built files, and the tests went back to 7/7 and 3/3.
- **Ablation 2: the compare-and-set predicate removed.** The update
became `where rowid = ?`. **Exactly one test went red**,
compare-and-set: `raced` was overwritten with `"bare"`. After the
restore, the file is 7/7 green again.

## Gates

`node scripts/pm/dispatch-gates.mjs --commands --repo
objectstack-ai/objectstack` at `74ff944` derived 62 families: the
dispatch's 56, plus `check:engine-double-contract`,
`check:objectql-double-limit`, `check:query-options-erasure`,
`check:type-check-coverage`, `check:type-check-debt` and
`check:where-matcher`. Reconciled with `--ran` (exit codes recorded):
**60 exit 0 and 2 NOT MEASURED** (both exit 3, a missing prerequisite).

- `check:dual-build-cjs-loads` needs every workspace package's `dist`,
and 78 published entry points have no `dist` here. As a narrowed probe:
the CJS and ESM entries of driver-sql and driver-turso both load, and
`recoverUnencodedJsonText` resolves from both driver-sql builds.
- `check:type-check-debt` is missing the built type entries of 23
dependencies. The two packages this diff touches type-check clean
(above).
- `check:plugin-teardown-shape --self-test` first exited 3 on the
shallow clone (its pinned fixture commit was unreachable). After
fetching that one commit it passed, and `check:lean-entry-closure`
passed after the `@objectstack/objectql` closure was built.
- `check:driver-conformance` gave the same line before (`2c1011b`) and
after (`74ff944`): `OK — 50 covered cell(s), 0 in the DEBT ledger, 0
exempt`.
- `check:durability-log-level` exit 0: 36 durability-critical seams, all
loud.
- Narrowed lint, not the repo-wide `pnpm lint`: `eslint
--no-inline-config --format json` over the 6 changed TS files gave 6
results, 0 errors and 0 warnings, at `74ff944`. Every one of the 6 falls
under the `packages/**/*.{ts,tsx,mts,cts}` blocks of
`eslint.config.mjs`, and the `.changeset` file is outside every lint
glob. The config enables no type-aware linting (no
`parserOptions.project`, no typed rules, as its own comment records), so
this diff cannot move the verdict for an untouched file.

## Acceptance notes

- **Behaviour on the wasm face, measured only.** sql.js reads text
through a decoder that drops a leading U+FEFF, and its bind path
truncates at an embedded NUL. Where the read differs from the stored
bytes, the compare-and-set does not match, so the cell is not written
and keeps reading as it did. The retired statement rewrote such cells,
because `json_quote()` works on the stored bytes. For a BOM-prefixed
JSON text, that would change this face's read from an array to a string.
That last point is reasoned from the two measured reads; it was not run
against the old code. The wasm face also used to run a zero-row `UPDATE`
on every sync for every json column. Its dirty flag is set by statement
kind, so the file-backed image was marked dirty each time. It now runs
an `UPDATE` only when a candidate is rejected. A committed wasm pin
would be a new file in `driver-sqlite-wasm`, which is outside this
card's declared file surface, so none was added.
- **Withheld cells cost a read on every sync.** A deep cell stays a
pre-filter candidate forever, so it is read and parsed on each sync,
with one `info` line per column. These cells are rare by shape.
- **The json pre-filter has the same embedded-NUL exception as the date
one.** `json_valid()` stops at a NUL, so a text that is valid JSON up to
a NUL is never selected. That is the safe direction, and the local doc
now says so. The remote module's header names only the date exception
("with one exception in the safe direction (the date bullet)"). This is
a gap in the docs, not a defect.
- **A cursor stall guard.** If a rowid comes back rounded (past 2^53)
and the cursor stops advancing, the pass stops, leaving the rest of the
column legacy, which reads the same.

---
_Generated by [Claude
Code](https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx)_

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/l tests tooling

Projects

None yet

1 participant