Repository navigation
Commit 2034975
fix(driver-turso): a Turso remote database enforces the storage.notNull a field declares (#23893)
Fixes #23847
Clause-②: no
## What was wrong
`RemoteTransport`, driver-turso's remote face and the `libsql://`
transport that hosted environments use, built every declared column
without `NOT NULL`. `buildCreateTableSQL` and both `ALTER TABLE … ADD
COLUMN` sites (in `syncSchema` and in `syncSchemasBatch`) wrote only the
storage type. So a field declared `storage: { notNull: true }` was
unenforced on every Turso remote database. The local face's
`SqlDriver.createColumn` has always created that column `NOT NULL`
(ADR-0113). The shared differ then read each such column as destructive
`tighten_not_null` drift, on a database the remote sync had only just
created.
## What changes
- **`RemoteTransport.columnDeclarationSQL(field)`** is now the one
renderer for every statement that materializes a declared field's column
on this face. It writes the storage type from `mapFieldTypeToSQL`, then
` NOT NULL` when `declaresColumnNotNull(field)` holds. The predicate
asks what `createColumn`'s own `declaresColumnNotNull` asks: whether
`storage.notNull` is truthy. That holds on every field type, multi-value
JSON columns included (the #17231 case), and `required` alone never
triggers it.
- **All three sites render through it:** `buildCreateTableSQL`, the
`syncSchema` ADD COLUMN and the `syncSchemasBatch` ADD COLUMN. The three
remote schema doors (`syncSchemasBatch`, `syncSchema`, `initObjects`)
all reach those statements.
- **A refused remote `ADD COLUMN` from `syncSchema` now names its
statement.** It uses knex's `SQL - message` form, as the local face's
error already does. libSQL's own message names no table and no column,
and the engine's per-object report names only the object. The error
object, its class and its code are unchanged.
- **`turso-driver.ts`, the caveat text only.** The remote face's
`driftReportCaveat` sentence, and the header docblock above it, said in
the present tense that the remote sync creates no `NOT NULL`. This
change makes that false, so those NOT NULL sentences now say that tables
created by an earlier release keep their nullable columns. The caveat
stays non-null. Every line carrying the "a destructive entry has no
remedy on this transport" clause, and its docblock twin, is
byte-identical to `main`; #23848 owns that clause (relay 6213357665,
accepted in 6213866355).
## Measurements
The committed test uses the instrument
`turso-remote-drift-detection.test.ts` established: a remote-mode
`TursoDriver` over a real `@libsql/client` `file:` client (libSQL
3.45.1), with the local face as the control. A second leg ran against a
real server: the official `sqld 0.24.32` (tursodatabase/libsql release
`libsql-server-v0.24.32`, sha256 `71720fc8…336879f`, matching its
published checksum), reached as `libsql://127.0.0.1:PORT?tls=0`. That
leg is a scratch suite and is not committed, because CI has no sqld.
**Positive control on unchanged code (`f7e836da1`):**
- sqld: each of the three doors creates `title` and `tags` (both
`storage.notNull`) with `notnull=0`, and `detectManagedDrift` answers
`tags:tighten_not_null` and `title:tighten_not_null`. The ADD COLUMN
path behaves the same. A populated table accepts the "notNull" column
silently, as nullable.
- `file:` client: the remote reader and the local face reading the same
file both report the same two `tighten_not_null` entries. A table the
local face created reads `[]`.
**On the fix (sqld re-run at `5a41f2167`):** every door gives
`title:nn=1 tags:nn=1 plain:nn=0` and drift `[]`, on CREATE TABLE and on
ADD COLUMN. A populated table refuses with `ALTER TABLE "…" ADD COLUMN
"added" TEXT NOT NULL - SQLITE_UNKNOWN: SQLite error: Cannot add a NOT
NULL column with default value NULL`, and the column stays absent.
**Ablation:** `scripts/ablation-replace.mjs` in WRAP mode replaced
`return declaresColumnNotNull(field) ? … NOT NULL … : colType;` with
`return colType;` (anchor 1 to 0, blob `039d3cd8` to `b133667a`), and
the new suite ran while mutated. Result: **10 of 10 red**. The restore
is proven: the blob equals HEAD's `039d3cd8` and `git diff HEAD` is
empty. The test reaches the subject through relative imports, so it
reads `src/` and no build sits between the mutation and the run.
## `turso-remote-not-null.test.ts`, 10 cases
1. CREATE TABLE through `syncSchemasBatch` / `syncSchema` /
`initObjects`: the remote `notnull` flags equal the local face's for the
same declarations (`title`, `amount`, `active` and multi-value `tags`
are 1; `required`-only `note`, `notNull: false` and plain are 0). The
remote and the local reader both report drift `[]`.
2. ADD COLUMN onto an existing empty table, through the same three
doors: the same result.
3. Enforcement: an insert that leaves `title` NULL is refused by the
database on both faces, and no row lands.
4. A populated table with no DEFAULT: both faces refuse (`Cannot add a
NOT NULL column with default value NULL`). The remote `syncSchema`
refusal names `ALTER TABLE "rows_t" ADD COLUMN "added" TEXT NOT NULL`,
and no nullable column is slipped in.
5. A **recorded asymmetry**, described in the next section.
## ADD COLUMN on a table that already holds rows
Once a table has a row, SQLite refuses `ADD COLUMN … NOT NULL` without a
non-NULL DEFAULT. An empty table accepts it. Both were measured on
libSQL 3.45.1 and on sqld 0.24.32. The local face sends the same
statement and gets the same refusal, measured as `alter table … add
column … text not null - Cannot add a NOT NULL column with default value
NULL`. So that object's sync fails loudly on both faces, and the engine
reports it per object: `ObjectQLPlugin` replays a failed batch per
object at warn first, so only the failing object loses its DDL. The `NOT
NULL` is never dropped to let the statement pass.
One case still differs, and it is about defaults rather than
nullability. The remote face emits no declared column DEFAULT for any
field, while `createColumn` emits a literal `defaultValue`
(`applyDeclaredColumnDefault`). So a `storage.notNull` field with a
literal `defaultValue`, added to a populated table, is accepted locally
(existing rows take the default) and refused here. Case 5 pins that as a
named asymmetry, to be inverted when this face emits declared defaults.
Whether it should, and from which shared source, goes back to the
dispatching seat as an open question. A third copy of the default rules
(the CLI's `declaredColumnDefault` is already the second) is a choice
for that seat, not for this PR.
## Not in this PR
- **Data at rest.** A remote database created before this release keeps
its nullable columns. An additive sync never tightens an existing
column, so that database's drift report still names those columns
`tighten_not_null`. No existing database is migrated here. That is why
the caveat stays, and why the `OS_ARTIFACT_URL` boot gate still does not
act on a remote report (#23848 is the other precondition).
## Acceptance notes
- **File surface.** The claim's surface was amended in 6213866355 to
cover the `turso-driver.ts` NOT NULL sentences. The caveat's existing
pin (`turso-remote-drift-detection.test.ts`, which expects the text to
contain `NOT NULL`) still holds.
- **Stale CLI comment.**
`packages/cli/src/utils/artifact-boot-migration.ts` has a code comment
whose present tense is now stale in the same way: "its schema sync does
not create a declared NOT NULL". It is a comment in another package, so
it is not edited here. carrier: #23848, which reworks that gate's remote
branch.
- **The predicate has two spellings.** driver-sql's
`declaresColumnNotNull` is module-private, so this transport spells the
same question itself. The new test pins their parity: remote `notnull`
flags equal the local face's for one declaration set. Exporting the
driver-sql predicate would give it one source; noted, not filed.
- **Builtin timestamp defaults differ by face (observation, reach not
measured).** From the dev's measurement run at `f7e836da1`: the remote
CREATE TABLE gives `created_at` / `updated_at` `TEXT DEFAULT
(datetime('now'))` (the zone-naive `YYYY-MM-DD HH:MM:SS` form), while
the local face gives `DEFAULT strftime('%Y-%m-%dT%H:%M:%fZ', 'now')`.
Whether any write path leaves those columns to the database default was
not measured, so this is noted, not filed. (Added by the `domain:engine`
seat 2 from the dev's report.)
- **Conformance ledger:** 55 covered, 0 DEBT, 0 exempt, both before
(`f7e836da1`) and after (`fcc038d2f`).
## Tests (head `fcc038d2f`)
- `pnpm --filter @objectstack/driver-turso test`: 91 files, 2425 passed,
33 skipped (VERDICT command-exit 0).
- `pnpm --filter @objectstack/driver-turso typecheck`: exit 0. `tsc
--listFiles` includes all 91 test files, the new one among them.
- eslint over the 3 changed `.ts` files: exit 0.
## Gates (head `fcc038d2f`)
- `node scripts/pm/dispatch-gates.mjs --commands` derived 67 families,
and all 67 exit 0. `--ran` reports 67 derived, 67 run, 0 NOT-MEASURED, 0
UNRUN.
- The four artifact-roster families whose roster sits in a directory
this diff touches were also run, and each exits 0:
`check-changeset-fixed`, `check:authz-resolver`,
`check:error-code-casing`, `check:filter-alias-parity`.
- The declared wide-population families, the type-check lanes and the
rest of the full battery are CI's to run.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01TQLy3Yq6vF4SDpTXdr3Uwi)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent 570f01d commit 2034975
4 files changed
Lines changed: 344 additions & 9 deletions
File tree
- .changeset
- packages/drivers/driver-turso/src
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
99 | 99 | | |
100 | 100 | | |
101 | 101 | | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
102 | 113 | | |
103 | 114 | | |
104 | 115 | | |
| |||
2098 | 2109 | | |
2099 | 2110 | | |
2100 | 2111 | | |
2101 | | - | |
2102 | | - | |
| 2112 | + | |
2103 | 2113 | | |
2104 | 2114 | | |
2105 | 2115 | | |
| |||
2230 | 2240 | | |
2231 | 2241 | | |
2232 | 2242 | | |
2233 | | - | |
2234 | | - | |
| 2243 | + | |
2235 | 2244 | | |
2236 | 2245 | | |
2237 | 2246 | | |
| |||
2396 | 2405 | | |
2397 | 2406 | | |
2398 | 2407 | | |
2399 | | - | |
2400 | | - | |
| 2408 | + | |
2401 | 2409 | | |
2402 | 2410 | | |
2403 | 2411 | | |
2404 | 2412 | | |
2405 | 2413 | | |
2406 | 2414 | | |
2407 | 2415 | | |
| 2416 | + | |
| 2417 | + | |
| 2418 | + | |
| 2419 | + | |
| 2420 | + | |
| 2421 | + | |
| 2422 | + | |
| 2423 | + | |
| 2424 | + | |
| 2425 | + | |
| 2426 | + | |
| 2427 | + | |
| 2428 | + | |
| 2429 | + | |
| 2430 | + | |
| 2431 | + | |
| 2432 | + | |
| 2433 | + | |
| 2434 | + | |
| 2435 | + | |
| 2436 | + | |
| 2437 | + | |
| 2438 | + | |
| 2439 | + | |
| 2440 | + | |
| 2441 | + | |
| 2442 | + | |
| 2443 | + | |
| 2444 | + | |
| 2445 | + | |
| 2446 | + | |
| 2447 | + | |
| 2448 | + | |
| 2449 | + | |
| 2450 | + | |
| 2451 | + | |
| 2452 | + | |
| 2453 | + | |
| 2454 | + | |
| 2455 | + | |
| 2456 | + | |
| 2457 | + | |
| 2458 | + | |
| 2459 | + | |
| 2460 | + | |
| 2461 | + | |
| 2462 | + | |
| 2463 | + | |
| 2464 | + | |
| 2465 | + | |
| 2466 | + | |
| 2467 | + | |
| 2468 | + | |
| 2469 | + | |
| 2470 | + | |
| 2471 | + | |
| 2472 | + | |
| 2473 | + | |
| 2474 | + | |
| 2475 | + | |
| 2476 | + | |
| 2477 | + | |
| 2478 | + | |
| 2479 | + | |
| 2480 | + | |
| 2481 | + | |
| 2482 | + | |
| 2483 | + | |
| 2484 | + | |
| 2485 | + | |
| 2486 | + | |
| 2487 | + | |
2408 | 2488 | | |
2409 | 2489 | | |
2410 | 2490 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
603 | 603 | | |
604 | 604 | | |
605 | 605 | | |
606 | | - | |
607 | | - | |
| 606 | + | |
| 607 | + | |
| 608 | + | |
| 609 | + | |
608 | 610 | | |
609 | 611 | | |
610 | 612 | | |
| |||
615 | 617 | | |
616 | 618 | | |
617 | 619 | | |
618 | | - | |
| 620 | + | |
| 621 | + | |
619 | 622 | | |
620 | 623 | | |
621 | 624 | | |
| |||
0 commit comments