Repository navigation
Commit e35c40a
Fixes #21113
Clause-②: yes (widening)
## Patch round 1 (review 5930100091)
The one blocking item was `@objectstack/driver-sql` at `patch` beside a
root `type` re-export (`AutoNumberReservation`). Branch taken: the
re-export is dropped and the two re-seed overrides in `turso-driver.ts`
are typed `Parameters` indexed over `SqlDriver`'s
`autoNumberValueExists` (its second parameter) and
`resyncSequenceToDataMax` (its first) — the premise (the protected
member's parameter type is reachable from the subclass without the
export) measured true: `driver-sql` typecheck exit 0, `driver-turso`
typecheck exit 0, no structural copy. `driver-sql`'s export list is
byte-identical to `main`, so `patch` stands; the changeset sentence that
named the export now reads "No export is added". `origin/main` merged
again (`fde553c50`, which also lands the `agentGuidance: false` opt-out
of #21151); re-run on the merged tree: closure build exit 0, both
typechecks exit 0, `driver-turso` `82 passed (82)` files / `2218 passed
| 33 skipped` (83 files / 2219 with the throwaway probe present),
`driver-sql` autonumber suites `66 passed | 2 skipped`, and the
changeset, ADR-0087, nul-bytes and published-files gates all exit 0.
Finding (b) was measured on the remote face as the review asked and
reproduces; it is in the round-1 report on #21113 for the seat to file.
## What this does
Disposition A of the remote-autonumber question, executed: a `create`,
`bulkCreate` or `upsert` on the Turso REMOTE transport that leaves an
`auto_number` slot empty now gets a generated value, issued atomically
in the database from the same persistent `_objectstack_sequences`
counter the local and embedded-replica faces draw from.
`supports.autonumber` stays `true` on the remote face and is now
honoured. The `NOT_IMPLEMENTED` / 501 refusal is gone; its suite is
converted into the generation suite.
Why now: the appetite door the refusal left shut (「for want of measured
demand」) was met by objectstack-ai/cloud#2531 — `POST
/api/v1/data/crm_account` answers `501` on a hosted tenant, whose
database is on this transport, so no object declaring an `auto_number`
field could get a new record on the hosted product. That card carries
the cloud half, `Blocked-by:` this one.
## How, and what is shared rather than copied
**Where the slots are filled.** On the driver, one layer above the
statement builder, exactly where the refusal was raised:
`RemoteTransport.create(object, data)` takes no schema and caches none
(pinned), while `TursoDriver` holds `autoNumberFields` from remote
schema sync. The remote `create` / `upsert` / `bulkCreate` call the
inherited `SqlDriver.fillAutoNumberFields` on a copy of the caller's
row, then hand the filled row to the transport, which builds the same
`INSERT` it built for a caller-supplied number.
**One semantics (H3).** The rules that decide WHICH counter a value is
drawn from and WHERE a cold counter starts are `SqlDriver`'s and are
called, never copied. Lifted into `protected` members in the sequence
region of `sql-driver.ts` (`:7532`–`:8190` on this head; nothing else in
that file): `resolveSequenceTenantId` (replaces three inline copies),
`defineSequencesTable` (the table's one definition — the remote face
compiles it to text with the connection-less Knex it already holds and
sends that), `maxAutonumberCounter` + `escapeLikePrefix` (the bootstrap
reading, suffix included, split from the Knex statement that fetches the
values), `sequencesTableName`, `autoNumberCollisionRetries`. The two
re-seed overrides take their parameter types through `Parameters`
indexed over `SqlDriver`'s protected members, so `driver-sql`'s export
list is unchanged. The format rendering and precedence were already
shared (`resolveAutonumberFormat` at registration, `renderAutonumber` /
`missingFieldValues` in `fillAutoNumberFields`), and are reached by
calling that method.
**What is this face's own: how the counter moves (H2).**
`TursoDriver.getNextSequenceValue` routes by transport. Local and
replica keep the inherited Knex transaction. Remote moves the counter in
ONE statement:
```
warm: UPDATE _objectstack_sequences SET last_value = last_value + 1 WHERE key_hash = ? RETURNING last_value
cold: scan the data table's MAX in JS by the shared reading, then
INSERT … VALUES (…, max + 1) ON CONFLICT (key_hash) DO UPDATE SET last_value = last_value + 1 RETURNING last_value
```
Each is a single SQLite statement serialised by the database's write
lock across connections and processes; no in-process state takes part
(`remoteSequencesTableEnsured` remembers only that the TABLE exists).
Two cold writers both scan and both INSERT; one wins the row, the
other's `ON CONFLICT` arm increments it.
H2 falsified on one point, reported rather than followed: the PM
expected the seed folded into the statement as a SQL `MAX(…)`. It is
not, because the bootstrap reading strips a declared suffix and reads
the digit run after the prefix (`readAutonumberCounter`, #6468), which a
dialect expression would be a second copy of — the exact collision H3
forbids. A Hrana batch was not needed either: the JS reading has to sit
between the scan and the insert, and both insert arms are atomic on
their own.
**The legs (H4).** `create`, `bulkCreate` and `upsert` all generate.
`bulkCreate` on the remote face now runs through the driver's own
`create` per row, which is what `RemoteTransport.bulkCreate` always did
underneath (it loops the transport's `create`: one INSERT and one
read-back per row, never one statement — so the old comment's
「all-or-nothing on this transport too」 was not true, and the statements
sent and the mid-batch failure state are unchanged). `upsert` reserves
before the statement, as `SqlDriver.upsert` does, and names the
autonumber columns to the transport as insert-only
(`RemoteTransport.upsert` gains an optional fifth argument, schema-free:
WHICH columns, not WHY), so a merge keeps the number already in the row
(#7011) and the insert leg — the one #7099 recorded as writing NULL —
gets its number. The reservation a merge does not use is a gap, never a
renumbering, the local face's rule byte for byte (pinned: seeded 42,
merge, next create is 44). `#7099`'s post-write warning is removed with
the leg it reported.
**Legacy shape and collision re-seed (H5), decided from the code.** The
pre-`key_hash` table: `SqlDriver.ensureSequencesKeyHashShape` rebuilds
it through a live Knex connection, which this face has none of, and a
raw-SQL rewrite would be a second copy of a migration; keying by the
legacy `(object, tenant_id, field)` rule would be a second keying rule
beside the shared one. So it is **refused**, `DATABASE_ERROR` / 500 with
the remedy in the message (open the database once through the local or
replica face), not cached as a verdict, caller-supplied numbers still
written. It cannot arise on a database this face created. The #5495
re-seed is **needed** on this face — the bypass paths that create a
stale counter (`isSystem` seed replay, `preserveAudit` import) are
exactly the seed/import paths, and `bulkCreate` is their common door —
and it is carried: `autoNumberValueExists` and `resyncSequenceToDataMax`
are overridden for the remote face (the forward-only move is one atomic
UPDATE guarded by `last_value` being below the observed MAX), and
`collidingAutoNumberReservations` is the inherited discriminator, so a
duplicate on a value the caller typed stays the caller's 409 (pinned).
**H6.** `Clause-②: yes (widening)` holds as measured: a create refused
501 today is accepted; no spec key moves; no export is added (the six
lifted members are `protected`, and the overrides are typed through
indexed types rather than a root re-export); `RemoteTransport.upsert`
gains an optional parameter. Changeset: `@objectstack/driver-turso`
minor, `@objectstack/driver-sql` patch.
## Tests
Head for every number below: `52ab7ad76` (the merge of `origin/main`
`1bd14c984`), unless stated.
- **Converted, not deleted:** `turso-remote-autonumber-refusal.test.ts`
→ `turso-remote-autonumber-generation.test.ts` (`git mv` then rewritten;
git's rename detection does not fire at this similarity, so the diff
reads D+A — the three-face structure, the controls, the `update()` /
plain-object / caller-supplied cases and the layer pins are the refusal
suite's own). 27 cases: generation on every leg incl. the #7099 leg;
caller-supplied numbers, merge, update, plain object untouched; the
two-face block (local Knex face and remote native-libsql face on ONE
file: numbers interleave 1,2,3,4 on one `key_hash` row equal to the
local face's `sequenceKeyHash`; per-tenant buckets; the #6468 suffix
seed read as 5 not 52026; the `{field}` refusal is the same sentence on
both faces and writes nothing); the #5495 re-seed on `create` and
`bulkCreate`; the caller's own 409; the legacy-shape refusal with `code`
+ `status`; local and replica still issue; `RemoteTransport.prototype`
still has no autonumber member and `create.length === 2`;
`supports.autonumber === true`.
- **The two boundary pins rewritten, not deleted**
(`turso-autonumber-resync.test.ts`,
`turso-autonumber-batch-resync.test.ts`): the transport-surface probe
verbatim, and the refusal half replaced by generation + re-seed on the
remote face.
- **Cross-process pin** `turso-remote-autonumber-concurrency.test.ts`:
two child processes (`tsx` over the package source, as `packages/spec`'s
process-boundary pins do; `tsx` added as a devDependency of
`driver-turso`), each with its own `@libsql/client` native `file:`
connection to one database file and its own remote-mode `TursoDriver`,
released together by a file barrier; 2 × 200 creates per round → 400
distinct, exactly 1..400, strictly increasing per writer, no failed
write, the table and the counter row agreeing. Those are the hard pins,
asserted on every round. The overlap EVIDENCE (the writer changes at
least twice in the global order) ends the loop on the first round that
shows it; after three rounds without one the test is skipped with a note
that says NOT MEASURED — because whether two processes overlap is the
scheduler's decision, measured here once as a strictly serial run (1
change) under another seat's full-package run on this shared box.
Final-head runs: 3/3 green standalone, measuring `400 writes · 400
distinct` with 31, 21 and 23 writer changes; inside the full package
run, 9. **Said plainly:** no `sqld` runs in this container; what is
measured is two processes on libSQL's engine under its write lock, not
an HTTP server.
- Driver-conformance ledger (lane commitment): identical before and
after — `50 covered cell(s), 0 in the DEBT ledger, 0 exempt`; the matrix
has no autonumber cell for any driver, so none moved.
Runs, all under `scripts/pm/os-verify-lock.sh` after the merge: `pnpm
--filter '@objectstack/driver-turso^...' build` exit 0 · `driver-sql`
typecheck exit 0 · `driver-turso` typecheck exit 0 · `driver-turso`
`vitest run`: `82 passed (82)` files, `2218 passed | 33 skipped (2251)`
· `driver-sql` `vitest run` in two halves of 108 files: `103 passed | 5
skipped`, `1389 passed | 100 skipped`; `102 passed | 6 skipped`, `1914
passed | 88 skipped` (every file; the autonumber-named nine also run on
their own: `66 passed | 2 skipped`).
## Reverse verification (predicted before each run; mutation and restore
proven on disk by `scripts/ablation-replace.mjs`, blob == HEAD and `git
diff HEAD` empty after each leg)
1. **Merge-set exclusion dropped** (`RemoteTransport.upsert` ignores
`insertOnlyColumns`). Predicted: exactly the two `[#7011]` merge pins
red. Measured: `2 failed | 25 passed`, both merge pins, `expected
'CASE-00043' to be 'CASE-00042'` and `expected 'CASE-00099' to be
'CASE-00042'` — the renumbering the exclusion exists to prevent.
2. **Warm path made non-atomic** (SELECT `last_value`, then UPDATE to
`+1`). Predicted: the distinct count in the concurrency pin drops below
N; the stub-backed generation suite stays green. Measured — **direction
reversed, kept rather than tidied**: the generation suite stayed green
(`27 passed`) as predicted, but the concurrency pin went red on its 「no
write failed」 assertion, not on the distinct count: the fixture's
tenant-scoped unique index refused the second row carrying a duplicated
number, and the forward-only re-seed could not outrun a writer that kept
reading the same stale value, so writes failed with `SQLITE_CONSTRAINT:
UNIQUE constraint failed: index
'uniq_crm_case_organization_id_case_number'` — 2 of 3 runs with no gap
between the two statements, 3 of 3 with a 1 ms gap (the order of one
HTTP round trip). On a declared-unique column a counter that hands out a
number twice surfaces as refused writes before duplicate rows; recorded
in the pin's docblock.
3. **Shared bootstrap reading zeroed** in `driver-sql`
(`maxAutonumberCounter`, reached through `dist/`: rebuilt,
`ablation-dist-preflight` present in 2 built files, exit 0). Predicted:
the bootstrap and re-seed pins red on BOTH faces. Measured: `10 failed |
25 passed` across the three suites — generation:
explicit-then-continues-above, merge-then-44, #6468 suffix, re-seed
`create`, re-seed `bulkCreate`; resync: LOCAL and REMOTE re-seed;
batch-resync: LOCAL `bulkCreate`, LOCAL `upsert`, REMOTE batch. Restore:
rebuilt, preflight `--absent` exit 0. A first attempt of this ablation
was a null operation (the tool refused because the replacement restated
the anchor; the preflight correctly reported the marker absent and that
green run is void) and was redone with a different anchor.
## Gates
Derived with `node scripts/pm/dispatch-gates.mjs --commands --repo
objectstack-ai/objectstack` on `52ab7ad76` (no paths; 73 commands). Run
on the final tree (`2c9d2985a`, every file identical to the commit): the
73 derived commands — **72 exited 0**, incl. `check:driver-conformance`,
`check:query-options-erasure` (holds: 67 unswept non-test sites, none
new — my first head had added one `as any` on a `find()` query in the
new concurrency pin, retyped as `DriverQuery`),
`check:test-source-alias`, `check:undeclared-dep-imports`,
`check:doc-authoring`, `check:nul-bytes`, `check:lean-entry-closure`
(measured after building `@objectstack/objectql`: 15 packages, 201/199
modules, admitted set held), `check:type-check-debt` (399.5 s
re-measure, none above record), `check:pm-skill-ratchet` (0 with
AGENTS.md at the merge-base blob — the turbo 2.11.5 agent-rules block,
#21146, had rewritten it once in this worktree and is restored).
`check:dts-closure` exited 1 on its first final-tree run naming
`plugin-email` and `plugin-security` (neither in this diff nor in its
closure) with `dist/index.d.ts` absent at that moment; a forced rebuild
of the two and a rerun exits 0 (71 packages, 167/167 declarations) —
concurrent turbo activity in the same tree, not this change. **NOT
MEASURED (1):** `check:dual-build-cjs-loads` exits 3 PREREQUISITE NOT
MET locally (78 packages without `dist/` in this worktree) — CI's.
`dispatch-gates --ran`: 73 derived, 72 run, 1 NOT-MEASURED declared, 0
UNRUN (exit 0). Seven further families (`agent-test-spelling`,
`docs-audit-scope`, `pm-governed-merges`, `pm-governed-prose`,
`pm-skill-id-lint`, `pm-skill-ratchet`, `required-contexts`) were
derived only while the turbo block sat in AGENTS.md; they were run
anyway and all exit 0. The derivation itself still lists what no local
run covers: the 53 artifact-roster families, 11 wide-population
families, 6 path-scheduled CI jobs and the type-check lanes.
`pnpm lint` narrowed, as a measurement: ① population = the lint script's
own `eslint . --no-inline-config`, under one `eslint.config.mjs`; ②
`--format json` on the changed source files: 9 files, 0 errors, 0
warnings; ③ invariance: the config enables no type-aware linting for any
file (`eslint.config.mjs` states it, no `parserOptions.project`), so
this diff moves no untouched file's verdict.
## Acceptance notes
- Both faces: SQLite's `LIKE` has no escape character unless `ESCAPE` is
declared, and the shared `escapeLikePrefix` writes backslashes. The
remote statement declares `ESCAPE '\'`; the local Knex builder in
`scanMaxNumericTail` does not (knex emits `like ?` with the escaped
binding and no `ESCAPE`, measured on the sqlite and pg dialects), so on
the local SQLite faces a format whose prefix contains `_` or `%` seeds
from 0 (measured: `SO\_%` matches nothing on SQLite without `ESCAPE`).
Pre-existing, outside this card's class; filed in the report as a
finding, not fixed here.
- `RemoteTransport.upsert` with caller `conflictKeys` leaves `id` in the
merge set (`"id" = excluded."id"`) — the #8622 shape the local face
closed. Not touched; noted for the lane.
- The README's remote refusal table loses its 「Record numbers」 row;
`content/docs/**` carries no statement of the remote refusal (grepped).
## 维护者速读(草稿)
- **改了什么:** Turso 远程传输面上,`create` / `bulkCreate` / `upsert` 留空的
`auto_number` 字段现在会生成编号,编号来自与本地面相同的持久计数表,在数据库里原子递增;原先的 501 拒绝改为生成。
- **为什么改:** 托管产品的租户库全部走远程传输,任何声明了 `auto_number` 字段的对象都无法新建记录(cloud#2531
实测:HotCRM 建客户答 501)。这正是 #6944 当初留下的「等实测需求」的门,现在门开了。
- **风险与代价(含回滚):**
多进程并发只靠数据库写锁串行,无进程内计数;两个面共用一张计数表、一套键规则,已在同一数据库文件上对拍。遗留形状的计数表在远程面上明确拒绝、不迁移。回滚即还原本
PR;已生成的编号留在数据里,计数表形状与本地面一致。
- **席位意见:** (留空)
- **你要做的:** 批准后 cloud#2531 推进 pin 并在托管环境复验建客户。
---
_Generated by [Claude
Code](https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG)_
---------
Co-authored-by: Claude Code <noreply@anthropic.com>
1 parent f0cc16e commit e35c40a
13 files changed
Lines changed: 1587 additions & 840 deletions
File tree
- .changeset
- packages/drivers
- driver-sql/src
- driver-turso
- src
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7347 | 7347 | | |
7348 | 7348 | | |
7349 | 7349 | | |
| 7350 | + | |
| 7351 | + | |
| 7352 | + | |
| 7353 | + | |
| 7354 | + | |
| 7355 | + | |
| 7356 | + | |
| 7357 | + | |
| 7358 | + | |
| 7359 | + | |
| 7360 | + | |
| 7361 | + | |
| 7362 | + | |
| 7363 | + | |
| 7364 | + | |
| 7365 | + | |
| 7366 | + | |
| 7367 | + | |
| 7368 | + | |
| 7369 | + | |
| 7370 | + | |
| 7371 | + | |
| 7372 | + | |
| 7373 | + | |
| 7374 | + | |
| 7375 | + | |
| 7376 | + | |
| 7377 | + | |
| 7378 | + | |
| 7379 | + | |
| 7380 | + | |
| 7381 | + | |
| 7382 | + | |
| 7383 | + | |
| 7384 | + | |
| 7385 | + | |
| 7386 | + | |
| 7387 | + | |
| 7388 | + | |
| 7389 | + | |
| 7390 | + | |
| 7391 | + | |
| 7392 | + | |
| 7393 | + | |
| 7394 | + | |
| 7395 | + | |
| 7396 | + | |
| 7397 | + | |
| 7398 | + | |
| 7399 | + | |
7350 | 7400 | | |
7351 | 7401 | | |
7352 | 7402 | | |
| |||
7356 | 7406 | | |
7357 | 7407 | | |
7358 | 7408 | | |
7359 | | - | |
7360 | | - | |
7361 | | - | |
7362 | | - | |
7363 | | - | |
7364 | | - | |
7365 | | - | |
7366 | | - | |
7367 | | - | |
7368 | | - | |
7369 | | - | |
| 7409 | + | |
7370 | 7410 | | |
7371 | 7411 | | |
7372 | 7412 | | |
| |||
7497 | 7537 | | |
7498 | 7538 | | |
7499 | 7539 | | |
7500 | | - | |
7501 | | - | |
| 7540 | + | |
7502 | 7541 | | |
7503 | 7542 | | |
7504 | 7543 | | |
7505 | 7544 | | |
| 7545 | + | |
| 7546 | + | |
| 7547 | + | |
| 7548 | + | |
| 7549 | + | |
| 7550 | + | |
| 7551 | + | |
| 7552 | + | |
| 7553 | + | |
| 7554 | + | |
| 7555 | + | |
| 7556 | + | |
| 7557 | + | |
| 7558 | + | |
| 7559 | + | |
| 7560 | + | |
| 7561 | + | |
| 7562 | + | |
| 7563 | + | |
| 7564 | + | |
7506 | 7565 | | |
7507 | 7566 | | |
7508 | | - | |
7509 | | - | |
| 7567 | + | |
7510 | 7568 | | |
7511 | 7569 | | |
7512 | 7570 | | |
| |||
7647 | 7705 | | |
7648 | 7706 | | |
7649 | 7707 | | |
7650 | | - | |
| 7708 | + | |
7651 | 7709 | | |
7652 | 7710 | | |
7653 | 7711 | | |
| |||
7907 | 7965 | | |
7908 | 7966 | | |
7909 | 7967 | | |
7910 | | - | |
7911 | | - | |
| 7968 | + | |
7912 | 7969 | | |
7913 | 7970 | | |
7914 | 7971 | | |
| |||
7942 | 7999 | | |
7943 | 8000 | | |
7944 | 8001 | | |
7945 | | - | |
7946 | | - | |
| 8002 | + | |
7947 | 8003 | | |
7948 | 8004 | | |
7949 | 8005 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
130 | 130 | | |
131 | 131 | | |
132 | 132 | | |
133 | | - | |
134 | 133 | | |
135 | 134 | | |
136 | 135 | | |
| |||
147 | 146 | | |
148 | 147 | | |
149 | 148 | | |
150 | | - | |
151 | | - | |
152 | | - | |
153 | | - | |
154 | | - | |
155 | | - | |
156 | 149 | | |
157 | 150 | | |
158 | 151 | | |
159 | 152 | | |
160 | 153 | | |
161 | 154 | | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
162 | 173 | | |
163 | 174 | | |
164 | 175 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
| 50 | + | |
50 | 51 | | |
51 | 52 | | |
52 | 53 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1595 | 1595 | | |
1596 | 1596 | | |
1597 | 1597 | | |
| 1598 | + | |
| 1599 | + | |
| 1600 | + | |
| 1601 | + | |
| 1602 | + | |
| 1603 | + | |
| 1604 | + | |
| 1605 | + | |
| 1606 | + | |
1598 | 1607 | | |
1599 | 1608 | | |
1600 | 1609 | | |
1601 | 1610 | | |
1602 | 1611 | | |
| 1612 | + | |
1603 | 1613 | | |
1604 | 1614 | | |
1605 | 1615 | | |
| |||
1618 | 1628 | | |
1619 | 1629 | | |
1620 | 1630 | | |
1621 | | - | |
| 1631 | + | |
1622 | 1632 | | |
1623 | 1633 | | |
1624 | 1634 | | |
| |||
0 commit comments