Skip to content

Commit e809f0b

Browse files
committed
Merge remote-tracking branch 'origin/main' into claude/issue-19332-g2b-remaining-g2-rows
2 parents 1603789 + 9801da1 commit e809f0b

68 files changed

Lines changed: 4552 additions & 240 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.changeset/20106-reclaim-space-full-freelist.md‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,4 @@ Clause-②: no
1414

1515
`SqliteWasmDriver` was already complete: its dialect steps every PRAGMA to the end (300 → 0 before and after this change).
1616

17-
On a file-backed database in WAL mode (the default) the database file shrinks once a checkpoint runs, and during the call the freed pages pass through the `-wal` file, which keeps its size until the last connection closes.
18-
1917
Nothing to migrate: `reclaimSpace()` keeps its signature, and a database whose `auto_vacuum` mode is not `INCREMENTAL` still reclaims nothing, as before.
Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
---
2+
"@objectstack/spec": minor
3+
---
4+
5+
feat(spec): declare the `$empty` filter operator — what 「is empty」 means, once, per field type — staged ahead of its executors (#20311)
6+
7+
Clause-②: yes (widening) — a declared operator slot and five exports are added. `FieldOperatorsSchema.parse({ $empty: true })` used to strip the undeclared key and now keeps it. The one refusal that comes with the declared type sits on a key nothing writes (see below).
8+
9+
**⚠️ Authoring `$empty` today is refused at query time.** The operator is declared but STAGED: it is deliberately absent from `FILTER_OPERATORS`, so no query executor answers it yet. A hand-written `{ "f": { "$empty": true } }` gets `INVALID_FILTER` / 400 from `driver-sql` (and the drivers that inherit its compiler), `driver-turso`'s remote transport, `driver-memory`, `driver-mongodb`, objectql `having` and the analytics `where` compiler; `READ_SCOPE_COMPILE_FAILED` / 500 (fail-closed) from the analytics read-scope SQL compiler; and `@objectstack/formula`'s write-side `matchesFilterCondition` answers `false` for every record, its fail-closed posture for an operator it has no arm for. Until each of those faces has its arm, write 「is empty」 with the view operator `is_empty`, which is unchanged.
10+
11+
**What the operator means.** Its description is the ruled per-type table (ruling B on #20311, spelled as an operator by ruling A on #20399):
12+
13+
| field type | `$empty: true` matches |
14+
|---|---|
15+
| text-like (`STRING_VALUE_TYPES`: text, textarea, email, url, phone, password, secret, markdown, html, richtext, code, color, signature, qrcode) | null or `''` |
16+
| multi-value (`isMultiValueField`: multiselect, checkboxes, tags, and select, radio, lookup, user, file or image with `multiple: true`) | null or `[]` |
17+
| every other type | null only |
18+
19+
`$empty: false` is the exact complement. A face that holds no field declaration (the formula matcher, objectql `having`) judges by the value: null, `''` and `[]` are empty.
20+
21+
**The one expansion every face calls**, exported from `@objectstack/spec/data`:
22+
23+
- `expandEmptyOperator(field)` — keyed on the field DEFINITION (type plus `multiple`), because a `lookup` is `null_only` and a `lookup` with `multiple: true` is `multi_value`. Returns one of the frozen `EMPTY_OPERATOR_ARMS` rows: `{ arm, emptyString, emptyList }` (`EmptyOperatorArm`, `EmptyOperatorExpansion`).
24+
- `isEmptyFilterValue(value, expansion?)` — the value-level half: with an expansion, the declared row; without one, the by-value reading for the declaration-free faces.
25+
26+
**What does not change.**
27+
28+
- The `is_empty` / `is_not_empty` view operators still lower to `{ "$null": true | false }`. A later change flips that lowering to `$empty` once every face answers it; no stored filter changes result in this release.
29+
- An empty list is still refused as an equality comparand: `{ "tags": [] }` and `{ "tags": { "$eq": [] } }` keep their refusal. The multi-value row lives in the operator precisely because it cannot be spelled as a lowered equality.
30+
- `FILTER_OPERATORS` is unchanged, so every executor that derives its accepted set from it (`driver-memory`'s gate among them) keeps refusing `$empty` rather than dropping it.
31+
32+
**One new refusal, on a key nothing writes.** A NON-boolean `$empty` (`"true"`, `1`, `null`) is refused where the declared boolean flags `$null` / `$exists` already are: at the operator slot, and at the save door (`FilterConditionSchema` and the analytics filter carriers that share its slot check), in the flags' own first sentence. `$empty` appears nowhere in this repository or in objectui's `main` before this change (0 occurrences in either).
33+
34+
**Stored sharing rules** (ruling B's landing measurement): the criteria sharing rules in this repository's examples and objectui's fixtures that use 「is empty」 are 0, and this release changes no lowering, so none changes result. Production sharing rules are NOT MEASURED: they are unreadable from here.
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
---
2+
"@objectstack/spec": minor
3+
---
4+
5+
`ComponentPropsMap` declares `action:button`, `action:group`, `action:menu`, `action:icon`, `element:definition-list` and `element:repeater` — six blocks in objectui's curated public vocabulary that had no row (#20371). Each row is strict from birth, with its key set measured from the objectui renderer's own read points at the `.objectui-sha` pin, not transcribed from objectui's `UIActionSchema`, the registrations' `inputs`, or this package's object-metadata `ActionSchema`.
6+
7+
Clause-②: yes
8+
9+
Six new declared rows on a published surface, and two types the `element:` vocabulary now answers for, so the accept set a consumer writes against grows. Nothing previously accepted by a declared row is refused and nothing is retired.
10+
11+
What changes at the authoring doors (`os validate` / `os build` / `os lint`):
12+
13+
- **`element:definition-list` and `element:repeater` are no longer refused as `component-type-unknown`.** Both sit inside the reserved `element:` namespace; with no enum member and no row, the vocabulary refused them although objectui registers, publishes and offers both in the Studio page designer. They join the `element:` vocabulary through their rows (no enum member), and a typo inside the namespace (`element:repeatr`) is still refused.
14+
- **The props gate now judges all six.** The four `action:*` types sat outside every reserved namespace, so an authored `properties` bag on them was skipped — a misspelled key parsed, stored and did nothing. Findings stay at the gate's existing warning tier.
15+
16+
Measured decisions worth knowing when you author these blocks:
17+
18+
- **`action:button` / `action:icon`** — `name` is optional (the renderer reads `name ?? label`). The executor is `actionType`; `type` inside `properties` is refused with a rename to `actionType` (on a page component `type` is the component itself). `visible` / `disabled` take a boolean, a CEL string or a `{ dialect, source }` envelope. The legacy `enabled` fallback and the host-only `autoTrigger` flag are refused with a prescription. `action:icon` reads no `size`. `objectName` names the object the action acts on (forwarded to the runner; omitted, the action acts on the page's object).
19+
- **`action:group` / `action:menu`** — `actions` is a LIST of action objects (a member's executor is its own `type`); a bare list of action names is refused. A member's `objectName` rides the member object; the containers themselves read no `objectName`. `action:group` reads no group-level `name`, so it is refused with a prescription. `variant` / `size` take the Button primitive's vocabulary; `primary` and `md` are accepted only on `action:button` (and `primary` on `action:icon`), where the renderer maps them.
20+
- **`element:definition-list`** — `items` of strict `{ term, description? }`; `columns` is the NUMBER `1` or `2` (the string `'2'` renders one column and is refused).
21+
- **`element:repeater`** — `object` is required; `filter` / `sort` take the family's one orthography (`ViewFilterRule[]`, `SortItem[]`); `fields` takes a field name or `{ field }` (an unrendered `label` is refused).
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
---
2+
"@objectstack/objectql": minor
3+
---
4+
5+
fix(objectql)!: a `progress` field's declared `min` / `max` are enforced on writes — a value outside them is refused with `min_value` / `max_value`, exactly as on `number` (#20386)
6+
7+
Clause-②: no (narrowing)
8+
9+
**BREAKING** — a narrowing of the write accept set on `@objectstack/objectql`, shipped as `minor` under the repo's launch-window convention (`check-changeset-no-major` refuses `major` until GA); the breaking-ness is carried by this banner and the ADR-0087 disposition, never by the level. Nothing an author writes changes spelling: `min` and `max` keep their keys, their type and their legality on every field type.
10+
11+
`FieldSchema.min` / `max` declare a check ("Checked on the WRITTEN value only") with no type exclusion, but the record validator returned for a `progress` field right after its finite-number check, above the bounds. So a `progress` field declaring `max: 100` stored `150`, and one declaring `min: 0` stored `-5`, with `201` on memory and SQLite, while a `number` field with the same bounds refused both. The bounds now bind on `progress` at the one place a write is judged.
12+
13+
**What a caller sees, before → after.** A `progress` write outside a declared bound: `201`, stored as sent → `400 VALIDATION_FAILED` with field code `max_value` (`constraint: { max }`) or `min_value` (`constraint: { min }`), nothing stored. That is the `number` field's answer, envelope for envelope, in all four locales. The REST create, batch, update and updateMany routes all answer it, and `validate` (the dry run) predicts it. A value inside the bounds, or on either bound (both are inclusive), writes exactly as before. Only a write that CARRIES the field is judged: a stored value outside a bound is never re-read and survives an update that does not send it.
14+
15+
The fix, when a write is refused: send a value inside the bounds, or widen or delete the field's `min` / `max` to match what it really holds.
16+
17+
⛔ Only the bounds. `scale` and `precision` stay unread on `progress`: each key's own contract names the types it binds on, and `progress` is in neither set, so `33.5` still writes into a `progress` field that declares `scale: 0`.
18+
19+
**Who is affected, measured** on `origin/main` `dc0ab6a2e`: the two example-app `progress` fields (`examples/app-showcase` `showcase_task.progress` and the field zoo's `f_progress`) both declare `min: 0, max: 100`, and every value their seeds and actions write (12 seed rows, one `progress: 100` action) is inside. The console's `progress` editor, objectui's `SliderField`, drives a Radix slider bounded by the field's declared `min` / `max`, so it cannot emit a value outside them.
20+
21+
<!-- adr-0087: not-required (no-migration-prescription) Nothing authorable moves: `min` and `max` keep their keys, their type (`z.number().optional()`) and their legality on every field type, `packages/spec` is untouched, and no stored metadata representation changes, so `objectstack migrate meta` has nothing to rewrite and the ledger has no row to gain. What narrows is the record validator's write accept set for values under bounds the field already declares, which is runtime behaviour, not an authored shape. The other categories are closed on facts: the package publishes (not `unpublished`); no ADR-0087 id is minted here and none covers this (not `registered` / `already-registered`); and runtime behaviour changes, not only a TypeScript declaration (not `runtime-interface-only` / `type-surface-only`). -->
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
---
2+
"@objectstack/driver-turso": minor
3+
---
4+
5+
`TursoDriver` in **remote** mode refuses a read over a missing table or a missing column with the same code the local mode answers, instead of answering "no rows" (#20424).
6+
7+
Clause-②: no (narrowing)
8+
9+
<!-- adr-0087: not-required (already-registered driver-sql-unresolvable-where-column-refused) That registered entry names `TursoDriver` among the `SqlDriver` subclasses whose reads now refuse an unresolvable column, and prescribes this change's remedy: name a column the table has, or run schema sync so a declared field exists as a column. The `aggregate` legs (a table that is really absent, a `groupBy` or aggregation column that is absent) are the same drifted-schema family with the same remedy, so no new migration entry is owed. -->
10+
11+
**BREAKING** — an accept-set narrowing on the remote face of `TursoDriver`'s read doors, shipped as `minor` under the launch-window convention (`check-changeset-no-major` refuses `major` until GA; breaking-ness is carried by this banner and the ADR-0087 disposition above, not by the level). Remote-face users meet this refusal for the first time here.
12+
13+
**FROM → TO.** A remote-face read (`aggregate`, `find`, `findOne`, `count`) that answered `[]` / `null` for a missing table or a missing column now refuses, as the local face does: `DATABASE_ERROR` / 500 for a table that is absent, `INVALID_FIELD` / 400 for a `groupBy` or aggregation column that is absent, `INVALID_FILTER` / 400 for a `where` column that is absent. **The fix:** run schema sync so the declared field has its column (or the object its table), or name a column the table has. `RemoteTransport.find` and `RemoteTransport.aggregate`, exported from the package root, now raise the backend's error where they answered `[]`.
14+
15+
**What was wrong.** Two catches in `RemoteTransport` read a backend "no such table" or "no such column" as an empty result. `aggregate` answered `[]` for both. `find` (and `findOne` through it) answered `[]` (`null`) for a missing column once its projection retry was spent, or when there was no projection to drop. So on a remote Turso database a schema drift or a missing table read as "there is no data", while the local mode of the same driver, over the same file, refused it. Measured with a local driver over the same libSQL file as the control, for a federated and for a managed object alike:
16+
17+
| read | local | remote before |
18+
|:--|:--|:--|
19+
| `aggregate` on a table that is really absent | `DATABASE_ERROR` / 500 | `[]` |
20+
| `aggregate` grouped by, or aggregating, a declared field whose column is absent | `INVALID_FIELD` / 400 | `[]` |
21+
| `aggregate` whose `where` names that field | `INVALID_FILTER` / 400 | `[]` |
22+
| `find` / `findOne` whose `where` names that field | `INVALID_FILTER` / 400 | `[]` / `null` |
23+
| `count` whose `where` names that field | `INVALID_FILTER` / 400 | `DATABASE_ERROR` / 500 |
24+
| `find` ordered by that field | the rows, unordered | `[]` |
25+
26+
**What changes, on the remote face only:**
27+
28+
- `aggregate`, `find`, `findOne` and `count` answer each row above the way the local face does. The backend's error is classified by the local face's own inherited seam, `SqlDriver.aggregateBackendFault`, and not by a second copy: an unresolvable column named by a `groupBy` or an aggregation is `INVALID_FIELD` / 400, one named by the `where` is `INVALID_FILTER` / 400, and anything else is `DATABASE_ERROR` / 500. The dialect text goes to the server log, never to the caller.
29+
- `find` keeps the local face's recovery ladder: a projection naming a column the table lacks is dropped first, then an ORDER BY on one, and the rows answer. A `where` is never dropped. Before, the ORDER BY rung was missing and the sort answered `[]`.
30+
- A refusal the transport raises while it compiles the statement (a filter or aggregate-vocabulary refusal, the timeout envelope) keeps its own code and status.
31+
- `RemoteTransport.find` and `RemoteTransport.aggregate`, used on their own, now raise the backend's error where they answered `[]`.
32+
33+
This is the refusal the registered migration entry `driver-sql-unresolvable-where-column-refused` already names for `driver-sql` "and its `TursoDriver` / `SqliteWasmDriver` subclasses": the remote face of `TursoDriver` now delivers it. **If a read now refuses for you:** the table or column it names is missing from the remote database. Run schema sync so the declared field has its column (or the object its table), or correct the name the query uses.
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
---
2+
'@objectstack/driver-sql': patch
3+
---
4+
5+
fix(driver-sql): `reclaimSpace()` returns the freed bytes from the SQLite `-wal` sidecar too, and never waits on another connection (#20426)
6+
7+
Clause-②: no
8+
9+
On a file-backed SQLite database in WAL mode, the default, `reclaimSpace()` returned the whole freelist but left the freed bytes in the `-wal` sidecar. At 25,754 free pages the database file went from 103,149,568 to 16,384 bytes while the `-wal` file went from 4,255,992 to 94,430,432 bytes, and it kept that size until the last connection closed. The lifecycle sweep calls this method after every sweep that deleted rows, and it reported the datasource as reclaimed.
10+
11+
On better-sqlite3 (`SqlDriver`, and `TursoDriver` in local mode) the vacuum now runs in chunks of a quarter of the connection's page cache, 1,000 pages at the default cache size, with a `PASSIVE` checkpoint after each chunk. One `TRUNCATE` checkpoint closes the call, taken with a busy timeout of 0, so it never waits on another connection. On the same database, file plus `-wal` goes from 107,405,560 to 16,384 bytes while the driver is still open.
12+
13+
When another connection holds a read transaction, the call still returns without waiting (47 to 66 ms measured; a `TRUNCATE` checkpoint that waits blocked the process for the connection's 5-second busy timeout). The pages are off the freelist, and their bytes leave the files at a later checkpoint. The call no longer grows the pair either: 107,405,560 bytes before and after, where the single statement grew it to 197,580,000.
14+
15+
A database in rollback-journal (`delete`) mode behaves as before. The remote `TursoDriver` route and `SqliteWasmDriver` are unchanged. Nothing to migrate: `reclaimSpace()` keeps its signature.

0 commit comments

Comments
 (0)