You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 7b1e4a4
Browse filesBrowse the repository at this point in the historyBrowse files
feat(spec,metadata-protocol): a stored filter the record-filter conversion leaves as stored is a TODO in os migrate meta --stored, not silence (#17321) (#20244)
Fixes#17321
Clause-②: yes
Delivers the rest of ruling `5644018752` item 2, the half that PR #20175
(`d7c024133e`) stopped at:
> 「A record carrying `$and` / `$or` / `$not` is passed through unchanged
and reported as a structured TODO naming the page/block and the
combinator — `os migrate meta --stored` prints the list; the operator
can answer "did it convert my row" from that output.」
This PR only adds reporting. Nothing is flattened. Nothing that was left
as stored starts converting. Every stored body is rewritten exactly as
it is on `main`: the entry's §1–§7 pins pass unchanged, and a new pin
compares the stack produced with a TODO sink to the one produced without
one.
Seam: `spec:ConversionTodoNotice (applyConversionsToStoredItem onTodo) →
runtime:metadata-protocol migrateStoredMetadata /
formatStoredMigrationReport (os migrate meta --stored, POST
/api/v1/meta/_migrate-stored)`
## The channel (`packages/spec/src/conversions/`)
- `CONVERSION_TODO_CODE` (`OS_METADATA_CONVERSION_TODO`),
`ConversionTodoDetail` (`path`, `from`, `reason`) and
`ConversionTodoNotice` add a third lane beside `onNotice` and
`onConflict`. The conflict twin is the precedent: the new lane has an
optional `ApplyConversionsOptions.onTodo`, and an entry reaches it
through `context.reportTodo`, exactly as it reaches
`context.reportConflict`. There is no thrown sentinel, no return-value
side channel and no global.
- `StoredConversionOptions` is `ApplyConversionsOptions` with only
`includeRetired` omitted, so `onTodo` reaches
`applyConversionsToStoredItem` with no new plumbing. Pinned in
`stored.test.ts`.
- The load-time consumers (`normalizeStackInput`, `validate`, `lint`)
are not wired. Only the stored pass passes a sink.
## The entry's decline branches (`registry.ts`,
`page-component-filter-record-to-rule-array` only)
The helpers now return `{ rules }` or `{ declined: why }` where they
used to return `undefined`. The verdict logic is unchanged: still
all-or-nothing, with the same accept set. Top-level `$` keys are now
judged before field keys, which moves only the reason: the combinator is
named even when a field key beside it would also decline. Every branch
that leaves a legacy form in place emits exactly one TODO. The reason
names the block (its type, plus its `id` when it has one) and what
blocks the rewrite. One pin per branch is in §8 of
`page-component-filter-record-to-rule-array.test.ts`:
| branch | reason names |
| --- | --- |
| record: a top-level combinator (`$and` / `$or` / `$not`, one or
several) | "carries the combinator `$or`", plus any other `$` key beside
it |
| record: a top-level `$` key that is not a combinator (`$text`) |
"carries the top-level key `$text`, which is not a field" |
| record: a `null` value | the key; the renderer skips it today, and an
`equals` rule would test for null |
| record: an array value, or a non-plain object value | the key and the
value kind |
| record: an empty operator object | the key |
| record: an operator outside the one table (`$null`, `$exists`,
`$regex`, `$Gt`) | the key and the operator |
| record: a nested object whose key is not an operator | the key and the
inner key |
| AST: an `and` / `or` group, or a list nesting one | "an `or` group" |
| AST: neither one comparison nor a flat list of them | the shape |
| AST: a comparison the AST refuses to lower (`["a",">"]`) | the
comparison and the first sentence of the AST's refusal |
| AST: an operator with no rule word (`like`, `ilike`) | the field and
the operator |
| door: a mapped rule that `ViewFilterRuleSchema` refuses | the rule and
the door's first issue |
| node: the block renders inline rows (`data: { provider: 'value' }`, a
`data` array, `staticData`) | the inline shape, and that the objectui
renderer this release pins cannot match a rule array on inline rows.
This is a limit of the pinned renderer, not a protocol fact:
objectui#10767 fixed it upstream, but `.objectui-sha` does not carry the
fix, so the decline stays. |
Branches argued NOT to emit:
- **A value that is not a legacy form at all**: an existing rule array,
`[]`, a string, a number, or a mixed list of a rule object and an AST
tuple (`isFilterAST` refuses the last). The conversion never claimed
these, and the door's own element-level refusal names them.
- **A `filter` on a component type outside the family**
(`record:related_list`), and **`defaultFilters` off `object-grid`**.
These are not this entry's doors.
- The lowered-shape guard (`!lowered || … || !(field in lowered)`)
shares the AST-operator reason. No input reached it separately in the
probes.
## The stored pass (`packages/metadata-protocol`)
- `convertStoredItemDetailed` forwards `onTodo` and returns `todos`.
- `migrateStoredMetadata`: a row carries its TODOs (`rows[].todos`, new
type `StoredMigrationTodo`) whatever its outcome. A row with nothing but
TODOs is now `skipped`, where it used to be `canonical`. Its reason says
that the chain rewrites nothing there, and why.
- `formatStoredMigrationReport` nests a `TODO` line (the conversion id,
the shape left as stored, the path and the reason) under each row
wherever the row is listed. It adds one closing `☐ TODO: N site(s) in M
row(s)` line and withholds "✓ Every row examined is already on protocol"
when any TODO exists. A report with no TODO renders byte-identically to
`main`.
**Clean-ness: the choice and why.** `storedMigrationClean` is unchanged
(`pending === 0 && failed === 0`). Its own doc defines the `skipped`
classes as "outside what a body-canonicalization pass can do … still
printed". It refuses to let them flip the verdict because no run of the
command could clear them ("a gate failing on a condition its own tool
has no lever for"). A TODO is exactly that, by ruling: the conversion
must not flatten. So TODOs never move the verdict in either direction. A
TODO-only row is `skipped`, and a row that also converts keeps the
outcome its notices give it. Exit semantics for rows without a TODO do
not move.
## Measured on the way: the two door kinds at the write path
The card, #20175's entry text and its changeset all say a record-form
filter left as stored "is refused at its `filter` door on its next
save". Through `saveMetaItem` that holds only for the binding: a
leftover in `dataSource.filter` (a declared key of the strict component
schema) fails the re-save. A leftover in `properties.filter` /
`properties.defaultFilters` (the open `properties` bag) is **re-saved
successfully**, because the component-props gate is advisory. Pinned in
`protocol.stored-migration.test.ts` ("MEASURED — the write path judges
the two door kinds differently"). So on `--apply` the mixed props-door
row is `rewritten`: its lossless filter persists, and the next run
reports the leftover as a `skipped` TODO row. The mixed binding-door row
is `failed`, and its TODO says why. My TODO text makes no refusal claim
("not the rule-array form its door declares"). The entry docblock and
`summary` I touch are corrected to match.
## Tests (numbers from the logs)
- `@objectstack/spec`: `vitest run --project local`, 542 files / 15964
passed (2 todo) at `23cd7bbed`. The conversion trio
(`page-component-filter-record-to-rule-array`, `stored`, `conversions`)
3 files / 313 passed at the final `7459968a6`. `pnpm typecheck` (tsc,
scripts and the test layer) exit 0 at `23cd7bbed`.
- `@objectstack/metadata-protocol`: full suite 189 passed / 3 skipped
files, 2706 tests at `23cd7bbed`. `protocol.stored-migration.test.ts`
30/30 at `7459968a6`. `pnpm typecheck` exit 0; its tsconfig includes
`src/**/*`, so the new test file is checked too.
- `@objectstack/runtime`: full suite in two shards at `7459968a6`: 140
files / 1790 tests, then 139 files / 2119 passed (1 skipped). `pnpm
typecheck` (tsc and the test layer) exit 0.
- `@objectstack/metadata-core` (conversion-layer importer): 16 files /
285 passed at `23cd7bbed`.
- `@objectstack/cli` (reads the report through
`formatStoredMigrationReport` / `storedMigrationClean`): not run
locally. No unit-layer test reads the stored report, and its one
stored-pass test, `meta.stored-flow-resolution.integration.test.ts`, is
flows-only (flows emit no TODO) and sits in the integration layer,
declared to CI.
**Ablation.** I removed the `onTodo` forwarding in
`convertStoredItemDetailed` through `scripts/ablation-replace.mjs`
(anchor 1→0, blob `46fff791a65d`→`d1b9bbfc0359`) and ran
`protocol.stored-migration.test.ts`: **4 failed / 26 passed**. The four
TODO pins went red (the combinator-only row read `canonical` again), and
both controls stayed green. The restore brought the blob back to HEAD
(`46fff791a65d`), and `git diff HEAD` is empty. No build or dist was
involved: the test imports `./protocol.js` from source.
## Gates
`node scripts/pm/dispatch-gates.mjs --commands --repo
objectstack-ai/objectstack` at `7459968a6` derived 109 commands.
Results: 106 exit 0 (`check:skill-examples` counted after building the
`client-react` closure it needs), 1 exit 1 (`check:empty-changeset`, see
below), and 2 exit 3 (`check:dual-build-cjs-loads` and
`check:type-check-debt`, which need every package's `dist`: NOT
MEASURED, and CI's). `--ran` reconciliation: 109 derived, 107 run, 2
NOT-MEASURED, 0 UNRUN, exit 0. `check:api-surface` and
`check:export-origins` are regenerated and show 3 added, 0 removed
(`CONVERSION_TODO_CODE`, `ConversionTodoDetail`,
`ConversionTodoNotice`). `check:dispatcher-error-vocabulary` passes with
the new foreign-vocabulary row.
## ⚠ A pending release note is corrected here: please confirm
`check:empty-changeset` is red on purpose. This is its DELIBERATE
CORRECTION class: this PR edits
`.changeset/17321-record-filter-d2-conversion.md` from #20175, which has
not been released yet.
1. Its sentence "`os migrate meta --stored` does not list these rows
yet: a row the conversion leaves as stored reports there as already on
protocol" is **made false by this PR**. It now reads "`os migrate meta
--stored` lists each filter left as stored as a TODO under its row."
2. Its sentence "…is refused at its `filter` door on its next save" is
**measured false** for the two `properties` doors (see the section
above). It now states the refusal per door kind.
Restoring the file from base would put a false sentence into the
release, so the gate stays red until a person confirms these two
corrections.
## Scope beyond the claimed file surface (declared)
- `content/docs/deployment/cli.mdx`: its `--stored` decline table says
it is "the operator-observable surface — what a run can actually
report". This PR makes a new row class observable, so without the new
row the table would become false. One row was added, and "exits `1`" is
scoped to "an old dialect this pass can convert". The
`migrateStoredMetadata` JSDoc names this table as its operator-facing
twin.
- `packages/metadata-protocol/src/index.ts`: one line re-exporting
`StoredMigrationTodo`, beside the `StoredMigrationNotice` it parallels.
- `.changeset/17321-record-filter-d2-conversion.md`: see above.
Not touched: `packages/cli`, `packages/metadata-core`, `protocol.ts`'s
`saveMetaItem` region, and `.objectui-sha`.
## Acceptance notes
- **`os migrate meta --stored` still ends in "Stored metadata is already
on protocol N — nothing to rewrite" under a TODO list.** The CLI's
closing line (`packages/cli/src/commands/migrate/meta.ts`, the `clean &&
!apply` branch) reads only `storedMigrationClean`. That is true for a
TODO-only run, and today already for a run whose only rows are
non-canonical-type skips. The formatter withholds its own "already on
protocol" line when a TODO exists, but the CLI line is outside this card
(`packages/cli`). It is reported to the seat as a finding.
- **The re-save refusal claim outside this PR's files**: the semantic
entry `18.element-data-source-and-object-block-filter-rule-array` and
its generated mirror in `migrations/registry.ts` say a stored
record-form filter "is refused at the `filter` door on its next save".
That is measured false for the two `properties` doors (see above). It is
reported to the seat as a release-text finding.
- **Dropping the inline-row decline once `.objectui-sha` carries
objectui#10767** is a follow-up the PM carries. This PR keeps the
decline and words its TODO as a limit of the pinned renderer.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01CiCTczDo7tGhafXjf61dUJ)_
---------
Co-authored-by: Claude <noreply@anthropic.com>
feat(spec,metadata-protocol): `os migrate meta --stored` lists every stored page filter the record-filter conversion leaves as stored, as a TODO naming the page, the block and why — the ADR-0087 D3 TODO channel (#17321, ruling B item 2)
7
+
8
+
**Clause-②: yes** — `@objectstack/spec` gains public exports (`CONVERSION_TODO_CODE`,
9
+
`ConversionTodoDetail`, `ConversionTodoNotice`, and the optional `ApplyConversionsOptions.onTodo`
10
+
and `ConversionContext.reportTodo`), and `@objectstack/metadata-protocol` gains
11
+
`StoredMigrationTodo` and `StoredMigrationRow.todos`. No door's accept set moves, and nothing
12
+
that was left as stored before starts converting: every stored body is rewritten exactly as it
13
+
was.
14
+
15
+
**What was silent.** The D2 conversion `page-component-filter-record-to-rule-array` leaves a
16
+
stored filter as stored wherever no lossless rule-array spelling exists — above all a record
17
+
carrying `$and` / `$or` / `$not`, which is never flattened. It emitted nothing for such a site,
18
+
and `os migrate meta --stored` reads conversion notices as its change signal, so a page whose
19
+
only legacy filter carried a combinator was reported as **already on protocol**.
20
+
21
+
**What it says now.** Each such site is a structured TODO (code `OS_METADATA_CONVERSION_TODO`)
22
+
carrying its path, the shape left in place, and a reason that names the block (its type, and its
23
+
`id` when it has one) and what blocks the rewrite — the combinator by name, the operator
24
+
(`$null`, `$exists`, an AST `like`), the null or array value, the rule the door would refuse, or
25
+
the inline rows the block renders. The stored pass lists them under their row, whatever the
26
+
row's outcome:
27
+
28
+
```text
29
+
⚠ 1 row(s) are outside this pass — each row's reason says why:
30
+
• page/pipeline_board [env-wide] — the conversion chain rewrites nothing here: it left 1 site(s) of this row as stored, …
31
+
TODO page-component-filter-record-to-rule-array: {"$or":[…]} left as stored at pages[0].regions[0].components[0].properties.filter — On the `object-kanban` block, this filter carries the combinator `$or`: …
32
+
☐ TODO: 1 site(s) in 1 row(s) are left as stored — no conversion can rewrite them without changing what they mean, so no run of this pass will. …
33
+
```
34
+
35
+
The same list is `rows[].todos` in `--json` and in the `POST /api/v1/meta/_migrate-stored`
36
+
report. A run with no TODO prints exactly what it printed before.
37
+
38
+
**Outcome and exit code.** A row whose only finding is TODOs has nothing to persist and is now
39
+
reported `skipped` (it was `canonical`). Like every other skip class it does not change the
40
+
run's exit code: no run of this pass can clear it, because the conversion must not flatten a
41
+
combinator — it is the hand rewrite's to decide. A row that also converts something keeps the
42
+
outcome its conversion gives it, with its TODOs listed beside its notices. Measured through the
43
+
write path: on `--apply`, a row whose leftover sits in a block's `properties.filter` or
44
+
`properties.defaultFilters` is rewritten (its lossless filters persist; the metadata API's save
45
+
does not refuse block props by component type), while a leftover in `dataSource.filter` fails
46
+
the save, and the row's TODO says why.
47
+
48
+
**For code calling the conversion layer.**`onTodo` and `reportTodo` are optional. Only the
49
+
stored-metadata pass passes a sink today; every other seam leaves the site as stored silently,
Copy file name to clipboardExpand all lines: content/docs/deployment/cli.mdx
+3-2Lines changed: 3 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1196,6 +1196,7 @@ can produce, because both supply a live one.
1196
1196
| Types with no repository write path (`agent`) | Their write path records no history and would force a draft live — a half-write is worse than leaving the row to the read path |
1197
1197
| Rows that still fail the current schema after conversion | That is a genuine contract violation, not chain-owned history. The write path's rejection is correct; fix the row in Studio |
1198
1198
| A flow whose rename the conflict guard refused | The old node-type token is a live name something else owns here. Rewriting would clobber that owner, so the row fails loudly naming the token — never a silent skip |
1199
+
| A site the conversion chain leaves as stored because no lossless rewrite exists — above all a page filter carrying `$and` / `$or` / `$not` | Flattening a combinator changes which rows the page selects, so it is never done. Each site is printed as a `TODO` line under its row — path, block, and what blocks the rewrite — whatever the row's outcome; a row with nothing but TODOs is reported `skipped`. It does not fail the run, since no run of this pass can clear it: rewrite each site by hand |
1199
1200
1200
1201
**Flows are covered, and cost one extra plugin.** Flow-node conversions carry an
1201
1202
open-namespace conflict guard that has to consult the *live* executor registry
@@ -1222,8 +1223,8 @@ something the platform can depend on. What running it buys is hygiene (cleaner
1222
1223
diffs, exports and history from here on, and the recurring boot notices go
1223
1224
quiet) plus one thing that was previously unobtainable: **you can assert it.**
1224
1225
A run with nothing left to do exits `0`; a deployment with rows still carrying
1225
-
an old dialect exits `1`. So "my metadata is on protocol N" becomes a check
1226
-
rather than a belief.
1226
+
an old dialect this pass can convert exits `1`. So "my metadata is on protocol N"
1227
+
becomes a check rather than a belief.
1227
1228
1228
1229
Note the division of labour with the default mode: `os migrate meta --from N`
1229
1230
lists the edits **an author's source** needs and reads no database; `--stored`
expect(text).toContain(`TODO page-component-filter-record-to-rule-array: ${JSON.stringify(COMBINATOR)} left as stored at pages[0].regions[0].components[0].properties.filter`);
0 commit comments