Skip to content

Commit d5b349d

Browse files
committed
docs(cli): name os migrate unmapped-columns as the conversion route for a retired field
The os migrate table and example block, a subsection with the three-step route and the door's answers, the queries page's conversion sentence, and the callout pointer that sits above its sentence, not below. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016GiHYRmLSNWTfbX9gVQkpz
1 parent 5b081d2 commit d5b349d

3 files changed

Lines changed: 67 additions & 2 deletions

File tree

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
---
2+
'@objectstack/cli': minor
3+
---
4+
5+
feat(cli): `os migrate unmapped-columns --object NAME` reads the values of a retired field's columns, keyed by record id, for a conversion before `os migrate apply --allow-destructive` drops them (#21573)
6+
7+
Clause-②: yes (widening)
8+
9+
- **What it reads.** The columns `os migrate plan` reports as `unmapped_column` for one object's table: a column that is still in the table and that no metadata declares, typically one a retired field left behind. The column set is the plan's own findings, from the same differ on the same read-only boot, so the command never reads a column the plan does not report. Each record is emitted as `{ id, values }`. `--json` prints one document, `{ database, object, table, columns, count, records, duration }`. The text face lists the columns and each record's values.
10+
- **Why it exists.** A read or a write through the engine now serves an object's declared fields only, and naming an undeclared column is refused. An app that moves a retired field's values into the field that replaced it reads them once with this command, writes them with its own script, and then drops the columns with `os migrate apply --allow-destructive`. That is the route the read and write narrowing in `@objectstack/objectql` names for this case.
11+
- **Operator-only and read-only.** It runs under the database credentials you pass (`--database-url`, else `OS_DATABASE_URL`, else the project database), and it reads every organization's rows. No REST route, API flag or per-request option serves these values, and the runtime doors are unchanged. It boots the way `os migrate plan` does: no schema DDL, no seed data, and no database file created.
12+
- **Values as stored.** An unmapped column has no declared type, so each value is emitted as the database client returns it, with no field-type decoding.
13+
- **Answers.** An object with no unmapped column, or with no table yet: empty work, exit 0. No SQL driver: `os migrate plan`'s own `no_sql_driver` answer, exit 0. An undeclared object name: `OBJECT_NOT_FOUND`, exit 1. An object the plan does not diff (federated, or bound to another datasource): refused, exit 1. A read that cannot be complete, such as one stopped by `--max-records`: refused, exit 1, and no partial set is emitted.
14+
- `MigrateUnmappedColumnsCommand` is exported from `@objectstack/cli` beside the other `os migrate` commands.
15+
16+
Nothing that ran before changes. This is a new command.

‎content/docs/data-modeling/queries.mdx‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -290,7 +290,11 @@ system columns (`id`, `created_at`, `updated_at`, and the tenant, owner and audi
290290
the registry adds). A column no metadata declares is never returned — for example one a
291291
retired field left in the table until `os migrate apply --allow-destructive` drops it — and
292292
naming it in `fields` on the data API is refused with `400 INVALID_FIELD`. To read such a column's values
293-
for a one-time conversion, run the conversion before the field is retired.
293+
for a one-time conversion, run the conversion before the field is retired, or, once it is
294+
retired, read them with the operator-only
295+
[`os migrate unmapped-columns`](/docs/deployment/cli#os-migrate-unmapped-columns) and convert
296+
them before `os migrate apply --allow-destructive` drops the column. No runtime door serves
297+
them.
294298

295299
### Nested / Related Fields
296300

‎content/docs/deployment/cli.mdx‎

Lines changed: 46 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -519,7 +519,7 @@ as the project's **default** datasource it is refused at boot the same way.
519519
**What it boots:**
520520
- Reads the artifact's `manifest`, `objects`, `views`, `flows`, …
521521
- Auto-registers the platform services declared in `requires: [...]` (e.g. `ai`, `automation`, `analytics`, `auth`, `ui`). Declaring a **service** capability (`automation`, `analytics`, `ai`, `audit`, …) is a *requirement*: if its provider package isn't installed, boot **fails fast** with a clear error instead of silently starting without a capability you asked for. (`auth` and `ui` are tier-gated with their own opt-in rules — `auth`'s secret-gated skip is described below.)
522-
- Auto-detects the driver from the database URL scheme (`libsql://`/`https://*.turso.*` → Turso — via the optional `@objectstack/driver-turso` package, and a loud failure with the install command when it is missing rather than a fallback to sqlite —, `postgres[ql]://`/`pg://` → pg, `mongodb[+srv]://` → MongoDB, otherwise sqlite; `:memory:` is SQLite's own in-memory database). `memory://` and `mingo://` are refused — see the callout below.
522+
- Auto-detects the driver from the database URL scheme (`libsql://`/`https://*.turso.*` → Turso — via the optional `@objectstack/driver-turso` package, and a loud failure with the install command when it is missing rather than a fallback to sqlite —, `postgres[ql]://`/`pg://` → pg, `mongodb[+srv]://` → MongoDB, otherwise sqlite; `:memory:` is SQLite's own in-memory database). `memory://` and `mingo://` are refused — see the callout above.
523523
- Runs standalone boot mode with one active environment.
524524
525525
**Authentication:**
@@ -853,6 +853,7 @@ diverges from the live schema, and the physical column wins at write time.
853853
| `os migrate plan` | Dry-run: show how the database has drifted from metadata, categorised safe / needs-confirm / destructive (no changes applied) |
854854
| `os migrate apply` | Reconcile the database to metadata. Applies loosening changes; destructive ones require `--allow-destructive` |
855855
| `os migrate multi-value-columns` | Migrate a stale `varchar`/`text` column to `json` where the field declares `multiple: true` — one of three drift ops `apply` never reconciles for you. Dry run by default; `--apply` runs the statement the finding prints |
856+
| `os migrate unmapped-columns` | Read the values of the columns `plan` reports as `unmapped_column` for one object, keyed by record id — the conversion route for a retired field's values before `apply --allow-destructive` drops its columns. Read-only, and operator-only: no runtime door serves these values |
856857
857858
```bash
858859
os migrate plan # Preview drift (no changes)
@@ -861,6 +862,7 @@ os migrate apply --yes # Skip the prompt (CI / scripts)
861862
os migrate apply --allow-destructive --yes # Also drop orphaned columns, tighten NOT NULL, narrow types
862863
os migrate apply --force # Migrate even though another process is using the database
863864
os migrate plan --json # Machine-readable output
865+
os migrate unmapped-columns --object contact --json # A retired field's stored values, keyed by record id (read-only)
864866
```
865867
866868
#### Nothing is written before you confirm
@@ -1048,6 +1050,49 @@ hook or an integration already copied into some *other* single-value column is
10481050
not something it looks for, and it is deliberately not something it will grow
10491051
into: that repair is specific to what your automations did with the value.
10501052
1053+
#### `os migrate unmapped-columns`
1054+
1055+
Retiring a field leaves its column in the table: the additive sync never drops
1056+
one, and `os migrate plan` reports it as `unmapped_column` until
1057+
`os migrate apply --allow-destructive` does. In between, the values are still
1058+
stored, and no runtime door serves them: a read or a write through the data API
1059+
returns the object's declared fields only, and naming the column in `fields` is
1060+
refused. When the values have to move into the field that replaced it, this is
1061+
the read:
1062+
1063+
```bash
1064+
os migrate unmapped-columns --object contact # The columns, and every record's values
1065+
os migrate unmapped-columns --object contact --json > out.json # The same, for a conversion script
1066+
os migrate unmapped-columns --object contact --max-records 1000000 --json
1067+
os migrate unmapped-columns --object contact --database-url postgres://…
1068+
```
1069+
1070+
The conversion route is three steps: read the values with this command, write
1071+
them into the declared fields with your own script, then run
1072+
`os migrate apply --allow-destructive` to drop the columns.
1073+
1074+
- **One column set.** It reads exactly the columns `os migrate plan` reports as
1075+
`unmapped_column` for that object's table: the same differ, the same boot.
1076+
Columns the differ never reports are never read either, such as the driver's
1077+
own `id`, `created_at` and `updated_at`.
1078+
- **Operator-only and read-only.** It runs under the database credentials you
1079+
pass, and covers every organization's rows. There is no REST route or API
1080+
flag behind it. It boots the way `plan` does, so it writes nothing, and it
1081+
drops nothing.
1082+
- **Values as stored.** An unmapped column has no declared type, so each value is
1083+
emitted as the database client returns it, with no field-type decoding: a
1084+
retired `json` field on SQLite reads as its stored text, a retired `boolean`
1085+
as `0` or `1`.
1086+
- **Empty work, exit 0**, for an object with no unmapped column, or with no table
1087+
in this database yet. `--json` prints one document:
1088+
`{ object, table, columns, count, records: [{ id, values }] }`.
1089+
- **Refused, exit 1**: an object name the deployment does not declare
1090+
(`OBJECT_NOT_FOUND`); an object `plan` does not diff (federated, or bound to
1091+
another datasource), whose empty answer would be unmeasured; and a read that
1092+
cannot be complete, such as one stopped by `--max-records`. A partial set is
1093+
never emitted, because a conversion over part of a table, followed by the
1094+
drop, loses the rest.
1095+
10511096
#### Data migrations
10521097
10531098
The commands above reconcile **schema**. A *data* migration rewrites rows, and

0 commit comments

Comments
 (0)