Skip to content

Commit 88b484e

Browse files
feat(objectql): resolve picklist references at runtime — served options, additive extensions, write validation, load-time refusal (#21047)
Fixes #19519 Clause-②: no The runtime layer of the shared `picklist` kind, phase 1 of #18164. The spec layer (#19518) is already on `main`. A select field that authors `picklist: 'industry'` is now served with that list's resolved options, a write is judged against them, and `picklistExtensions` from other packages merge into the list additively. Dispatched by the `domain:engine` seat 2 PM under claim 5923202473. Dev session `session_01MZu5JqVPacMktogpMxq9Xu`. ## What changed **Load** (`packages/objectql/src/engine.ts`, `packages/metadata/src/plugin.ts`) - `picklists` and `picklistExtensions` join `METADATA_ARRAY_KEYS`, first in the list. Both registration seams (manifest and nested plugin) reach them through the shared `registerMetadataCollections` body. - A `picklistExtensions` entry is dispatched to `SchemaRegistry.registerPicklistExtension`, never registered as an item of its own. - The artifact door maps `picklists` to the `picklist` kind, so `GET /meta/picklist/NAME` serves the list on an artifact boot. **Merge** (`packages/objectql/src/registry.ts`, new `picklist-resolution.ts`) - A list's options are its own, followed by each extension's. - A value the list already carries is refused with `422 INVALID_METADATA`, naming both declarations. This holds in either registration order: list then extension, or extension then list. It is never last-wins. - A package that registers again replaces its own contribution. Uninstalling a package removes its values. **Serve** - The registry's object fold (`foldExtendersOntoDefinition`, shared by `resolveObject`, the owner layer and `foldObjectExtendersOnto`) writes the resolved options onto every field that names a list, and keeps `picklist` (`PicklistServedFieldSchema`). - Every object read and the write door therefore read one resolved set. That covers registry-sourced bodies and `sys_metadata` bodies, which the protocol folds through `foldObjectExtendersOnto`. - Resolution is re-derived on every fold, never trusted. A stale served copy (the boot bridge writes resolved bodies into the metadata service) gets the current list. When the list is gone, its options are removed, so the write door refuses rather than accepting a stale value. - An object with no picklist-bound field is returned by reference, so its read is byte-identical to before. - Locale: the existing `translateObject` / `translatePicklist` resolvers relabel the resolved options per request. The dogfood case drives this end to end. **Unknown name: a load-time error** (`packages/objectql/src/plugin.ts`) - At `kernel:ready` every package has registered, so "not declared" is final (AGENTS.md, startup registry reads). A packaged field naming a list nothing declares fails the boot with `INVALID_METADATA`, naming every such field and its package. A `picklistExtensions` entry extending such a list fails the same way. - After that, the vocabulary is sealed. An artifact registered through the `manifest` service is checked before any of it registers, so a refused install registers nothing. - Tenant overlays are never judged at boot: a stored row cannot fail a boot. The write door still refuses every value of such a field. **Write validation** (`packages/objectql/src/validation/record-validator.ts`) - The door that judges inline `options` judges the resolved set, and the refusal names the list. The wire code stays `invalid_option`. - Three message keys carry the sentence, in en, zh-CN, ja-JP and es-ES (`packages/spec/src/system/validation-message.ts`): `invalid_option_picklist`, `invalid_option_value_picklist` and `invalid_option_picklist_unresolved`. They change the message text only and never reach the wire. - A picklist-bound field with no resolved options accepts no value. It is never read as free-form. **Error codes (H5).** Both refusals reuse `INVALID_METADATA`. `@objectstack/objectql` already emits it in the ADR-0112 ledger. No new code, so `Clause-②: no` stands. **Enumerations and ledgers** - `scripts/check-stack-collection-maps.mjs`: the `METADATA_ARRAY_KEYS` PENDING row is retired. The `ARTIFACT_FIELD_TO_TYPE` row now carries `picklistExtensions` only, with a reason: it is merged by the registry and is not a kind. `APP_CATEGORY_KEYS` / `SECURITY_FIELDS` keep their DELIBERATE rows. One is an app-payload heuristic, the other a four-collection security subset, and neither should list picklists. - `serializers/typescript-serializer.ts` annotates `picklist` as `Picklist` (data). - Liveness: `field.picklist` is `live`, without `authorWarn`. The `picklist` kind's rows and `translation.picklists` are `live`. Count shards are regenerated. ## Tests - `packages/objectql/src/engine-picklist.test.ts` (25), real engine and registry: - served shape on two objects; - load-order independence, and a nested plugin's lists; - duplicate refusal in four shapes, plus the replay; - uninstall; - unknown-name and orphan-extension audit, attributed per package; - write accept, write refuse naming the list, and the unresolved refusal; - option default from the list; - stale-copy handling, including both `fields` spellings. - `packages/objectql/src/plugin-picklist-boot-audit.test.ts` (6), on a real `ObjectKernel`: - boot refused for an unknown field reference and for an orphan extension; - boot accepted when the list arrives from a package registered later; - post-boot artifacts refused before registering, or accepted. - `packages/objectql/src/protocol-picklist-served-roundtrip.test.ts` (2): the protocol read serves the resolved options, and writing the served body back is refused (see H3). - `packages/rest/src/import-template-route.test.ts`: the parity battery gains `picklist_option_default`. The template's object read and the engine's import door must both see the list's `default: true` option, or the star and the refusal disagree. - `packages/qa/dogfood/test/picklist-shared-across-objects.dogfood.test.ts` (5), the ADR-0054 runtime proof: - one list, two objects, two packages composed into one artifact; - the extension's value is accepted on object A; - a value outside the set is refused on object B, naming the list; - `Accept-Language: zh-CN` relabels both objects' options and the served list item. - `packages/metadata/src/serializers/typescript-serializer-annotation.test.ts` gains the `picklist` representative. - `packages/lint/src/lint-liveness-properties.test.ts`: the pin "the shipped field ledger warns on `picklist` alone" is now false, because this change takes the row out of `authorWarn`. It now expects an empty set (a test-only edit outside the claimed surface; see Acceptance notes). Ablation, run once and not kept: replacing `return this.resolvePicklistFields(merged);` with `return merged;` in `registry.ts`, through `scripts/ablation-replace.mjs`, turned `engine-picklist.test.ts` red (8 failed, 16 passed of 24 at that commit). The restore was proven: the blob equals HEAD `24f6934320bf` and `git diff HEAD` is empty. ## Open questions for the seat 1. **H3, the served-body round trip: two readings are live.** Measured in `protocol-picklist-served-roundtrip.test.ts`: `GET /meta/object/NAME` serves `picklist` and `options` together, and a `PUT` of that body is refused `422 INVALID_METADATA` by `FieldSchema`, with its prescription. - This PR keeps refusing. That is what `PicklistServedFieldSchema`'s doc declares, and stripping at the write door would be consumer-side tolerance. - The cost: a Studio editor that writes back what it read is refused on any object with a picklist-bound field until it drops `options` (a Studio-side change, phase 2). - The alternative is to strip `options` at the write door when they equal the resolved list. It would be declared and tested, but it is still a second shape accepted by an authoring door. 2. **`resolveObjectFieldLabels`** (the field-labels endpoint) reads only the bundle and has no field-to-picklist binding. So inherited picklist option labels are still missing there. Fixing it means changing `service-i18n` and the runtime i18n dispatcher, which this claim does not cover. The served object (`translateObject`) does carry them. 3. **A pending release note, corrected here: needs confirmation.** `.changeset/19518-picklist-kind.md` (the spec layer's note, still pending) said: "This release does not resolve the reference. Until the runtime does, a picklist-bound field is served without options, and the liveness ledger grades the key `planned` and warns an author who writes it." With this PR in the same release, that sentence is false. This PR replaces it with "The runtime resolves the reference onto that served field; see the picklist runtime entry of this release." `check-empty-changeset` refuses any change to a changeset this PR did not add, and stays red until someone confirms the correction. That is its deliberate-correction path: restoring the old sentence would republish a false one. The alternative is to restore it and let the release author reconcile the two notes. ## Gates `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` on this branch derived 114 families, and all 114 were run. - `--ran` reconciliation: 113 run, 1 NOT-MEASURED. `check:dual-build-cjs-loads` exited 3: its prerequisite is `dist/` for eight packages this diff does not touch, and CI builds them. - `check-empty-changeset` exits 1: question 3 above. - `check:platform-checklist` exits 1: an `ABSENT SYMBOL` for `plugin-auth`'s `twoFactor` in `areas/identity-auth.json`. It reproduces identically on `origin/main` at `2f2fa11d7`, so it is not this PR's. - `check-engine-split-ratio` and `check-plugin-teardown-shape --self-test` refused on the shallow clone, then passed after the deepening each prescribes. - `check:objectql-double-limit` and `check:slot-lookup` caught two new test doubles in this PR. Both were fixed and re-run green. Local runs: - `objectql`: `pnpm test`, 351 files / 6851 tests green, and `pnpm typecheck` green. - `metadata`: `pnpm test`, 836 green, and `pnpm typecheck` green. - Typecheck green for `rest` and `dogfood`. - `spec` `validation-message.test.ts` green. - `lint` `lint-liveness-properties.test.ts` green. - The `metadata-protocol` picklist-adjacent files green. - `check:liveness`, `check:stack-collection-maps`, `check:startup-registry-verdict`, `check:durability-log-level`, `check:doc-authoring` and `check:nul-bytes` green. - eslint on the 16 touched TS/MJS files: 0 errors, 0 warnings. ## Acceptance notes - **Outside the claimed surface, changed because this PR makes them false:** - the lint pin above; - one sentence of the pending `.changeset/19518-picklist-kind.md`, which said this release does not resolve the reference; - the `translation.json` / `README.md` liveness rows for picklists. - **Not changed:** - `metadata-core`'s `MetadataTypeSchema` (it already lacks `seed`, nothing reads `picklist` through it, and widening a published enum with no consumer is surface without pull); - `runtime` `app-plugin.ts` (see above); - `examples/app-showcase/src/coverage.ts` (its `picklistExtensions` waiver now reads as stale; demonstrating it in the showcase is example-app work); - the platform-checklist items `records-forms.picklist-*` (the checklist seat re-runs them). - **Merged `origin/main` once before this PR, cleanly.** It brought the CLI's author-time picklist reference check from another card. That door and this runtime audit agree: the CLI refuses within one stack, and the runtime refuses across packages at boot. - HMR: a picklist edited while `os dev` runs reaches the registry only through a full manifest re-registration. The `metadata:reloaded` ingest re-registers objects, not lists. --- _Generated by [Claude Code](https://claude.ai/code/session_01MZu5JqVPacMktogpMxq9Xu)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 63d1a7c commit 88b484e

26 files changed

Lines changed: 1579 additions & 78 deletions

‎.changeset/19518-picklist-kind.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ Clause-②: yes (widening)
1111

1212
- **The kind.** `PicklistSchema` — `{ name, label, description?, options }`, where `options` is the field option shape (`SelectOptionSchema`) reused as is. Authored in a package as `*.picklist.ts` (`definePicklist`) or `defineStack({ picklists })`. It is a registered kind (`MetadataTypeSchema`, `DEFAULT_METADATA_TYPE_REGISTRY`, `getMetadataTypeSchema('picklist')`) that loads before `object`. It is package-owned, so a runtime create or a per-organization overlay is refused.
1313
- **The reference.** `Field.select({ picklist: 'industry' })` adds a `picklist` key to `FieldSchema`. It is valid on the option types only (select, radio, multiselect, checkboxes, tags). A field that declares both `picklist` and `options` is refused at `options`, with a prescription. The functional-completeness predicate counts a `picklist` reference as the field's option source.
14-
- **The served shape.** `PicklistServedFieldSchema` declares what a client reads for a picklist-bound field: the resolved `options` next to the `picklist` that names the list. This release does not resolve the reference. Until the runtime does, a picklist-bound field is served without options, and the liveness ledger grades the key `planned` and warns an author who writes it.
14+
- **The served shape.** `PicklistServedFieldSchema` declares what a client reads for a picklist-bound field: the resolved `options` next to the `picklist` that names the list. The runtime resolves the reference onto that served field; see the picklist runtime entry of this release.
1515
- **Extensions.** `defineStack({ picklistExtensions: [{ extend, options }] })` adds options to a picklist that another package owns. It can only add; removing or renaming a value stays with the owning package.
1616
- **Translation.** `TranslationData` gains `picklists.<name>.{ label?, options: { value: label } }`. `translatePicklist` translates a served picklist item. `translateObject` gives a picklist-bound field the list's option labels, and a field-level `options` entry still wins over them.
1717
- **Studio type label.** `@objectstack/platform-objects` carries the `picklist` type's label and description in its metadata-forms translation bundles (en, zh-CN, ja-JP, es-ES).
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
---
2+
'@objectstack/objectql': minor
3+
'@objectstack/metadata': patch
4+
'@objectstack/spec': patch
5+
---
6+
7+
feat(objectql): the runtime resolves a field's `picklist` onto its served options, validates writes against the resolved list, and merges `picklistExtensions` additively
8+
9+
Clause-②: no
10+
11+
- **Load.** `defineStack({ picklists })` and `defineStack({ picklistExtensions })` now register, from a manifest and from a nested plugin, through the same registration seam as every other collection. The compiled-artifact door registers `picklists` as `picklist` items, so `GET /meta/picklist` serves them on an artifact boot.
12+
- **Merge.** A picklist's options are its own, followed by the options every `picklistExtensions` entry adds. A value the list already carries is refused with `422 INVALID_METADATA`, which names both declarations, whichever of the two registered first. The later declaration never replaces the earlier one. A package that registers again replaces its own extension. Uninstalling a package removes the values it added.
13+
- **Serve.** A field with `picklist: 'NAME'` is served with the resolved options written onto it and `picklist` kept (`PicklistServedFieldSchema`), on every object read, including objects stored in `sys_metadata`. The list's translations (`picklists.NAME.options.VALUE`) relabel those options per request locale. An option marked `default: true` in the list fills an omitted field on insert, as an inline option does, and the import template reads it the same way.
14+
- **Unknown name.** A packaged field that names a picklist no loaded package declares fails the boot at `kernel:ready` with `INVALID_METADATA`, and so does a `picklistExtensions` entry that extends such a list. The error names every such field or extension and the package that declared it. After boot, an artifact registered through the `manifest` service is checked before any of it registers. A field whose list does not resolve is served with no options and accepts no value.
15+
- **Write validation.** The write door judges a picklist-bound field against the resolved options, and its refusal names the picklist. The wire code stays `invalid_option`. The validation message catalog gains three message keys for this (`invalid_option_picklist`, `invalid_option_value_picklist`, `invalid_option_picklist_unresolved`) in en, zh-CN, ja-JP and es-ES. They change the message text only, never the wire.
16+
- **Writing the served body back.** The served body carries `picklist` and `options` together. Writing it back through the metadata door is still refused, with the prescription to drop `options`, as `FieldSchema` declares. Nothing strips it on the write side.
17+
- **Ledger.** `field.picklist`, the `picklist` kind's rows and `translation.picklists` are `live`. `field.picklist` no longer carries `authorWarn`, so `os lint` / `os validate` stop warning an author who writes it.

‎packages/lint/src/lint-liveness-properties.test.ts‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1432,10 +1432,11 @@ describe('the object/field walk, against a synthetic ledger directory (#19268)',
14321432
// If `field.json` ever warns again these two flip, and the block above
14331433
// ("field walk: a malformed `fields` array …") can take its real subject back
14341434
// — but this block keeps working either way, which is the point.
1435-
it('the SHIPPED field ledger warns on `picklist` alone — which is why the walk needs a subject of its own', () => {
1436-
// `picklist` is `planned` + `authorWarn` until the server resolves picklist
1437-
// references; the synthetic slot below is still warned by nothing shipped.
1438-
expect([...authorWarnedProperties('field')]).toEqual(['picklist']);
1435+
it('the SHIPPED field ledger warns on nothing — which is why the walk needs a subject of its own', () => {
1436+
// `picklist` was the last warned row (`planned` + `authorWarn`) and left
1437+
// when the server began resolving picklist references (`live`); the
1438+
// synthetic slot below is warned by nothing shipped either.
1439+
expect([...authorWarnedProperties('field')]).toEqual([]);
14391440
expect(
14401441
lintLivenessProperties({
14411442
objects: [{ name: 'widget', fields: [{ name: 'a', synthWarnedSlot: true }] }],

‎packages/metadata-protocol/src/reference-sites.derivation.test.ts‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -211,3 +211,14 @@ describe('[#9190] derived reference sites — derivation is a pure function of t
211211
expect(flatten(a)).toEqual(flatten(REFERENCE_SITES));
212212
});
213213
});
214+
215+
describe('a field\'s `picklist` is a reference site of the `picklist` kind', () => {
216+
// The admin "Used by" panel and the delete-safety check read this index. A
217+
// select field that names a shared list (`Field.select({ picklist })`)
218+
// must count as a use of that list, or the list reads as safe to delete
219+
// while objects take their options from it. Derived by the naming rule —
220+
// the property spells the target — so no row exists to forget.
221+
it('object.fields{}.picklist points at the picklist it names', () => {
222+
expect(sitesFor('picklist')).toContainEqual({ fromType: 'object', property: 'picklist' });
223+
});
224+
});

‎packages/metadata/src/plugin.ts‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -178,6 +178,12 @@ const ARTIFACT_FIELD_TO_TYPE: Record<string, string> = {
178178
ragPipelines: 'rag_pipeline',
179179
hooks: 'hook',
180180
mappings: 'mapping',
181+
// Shared option lists. Registered as items here so the artifact boot
182+
// serves `GET /meta/picklist` like every other kind; what a FIELD is
183+
// served with is resolved by the ObjectQL registry, which also merges
184+
// `picklistExtensions` into the list they name — that collection is not a
185+
// kind of its own and has no entry here (see check:stack-collection-maps).
186+
picklists: 'picklist',
181187
analyticsCubes: 'analytics_cube',
182188
connectors: 'connector',
183189
emailTemplates: 'email_template',

‎packages/metadata/src/serializers/typescript-serializer-annotation.test.ts‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,7 @@ const REPRESENTATIVE: Record<string, Record<string, unknown>> = {
4646
hook: { name: 'account_audit', object: 'account', events: ['beforeInsert'], handler: 'audit_account' },
4747
seed: { object: 'account', records: [{ name: 'Acme' }] },
4848
mapping: { name: 'account_import', targetObject: 'account', fieldMapping: [] },
49+
picklist: { name: 'industry', label: 'Industry', options: [{ label: 'Technology', value: 'technology' }] },
4950
datasource: { name: 'warehouse', driver: 'sqlite', config: {} },
5051
analytics_cube: { name: 'account_cube', sql: 'account', measures: {}, dimensions: {} },
5152
page: { name: 'account_home', label: 'Account Home', regions: [] },
@@ -81,7 +82,7 @@ const REPRESENTATIVE: Record<string, Record<string, unknown>> = {
8182
*/
8283
const BOUND_SCHEMA: Record<string, string> = {
8384
object: 'ObjectSchema', field: 'FieldSchema', hook: 'HookSchema', seed: 'SeedSchema',
84-
mapping: 'MappingSchema', datasource: 'DatasourceSchema', analytics_cube: 'CubeSchema',
85+
mapping: 'MappingSchema', picklist: 'PicklistSchema', datasource: 'DatasourceSchema', analytics_cube: 'CubeSchema',
8586
page: 'PageSchema', dashboard: 'DashboardSchema', app: 'AppSchema', action: 'ActionSchema',
8687
report: 'ReportSchema', dataset: 'DatasetSchema', flow: 'FlowSchema', webhook: 'WebhookSchema',
8788
job: 'JobSchema', translation: 'TranslationItemSchema', email_template: 'EmailTemplateDefinitionSchema',

‎packages/metadata/src/serializers/typescript-serializer.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,7 @@ const ANNOTATION_BY_METADATA_TYPE: ReadonlyMap<string, readonly [typeName: strin
5353
['hook', ['Hook', 'data']],
5454
['seed', ['Seed', 'data']],
5555
['mapping', ['Mapping', 'data']],
56+
['picklist', ['Picklist', 'data']],
5657
['datasource', ['Datasource', 'data']],
5758
['analytics_cube', ['Cube', 'data']],
5859
['page', ['Page', 'ui']],

0 commit comments

Comments
 (0)