Skip to content

Commit cef89b8

Browse files
committed
Merge origin/main into claude/issue-20432-skipped-index-durability
Claude-Session: https://claude.ai/code/session_01N8TPEsoJxPsdSdNKGnNGEN Co-authored-by: Claude <noreply@anthropic.com>
2 parents 569f5be + 0bbe400 commit cef89b8

63 files changed

Lines changed: 2063 additions & 414 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
---
2+
'@objectstack/spec': patch
3+
---
4+
5+
fix(spec): `os migrate meta` guidance for the `datasource-*`, `filter-*`, `action-*`, `data-*` and `element-*` migration entries states each lesson in words instead of citing tracker numbers
6+
7+
Clause-②: no
8+
9+
The ADR-0087 semantic entries of the `datasource-*` family (the publish-time credential,
10+
placeholder and URL refusals, and the bound-secret pairs a mongo datasource cannot use),
11+
the `filter-*` family (the retired `$regex`, the `$between` endpoint refusals and the
12+
comparand shapes the save door now refuses), the `action-*` family (the retired
13+
descriptor key, the `resumeAuthority` default flip, the action-session rename, the bulk
14+
dispatch contract and the engine facade's query envelope), the `data-*` family (the
15+
retired driver and engine contract members, the retired field-changed event and two
16+
duration keys renamed with their unit) and the `element-*` family (the filter rule array
17+
at the page binding and the element and block doors) are printed by `os migrate meta` as
18+
the header, `why:` and `verify:` lines of a manual change. Their text sent the reader to
19+
issue-tracker, decision-batch and ruling-record numbers — some of which no longer
20+
resolve, and some in other repositories — for what a ruling, measurement or fix had
21+
decided; it now says what was decided, in the sentence being read. ADR ids are kept.
22+
23+
Text only: no entry id, `surface`, `from` / `to`, conversion or matching logic changes,
24+
and the chain rewrites exactly what it rewrote before. The generated migration registry,
25+
`spec-changes.json` and the protocol upgrade guide carry the same text.

‎.changeset/20289-os-test-names-tags.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,4 +14,4 @@ A Quality Protocol suite's `name`, each scenario's `name` and `description`, and
1414
- **`--tags TAG[,TAG...]`** runs only the scenarios carrying AT LEAST ONE of the listed tags (any-of, exact, case-sensitive) — the comma-list reading of Odoo's `--test-tags` and the everyday use of Playwright's `--grep @a|@b`. With the flag, an untagged scenario is left out. Left-out scenarios are **deselected**: not run, counted on the summary (`--tags smoke selected 1 of 4 scenarios; 3 deselected (not run, not counted as passed).`), never counted as passed. A requested tag that no loaded scenario carries is named on the summary. An empty entry (`--tags smoke,`) is refused before anything runs. Without the flag nothing changes: every scenario runs.
1515
- **Exit status.** A selection that matches no scenario takes the posture an empty pattern already has: exit `0` with `No scenario matched --tags …`, and exit `1` under `--fail-on-empty`, whose description now covers both cases. The `Found N test suites.` line and the `SUCCESS: All N scenarios passed.` / `FAILED: …` summary lines keep their spelling.
1616
- **`@objectstack/core`:** `QA.TestResult` gains `scenarioName` and `description` on every result, and `suiteName` on every result `runSuite` produces (absent only from a lone `runScenario` call, which has no suite).
17-
- **`@objectstack/spec`:** `TestScenario.requires` (`params`, `plugins`) is still checked by nothing — its describe() now says **NOT CHECKED** instead of reading as a guard, so a scenario that declares a plugin the target lacks still runs, and the unmet requirement surfaces only as whatever failure it causes, if any. The liveness ledger (`liveness/qa.json`) moves the four keys above to `live`, citing their readers.
17+
- **`@objectstack/spec`:** the liveness ledger (`liveness/qa.json`) moves the four keys above to `live`, citing their readers. `TestScenario.requires`, the family's fifth key, is checked in this same release and has its own note: an unmet `params` or `services` entry skips the scenario with its reason, and `requires.plugins` is retired into `requires.services`.
Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
---
2+
'@objectstack/spec': minor
3+
'@objectstack/core': minor
4+
'@objectstack/cli': minor
5+
---
6+
7+
feat(spec,core,cli)!: a scenario's `requires` is checked before it runs — unmet `params` or `services` SKIP it with a reason; `requires.plugins` is retired into `requires.services` (#20289)
8+
9+
Clause-②: yes (narrowing)
10+
11+
**BREAKING** — shipped as `minor` under the launch-window convention
12+
(`check-changeset-no-major` refuses `major` until GA; breaking-ness is carried by
13+
this banner, the `(narrowing)` arm above and the ADR-0087 disposition below,
14+
never by the level).
15+
16+
A Quality Protocol scenario's `requires` block declared preconditions —
17+
`params` (environment variables) and `plugins` (plugins that must be loaded) —
18+
that nothing checked: measured on a stub target, a scenario naming a missing
19+
plugin and an unset variable reported PASSED exactly like its no-requirements
20+
control. ADR-0049 enforce-or-remove, verdict ENFORCE (the mainstream has
21+
declared preconditions: JUnit `@EnabledIfEnvironmentVariable`, pytest `skipif`),
22+
ruled B for the shape: each key is judged against something `os test` can
23+
actually observe.
24+
25+
- **`requires.params`** — each variable must be set to a non-empty value in the
26+
environment of the process running `os test` (not the target server's, which a
27+
suite cannot see). An empty value counts as unset: an unconfigured CI secret
28+
arrives as an empty string.
29+
- **`requires.services`** (new) — each entry is a discovery service key
30+
(`CoreServiceName`: `auth`, `automation`, `analytics`, `ai`, `storage`, …; a
31+
misspelling is refused when the suite loads) that the target must declare
32+
`enabled` with status `available` in its discovery document (ADR-0076 D12).
33+
It is read from the discovery request the HTTP adapter already makes once per
34+
run; a suite that requires no service issues no extra request.
35+
- **SKIPPED.** A scenario with an unmet entry runs no step — `setup` included —
36+
and `os test` prints it with its reason, naming every unmet entry and, for a
37+
service, the services the target does declare available:
38+
`Skipped: requires.services 'ai' is not available on the target (enabled: false, status: unavailable). The target declares available: auth, data, metadata.`
39+
It is counted on its own — `SUCCESS: 3 scenarios passed. 1 skipped (not run, not counted as passed).` —
40+
and never as passed. Skips alone exit `0`; a run in which EVERY selected
41+
scenario was skipped prints `No scenario ran: …` instead of `SUCCESS`, exits
42+
`0`, and exits `1` under `--fail-on-empty`. With nothing skipped, the summary
43+
lines keep their spelling.
44+
- **`@objectstack/core`:** `QA.TestResult` gains `status` (`'passed' | 'failed' | 'skipped'`)
45+
and, on a skipped result, `skipped` (`reason`, `unmet[]`, `availableServices`);
46+
`passed` stays and is `false` on a skip. `TestRunner` takes an optional
47+
`{ env }` (default: this process's environment), and `TestExecutionAdapter`
48+
gains an optional `readTargetServices()` — `HttpTestAdapter` answers it from
49+
its one discovery probe. An adapter without it skips a service requirement
50+
rather than running it.
51+
52+
```
53+
FROM { "id": "ai-summary", "requires": { "plugins": ["@objectstack/service-ai"] }, "steps": [...] }
54+
-> ran anyway; the missing plugin surfaced as whatever failure it caused, or passed
55+
TO -> os test refuses the suite at load:
56+
✗ scenarios.0.requires.plugins: `scenarios[].requires.plugins` was removed in
57+
@objectstack/spec 17.5.0 (ADR-0049 enforce-or-remove) — nothing ever checked it: …
58+
Delete the key and name the service the scenario needs in `requires.services`, …
59+
Plugin → service: @objectstack/service-analytics → analytics, @objectstack/plugin-auth → auth, …
60+
61+
FROM { "requires": { "services": ["ai"] }, … } (new key)
62+
TO -> against a target whose discovery does not declare `ai` enabled and available:
63+
⏭️ Scenario: Summarise an account [ai-summary] (skipped)
64+
Skipped: requires.services 'ai' is not available on the target (…). The target declares available: …
65+
```
66+
67+
**Fix.** `requires.plugins: ["<package>"]` → `requires.services: ["<service>"]`,
68+
using the mapping the refusal prints (derived from `CORE_SERVICE_PROVIDER`, the
69+
provider table discovery itself reports): `@objectstack/plugin-auth` → `auth`,
70+
`@objectstack/service-analytics` → `analytics`, `@objectstack/service-automation`
71+
→ `automation`, `@objectstack/service-storage` → `storage`, and so on; the `ai`
72+
service is provided by ObjectStack Cloud/Enterprise. A plugin that fills no
73+
discovery service slot has no service to require — gate that scenario with a
74+
`params` variable or select it with `--tags`. `tsc` refuses `plugins` at a typed
75+
authoring site (its input type is `never`). A `TestResult` consumer that counted
76+
`!passed` as a failure should read `status` — a skipped result is `passed: false`
77+
and is not a failure.
78+
79+
**What does not change.** A scenario without `requires` runs exactly as before,
80+
and a suite that requires no service issues no discovery request it did not
81+
already issue.
82+
83+
### The retirement kit
84+
85+
- **Schema.** `TestScenarioSchema.requires` is a non-strict `z.object()`, so
86+
`plugins` is a `retiredKey()` tombstone carrying its prescription (a bare
87+
deletion would have stripped it in silence); `services` is new, closed over
88+
`CoreServiceName`.
89+
- **ADR-0087.** `RETIRED_KEYS_BY_MAJOR[18]` gains `qa/TestScenario:requires.plugins`.
90+
No D2 conversion: a QA suite is a loose JSON file `os test` loads, never a
91+
stack collection member or a stored row. The family's D3 entry,
92+
`qa-scenario-requires-plugins-retired`, carries the prescription to
93+
`os migrate meta` and the upgrade guide.
94+
- **Ledger and docs.** `liveness/qa.json` moves `qa.scenarios.requires` from
95+
`dead` to `live`, citing the runner's judgement and the adapter as producer;
96+
`state-counts.md` moves `qa` to 9 live / 0 dead. The `os test` section of the
97+
CLI reference documents the check, the skip line and the exit posture, and the
98+
generated `qa/testing` reference page is regenerated.
99+
100+
<!-- adr-0087: registered qa-scenario-requires-plugins-retired -->

‎content/docs/deployment/cli.mdx‎

Lines changed: 32 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1715,9 +1715,38 @@ out are **deselected**: not run, counted on the summary
17151715
and never counted as passed. A listed tag that no loaded scenario carries is named
17161716
on the summary, since a typo narrows the run without failing it, and an empty entry
17171717
(`--tags smoke,`) is refused before anything runs. Without the flag every scenario
1718-
runs. A scenario's `requires` block is **not checked**: a scenario that names a
1719-
plugin the target does not load still runs, and the unmet requirement surfaces
1720-
only as whatever failure it causes, if any — the scenario can still pass.
1718+
runs.
1719+
1720+
**A scenario's `requires` is checked before its first step.** Two keys, each
1721+
judged against something `os test` can observe:
1722+
1723+
- `requires.params` — environment variables that must be set to a non-empty
1724+
value **in the process running `os test`** (not on the target server, which a
1725+
suite cannot see): `"params": ["BILLING_SANDBOX_KEY"]`.
1726+
- `requires.services` — service keys the target must declare in its discovery
1727+
document as `enabled` with status `available`, read from the same discovery
1728+
request the runner already makes once per run: `"services": ["ai", "analytics"]`.
1729+
The keys are the discovery service names (`auth`, `automation`, `storage`, …),
1730+
and a misspelled one is refused when the suite loads.
1731+
1732+
A scenario with an unmet entry is **skipped**: none of its steps runs — `setup`
1733+
included — and its line says why, naming every unmet entry and, for a service,
1734+
the services the target does declare available:
1735+
1736+
```text
1737+
⏭️ Scenario: Summarise an account with the AI service [ai-summary] (skipped)
1738+
Skipped: requires.services 'ai' is not available on the target (enabled: false, status: unavailable). The target declares available: auth, data, metadata.
1739+
```
1740+
1741+
A skipped scenario is counted on its own —
1742+
`SUCCESS: 3 scenarios passed. 1 skipped (not run, not counted as passed).` — and
1743+
is never counted as passed. Skips alone do not fail a run, but a run in which
1744+
**every** selected scenario was skipped proved nothing: it prints
1745+
`No scenario ran: all 2 selected scenarios were skipped on unmet requirements.`
1746+
instead of `SUCCESS`, exits **0**, and exits **1** under `--fail-on-empty`. The
1747+
retired `requires.plugins` is refused when the suite loads, with the plugin →
1748+
service mapping in its message: no served surface lists loaded plugins, so it
1749+
was never checkable, and `requires.services` asks the question it stood for.
17211750
17221751
**A pattern that matches no suite is not a failure by default.** The run prints
17231752
`Found 0 test suites.` — the same machine-readable line a full run prints, so a

‎content/docs/references/qa/testing.mdx‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -116,7 +116,7 @@ A complete test scenario with setup, execution steps, and teardown
116116
| **setup** | `{ name: string; description?: string; action: object; assertions?: object[]; … }[]` | optional | Steps to run before main test (preconditions) |
117117
| **steps** | `{ name: string; description?: string; action: object; assertions?: object[]; … }[]` | ✅ | Main test sequence to execute |
118118
| **teardown** | `{ name: string; description?: string; action: object; assertions?: object[]; … }[]` | optional | Steps to cleanup after test execution |
119-
| **requires** | `{ params?: string[]; plugins?: string[] }` | optional | Environment requirements for this scenario. NOT CHECKED by `os test` or the core TestRunner: the scenario runs whether or not they hold, and an unmet requirement surfaces only as the failure it causes |
119+
| **requires** | `{ params?: string[]; services?: Enum<'metadata' \| 'data' \| 'auth' \| 'storage' \| 'file-storage' \| 'search' \| 'cache' \| …>[] }` | optional | Preconditions judged before the scenario's first step (setup included). Every entry must hold, or the scenario is SKIPPED with a reason naming each unmet entry — counted separately by `os test` and never counted as passed |
120120

121121
### Nested Shape: `TestScenario.setup[number]`
122122

@@ -158,8 +158,9 @@ A single step in a test scenario, consisting of an action and optional assertion
158158

159159
| Property | Type | Required | Description |
160160
| :--- | :--- | :--- | :--- |
161-
| **params** | `string[]` | optional | Environment variables or parameters the scenario needs. Declared only: nothing checks them before the scenario runs |
162-
| **plugins** | `string[]` | optional | Plugins the scenario needs loaded on the target. Declared only: nothing checks them before the scenario runs |
161+
| **params** | `string[]` | optional | Environment variables that must be set to a non-empty value in the process running `os test` — the runner's own environment, not the target server's. An unset or empty variable skips the scenario (SKIPPED, never passed) with a reason naming the variable |
162+
| **services** | `Enum<'metadata' \| 'data' \| 'auth' \| 'storage' \| 'file-storage' \| 'search' \| 'cache' \| …>[]` | optional | Services the target must declare in its discovery document as `enabled` with status `available` (ADR-0076 D12), read from the discovery document `os test` already fetches once per run. An entry the target does not declare that way skips the scenario (SKIPPED, never passed) with a reason naming the service and the services the target does declare available |
163+
| **plugins** | `never` | optional | [REMOVED] `scenarios[].requires.plugins` was removed in @objectstack/spec 17.5.0 (ADR-0049 enforce-or-remove) — nothing ever checked it: `os test` reaches its target over HTTP and no served surface lists the loaded plugins, so a scenario naming a missing plugin ran anyway. Delete the key and name the service the scenario needs in `requires.services`, which is judged against the services the target's discovery document declares enabled and available — an unmet entry skips the scenario and says why. Plugin → service: @objectstack/service-analytics → analytics, @objectstack/plugin-auth → auth, @objectstack/service-automation → automation, @objectstack/service-cache → cache, @objectstack/service-queue → queue, @objectstack/service-job → job, @objectstack/service-realtime → realtime, @objectstack/service-storage → storage, @objectstack/service-i18n → i18n, @objectstack/service-messaging → notification, @objectstack/metadata-protocol → ui. A plugin not listed fills no discovery service slot, so there is no service to require for it. |
163164

164165

165166
---
@@ -224,7 +225,7 @@ A complete test scenario with setup, execution steps, and teardown
224225
| **setup** | `{ name: string; description?: string; action: object; assertions?: object[]; … }[]` | optional | Steps to run before main test (preconditions) |
225226
| **steps** | `{ name: string; description?: string; action: object; assertions?: object[]; … }[]` | ✅ | Main test sequence to execute |
226227
| **teardown** | `{ name: string; description?: string; action: object; assertions?: object[]; … }[]` | optional | Steps to cleanup after test execution |
227-
| **requires** | `{ params?: string[]; plugins?: string[] }` | optional | Environment requirements for this scenario. NOT CHECKED by `os test` or the core TestRunner: the scenario runs whether or not they hold, and an unmet requirement surfaces only as the failure it causes |
228+
| **requires** | `{ params?: string[]; services?: Enum<'metadata' \| 'data' \| 'auth' \| 'storage' \| 'file-storage' \| 'search' \| 'cache' \| …>[] }` | optional | Preconditions judged before the scenario's first step (setup included). Every entry must hold, or the scenario is SKIPPED with a reason naming each unmet entry — counted separately by `os test` and never counted as passed |
228229

229230

230231
---

0 commit comments

Comments
 (0)