Skip to content

Commit 5fd4855

Browse files
docs(qa): re-point eight checklist items the 17.6.0 release-verification run proved stale (#21339)
Docs-only checklist revision from the 17.6.0 release-verification run #21330 (subject `617f25f8`, Console pin `31971ff1e28f`). Every change follows the checklist README's lifecycle rule: the item's `revision` bumps and one `history` entry says what changed, why, and cites the run. No product code, no `content/docs/**`, no generated file. ## Stale clauses the run proved (each a FAIL in #21330 with disposition stale-clause / assertion-defect) | item | rev | evidence | changed by | |---|---|---|---| | `access-security.audit-log-browser` | 2 → 3 | admin `GET /data/sys_audit_log?filter={"action":"delete"}` → 0 rows; the row is stored with correct attribution | `30c530e5` (#21194): the ledger serves a non-system reader, admins included, only rows about records it can read | | `api-backend.filter-comparand-conformance` | 2 → 3 | POST `/query` → 400 `VALIDATION_FAILED` at `query.where.f_number.$eq`; GET `$filter` and engine → 400 `INVALID_FILTER`; no door returns rows | #20116 (`cfc3bcf1` #20247, `dd1b8031` #20325) — the split query-contract-matrix rev 3 already records | | `api-backend.date-range-preset-matrix` | 1 → 2 | equality `{"signed_on":"today"}` → 400 `INVALID_FILTER` (temporal door); `$gte:"this_week"` → 400 with `bareDateRangePresetComparandMessage` | by design: `18.filter-preset-ordering-comparand-refused.ts` judges ordering positions only | | `records-forms.import-transform-matrix` | 1 → 2 | 400 `UNSUPPORTED_TRANSFORM` names the missing sandbox, 0 rows — but no `framework#2611` | `f115b1f` (#21188): refusals state decisions in words, not tracker numbers | | `studio-authoring.view-authoring-live` | 1 → 2 | `GET /meta/view?object=repair_asset` serves `repair_asset.default` / `repair_asset.form` with the authored config; container name 0 hits | by design: `expandViewContainer` (#7163, #7736, #13407) | ## Expected-fail notes 17.6.0 has made pass (clauses held in #21330; only their framing was stale) | item | rev | measured | fixed by | |---|---|---|---| | `automation.packaged-flow-subflow-disable-refusal` | 1 → 2 | caller off → child's disable retry 200, ledger `active=false`; caller-first enable 409 `RESOURCE_CONFLICT` | `36d043b` #20724, `0d9349f` #20759; the enable guard is `679f95e` #20711 (step 6 now enables the child first) | | `automation.packaged-flow-clone-contract` | 1 → 2 | clone survives a cold restart and fires; still unreachable from Studio | durability `cb4c31d` #20907; reachability now filed as #21332 (clause unchanged, still expected to fail) | | `access-security.packaged-flow-write-door-parity` | 1 → 2 | `PUT` / `DELETE /automation/showcase_urgent_task_alert` → 403 `NOT_OVERRIDABLE`, flow unchanged | `4b45afae` (#20817); knownGap names the existing pin `packaged-flow-write-door-parity.dogfood.test.ts` | No clause was weakened: each still refuses the original failure mode (rows returned, a served delete row, a 200-with-zero-rows), and the clone clause keeps its expected fail. ## Validation - `node scripts/check-platform-checklist.mjs` → `OK — 15 areas, 269 items (265 active, 2 planned)`; symbol anchors and line-citation sweep green. - `api-backend.json` is re-serialized in its existing canonical 2-space form; the other four files are edited in place in their existing mixed formatting. Not in this PR (listed on #21330's close-out instead): the other checklist-accuracy findings the run collected, and the two `planned` picklist items, which can only be promoted by a run in which they pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_018zT8d8NpiQ1ExhuNd5TxY6 Co-authored-by: Claude <noreply@anthropic.com>
1 parent 37a0148 commit 5fd4855

5 files changed

Lines changed: 58 additions & 40 deletions

File tree

‎docs/qa/platform-checklist/areas/access-security.json‎

Lines changed: 18 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1199,7 +1199,7 @@
11991199
"title": "The audit-log browser surfaces attributable events over sys_audit_log with correct actor/object, filters, and a before/after payload drawer",
12001200
"since": "v16",
12011201
"status": "active",
1202-
"revision": 2,
1202+
"revision": 3,
12031203
"priority": "P1",
12041204
"surface": "mixed",
12051205
"personas": [
@@ -1220,21 +1220,21 @@
12201220
"open /system/audit-log; wait for the table to settle; screenshot",
12211221
"locate all three events in the page and read their action / object / actor cells",
12221222
"narrow to writes and confirm the narrowing is server-side; capture the re-issued /api/v1/data/sys_audit_log request with its $filter. ⚠️ There is no Action dropdown on this console: use the prebuilt filtered views (Recent / Writes / Auth / Config) or the generic Filter Records → Where → Add filter builder to constrain action",
1223-
"click the delete row → the side drawer opens; screenshot the Before (old_value) / After (new_value) JSON panels",
1223+
"click an UPDATE row (e.g. PATCH a showcase_contact first) → the side drawer opens; screenshot the Before (old_value) / After (new_value) JSON panels. Since 17.6.0 no delete row is served to the page (#21194), so the drawer is checked on an update row",
12241224
"cross-check the API twin directly: GET /api/v1/data/sys_audit_log?$filter=... for each of the three actions and compare actor/object/action to the page",
12251225
"attempt to forge the trail: POST and PATCH /api/v1/data/sys_audit_log — both must be refused (get+list only)"
12261226
],
12271227
"acceptance": [
12281228
{
1229-
"clause": "all three ops produce audit rows with the correct action, actor and target: login→action 'login' attributed to the member; delete→action 'delete' with object_name showcase_task + record_id; settings write→action 'config_change'",
1229+
"clause": "all three ops produce audit rows with the correct action, actor and target: login→action 'login' attributed to the member; settings write→action 'config_change' — both SERVED on the data API; delete→action 'delete' with object_name showcase_task + record_id is WRITTEN but, since 17.6.0, NOT served to a non-system reader (administrators included): the ledger serves only rows about records the reader can read, and a deleted record is readable by nobody (30c530e5, #21194)",
12301230
"oracle": "api",
1231-
"verify": "GET /api/v1/data/sys_audit_log returns the three rows; action/actor(user_id)/object_name/record_id match what each op did (fields per packages/plugins/plugin-audit/src/objects/sys-audit-log.object.ts)",
1232-
"evidence": "the three audit rows"
1231+
"verify": "GET /api/v1/data/sys_audit_log returns the login and config_change rows with action/actor(user_id)/object_name/record_id matching what each op did (fields per packages/plugins/plugin-audit/src/objects/sys-audit-log.object.ts), and returns NO row for $filter action='delete' even to the admin who deleted; the delete row's existence and attribution are proven from the store itself (the audit table in the telemetry database, read under system context or with the server stopped) — a delete row SERVED on the data API is the FAIL, and so is a delete row missing from the store",
1232+
"evidence": "the two served rows + the delete row's stored copy + the empty delete-filtered read"
12331233
},
12341234
{
12351235
"clause": "the browser renders those same rows: after a screenshot confirms the table painted, the DOM rows carry the same action/actor/object the API returned — the page shows server truth, not a recomputation",
12361236
"oracle": "dom",
1237-
"verify": "post-screenshot, the three rows' Action/Object/Actor cells equal the API values (read the DOM only after render is confirmed — hydration-race)",
1237+
"verify": "post-screenshot, the served rows' Action/Object/Actor cells equal the API values (read the DOM only after render is confirmed — hydration-race)",
12381238
"evidence": "screenshot + the row DOM read"
12391239
},
12401240
{
@@ -1250,9 +1250,9 @@
12501250
"evidence": "the drawer screenshot annotated 'before/after panels, not a diff'"
12511251
},
12521252
{
1253-
"clause": "the API twin reconciles with the page: GET /api/v1/data/sys_audit_log returns the same three events with matching actor/object — a page row without a backing API row (or vice versa) is a FAIL",
1253+
"clause": "the API twin reconciles with the page: GET /api/v1/data/sys_audit_log returns the same served events (login, config_change) with matching actor/object, and both omit the delete — a page row without a backing API row (or vice versa) is a FAIL",
12541254
"oracle": "api",
1255-
"verify": "field-by-field compare of the page's three rows against the /api/v1/data/sys_audit_log bodies",
1255+
"verify": "field-by-field compare of the page's rows against the /api/v1/data/sys_audit_log bodies",
12561256
"evidence": "API list vs page rows"
12571257
},
12581258
{
@@ -1283,7 +1283,8 @@
12831283
"change": "new — audit-log browser over sys_audit_log: attributable events, server-side filter, before/after payload drawer, API cross-check, append-only guard",
12841284
"ref": "claude/platform-test-checklist-ocwugl"
12851285
},
1286-
{ "revision": 2, "date": "2026-08-18", "change": "re-pointed clause 2 and step 5 at controls that exist. The item named an 'Action filter' dropdown; this console offers prebuilt filtered views (Recent / Writes / Auth / Config) plus a generic Filter Records / Where / Add filter builder. Server-side narrowing — the property the clause is actually for — is provable through the Writes view; naming a control that does not exist invites the absence-inference trap (#9453 CF-9)", "ref": "#9386" }
1286+
{ "revision": 2, "date": "2026-08-18", "change": "re-pointed clause 2 and step 5 at controls that exist. The item named an 'Action filter' dropdown; this console offers prebuilt filtered views (Recent / Writes / Auth / Config) plus a generic Filter Records / Where / Add filter builder. Server-side narrowing — the property the clause is actually for — is provable through the Writes view; naming a control that does not exist invites the absence-inference trap (#9453 CF-9)", "ref": "#9386" },
1287+
{ "revision": 3, "date": "2026-10-02", "change": "re-pointed clause 1 (and the clauses that counted its three rows) at 17.6.0's ledger read rule. 30c530e5 (#21194) serves a non-system reader, administrators included, only audit rows about records it can read, so the delete row is written but never served on the data API; the 17.6.0 release-verification run measured the admin's delete-filtered read at 0 rows while the stored row carried correct attribution. The clause now asserts the login and config_change rows served, the delete row stored and NOT served; step 6's drawer check moves to an update row, since no delete row reaches the page", "ref": "#21330" }
12871288
]
12881289
},
12891290
{
@@ -2273,10 +2274,10 @@
22732274
},
22742275
{
22752276
"id": "access-security.packaged-flow-write-door-parity",
2276-
"title": "Write-door parity on a packaged flow: PUT/DELETE /automation/:name must refuse the same packaged artifact that PUT /meta/flow/:name refuses (ADR-0126 §2 locked base) — EXPECTED FAIL today, the /automation door sails through on manage_metadata alone",
2277+
"title": "Write-door parity on a packaged flow: PUT/DELETE /automation/:name must refuse the same packaged artifact that PUT /meta/flow/:name refuses (ADR-0126 §2 locked base)",
22772278
"since": "v17",
22782279
"status": "active",
2279-
"revision": 1,
2280+
"revision": 2,
22802281
"priority": "P1",
22812282
"surface": "api",
22822283
"personas": [
@@ -2289,15 +2290,15 @@
22892290
"capture the flow's full current definition BEFORE any probe (GET /api/v1/automation/showcase_urgent_task_alert) — it is the restore payload"
22902291
],
22912292
"knownGaps": [
2292-
"the acceptance is the ADR-0126 §2 PARITY PROMISE ('the packaged base is locked — in-place edit refused loudly at the write door'), not today's behavior: as of #12438 the /automation door has NO lock — PUT/DELETE /automation/:name reach registerFlow/unregisterFlow with only the manage_metadata authoring gate in front (packages/runtime/src/domains/automation.ts), and the engine has zero lock/provenance check on that path (packages/services/service-automation/src/engine.ts). Clauses 2-3 are EXPECTED FAILS: a 200 there, where /meta refuses the same artifact, is the product finding, tracked centrally in FOLLOW-UPS (#12438). Keep the parity promise as the acceptance so the item flips green when the door is locked, without a rewrite"
2293+
"the acceptance is the ADR-0126 §2 PARITY PROMISE ('the packaged base is locked — in-place edit refused loudly at the write door'). It was an expected fail when authored (#12438: the /automation door had no lock); 4b45afae (#20817) gave the /automation write doors the packaged-base lock the /meta door keeps, and on 17.6.0 all three doors answer 403 NOT_OVERRIDABLE with the same lock message. The permanent pin is packages/qa/dogfood/test/packaged-flow-write-door-parity.dogfood.test.ts"
22932294
]
22942295
},
22952296
"steps": [
22962297
"boot showcase isolated; admin session; GET /api/v1/automation/showcase_urgent_task_alert and store the full definition (the restore payload); also GET /api/v1/meta/flow/showcase_urgent_task_alert?layers=true to prove the artifact is package-backed (populated code layer, _packageId com.example.showcase)",
22972298
"control leg, the /meta door: PUT /api/v1/meta/flow/showcase_urgent_task_alert with a trivially modified copy of the definition (e.g. label suffix) — capture status, code and WHICH layer answered; repeat with ?package=com.example.showcase and capture that code too",
22982299
"probe leg 1: PUT /api/v1/automation/showcase_urgent_task_alert with the same trivially modified definition — capture status and, if 2xx, GET the flow back to prove the live registration mutated",
22992300
"probe leg 2: DELETE /api/v1/automation/showcase_urgent_task_alert — capture status and, if 2xx, confirm GET /api/v1/automation/showcase_urgent_task_alert now 404s (the shipped flow is gone from the live engine)",
2300-
"RESTORE, unconditionally: PUT /api/v1/automation/showcase_urgent_task_alert with the stored original definition (re-registering is the cheap path; a cold restart's boot flow pull is the fallback), then GET it back and diff against the stored capture — byte-identical",
2301+
"RESTORE, only if a probe mutated anything: diff GET /api/v1/automation/showcase_urgent_task_alert against the stored capture; if it differs (or 404s), restore with a cold restart over the same database (the boot flow pull re-registers the packaged body — a restore PUT is itself refused 403 once the door is locked) and diff again — byte-identical",
23012302
"verify the flow still fires: trigger its record-change mutation once and confirm a run appears (the restore must revive the trigger binding, not just the definition read)"
23022303
],
23032304
"acceptance": [
@@ -2308,13 +2309,13 @@
23082309
"evidence": "both PUT traces (with and without ?package=) + the before/after /meta reads"
23092310
},
23102311
{
2311-
"clause": "parity, update door: PUT /api/v1/automation/showcase_urgent_task_alert against the SAME packaged artifact is refused — ⚠️ EXPECTED FAIL today: the door runs only the manage_metadata authoring gate (automation.ts) and registerFlow re-registers with no lock or provenance check (engine.ts), so a 200 here while /meta refused the identical artifact IS the finding. Record the fail with both traces side by side; the defect is tracked centrally in FOLLOW-UPS (#12438) — do not re-file it per run",
2312+
"clause": "parity, update door: PUT /api/v1/automation/showcase_urgent_task_alert against the SAME packaged artifact is refused with the same lock the /meta door applies (4b45afae, #20817). A 200 here while /meta refused the identical artifact is the fail — record both traces side by side",
23122313
"oracle": "api",
23132314
"verify": "same admin session, same artifact, same-shape body at both doors; the verdicts must MATCH. A 2xx on /automation with a mutated GET read-back, paired with the /meta refusal from clause 1, is a fail of this clause and the expected present-day outcome",
23142315
"evidence": "the /automation PUT trace + the mutated (or unchanged) GET read-back, paired with clause 1's refusal"
23152316
},
23162317
{
2317-
"clause": "parity, delete door: DELETE /api/v1/automation/showcase_urgent_task_alert is refused for the same reason — ⚠️ EXPECTED FAIL today (unregisterFlow, engine.ts, removes the shipped flow from the live engine with no check; 'delete first, refuse second' is the exact shape the #10145 measurement recorded at this door before the capability gate existed, and the lock half is still missing)",
2318+
"clause": "parity, delete door: DELETE /api/v1/automation/showcase_urgent_task_alert is refused for the same reason (4b45afae, #20817) — a 200 that removes the shipped flow from the live engine is the fail ('delete first, refuse second' is the shape the #10145 measurement recorded at this door before the lock existed)",
23182319
"oracle": "api",
23192320
"verify": "DELETE answers >=400 and the flow still serves; a 200 followed by a 404 on GET /api/v1/automation/showcase_urgent_task_alert is the fail (and the deletion this item's restore step exists to undo)",
23202321
"evidence": "the DELETE trace + the follow-up GET"
@@ -2349,7 +2350,8 @@
23492350
"date": "2026-08-26",
23502351
"change": "new — #12438 measured that PUT/DELETE /automation/:name re-register/unregister a PACKAGED flow on manage_metadata alone while PUT /meta/flow/:name refuses the same artifact: the ADR-0126 §2 lock is unimplemented at the /automation door. Authored as a door-parity item with the parity promise as the acceptance and the present-day 200 recorded as an expected fail (tracked centrally in FOLLOW-UPS), plus a mandatory restore step because the probe mutates a live engine registration",
23512352
"ref": "#12438"
2352-
}
2353+
},
2354+
{ "revision": 2, "date": "2026-10-02", "change": "retired the expected-fail framing: 4b45afae (#20817) locked the /automation write doors against the packaged base, and the 17.6.0 release-verification run measured PUT and DELETE /automation/showcase_urgent_task_alert answering 403 NOT_OVERRIDABLE with the /meta door's lock message, the flow unchanged and still firing. Title, knownGaps and clauses 2-3 now state the parity as holding; the restore step is conditional (a restore PUT is itself refused), and the knownGap names the permanent pin packaged-flow-write-door-parity.dogfood.test.ts", "ref": "#21330" }
23532355
]
23542356
},
23552357
{

0 commit comments

Comments
 (0)