Skip to content

Commit be8561d

Browse files
baozhoutaoclaude
andauthored
feat(qa): AI 可执行的平台功能标准测试清单 — 15 域 178 项 + 能力覆盖 ratchet + CI 门禁 (#6557)
* feat(qa): 平台功能标准测试清单 — AI 可执行的常备台账 (docs/qa/platform-checklist) Release verification lived in one-off shapes — a checkbox issue per release (#3358) and a hand table (docs/plans/release-15.1-test-plan.md) — that could not accumulate: items were not reusable across releases, ticks had no revision to pin against, and every fixture gap a sweep found (#3408/#3409/#3415) was rediscovered from prose. This lands the standing, machine-readable successor: - areas/*.json — 9 areas, 32 items seeded from the #3358 evidence runs. Each item: clause-grained acceptance (clause + oracle + verify + evidence), declared fixtures with knownGaps, negative sides (both sides of every gate), false-positive traps, immutable <area>.<slug> ids, revision + history for append/change governance, `automated` refs for the 🤖 lane. - README.md — item anatomy; append/change/retire lifecycle (append-only, never delete); release sweep = filter over the ledger (since:vN ∪ P0); relationship to dogfood-verification, @objectstack/verify, showcase coverage.ts, e2e-live, and why spec's inert TestScenarioSchema is deliberately not reused. - RUNNER.md — AI execution protocol: verdict vocabulary with first-class `partial`/`blocked`, oracle hierarchy (server truth > screenshot > DOM), fail requires reproduction ×2 + automation self-check + filed issue, trap vocabulary distilled from the #3358 retractions, durable run records under runs/ pinned to item revisions. - scripts/check-platform-checklist.mjs (+ lint.yml gate, house style): structural invariants only — unique immutable ids, revision == last history entry, every acceptance clause names its oracle. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YD9f6FYyMraUWYeJf53V43 * feat(qa): 补齐清单缺失能力域 — ai 与 integration-system (11 域 46 项) Closes the two coverage holes the seed import left: nothing covered the AI metadata kinds (agent/tool/skill, MCP surfaces) or the integration/system services (declarative connectors, webhooks, jobs, email templates). - areas/ai.json — agent/tool/skill metadata round-trip (variants matrix), MCP HTTP transport both-sides (enabled 501/off + /mcp/skill public), stdio fail-closed + RLS/FLS parity (from #3358 §9), run_action ai.exposed gate + audit (15.1 §A9), validate_expression. Showcase ships no AI seeds (ADR-0063) — fixture requirements declared explicitly. - areas/integration-system.json — declarative connector lifecycle from the 15.1 §B rows (#2994/#3062 boot materialization, #3049 degraded husk + atomic recovery, #3059 stdio default-deny allowlist, #3024 spec-path escape rejection, #2985 descriptor-only boot audit, objectui#2563 designer picker), webhook live-fire + retired-trigger build gate, job scheduled run, email-template variable rendering. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YD9f6FYyMraUWYeJf53V43 * feat(qa): 深化 dashboards 与 i18n 域到深测契约 (rev 2) First two areas rewritten to the deep-test contract: concrete draft-door/API steps, clause-grained acceptance citing the exact spec sources (dashboard.zod.ts LEGACY key list as a variants matrix, chart.zod.ts wrong-layer guidance, translation.zod.ts metadataForms group), rejected-draft-not-persisted checks, per-recipient locale controls, and round-trip/symmetry clauses. Existing items bumped to revision 2 with history entries; new items added per area. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YD9f6FYyMraUWYeJf53V43 * feat(qa): 深化 identity-auth/automation/access-security/records-forms 域(检查点) Progressive deepening checkpoint — validator green at 11 areas / 74 items. identity-auth complete (auth-method matrix from AuthPluginConfigSchema + PUBLIC_AUTH_FEATURES, delegated-admin invitation gates ADR-0105 D8, admin lifecycle ops with audit attribution); automation and access-security rewritten to the deep contract (ExecutionStepLogSchema region tags, runRlsProofs delegation with skip-audit clause, readonlyWhen locks, owner-anchor forge/transfer/disown probes); records-forms in progress by its agent. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YD9f6FYyMraUWYeJf53V43 * feat(qa): 深化 platform-core 与 approvals 域(检查点,80 项) platform-core: /ready probe, authored-nav-vs-served diff, new seed-integrity item (authored row baselines, verbatim value spot-diff, upsert idempotence — the #3408/#3415 silent-seed class as first-class FAILs), metadata registry serving + draft→publish round-trip with deny gates. approvals: per-group signoff now pins the exact seed wiring (seed-approval-demo.ts distinct holders, run resume down the approve edge, actor derived from session #3800), quorum keeps its fixture blocker plus a runnable clamp-contrast clause. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YD9f6FYyMraUWYeJf53V43 * feat(qa): 平台测试清单缺口全排查 — 补齐至 170 项,修正 3 处错误豁免 Five-angle capability-coverage sweep (console UI surfaces · spec enums · routes/settings · built-in apps page-by-page · docs claims) closed a class of systematic omissions and grew the checklist 84 → 170 items across 15 areas, authored by 8 area-partitioned writer agents grounded in real source. Headline gaps closed: - The three built-in console apps (Setup/Studio/Account) had ZERO nav coverage — nav-surfaces-render only swept the showcase app. Added builtin-apps-nav-render (P0) + settings-hub-roundtrip, session-list-revoke, api-key-ui-lifecycle, org/team management, oauth/consent, audit-log-browser, notification-center, package-lifecycle, app-management, and more. - Three coverage.json waivers were FACTUALLY STALE and are corrected to real items: api (showcase authors 2 live declarative endpoints), datasource (read-only SQLite federation fixture), mapping (showcase_inquiry_feed). Ratchet now 25 kinds mapped / 3 waived (book/doc display-only, hook indirect). - sharing-rules-widen (the marquee gap): criteria rules → materialized sys_record_share, explain attribution, §A13 seed-not-clobber. - Enumerable-surface matrices previously missing: validation rule types (6), action locations (7, ACTION_LOCATIONS), gantt/kanban/calendar drag persistence, bulk select-all-matching, adhoc filter/sort builder, grid personalization, lookup create-new, concurrent-edit OCC-409, discussion @mentions, and route-ledger-live-parity extended to all ledgers (#3361 class). Every item stays honest: 8 blocked items name the missing fixture rather than faking coverage; product defects found while grounding (saved-report schedule cross-owner delete, AppManagement client-only stubs, undo without ifMatch) are encoded as expected-fail probes and catalogued in FOLLOW-UPS.md for maintainer decision, alongside docs-drift findings (retired capabilities still advertised). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YD9f6FYyMraUWYeJf53V43 * feat(qa): 变体新鲜度门 + hook/页面/字段下钻 + sweep 自动化配方(177 项) Automation to keep the checklist current with minimal human effort, plus a deep-drill round on the surfaces a maintainer asked about (objects, field types, hooks, custom pages, cascading fields). Automation: - scripts/check-platform-checklist.mjs gains a VARIANTS-FRESHNESS ratchet: a matrix item pins the spec enum it was authored against via `enumSource` {file, export, expect}; the validator extracts the enum's current member count from spec source and fails CI when it drifts, with a precise fix instruction. Closes the gap the kind-level coverage ratchet left — the latter catches a NEW metadata kind, enumSource catches a NEW value inside an existing kind's enum. Pinned: field types (49), chart types (20), action locations (7), webhook triggers (5), flow node types (20). - SWEEP.md — the five-angle gap-hunt as a committed runbook, so "跑一轮 coverage sweep" is an AI-executable refresh (the human-effort half); the deterministic gates are the zero-human half. Deep-drill items (the 4th stale waiver corrected → coverage 26 mapped / 2 waived, only book/doc display-only remain): - records-forms.object-hook-lifecycle (un-waives `hook` — showcase ships 4 observable hooks): 6 lifecycle events, CEL previous-vs-record transition semantics (#4770/#4784), async+retry, onError abort-rollback vs log, priority. - studio-authoring custom-page items: declarative block composition + data binding, page variables + action interpolation, html-parsed vs react-executed source tiers with the OS_PAGE_REACT capability gate. - records-forms field drills: per-type write-path constraint enforcement (22 variants, exact ADR-0114 codes), multilevel cascade clear-semantics, encrypted field fail-closed. Two server-side integrity gaps found → FOLLOW-UPS D7/D8. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YD9f6FYyMraUWYeJf53V43 * feat(qa): coverage-sweep 内部 skill — 一句话手动触发清单缺口重扫 Manual trigger for the five-angle checklist gap hunt: the maintainer says "跑一轮 coverage sweep" (or asks "X 有测试吗" needing verification) and the AI session loads the orchestration contract — worktree-first, five read-only hunters in parallel, per-area writer partitioning, central coverage.json reconciliation with waiver re-audit, validator-green landing, FOLLOW-UPS for defects. Canonical method stays in docs/qa/platform-checklist/SWEEP.md; the skill is the trigger + contract, not a second copy. metadata.internal: true per template-consistency (sibling to dogfood-verification/pm-dispatch). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YD9f6FYyMraUWYeJf53V43 * docs(qa): FOLLOW-UPS 头部措辞随提交状态更新 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YD9f6FYyMraUWYeJf53V43 * docs(qa): 首次真实执行记录 — platform-core.console-login (pass) + 证据 Demonstrates the runs/ format end-to-end: an opus runner drove the item in headless Chromium against an isolated showcase boot. Verdict pass across all 6 clauses + the wrong-password negative, each clause pinned to a server-truth oracle (sign-in/get-session status, on-the-wire 401 deny side) and a captured screenshot. runs/evidence/2026-08-08-console-login/ carries the 8 screenshots + 2 network traces. The record also shows the protocol earning its keep: the runner's automation self-check (RUNNER rule 2) caught a false "dead shell" P0 — a cookie-only clear left the console authed via its localStorage bearer token; only a full credential clear produced the real redirect-to-login. And it surfaced a genuine run-time precondition (the vendored /_console dist builds separately from the showcase workspace closure), now recorded in the env block rather than faked around. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YD9f6FYyMraUWYeJf53V43 * docs(qa): 测试结果不进仓库 — runs/ 改为格式契约 + git-ignore 结果 Per maintainer decision: a run record is OUTPUT about one build, not source. Removed the committed console-login run record + evidence PNGs, git-ignore runs/* (keeping only README + .gitignore), and rewrote the RUNNER/README guidance so results live in the executing environment (CI artifact / tracking issue / external QA store), never in git. The checklist under areas/ stays the durable version-controlled contract; a run is a transient assertion about a build and belongs with that build's other artifacts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YD9f6FYyMraUWYeJf53V43 * feat(qa): checklist-run 执行 skill + 选择器解析脚本 The execution counterpart to coverage-sweep (which authors items). Lets the maintainer test a specified feature point or feature FILE on demand: - scripts/checklist-select.mjs — deterministic selector resolver. Turns an item id / area: / capability: / priority: / surface: / since:vN / **file:<path>** into the concrete runnable item set. `file:` maps a framework source file to the items whose `source[]` cites it ("test whatever covers this file"), with narrowest-useful precedence (exact/basename before dir-level). Blocked items excluded by default. Carries a --self-test (12 cases) now wired into the check:platform-checklist gate, so the resolver stays healthy under the same CI. - .claude/skills/checklist-run/SKILL.md — the runner skill (internal, marked internal:true per template-consistency.test.ts). Trigger + selection + isolation/parallelism contract; defers the judging protocol to RUNNER.md and the boot know-how to dogfood-verification. Plans the run by surface (build the vendored console only for browser/mixed items), isolates per item (own port + file DB), dispatches opus runner subagents few-at-a-time for browser items, and keeps results OUT of the repo per the runs/ policy. README documents the two-skill split (coverage-sweep authors, checklist-run executes) and the selector grammar. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YD9f6FYyMraUWYeJf53V43 * chore(qa): 平台测试清单校验改为定期手动,从 CI 移除(维护者决定) By maintainer decision the platform test checklist is a QA ledger, not a per-PR code gate. Removed the `check:platform-checklist` step from lint.yml so no unrelated PR is ever blocked by checklist drift; the script stays for manual / periodic use (`pnpm check:platform-checklist`), run before a release or after a large platform surface lands. README + the lint.yml note updated to say so. Trade-off understood and accepted: a NEW metadata kind landing on main (like `capability` did) will no longer fail a PR — it surfaces at the next manual run instead. The coverage ratchet still exists; it just isn't a blocking gate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YD9f6FYyMraUWYeJf53V43 * docs(qa): README 记录「运行节奏」并全面校准去-CI 后的措辞 Added an 'Operating cadence — when to run this (it is NOT in CI)' section to the checklist README: run `pnpm check:platform-checklist` before a release, after a large platform surface lands, when touching the checklist, or beside a coverage-sweep/checklist-run — with the trade-off (a new kind/enum that lands between runs is caught at the next manual run, not on merge) spelled out and the one-line re-add if latency ever matters more than PR independence. Corrected every now-stale "fails CI" reference across README, coverage.json $comment, and 4 area-file history entries to say the manual check flags/​catches the drift instead of blocking a PR. Validator green (178 items, 27/2). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YD9f6FYyMraUWYeJf53V43 --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 64d764e commit be8561d

28 files changed

Lines changed: 17400 additions & 0 deletions
Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
---
2+
name: checklist-run
3+
description: >
4+
Execute the platform test checklist (docs/qa/platform-checklist/) against a real
5+
running app and produce a run record. Use whenever the maintainer says "测一下
6+
<功能>", "跑这个测试项", "run the checklist for <area/item>", "test this feature
7+
file", "验证 <功能点>", or points at a framework source file and asks whether it
8+
still works. Takes a SELECTOR (item id · area · capability kind · priority · a
9+
release · or a source-file path) and drives every matched item through its steps
10+
following RUNNER.md. The companion to `coverage-sweep` (which AUTHORS items); this
11+
one RUNS them. NOT a customer-published skill — internal agent tooling (lives in
12+
.claude/, never in the published `skills/` dir).
13+
metadata:
14+
# Hides this skill from interactive `npx skills add objectstack-ai/objectstack`
15+
# discovery — every SKILL.md outside `skills/` must carry this marker
16+
# (template-consistency.test.ts enforces it).
17+
internal: true
18+
---
19+
20+
# Checklist run — execute selected items against a live app
21+
22+
You resolve a **selector** to a set of checklist items, boot the app in isolation, drive
23+
each item's steps in the browser / over the API, and emit a **run record**. The method
24+
for judging each clause (verdicts, oracle hierarchy, evidence, the anti-false-positive
25+
self-check) is **`docs/qa/platform-checklist/RUNNER.md`** — read it first and obey it;
26+
this skill is the trigger, the selection contract, and the isolation/parallelism plan,
27+
not a second copy of the runner protocol.
28+
29+
Environment know-how (boot, the dist build model, the vendored-console trap, browser
30+
escape hatches) is the **`dogfood-verification`** skill — read it too. You are not
31+
reinventing how to boot; you are executing a specific list against a boot.
32+
33+
## 0. Resolve the selector — deterministic, no guessing
34+
35+
Never eyeball which items to run. Ask the resolver:
36+
37+
```
38+
node scripts/checklist-select.mjs <selector> --json
39+
```
40+
41+
Selectors (one per run):
42+
43+
| selector | runs |
44+
|---|---|
45+
| `platform-core.console-login` (bare id) | that one item |
46+
| `area:records-forms` (or bare `records-forms`) | every item in the area |
47+
| `capability:hook` | items mapped to a metadata kind in `coverage.json` |
48+
| `priority:P0` | the standing smoke |
49+
| `surface:api` | every API-surface item (cheap — no browser build needed) |
50+
| `since:v17` | everything introduced in a release (the release-sweep filter) |
51+
| **`file:packages/plugins/plugin-approvals/src/approval-service.ts`** | **items whose `source[]` cites that file — "test whatever covers this file"** |
52+
| `all` | the whole checklist |
53+
54+
`--json` gives the runnable list (id · priority · surface · revision). **Blocked items are
55+
excluded by default** — they can't run on stock fixtures; pass `--include-blocked` only to
56+
record them as `blocked` with their fixture reason. **Pin the `revision`** the resolver
57+
reports into the run record: a verdict is only valid for the revision it ran against.
58+
59+
## 1. Plan the run by surface — build only what you need
60+
61+
Read the matched items' `surface`:
62+
63+
- **All `api` / `build` / `cli`** → no console build. Boot the framework (`objectstack dev`)
64+
and drive REST/CLI. Fast (~minutes).
65+
- **Any `browser` / `mixed`** → you need the vendored console dist. It builds SEPARATELY
66+
from the showcase workspace closure (`pnpm objectui:build` from the pinned `.objectui-sha`);
67+
the first boot 404s `/_console/` until it exists (dogfood §2 — a real precondition, record
68+
it, don't fake a block). Budget the build (~10–30 min on a cold monorepo); it dominates
69+
wall time, the browser driving is minutes.
70+
71+
Build once, up front, for the whole run.
72+
73+
## 2. Isolate, then execute (per dogfood §0)
74+
75+
- Own free non-default port + own file DB **per concurrently-running item**
76+
(`--seed-admin -d file:/tmp/<run>/<item>.db`). Two runs sharing a port/DB/browser tab is
77+
the `shared-browser-tab` trap.
78+
- **Parallelism:** fan API-surface items out in parallel (each its own port, cheap). Run
79+
browser items **few-at-a-time** (2–3), each its own port + browser context — a single
80+
machine's CPU and one shared display contend past that. When dispatching runner
81+
subagents, **they must be `opus`**, each given: the item JSON, RUNNER.md, the
82+
dogfood skill, its own port/DB, and the results-out-of-repo rule (§4).
83+
- Execute each item's `steps` faithfully; judge each `acceptance` clause and each
84+
`negative` against its declared `oracle`, capturing the `evidence` the clause names.
85+
**Server truth outranks pixels; DOM only after a screenshot confirms render; a `fail`
86+
needs reproduction ×2 + the automation self-check + a filed issue** (RUNNER §rules).
87+
88+
## 3. When the run teaches you something about the ITEM
89+
90+
A run that discovers the item's `steps` are wrong (a moved route, a renamed key, an
91+
expiry path that needs localStorage cleared too) is the checklist working. That is a
92+
checklist EDIT — do it in a **worktree** (PD#11): revise the item, bump `revision`, append
93+
a `history` entry, keep `node scripts/check-platform-checklist.mjs` green, and land it on a
94+
task branch. Product defects found while running go to `FOLLOW-UPS.md` (or a filed issue)
95+
as expected-fail probes — never tick a clause green over a real defect.
96+
97+
## 4. The run record — results do NOT go in the repo
98+
99+
Write one JSON per run in the shape RUNNER.md defines (env with framework sha +
100+
`.objectui-sha` + port + db; per-clause verdicts each naming its evidence; derived item
101+
verdict; issues). **`runs/` is git-ignored** — the record and its screenshots stay in the
102+
executing environment / the tracking issue / an external QA store, never committed. The
103+
committed source is the checklist under `areas/`; a run is a dated assertion about one
104+
build and belongs with that build's artifacts. Report the per-clause verdict table + the
105+
evidence paths + the env-setup-vs-test time split back to the maintainer.
106+
107+
## Guardrails
108+
109+
- **Don't fake coverage.** Missing fixture → `blocked(fixture)` with the reason; unbuilt
110+
console → build it or record `blocked(environment)`; a half-proven item is `partial`,
111+
not `pass`. A blocked verdict WITH evidence is a successful run; a faked pass is not.
112+
- **Don't run blocked items as if runnable** — the resolver hides them for this reason.
113+
- **One selector, one run record.** For a release sweep, run `since:vN` and `priority:P0`
114+
as separate records rather than smearing them together.
Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
---
2+
name: coverage-sweep
3+
description: >
4+
Re-audit the platform test checklist (docs/qa/platform-checklist/) for coverage
5+
gaps and author the missing items — the five-angle capability sweep. Use whenever
6+
the maintainer says "跑一轮 coverage sweep", "run a coverage sweep", "排查测试清单
7+
遗漏", "审计测试覆盖", or asks whether some platform surface "有测试吗" and the
8+
answer needs verifying rather than recalling. Also the right tool after a large
9+
platform surface lands or before a major release. NOT a customer-published skill —
10+
this is internal agent tooling (lives in .claude/, never in the published
11+
`skills/` dir).
12+
metadata:
13+
# Hides this skill from interactive `npx skills add objectstack-ai/objectstack`
14+
# discovery — every SKILL.md outside `skills/` must carry this marker
15+
# (template-consistency.test.ts enforces it).
16+
internal: true
17+
---
18+
19+
# Coverage sweep — keep the platform test checklist honest
20+
21+
The canonical method lives in **`docs/qa/platform-checklist/SWEEP.md`** — read it
22+
first and follow it; this skill is the trigger and the orchestration contract, not a
23+
second copy of the procedure.
24+
25+
## What you are producing
26+
27+
A delta on `docs/qa/platform-checklist/`: new/extended items in `areas/*.json`, a
28+
reconciled `coverage.json`, defects/docs-drift appended to `FOLLOW-UPS.md` — all
29+
validating green under `node scripts/check-platform-checklist.mjs`, landed on a task
30+
branch per AGENTS.md (worktree-first, PD#11).
31+
32+
## Orchestration contract
33+
34+
1. **Worktree first** (PD#11): `git worktree add ../objectstack-<task> -b <branch> main`.
35+
All edits there. Read the checklist's current state before dispatching anything.
36+
2. **Five READ-ONLY gap hunters in parallel** — one per SWEEP.md angle (console UI /
37+
spec enums / routes & runtime / built-in apps / docs claims). Each gets: the current
38+
item-id list, the already-known waivers and blocked items (don't re-report), and the
39+
output contract `surface | evidence path | coverage verdict | proposed id | sketch |
40+
fixture?`. Hunters write NO files.
41+
3. **Dedupe into a scratch register** (delete it before landing). Cross-angle
42+
duplicates are high-priority signal, not noise.
43+
4. **Per-area writer agents** — one agent per `areas/*.json` file so writers never
44+
collide; nobody but the orchestrator touches `coverage.json` or `scripts/`.
45+
Every item follows README.md's deep-test contract; missing fixtures become
46+
`blocked`/`knownGaps`, never faked coverage. Writers ground every endpoint, enum,
47+
and error code in source before asserting — treat this skill's own briefs as
48+
hypotheses, source as truth.
49+
5. **Reconcile centrally**: un-waive any kind a hunter proved has a stock fixture
50+
(four of six waivers were stale in the 2026-08 sweep — re-audit every waiver every
51+
time), map new items in `coverage.json`, pin `enumSource` on any new variants
52+
matrix (see README "Variants stay fresh automatically").
53+
6. **Validate + land**: validator green, then commit on the task branch. Product
54+
defects and docs-drift go to `FOLLOW-UPS.md`; security-sensitive findings are
55+
NEVER filed publicly without the maintainer's decision.
56+
57+
## Scale guidance
58+
59+
A full sweep is ~5 hunter + ~8 writer agents. For a scoped question ("X 有测试吗?"),
60+
run ONE hunter on the relevant angle, verify against the checklist, and author only
61+
what's missing — same contract, smaller fleet.

‎.github/workflows/lint.yml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -193,6 +193,14 @@ jobs:
193193
- name: ADR anchors + number uniqueness (governed code names its decision)
194194
run: pnpm check:adr-anchors
195195

196+
# NOTE: the standing platform test checklist (docs/qa/platform-checklist/)
197+
# is validated by `pnpm check:platform-checklist`, but by MAINTAINER
198+
# DECISION it is NOT wired into CI — it runs on a periodic manual cadence
199+
# (before a release / after a large platform surface lands), not on every
200+
# PR. The checklist is a QA ledger, not a code gate; keeping it out of the
201+
# per-PR path means an unrelated PR is never blocked by checklist drift.
202+
# Run it by hand: `pnpm check:platform-checklist`. See that dir's README.
203+
196204
# #3280/#3290 org-identifier guard: `organizationId` is the blessed
197205
# developer-facing name for the caller's active org in hook/action bodies;
198206
# the `session.tenantId` alias was REMOVED in v11 (#3290). Keeps our own
Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
# Follow-ups — open items from the capability-coverage sweep (2026-08-08)
2+
3+
Decision register from the capability-coverage sweep. The gap items found by the
4+
five-angle sweep have all been authored into `areas/*.json` (checklist grew 84 → 170
5+
items; the `api`/`datasource`/`mapping` coverage waivers were corrected). What remains
6+
here is what the sweep surfaced that is **not** a checklist item: product defects to
7+
decide on, and docs that promise retired capabilities.
8+
9+
## 1. Product defects found during the sweep (decide handling)
10+
11+
These are real runtime/UI defects the gap hunters hit while grounding items. Each is
12+
captured inside the relevant checklist item as an **expected-fail probe** (so a run
13+
records the actual behavior instead of ticking green), but they are defects, not test
14+
gaps. Security-sensitive ones were deliberately NOT filed publicly — your call.
15+
16+
| # | defect | evidence | captured in | sensitivity |
17+
|---|---|---|---|---|
18+
| D1 | **Saved-report schedule routes lack an owner check** — `report-service.ts` `unscheduleReport`/`listSchedules` ignore `_context`, so user B can delete user A's report schedule (cross-owner destructive access). The read/run/delete routes ARE owner-gated (deny-as-404); only the schedule routes leak. | packages/plugins/plugin-reports/src/report-service.ts (unscheduleReport/listSchedules) | dashboards.saved-report-ownership (known-gap probe clause) | **SECURITY — not filed publicly; awaiting your decision** |
19+
| D2 | **AppManagementPage enable/disable/set-default/delete are client-only stubs** — the handlers call `toast.success()` with a `TODO: Replace with real API call` and issue no request; an admin sees "success" while nothing changes. | objectui apps/console/src/pages/system/AppManagementPage.tsx | platform-core.app-management-toggle (expected-fail probe) | UX-integrity — safe to file |
20+
| D3 | **`useGlobalUndo.executeOp` issues a bare `ds.update` with no `ifMatch`** — record undo can silently clobber a concurrent edit (no OCC guard on the undo path). | objectui react/src/hooks/useGlobalUndo.ts | records-forms.record-edit-undo (observe-and-flag clause) | correctness — safe to file |
21+
| D4 | **`SharedViewLink` builds dead `/share/<object>/<view>?token=` URLs** — client-generated token, no matching console route (only `/s/:token`), no server persistence. Registered but unused. | objectui plugin-view/src/SharedViewLink.tsx | — (not an item; demo-grade) | low — file a cleanup issue |
22+
| D5 | **List "Share" button is a no-op** — renders when `schema.sharing` is set but has no onClick. | objectui plugin-list/src/ListView.tsx | — | low — file a cleanup issue |
23+
| D6 | **`/api/v1/datasources` admin CRUD has no route ledger** — mounted by serve.ts, absent from rest-route-ledger.ts (tranche-3 discipline gap). | packages/services/service-datasource/src/admin-routes.ts | integration-system.datasource-admin-lifecycle (source note) | low — internal discipline |
24+
| D7 | **Parent-only PATCH does not revalidate a stale dependent child** — `evaluateOptionVisibility` skips fields absent from the payload, so changing only the parent leaves a now-invalid child value in place server-side; integrity rests entirely on the client clear. | packages/objectql/src/validation/rule-validator.ts (`!(name in data) continue`) | records-forms.cascading-multilevel-and-clear (knownGap) | integrity — safe to file |
25+
| D8 | **Lookup cascade scope is existence-only server-side** — `assertReferencesResolve` accepts any EXISTING id regardless of `lookupFilters` scope (a cross-account contact that exists is accepted on direct POST). May be by-design (filters = UI courtesy) — needs a maintainer ruling: declared ≠ enforced, or documented courtesy. | packages/objectql/src/engine.ts (assertReferencesResolve) | records-forms.cascading-multilevel-and-clear (knownGap) | integrity/design — needs ruling |
26+
27+
## 2. Docs promise capabilities the runtime doesn't deliver (PD#10, docs side — file docs issues)
28+
29+
The capability docs advertise features that were retired or never shipped. Under Prime
30+
Directive #10 ("never advertise a capability the runtime doesn't deliver") these are
31+
docs bugs, not checklist items.
32+
33+
- **Recycle bin / soft delete** — promised in `content/docs/capabilities/{data,integrations}.mdx`,
34+
but `enable.trash` was RETIRED ("every delete has always been a hard delete; soft delete
35+
parked at #3146", object.zod.ts retired-key guidance). → fix docs or ship the feature.
36+
- **Recently-visited lists** — `enable.mru` retired/never implemented. The console DOES ship a
37+
recents rail (UnifiedSidebar) — reconcile whether the doc claim maps to that surface or a dead one.
38+
- **TV display pages / discussion threads** — promised (analytics.mdx, build-without-code.mdx).
39+
Discussion = the real chatter surface (now covered by records-forms.record-discussion-mentions);
40+
display pages have no spec surface found → confirm removal or file.
41+
- **Five data-depth scopes** (permissions.mdx) — `own_and_reports`/`unit`/`unit_and_below` are
42+
ENTERPRISE (hierarchy-security). The open checklist correctly drives own/org only. Optional both-sides
43+
probe: authoring an intermediate depth in the open edition must degrade LOUDLY (ADR-0049), not
44+
silently to `own` — could become a checklist item if you want it.
45+
46+
## 3. Fixtures worth adding (would un-block currently-blocked items)
47+
48+
The 8 `blocked` items are blocked on missing stock fixtures, not on the platform. Adding
49+
these to the showcase would make them runnable:
50+
51+
- a `publicSharing.enabled` object → unblocks `access-security.share-link-capability-tokens`.
52+
- one configured OIDC/social IdP → unblocks `identity-auth.oauth-app-consent-loop`,
53+
`linked-accounts-social`, and the existing `sso-enforced-first-paint`.
54+
- a gantt view with `dependenciesField` + `lockField` + `parentField` → unblocks the
55+
fixture-gated variants of `records-forms.gantt-interactions`.
56+
- a not-auto-bound audience suggestion → unblocks the confirm/dismiss half of
57+
`access-security.suggested-binding-loop`.
58+
- the `IMPORT_CONSOLE_LIVE` import-harness backend → unblocks `records-forms.import-job-undo-cancel`.
59+
- an approval-escalation clock-control/`runEscalations()` harness → unblocks `approvals.sla-escalation`.
60+
- a second signed-up (non-admin) user in seeds, or a documented sign-up step in the runner →
61+
removes the recurring "needs a 2nd user" knownGap on several persona-gated items.
62+
63+
## 4. Notes
64+
65+
- `PENDING-GAPS.md` (the full deduped gap register that drove the authoring) can be deleted
66+
once you've reviewed §1–§2 above — it was scaffolding; this file is the durable residue.
67+
- The checklist itself (`areas/*.json`, `coverage.json`, `README.md`, `RUNNER.md`,
68+
`scripts/check-platform-checklist.mjs`) ships in this branch; this file carries the
69+
decisions that remain with the maintainer.

0 commit comments

Comments
 (0)