Repository navigation
docs(skills): ship at least three json evals per published skill in the objectui fixture shape - #19721
Merged
Conversation
…he objectui fixture shape Every one of the ten skills under skills/ now carries evals/*.json in the objectui shape (skill_name + evals[] with prompt / expected_output / files / assertions.must_contain / must_not_contain), one eval per stated "use when" trigger family plus at least one anti-pattern case per skill. The objectstack-data README loses its placeholder sentence and lists the fixtures its candidate-scenario list became; the objectstack-automation markdown case is converted to a json eval and kept beside the markdown (its ratchet row stays satisfied). No SKILL.md, rules/** or references/** line changes. Claude-Session: https://claude.ai/code/session_0129ZpnaBcYZZ51rCvQiXg6C Co-authored-by: Claude <noreply@anthropic.com>
os-support-ai
added
the
skip-changeset
Claude
github-actions
Bot
added
size/l
documentation
Collaborator
Author
|
Review reading from the dispatching seat (director, summon #27, CI on Content review (against GitHub, not the report): 13 files, all under Waiting on one word from the maintainer (asked in chat): A — this PR gains one commit adding the 11 Generated by Claude Code |
This was referenced Sep 22, 2026
…-skills token ratchet One CEILINGS row per new skills/*/evals/*.json file, each pinned at its measured count on the tree that added it (zero headroom), in a block that quotes the maintainer's batch #213 item 2 ruling and the follow-up word recorded as issue comment 5776586573. No existing row moves, the priced population is unchanged, and evals/** stays priced. Claude-Session: https://claude.ai/code/session_0129ZpnaBcYZZ51rCvQiXg6C Co-authored-by: Claude <noreply@anthropic.com>
Collaborator
Author
维护者速读(终稿)席位:总监席(summon #27, 改了什么:10 个发布 skill 每个新增 为什么改:您在批 #213 同意「先补 evals,再切分超长 SKILL.md」,evals 是切分前后「切了没变差」的量具;天花板行是您第二次回字「天花板行按照你的意见就行」的执行(A 方案)。 风险与代价:发布包多 11 个文件、约 12k token,只在有人主动读 evals 时进上下文;棘轮把它们全部计价,以后每个文件只能缩不能长。回滚 = revert 一个 squash 提交。 席位意见:通过。内容抽查过,CI 全绿,不改任何 SKILL.md 文本;唯一带判断的是把 evals 计入棘轮,这是您定的。 你要做的:合并本 PR(Squash)。合并后 #19714 自动关闭,#19715(切分卡)解锁进 skills 车道。 Generated by Claude Code |
os-zhuang
approved these changes
Sep 22, 2026
os-zhuang
marked this pull request as ready for review
September 22, 2026 13:33
akarma-synetal
pushed a commit
to akarma-synetal/framework
that referenced
this pull request
Sep 28, 2026
…ferences, TOC the seven long reference files, trim data's description (objectstack-ai#19738) Fixes objectstack-ai#19715 Clause-②: no ## 维护者速读(草稿) **改了什么** — 四个超过 500 行的已发布 `SKILL.md`(platform 1223 → 487 · automation 961 → 448 · data 854 → 469 · upgrade 600 → 488)按标题边界切分:每个被移走的段落与代码块**逐字节**落到该技能自己的 `references/` 下(新增 9 个参考文件),原位只留一行指针;七个超过 300 行的参考文件顶部各加 5 行目录(锚到已有标题);`objectstack-data` 的 `description` 压到 689 字符,触发短语与排除句原样保留、只删括号枚举。零删内容:20 个移动块中 17 个字节相同,3 个仅改了 10 处相对链接路径(逐条列在下文)。 **为什么改** — 决策批次 objectstack-ai#213 第 1 项,维护者裁决「1 2 4 同意」:超长 SKILL.md 会被 `head` 预读截断,示例与运维尾段读起来像第二个技能;切成「路由 + 规则」后入口文件只装规则、决策表与指针,长示例按需跟指针再读。 **风险与代价(含回滚)** — ① 直接读 `SKILL.md` 单文件时,105 个 eval 断言词里有 18 个(从 103 降到 85)现在只经指针可达;整棵技能树(SKILL.md + rules + references)的读数 105/105 前后不变,方法与局限见下文。② 为了到 500 行,platform 的第二部分(插件开发)与 automation 的「状态机与审批」整段下沉,其中含少量决策表——它们仍在树里,入口页保留指向 `rules/*.md` 的路由行。③ 三个门禁脚本的数据行跟着文本走(token 天花板行、role-word 基线两行搬家、scaffold 政策的载体路径),没有新增门禁。④ 令牌总量 +2058(指针行 + 目录行),四个入口行已按落地值下锁,七个目录文件中 4 行按裁决上调、3 行有余量按落地值下锁。回滚:整个 PR 一次 revert 即可,没有生成物或状态迁移。 **席位意见** — **建议批准并合并,但先看一眼「Sections that left SKILL.md」。** 裁决的八条形状逐条核过:零删内容(20 个移动块 17 个字节相同、3 个只改了 8 处相对链接;席位另做了四个技能的整行多重集比对,除链接与 description 外没有任何一行离开树)、四个入口文件全部 ≤ 500、data 的 description 689 字且触发短语与排除句原文保留、七个长参考文件各 5 行目录且锚点全部可达、净 +77 ≤ +80、无新门禁、changeset 按 objectstack-ai#19721 惯例。evals 读数按机械法前后树列 105 / 105 不变;入口页单读 103 → 85,离开的 18 个词都在指针可达处 —— 这是切分的代价,不是丢失。token 棘轮四行入口下锁、九个新文件零余量钉住、四行目录文件按裁决引文上调。第 0 轮 CI 红是本 PR 的:`packages/rest` 的仓级测试钉住 automation 入口页必须写出状态内省路由,切分把唯一拼写搬走了;第 1 轮把拼写放回指针行(净 0 行,不改测试),席位在补丁头上重跑该测试 8 / 8 绿。留给你的一个判断:为了到 500 行,platform 的插件开发与运维两大部分、automation 的状态机与审批、data 的 seeds / 字段组 / lint 整段下沉,其中含少量决策表 —— 它们仍在树里、入口页留有路由行;如认为某段必须留在入口页,点名即可,dev 按行预算换一段下沉。复核记录:PR 评论 5786766593(PASS,在席按服务档渲染);ACCEPT 在卡 objectstack-ai#19715。CI 读数:32 latest-per-name check runs — 25 success, 7 skipped, 0 in progress, 0 other。 **你要做的** — 这是 Tier H(`skills/**`)受管面:确认「哪些规则段可以离开入口页」符合你的意图(见「Sections that left SKILL.md」),然后手工合并;如认为某段必须留在入口页,请点名,我按行预算再换一段下沉。 --- ## What this does The four published `SKILL.md` files over 500 lines are split at heading boundaries into routing + rules. Every moved paragraph and code block lands **verbatim** under that skill's `references/` (nine new files) with a one-line pointer left where it was; the seven reference files over 300 lines gain a 5-line table of contents anchored to their existing headings; `objectstack-data`'s `description` shrinks to 689 characters with the same trigger phrases and exclusion sentence. The token ratchet re-locks the four entry rows at their landed counts, pins the nine new files at theirs, and raises the four TOC'd rows that carried no headroom by exactly the added tokens under the ruling. `skills/README.md` and `content/docs/ai/skills-reference.mdx` are regenerated from the trimmed frontmatter by `gen:skill-docs`. Premise re-measured on `origin/main` `106d4c8dd` before editing: the four line counts (1223 · 961 · 854 · 600) hold; the seven files carried no TOC; data's `description` measured **1003** characters by a YAML folded-scalar parse (the card's 1,023 was a different counting; either way it sat against the 1,024 cap). `premise_still_valid: true`. ## Readings after (head `011dfd603`) | File | Before | After | |:--|--:|--:| | `skills/objectstack-platform/SKILL.md` | 1223 | **487** | | `skills/objectstack-automation/SKILL.md` | 961 | **448** | | `skills/objectstack-data/SKILL.md` | 854 | **469** | | `skills/objectstack-upgrade/SKILL.md` | 600 | **488** | | data `description` (YAML folded, trailing newline excluded) | 1003 chars | **689 chars** | | net lines across `skills/**` (tracked +77 / −1781, new files +1781) | — | **+77** (budget ≤ +80) | ## Diff proof — every moved block (source lines at `106d4c8dd` → destination), verified byte-for-byte against the base blob | # | Source (lines, count) | Heading(s) | Destination (lines) | Bytes | |:-:|:--|:--|:--|:--| | 1 | platform `SKILL.md` 39–68 (30) | `### Minimal Example` | `references/bootstrap.md` 3–32 | verbatim | | 2 | platform 116–157 (42) | `### Map Format (Key → Name)` · `### Barrel Import Pattern` | `references/bootstrap.md` 34–75 | verbatim | | 3 | platform 272–280 (9) | `### Scaffolding Command` | `references/bootstrap.md` 77–85 | verbatim | | 4 | platform 498–527 (30) | `### Plugin Loading Order Matters` · `### Programmatic Bootstrap (Without CLI)` | `references/bootstrap.md` 87–116 | verbatim | | 5 | platform 531–542 (12) | `## Multi-App Composition` | `references/bootstrap.md` 118–129 | verbatim | | 6 | platform 585–644 (60) | `## Complete Working Example` | `references/bootstrap.md` 131–190 | verbatim | | 7 | platform 649–1007 (359) | `# Part 2 — Plugin Development & Kernel Extension` (whole) | `references/plugin-development.md` 1–359 | 6 link paths re-pathed, else verbatim | | 8 | platform 1012–1213 (202) | `# Part 3 — Operations: CLI, Testing, Deployment` (whole) | `references/operations.md` 1–202 | verbatim | | 9 | automation 143–209 (67) | `### Flow Example — Auto-Escalate Overdue Cases` | `references/examples-flows.md` 3–69 | verbatim | | 10 | automation 378–760 (383) | `## State Machines & Approvals` (whole) | `references/state-machines-and-approvals.md` 1–383 | verbatim | | 11 | automation 851–916 (66) | `### Time-relative triggers — scheduled per-record date sweep` | `references/examples-flows.md` 71–136 | verbatim | | 12 | data 184–230 (47) | `## Field Groups (MVP)` | `references/examples-objects.md` 3–49 | verbatim | | 13 | data 300–359 (60) | `## Quick-Start Template` | `references/examples-objects.md` 51–110 | 1 link path re-pathed | | 14 | data 499–525 (27) | `### Lifecycle Hooks` | `references/examples-objects.md` 112–138 | 1 link path re-pathed | | 15 | data 565–622 (58) | `## Metadata Protection (\`protection\`)` | `references/examples-objects.md` 140–197 | verbatim | | 16 | data 626–767 (142) | `## Seed Data & Fixtures (\`defineSeed()\`)` (whole) | `references/seeds.md` 1–142 | verbatim | | 17 | data 771–823 (53) | `## Linting & Generation Quality` | `references/lint-rules.md` 1–53 | verbatim | | 18 | upgrade 269–301 (33) | `### 2.3 A worked R1 — the retired field-mapping \`transform\`` | `references/examples-upgrade.md` 3–35 | verbatim | | 19 | upgrade 449–496 (48) | `### 3.4 The report — the human half` | `references/examples-upgrade.md` 37–84 | verbatim | | 20 | upgrade 540–573 (34) | `## The v17-canonical shapes, compiled` | `references/examples-upgrade.md` 86–119 | verbatim | Method: for each row the base bytes (`git show 106d4c8:path`, the listed lines) were hashed against the destination's listed lines — 17 rows identical, 3 rows differ only on the link-path lines below. The upgrade `a`-tag anchors (`decide-alone-or-ask`, `reverse-check`) stayed in `SKILL.md` because they sit just outside the moved ranges. The four new multi-block files carry a one-line H1; the five single-block files begin with the moved heading itself. **The only non-verbatim bytes — 10 link-path retargets** (a relative link crossing a move boundary would otherwise dangle): - platform `SKILL.md` (staying text): `#verify-your-work`, `#ports--networking`, `#part-3--operations-cli-testing-deployment` → `./references/operations.md#…` (3) - `references/plugin-development.md` (moved text, lines 656/657/658/738/760/791 at base): `./rules/plugin-lifecycle.md` → `../rules/…` (2), `./rules/service-registry.md` → `../rules/…` (2), `./references/plugin-hooks.md` → `./plugin-hooks.md` (2), `../objectstack-data/SKILL.md` → `../../objectstack-data/SKILL.md` (1) - data `SKILL.md` (staying text): `#field-groups-mvp` → `./references/examples-objects.md#field-groups-mvp` (1) - `references/examples-objects.md` (moved text, base lines 359 and 523): `./rules/indexing.md` → `../rules/indexing.md` (1), `./references/data-hooks.md` → `./data-hooks.md` (1) Every intra-file anchor in the 15 touched markdown files resolves (github-slugger, the repo's slug authority; 100 anchors, 0 unresolved) and all 104 relative links under `skills/**` resolve to existing files (the two `./crm_index.md` / `./crm_user_guide.md` spellings in `objectstack-ui/rules/pages.md` are illustrative example text on the base, untouched). ## Sections that left SKILL.md, and why the route had to widen The card's route (platform's ops tail + the two 60-line complete examples) reaches 894 lines on platform — the fenced code in the four files totals 462 / 310 / 223 / 162 lines against the 723 / 461 / 354 / 100 that had to go, so examples alone cannot reach 500 on three of the four. The cut therefore also moves whole self-contained parts: platform Part 2 (plugin development, which already has its own `rules/*.md` and `references/plugin-hooks.md` — the entry keeps a routing line to them), automation's State Machines & Approvals, data's Seed Data & Fixtures, Field Groups and Linting. Decision tables that now live behind a pointer: platform's ObjectKernel vs LiteKernel, Plugin Loading Order, Well-Known Plugin Names, MetadataPlugin boundary, Feature Flags, the ops tables; automation's Approver Types, Node Config, Branching, Best Practices and the state-machine rule; data's seed tables and the lint-rule table. Every pointer names the destination and the moved headings (heading names only — no paraphrase). If the maintainer wants a named section back in the entry, it costs its line count against the 500 ceiling; say which and I swap. ## Evals reading (card item 5) — method, before / after, limit There is no executable runner in this repo: the fixtures `skills/*/evals/*.json` are read by `scripts/check-skills-token-ratchet.mjs` for pricing only (no script consumes `must_contain`). The reading is therefore the mechanical one the dispatch names: for every eval of the four skills, each `must_contain` term's presence (substring, case as written) in that skill's `SKILL.md` alone and in the skill tree (`SKILL.md` + `rules/**` + `references/**`), on `106d4c8dd` and on this head. Script: `eval-reading.mjs` (in the scratchpad, not committed). | Skill | SKILL.md alone, before | SKILL.md alone, after | Tree, before | Tree, after | |:--|--:|--:|--:|--:| | objectstack-platform | 24/24 | 18/24 | 24/24 | 24/24 | | objectstack-automation | 26/26 | 20/26 | 26/26 | 26/26 | | objectstack-data | 35/37 | 31/37 | 37/37 | 37/37 | | objectstack-upgrade | 18/18 | 16/18 | 18/18 | 18/18 | | **total** | **103/105** | **85/105** | **105/105** | **105/105** | The tree column — the reading of record for a skill whose entry routes to references — is unchanged for all 105 terms. The 18 terms that left `SKILL.md` alone, each with the pointer on the eval prompt's path (the two data terms already absent on the base, `schemaMode` and `type: 'secret'`, live in `rules/datasources.md` and `rules/security.md` and are untouched): | Eval | Term | Now in | Pointer in SKILL.md | |:--|:--|:--|:--| | platform objectstack-ai#2 (audit plugin) | `registerService`, `ctx.hook(`, `metadata:reloaded`, `init(`, `destroy` | `references/plugin-development.md` (+ `rules/plugin-lifecycle.md`, `references/plugin-hooks.md`) | the Part 2 pointer, which also names `rules/plugin-lifecycle.md` · `rules/service-registry.md` · `references/plugin-hooks.md` | | platform objectstack-ai#6 (production deploy) | `OS_TRUSTED_ORIGINS` | `references/operations.md` | the Part 3 pointer (names Ports & networking · Deployment targets) | | automation objectstack-ai#1 (nightly sweep) | `defineFlow`, `type: 'schedule'` | `references/examples-flows.md` | the Flow Example pointer; the Flow Types table row `schedule` stays in the entry | | automation objectstack-ai#3 (approval + revise) | `type: 'approval'`, `type: 'back'`, `maxRevisions` | `references/state-machines-and-approvals.md` | the State Machines & Approvals pointer (names Send-back for revision · Node Config); `approval_revise` and `revise` stay in the entry | | automation objectstack-ai#5 (function step) | `timeoutMs` | `references/examples-flows.md` | the Flow Example pointer | | data objectstack-ai#4 (seeds) | `env:`, `daysFromNow` | `references/seeds.md` | the Seed Data pointer (names Dynamic values (CEL)); `defineSeed`, `externalId`, `upsert` stay in the entry | | data objectstack-ai#2 (invoice lines) | `deleteBehavior`, `inlineEdit` | `rules/relationships.md`, `rules/field-types.md`, `references/lint-rules.md` | the Relationship Patterns table's link to `rules/relationships.md` (unchanged) | | upgrade objectstack-ai#2 (conditionalRequired) | `requiredWhen` | `references/examples-upgrade.md` | the v17-canonical shapes pointer | | upgrade objectstack-ai#3 (transform residue) | `never executed` | `references/examples-upgrade.md` | the worked R1 pointer | Limit of the method: substring presence is not a graded run — it cannot say whether an agent given the prompt would follow the pointer. It is the reading the card can have today; no runner was built (item 6). No assertion was tuned. ## Token ratchet (`scripts/check-skills-token-ratchet.mjs`, convention ceil(utf8 bytes / 4)) Bundle total 152,338 → **154,396** (+2,058: the 20 pointer lines, seven 5-line TOCs, four new-file titles, minus the description trim); ratcheted ceiling sum 157,621 → 154,915 (the four entry re-locks bank their shrink). Gate and `--self-test` green on this head (54 authored files within ceilings, 65 self-test cases). | Row | Ceiling before → after | Kind | |:--|:--|:--| | `objectstack-platform/SKILL.md` | 12984 → 5833 | re-lock at landed count | | `objectstack-automation/SKILL.md` | 12768 → 5762 | re-lock | | `objectstack-data/SKILL.md` | 10009 → 6128 | re-lock | | `objectstack-upgrade/SKILL.md` | 8333 → 6193 | re-lock | | `objectstack-data/rules/field-types.md` | 3032 → 3158 | **raise** +126 (TOC; zero headroom) — ruling cited in the row | | `objectstack-ui/rules/dashboards.md` | 6090 → 6252 | **raise** +162 (TOC; zero headroom) | | `objectstack-ui/rules/list-views.md` | 3011 → 3141 | **raise** +130 (TOC +132, 2 absorbed) | | `objectstack-ui/rules/pages.md` | 5501 → 5692 | **raise** +191 (TOC; zero headroom) | | `objectstack-data/references/data-hooks.md` | 12611 → 10066 | re-lock (TOC +182 landed inside 2,727 headroom) | | `objectstack-data/rules/relationships.md` | 3778 → 3676 | re-lock (TOC +154 inside 256 headroom) | | `objectstack-data/rules/validation.md` | 3109 → 2756 | re-lock (TOC +155 inside 508 headroom) | | 9 new rows | `examples-flows` 1436 · `state-machines-and-approvals` 5569 · `examples-objects` 1779 · `lint-rules` 970 · `seeds` 1299 · `bootstrap` 1351 · `operations` 3125 · `plugin-development` 3137 · `examples-upgrade` 1197 | pinned AT landed count, zero headroom | The dispatch's assumption that all seven TOC'd rows sat at zero headroom was false for four of them; those four are re-locked (a lowering, always legitimate) rather than raised. ## Other gate data that followed the moved text (no new gate, ratchet or check script) - `scripts/role-word-baseline.json`: the automation entry's one `role` occurrence (the Approver Types row) and the upgrade entry's one (`role:` in the v17 agent shape) moved with their sections; the two ledger rows moved with them. 44 files / 123 occurrences before and after — a relocation, not an expansion; `check:role-word` green. - `scripts/sync-scaffold-emission-policy.mjs`: the published-restatement row that locates the Complete Working Example's `package.json` fence by heading now points at `references/bootstrap.md`, where the fence lives; `check:scaffold-emission-policy` green (2 carriers). - Generator-owned files: `check:skill-refs` stays green without regeneration — `build-skill-references.ts` owns only `_index.md`, `*.zod.ts` and subfolders under `references/`, so the new top-level hand-written files are inputs it tolerates; `_index.md` lists Zod schemas only. `gen:skill-docs` regenerated `skills/README.md` and `content/docs/ai/skills-reference.mdx` (the one file outside `skills/**` and the gate scripts). - `check-skill-identifier-liveness` leg-2 bindings (`## Seed Data` on platform, `### Access depth …` on data) stayed in their entries. ## Gates (captured exit before any pipe; union derived by `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands` on `011dfd603`, 70 families, `--ran` reconciles 70/70 with exit codes) All 70 derived families exit 0 on `011dfd603`, plus `pnpm --filter @objectstack/spec check:generated` (15 artifacts up to date after the `origin/main` merge). The ones that read a build were run after `pnpm --filter '@objectstack/lint...' build` and `pnpm --filter '@objectstack/client-react...' build` under the shared verify lock: `check:skill-examples` — "258 prose examples type-check across 3 surface(s)" (the moved os:check blocks included); `check:docs`; `check:docs-transcript-drift`; the two `@objectstack/lint` doc checks. Highlights: `check:skills-token-ratchet` + self-test ✓ · `check:skill-docs` ✓ · `check:skill-refs` ✓ · `check:skill-identifier-liveness` ✓ · `check:skill-top-level-keys` ✓ · `check:skill-compatibility` ✓ · `check:skill-frame-sync` ✓ · `check:doc-authoring` ✓ · `check:role-word` ✓ · `check:scaffold-emission-policy` ✓ · `check:nul-bytes` ✓ · `check:pm-dispatch-gates` (1905 cases) ✓ · `bare-root-worklist --self-test` ✓ (the `skills/*/references/**` liveness/precision pin holds over the enlarged population). `pnpm lint` (repo-wide eslint) is CI's run: the diff touches no `.ts`/`.js` source, only markdown, one JSON ledger and two `.mjs` gate scripts whose own self-tests ran green. Branch merged `origin/main` `170fd836a` (three spec commits, no overlap with this diff) before opening; `packages/spec` was rebuilt after the merge. ## Changeset `skip-changeset`, per the last merged `skills/**` PR (objectstack-ai#19721: `documentation` · `skip-changeset`, Clause-②: no) and measured: no workspace package's `files[]` ships `skills/` (positive control: `packages/cli` ships `bin`). `Clause-②: no`. Tier H — draft, the maintainer merges by hand. ## Acceptance notes - Two data eval terms were already reachable only through `rules/` on the base (`schemaMode`, `type: 'secret'`) — noted, not a card. - `objectstack-ui/rules/pages.md` lines 356/448 carry `./crm_index.md`-style example links that resolve to nothing; they are illustrative package-docs text inside the Docs section, untouched here — noted, not a card. --- _Generated by [Claude Code](https://claude.ai/code/session_01Wnstp2kTth7sGXfr8fXypc)_ --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #19714
Clause-②: no
Patch round 2026-09-22: commit 41844ce adds the 11 ratchet rows (one per new evals json, pinned at its measured count, zero headroom) under the maintainer's ruling 「天花板行按照你的意见就行」 — issue comment 5776586573, batch #213 item 2, letter A of the open question below;
check-skills-token-ratchet.mjsand its--self-testare green on that head, so the «Token ratchet» section below describes the state before this commit.What this does
Every one of the ten published skills under
skills/now shipsevals/*.jsonfixtures in the objectui shape —{ skill_name, evals: [{ id, prompt, expected_output, files, assertions: { must_contain, must_not_contain } }] }— copied verbatim fromskills/objectui/evals/app-composition.jsonin the sibling repo. 11 new json files, 54 new evals (59 with the 5 pre-existing objectstack-ui ones); every skill has at least 3, one per stated «use when» trigger family in its description, and at least one anti-pattern case whosemust_not_containnames the wrong spelling the skill exists to prevent (conditionalRequired:,type: 'unique',deleteBehavior: 'set_null',many_to_many,next:on flow nodes,{{in flow values,o: {translation shape,cursor:/distinct: truequery keys,sourceView,data:beforeInsertkernel hook,defineAgent(for third parties,??alias on an upgrade, …).Prompts are written as realistic authoring requests with business nouns;
expected_outputstates the shape an agent following the skill must emit. Everymust_containstring is something the skill's own text tells the agent to write — the full token → file:line citation list is at the bottom of this body (253 tokens, 0 untaught).Two decisions the card left to the dev, stated here:
evals/approvals/test-revise-loop.md): converted to json (flows-triggers-approvals.jsoneval 3 is the same budget_approval scenario, same required shape) and kept beside it, unchanged. Reason:SKILL.mdlines 954–955 link the markdown file by path (no SKILL.md line may change in this PR), and the token ratchet carries aCEILINGSrow for that path — a missing file is RED there, so deleting it would need a gate edit.skip-changeset. Measured against the last three merged skills-only PRs found bygit log origin/main --oneline -- skills/after deepening the shallow clone (docs(skills): pages.md rule 2 states the enforceable reason, not the retired ADR-0048 claim #19445, docs(skills): citePLATFORM_CAPABILITY_TOKENSinstead of restating its count in objectstack-platform #18839, docs(skills): navigation.md states the measured behaviour of colSpan and span: 'full', drops the deprecated claim #18674): each carriedskip-changesetand no.changeset/*.md.skills/**ships throughnpx skills add, not through any released package'sfiles[](nopackages/**/package.jsonlists it), so nothing published from an npm package changes.Files
evals/skills-tools-knowledge.json(new)*.skill.ts1 · LLM provider 2 · tools a skill grants 1, 4 · knowledge source / RAG 3 · anti: agent authoring 5evals/endpoints-auth-routes.json(new)*.endpoint.ts1, 2 · auth providers 4 · custom routes 3 · REST generator 5 · anti:transform:,'restore'1, 5evals/flows-triggers-approvals.json(new) +evals/approvals/test-revise-loop.md(kept)*.flow.ts1, 5 · event-driven rule 2 · approval chain 3 · screen flow / wizard 4 · anti:next:,values:,{{1, 2evals/objects-fields-relationships.json,evals/hooks-security-seeds-datasources.json(new);evals/README.mdrewritten*.object.ts/ field types 1 · relationships 2 · validation 3 ·visibleWhenrules 4 · hooks 5 · access control 6 · external database 7 · seeds / demo data 8 · anti:type: 'password',many_to_many,type: 'unique',conditionalRequired:1–4evals/cel-predicates-formulas.json(new)F/P/celtemplates 1, 3, 4 · «how do I express X» 2, 5 · anti:{record.,ISCHANGED(,new Date(1–5evals/bundles-locales-coverage.json(new)*.translation.ts1 ·translationitem 3 · generated bundle from a plugin 4 · new locale 2 · missing-translation warnings 5 · anti:o: {, ICU plural 1, 3evals/config-plugins-ops.json(new)objectstack.config.ts1 · plugin 2 · capability on 3 · Hono mount 4 ·osCLI 5 · deployment 6 · anti:workflows:,data:beforeInsert,new DriverPlugin1, 2, 6evals/filters-pagination-search.json(new)cursor:,distinct: true,fuzzy2–4evals/views-apps-actions-pages.json(new) +evals/analytics-inline-vs-dataset.json(unchanged, 5);evals/README.mdlists both*.view.ts1 ·*.app.ts2 ·*.action.ts3 ·*.page.ts+src/docs/*.md4 · interface page +*.report.ts5 · wizard form 6 ·*.dashboard.ts/*.dataset.tsexisting 1–5 · anti:groupBy:,sourceView,{record.1, 3, 5evals/protocol-major-upgrade.json(new)[REMOVED]prescription 2 · residue decision 3 · anti:sed -i,??1, 2No
SKILL.md,rules/**orreferences/**line changes (net 0 on everySKILL.md, measured withgit diff --numstat a251aaa -- 'skills/*/SKILL.md'→ empty).Verification
Run on 02b931d (
git rev-parse --short HEAD), worktreeobjectstack-issue-19714, basea251aaa.skills/*/evals/*.json: 12 files parse;skill_nameequals the frontmatternamein all 12; ids unique per file; every eval carriesid/prompt/expected_output/files/assertions.must_contain/must_not_contain; no emptymust_contain.dispatch-gateschange-set reading:changed lines: 630 (+613 / -17; 13 file(s))→ net +596 acrossskills/**/evals/**(budget ≤ +900);SKILL.mdnet 0.grep -rn evalsoverpackage.json/scripts/finds none here, and in objectui only the static oraclescripts/check-skill-eval-tokens.mjs(pnpm check:skill-eval-tokens), which checks that everymust_containtoken is taught as a whole token somewhere in the bundle's markdown. I invoked its exportedanalyze({ root })against this tree (the CLI is pinned to objectui's own root):bundles: 10 · preconditions: [] · shapeFindings: [] · rows: 253 · rows with zero whole-token hits (bundle-wide): 0. Per the card, no runner was built.must_containtokens, each located as a whole token in its own skill's hand-authored text (SKILL.md first, thenrules/*.md, then hand-authoredreferences/*.md; generated_index.md/react-blocks.mdexcluded); list at the bottom.grep -naP '[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]'→ no hits.Gates (derived by
node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands, 23 commands;--ranreconciliation with exit codes: 23 derived, 23 run, 0 NOT-MEASURED, 0 unrun)node scripts/check-ci-filter-parity.mjsnode scripts/check-closing-keyword-parity.mjs(+--self-test)node scripts/check-comment-mask-corpus.mjsnode scripts/check-doc-route-spelling.mjs --advisory(+--self-test)node scripts/check-skills-token-ratchet.mjs11 of 55 published bundle file(s) failed their check— every one is a NEW file «carrying no ceiling»; every pre-existing row is green (data README 142/143, ui README 287/289, analytics json 1102/1102, revise-loop md 550/1329). See «Token ratchet» below.node scripts/check-skills-token-ratchet.mjs --self-testevery discovered AUTHORED file carries a ceiling today, listing exactly the same 11 files; 65/65 pass on the pristine base tree (control run in the shared checkout at a251aaa). Same cause, not a second finding.pnpm --filter @objectstack/lint run check:doc-formula-expressionsPREREQUISITE NOT MET(formula + lint not built — identical on the pristine base tree); afterturbo run build --filter=@objectstack/formula --filter=@objectstack/lintunderos-verify-lock.sh(VERDICT command-exit 0, held 190s), the gate ran and passedpnpm check:agent-test-spellingpnpm check:corpus-claim-driftpnpm check:cross-package-test-inputspnpm check:doc-authoringpnpm check:driver-memory-censuspnpm check:gitlink-declaredpnpm check:nul-bytespnpm check:pm-governed-mergespnpm check:refd-timer-probepnpm check:role-wordpnpm check:skill-compatibilitypnpm check:skill-frame-syncpnpm check:skill-identifier-livenesspnpm check:watch-hint-literalNot run locally (CI-owned): the repo-wide
pnpm lintsweep, the type-check lanes, theTest Coreshards — this diff touches no package source. Exit codes were captured before any pipe (cmd log-redirect; EXIT=$?).Token ratchet — the one thing only the maintainer can lift
scripts/check-skills-token-ratchet.mjsenumerates every hand-authored file underskills/and reds on a file with noCEILINGSrow; its remedy text carries the⛔ MAINTAINER-ONLYmarker («adding a CEILINGS row prices new text into the bundle that ships to every customer project, which is the maintainer ruling this ratchet implements — not a step an author takes while landing the file»). The card's ruled shape forbids touching any gate script in this PR and the dispatch word says «⛔ 不改门禁本身», so this PR does not edit the script; theLint & Repo Gatescontext will stay red on it until the rows land.The authorisation the ratchet asks for exists — the maintainer's decision-batch #213 ruling, verbatim: 「1 2 4 同意」 on item 2 「objectstack 各 skill 先补 3 题 evals(复用 objectui 的 json 形状),先于第 1 项,好量化『切了没变差』」 — and the rows are mechanical, measured on this head in the ratchet's own
ceil(utf8 bytes / 4)convention, each initialised AT its measurement (zero headroom, the same terms as the #12392 rows):Sum of the 11 rows: 11965 tokens. Bundle total on this head per the ratchet's own price tag: 152338 (ratcheted authored 141361 vs 145656 ceilings). Whoever lands the rows (a maintainer-authorised follow-up, or this PR if the seat is told to add them) also clears the self-test's live-tree pin — no second edit needed.
Acceptance notes
must_not_containentries are spelled in their AUTHORED form (conditionalRequired:with the colon,type: 'unique','project_id.name'quoted) so a correct answer that merely names the rule while following it is not flagged — the same convention objectui adopted. A few (ISBLANK(record.,??,sourceView) can still trip an otherwise-correct answer that quotes the anti-pattern; treat such a flag as a review cue, as the pre-existing ui fixture's note says. Not filed: fixture design, not a defect.check-skill-eval-tokens.mjsis pinned to its own repo root on the CLI path but exportsanalyze({ root }), which is how the reading above was taken. A gate of that shape for this repo is a separate card, if ever — the ruled shape says no new gate here. Not filed: out of scope, no defect.scripts/pm/dispatch-gates.mjs --ranasks forcommand :: exit Nrows to count NOT-MEASURED itself; my first record carried bare commands, so its «0 NOT-MEASURED» was the runner's claim — re-recorded with exit codes, it derives the same zero. Not filed: usage, not a defect.维护者速读(草稿)
改了什么 — 十个已发布 skill 全部补上了 objectui 同款 json 形状的 evals(11 个新文件,54 题,每个 skill ≥ 3 题、每个「use when」触发族至少一题、每个 skill 至少一题反模式);objectstack-data 的「Not yet implemented」占位 README 改成了真实 fixture 清单;automation 的 markdown 用例转成 json 并保留原 md。没有动任何 SKILL.md / rules / references 的一行。
为什么改 — 批次 #213 第 2 项:「objectstack 各 skill 先补 3 题 evals(复用 objectui 的 json 形状),先于第 1 项,好量化『切了没变差』」。这些题是接下来拆 SKILL.md(姊妹卡)前后各跑一次的量尺。
风险与代价(含回滚) — 发布面只多了 evals 文件(随
npx skills add一起发运,约 11965 tokens)。唯一红着的门禁是check:skills-token-ratchet:11 个新文件没有 CEILINGS 行,而加行是「MAINTAINER-ONLY」,本 PR 按裁决没碰门禁脚本。行的内容已在上面按实测列好。回滚 = revert 这一个 commit(02b931d),无数据、无 API 影响。席位意见 — (留空,由席位定稿)
你要做的 — ① 决定 11 行 CEILINGS 落在哪里:让席位在本 PR 追加一个 commit,或另开一个引用本裁决的 follow-up;② 人合本 PR(Tier H)。
Token citations (must_contain → first whole-token occurrence in the skill's own text)
Generated by Claude Code
Generated by Claude Code