Skip to content

docs(skills): objectstack-ai says guardrails is enforced per turn and output validation lives on action.ai.outputSchema - #21302

Merged
os-zhuang merged 1 commit into
mainfrom
claude/issue-21288-ai-skill-guardrails-outputschema-enforced
Oct 2, 2026
Merged

os-zhuang merged 1 commit into
mainfrom
claude/issue-21288-ai-skill-guardrails-outputschema-enforced

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Fixes #21288

Clause-②: no

What this does

Two sentences in the published objectstack-ai skill said the opposite of what the cloud AI runtime now does. Both are rewritten to state only what the readings on the parent card prove. The skill text cites no card number (it cites ADRs, as the rest of the file does); the readings are cited here.

  1. ToolSchema paragraph, the outputSchema clause (lines 279–281 at 4e6dc233). It said the key's top-level keys are folded into the tool description shown to the model and outputs are not validated. It now says nothing reads the key on a tool record, and steers the author to ai.outputSchema on the action, which the AI runtime ☁️ enforces by withholding a result that does not conform.
  2. Common Pitfalls 1 (lines 309–311 at 4e6dc233). It said guardrails / memory / structuredOutput are declared only, no runtime reads them, and real limits come from the quota service. It now says guardrails is enforced per user turn by the AI runtime ☁️ (token and time limits; blocked tool names and categories refused) but is a limit, not an approval, and that memory is declared only, no runtime reads it. The pitfall's heading and its second half (what an enforced gate is) are unchanged.

Line budget (the skills/** discipline)

reading before (4e6dc233) after (8f4222b5)
skills/objectstack-ai/SKILL.md lines 416 418 (net +2; budget was net ≤ +2)
the same file in the ratchet's unit, ceil(utf8 bytes / 4) 5444 5486 (ceiling 6806, headroom 1320)
whole catalog, every skills/*/SKILL.md, lines 4395 4397
whole catalog, every skills/*/SKILL.md, ceil(bytes / 4) 52087 52129

Each position pays +1 line; no neighbouring line is re-wrapped (the diff is 7 insertions, 5 deletions, inside the two clauses only). The dispatch gave the file as 337 lines; at 4e6dc233 it is 416 (git show 4e6dc233:skills/objectstack-ai/SKILL.md | wc -l).

Verification (at 8f4222b5, worktree objectstack-issue-21288)

  • node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands derived 24 commands from the merge base 4e6dc233 (1 path, +7/−5). Each ran with its exit code captured before any pipe; --ran answered: "✓ dispatch-gates --ran: 24 derived famil(ies) accounted for — 24 run, 0 NOT-MEASURED".
  • node scripts/check-skills-token-ratchet.mjs exit 0: "✓ check-skills-token-ratchet: skills/objectstack-ai/SKILL.md is 5486 tokens (ceiling 6806; headroom 1320)".
  • Exit 0 each: pnpm --filter @objectstack/spec run check:skill-docs, pnpm check:skill-identifier-liveness, pnpm check:skill-frame-sync, pnpm check:skill-compatibility, pnpm check:corpus-claim-drift, pnpm check:doc-authoring, pnpm check:role-word, pnpm check:nul-bytes, pnpm check:pm-governed-merges, pnpm check:gitlink-declared, pnpm check:agent-test-spelling, pnpm check:cross-package-test-inputs, pnpm check:driver-memory-census, pnpm check:refd-timer-probe, pnpm check:watch-hint-literal, and the direct node scripts/check-*.mjs rows (ci-filter-parity, closing-keyword-parity and its self-test, comment-mask-corpus, doc-route-spelling advisory and self-test).
  • pnpm --filter @objectstack/lint run check:doc-formula-expressions: first run exit 3, PREREQUISITE NOT MET (@objectstack/formula and @objectstack/lint not built), nothing measured; after pnpm exec turbo run build --filter=@objectstack/formula --filter=@objectstack/lint under the shared verify lock, exit 0.
  • Beyond the derivation, because it type-checks the os:check blocks in skills/**: pnpm --filter @objectstack/spec run check:skill-examples exit 0, "✅ 259 prose examples type-check across 3 surface(s)" (needed @objectstack/spec, @objectstack/client and @objectstack/client-react built first, all under the lock).
  • NOT MEASURED locally, CI's own: the whole-root scans (pnpm lint), the type-check lanes and Test Core; the --ran footer names them as outside the derived total.

Changeset

skip-changeset. skills/** is in no published package's files[]: a scan of every package.json in the tree for a files[] entry naming skills or content/docs finds zero; the positive control @objectstack/spec lists dist, json-schema, liveness, prompts, llms.txt, README.md, src/**/*.zod.ts, CHANGELOG.md, api-surface, spec-changes.json. The catalog ships to customer projects through npx skills add, not through npm.

Governance

Tier H: skills/** is a governed surface (Prime Directive #14). This PR stays draft and lands only after an authorized APPROVED review; no seat readies, queues or arms auto-merge on it. The sibling docs line of the same card is a separate, ordinary PR on claude/issue-21288-actions-mdx-outputschema, which carries no closing keyword.

Acceptance notes

维护者速读(草稿)


Generated by Claude Code

… output validation lives on action.ai.outputSchema

The ToolSchema paragraph no longer says a tool's outputSchema keys are folded
into the model-facing description; it says nothing reads the key on a tool
record and steers the author to ai.outputSchema on the action, which the AI
runtime enforces by withholding a non-conforming result. Common Pitfalls 1 no
longer lists guardrails as declared-only: it is enforced per user turn (token
and time limits, blocked tool names and categories refused) but is a limit, not
an approval. memory stays described as declared-only.

Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FNKm1SmPpuJASnbjxWGtsJ
@github-actions github-actions Bot added size/s documentation Improvements or additions to documentation labels Oct 2, 2026
@objectstack-fleet objectstack-fleet Bot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Oct 2, 2026
@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Contract review

Served-tier: CONTRACT_REVIEW_TIER
Head-sha: 8f4222b5c95d648ce13e0653a81ff7a5157dcb5b
Local-runs: none

Face: governed skill text (skills/objectstack-ai/SKILL.md; skills/**, the published catalog), reviewed in-seat by the dispatching seat at the constant's tier. Inputs: #21288 (body and every comment: triage 5944149289, the claim 5944279390, the report 5944669125), this PR's body, its file list (1 file, +7 / −5) and patch against main at 4e6dc233, the three cloud readings on #20274 the card rests on (5940785002, 5943158888, 5943779795), and the check-runs on this head as read at this record. The sibling PR #21303 (content/docs/protocol/objectui/actions.mdx, non-governed, Refs #21288) is reviewed in the ACCEPT on the card, not here.

① Derived judgments

  1. Scope: the outputSchema parenthetical of the ToolSchema paragraph (:279–:281) and Common Pitfalls 1 (:310–:313), nothing else; 416 → 418 lines (+1 per position, net +2 of +2); no neighbouring line re-wrapped. RIGHT.
  2. The deleted claim 「its top-level keys are folded into the tool description shown to the model, and outputs are not validated」: reading 5943158888 — compileOutputContract reads action.ai.outputSchema directly (action-tools.ts:618); AIToolDefinition.outputSchema is only a copy; nothing loads authored tool metadata into a tool definition; so the fold is gone and the validation exists, on the action. The new text 「nothing reads it on a tool record; declare the schema as ai.outputSchema on the action instead, which the AI runtime ☁️ enforces by withholding a result that does not conform」 says exactly that and keeps ⚠️ experimental on the tool-record key (its liveness row stays experimental, the same reading). RIGHT.
  3. Pitfall 1, guardrails: reading 5940785002 — TurnGovernor applies the agent's token and time limits per user turn in all three chat paths, and matchBlockedTopic refuses an exact tool-name / action match from blockedTopics; the row moved to live. 「enforced per user turn by the AI runtime ☁️ — token and time limits, blocked tool names and categories refused — but it is a limit, not an approval」 follows the reading and keeps the pitfall's point (an enforced limit is still not an approval gate; the gate list that follows is unchanged). RIGHT.
  4. memory stays 「declared only — no runtime reads it」 (reading 5940785002 §2: agent.memory stays experimental). RIGHT.
  5. structuredOutput is removed from the declared-only enumeration and nothing is written in its place: the PM premise (0 lines, no new semantics) held, and its enforced semantics wait on spec(ai): agent.structuredOutput is now enforced by cloud, which refuses four authorable members (regex / grammar / xml formats, the coerce_types step) — retire them or build them, then flip the row #21277 as triage sequenced. A false membership is gone; no claim replaces it. RIGHT.
  6. No issue number enters the skill (the file cites ADRs only); the readings are cited in the PR body. RIGHT.
  7. Token ratchet (the dev's run at this head): 5444 → 5486 of 6806 (headroom 1320); catalog skills/*/SKILL.md 4395 → 4397 lines. Lint & Repo Gates re-measures on CI. RIGHT, pending that check.
  8. Dev readings, noted not filed, each with a carrier: the fold claim still stands in packages/spec/src/ai/tool.zod.ts:195, liveness/tool.json:49, content/docs/ai/tools.mdx:148 and two generated reference pages — PR feat(spec): agent.guardrails is live, enforced by the cloud AI runtime; tool.outputSchema steers authors to action.ai.outputSchema #21280 (open) rewrites them; the skill owes one structuredOutput sentence after spec(ai): agent.structuredOutput is now enforced by cloud, which refuses four authorable members (regex / grammar / xml formats, the coerce_types step) — retire them or build them, then flip the row #21277; action.zod.ts types action.ai.outputSchema as an open record so authoring accepts a schema the runtime refuses at call time — the spec lane's (same shape as spec(ai): agent.structuredOutput is now enforced by cloud, which refuses four authorable members (regex / grammar / xml formats, the coerce_types step) — retire them or build them, then flip the row #21277). RIGHT.

② Semver level

Docs-only in a governed surface no package ships; skip-changeset correctly applied; Clause-②: no. Consistent.

③ Boundary flags

  • The dispatch gave the file as 337 lines; the tree holds 416 — the seat's grep -c . reading dropped blank lines. The budget was stated as net lines and holds; corrected here.
  • check:skill-examples was not in the derived list and was run as an extra (exit 0, 259 prose examples type-check). Accepted.
  • One card, two PRs per triage's routing: PR docs(protocol): the actions.mdx outputSchema bullet says the cloud AI runtime validates the action result #21303 binds no closing keyword (Refs #21288; check-closing-keyword-parity --body exit 0), so No other open PR may claim the same issue holds for both. Accepted.
  • Check-runs on 8f4222b5 at this record: 14 success, 10 skipped, 5 in_progress (Lint & Repo Gates, Test Core 1/6, 2/6, two Type Check lanes), 0 failure. Tier H landing: an authorized APPROVED review, then every check green on the landing head; no other open PR touches this file.
  • open_questions: none.

Implemented-by: claude/issue-21288-ai-skill-guardrails-outputschema-enforced
Reviewed-by: session_01FNKm1SmPpuJASnbjxWGtsJ

VERDICT: PASS

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

维护者速读(终稿)

席位(skills 席 1,session_01FNKm1SmPpuJASnbjxWGtsJ)对照自读的 diff 与 #20274 上 cloud 席的三条读数(5940785002、5943158888、5943779795)校正 dev 草稿后定稿,2026-10-02T03:03Z。

改了什么。 对外发布的 skills/objectstack-ai/SKILL.md 两处:① ToolSchema 段里 outputSchema 的括注,原说「顶层键会折进给模型看的工具描述,输出不校验」,改为:工具记录上没有任何运行时读它;schema 应声明在 action 的 ai.outputSchema 上,云端 AI 运行时会校验返回值、扣住不合规的结果。② Common Pitfalls 1,原说 guardrails / memory / structuredOutput 「只是声明,没有运行时读它们」,改为:guardrails 已由 AI 运行时按每轮强制(token 与时间上限;被封禁的工具名与类别拒绝),但它是限额不是审批;memory 仍只是声明;structuredOutput 从该枚举里删掉、不另写新句(其语义等 #21277 裁决)。净增 2 行(预算 +2),token 5444 → 5486(上限 6806)。同卡另有一个普通 PR #21303 改 content/docs/protocol/objectui/actions.mdx 的一条 bullet(非受管,由席位走队列落地)。

为什么改。 这是 AI 作者写 agent/tool 元数据时读的教材。cloud 已落地 guardrails 与 action 返回值校验(cloud 235c5b29、cb62c3ea),旧句把一个已强制的控制说成无效、把校验位置指错,照它写的 AI 会绕开真正的闸、或把 schema 写在没人读的地方。北极星规则 4:给 AI 的文档说错一句即产品缺陷。

风险与代价(含回滚)。 纯文档,不动发布包与运行时;风险只在措辞与读数是否对齐:席位逐条对照三条读数,新句只说读数证明了的事;structuredOutput 处只删不写,避免 #21277 裁后再改一次。席内达档契约复核 PASS(见本 PR 评论)。回滚 = revert 本 PR。

席位意见。 建议批准。无同文件在飞 PR。spec 侧 action.ai.outputSchema 仍是开放 record(作者能写进运行时会拒的 schema)—— 与 #21277 同形,已记入 PR #21303 的 Acceptance notes,留给 spec 车道。

你要做的(一个动作)。 在本 PR 上留一条 APPROVED review(os-zhuang / hotlong 任一)。之后由席位清标、翻 ready、挂 auto-merge 入队;不需要你合并。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/s skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

3 participants