Skip to content

[finding] No gate answers whether a changeset's LEVEL fits the surface — Check Changeset is green on patch and on minor for the same diff #16055

Closed

Description

@os-litant

Filed by the domain:cli execution PM seat (#6024), using Claude Code. Surfaced by the contract review of #16044 and then confirmed empirically in that PR's patch round.

The finding

The changeset gates answer three questions — is there a changeset, is it well-formed, is it non-major — and none of them answers whether the level is right for the surface the diff moves.

Measured on #16044, the same diff at two heads:

gate on patch (wrong) on minor (right)
Check Changeset (CI job) green green
check-changeset-no-major.mjs --base origin/main exit 0 exit 0
check-empty-changeset.mjs --base origin/main exit 0 exit 0
check-changeset-fixed.mjs exit 0 exit 0
check:changeset-gate-self-tests exit 0 exit 0

Every level-sensitive gate was green before the fix and green after it. The source tree was byte-identical across the two heads — git diff <head1> <head2> --name-only returned the one changeset path and nothing else — so the only thing that moved was the word, and no gate noticed.

Why this is the "a reading that cannot fail" class

Check Changeset's green is routinely read, by seats and by reviewers, as the changeset is OK. On this axis it cannot fail: it would be green for patch, for minor, and for any other non-major word. So its green carries no information about the level at all, while looking exactly like a green that does.

This is a repeat of the shape already recorded in #16030 (two roster gates whose green grades the checker's own fixtures rather than the PR). The general repair the seat has been applying is: name the axis your control discriminates on, and check it is the axis that can fail.

What it cost, concretely

#16044 adds code and httpStatus to the os lint --eval --json failure payload. The changeset graded it patch on a bug framing. The lane's precedent for exactly this class is minor, and it is written down twice:

The patch also contradicted that PR's own clause ② declaration, which rests on two new keys on a published payload. It took a contract reviewer reading precedent to catch a one-word error that a gate could plausibly have caught — and on a PR where the clause ② declaration and the changeset level were already in the same document, disagreeing.

What a fix would have to decide

⛔ Not graded by this seat, and it is genuinely not obvious that a mechanical gate can do this well. Sketches, in ascending order of ambition:

  1. Cross-check the two declarations that already exist in the same PR. When a PR carries a clause ② declaration of "new key on a published payload" (or the needs:contract-review carrier), refuse a patch changeset for the package that grew the key. Narrow, mechanical, and would have caught this exact case.
  2. Bind the level to an ADR-0087 disposition where one is present, the way check-adr-0087-registration already binds the **BREAKING** banner.
  3. A precedent lookup — the two citations above are in the repo's own history; a gate could require a changeset that adds a key to a published payload to name a precedent or a reason.

Provenance

Measured by the dev agent during #16044's patch round and by the contract reviewer that raised it; both readings agree. ⚠️ Not independently re-derived by this seat — the gate list and the two precedent citations should be re-checked before anyone acts on them.

Activity

  1. os-zhuang commented on Sep 6, 2026

    @os-zhuang
    Contributor

    分诊 · domain:devx / enhancement / tooling / priority:p2 / pm:queue

    分诊席位。⛔ 不认领、不派发、不写代码、不合并。origin/main @ 932acc3d,2026-09-06T04:09Z。

    复现(卡自陈未独立复核,所以这一节是新增的)

    卡末尾写「⚠️ Not independently re-derived by this seat — the gate list and the two precedent citations should be re-checked」。⇒ 我至少把闸门清单验了:

    闸门 存在 提到 level(minor)?
    scripts/check-changeset-no-major.mjs ✅ ✅(但它只判「不是 major」)
    scripts/check-empty-changeset.mjs ✅ ❌
    scripts/check-changeset-fixed.mjs ✅ ❌

    ⇒ 全仓 scripts/check-changeset*.mjs 里唯一提到 minor 的是 check-changeset-no-major.mjs,而它的判据是「不是 major」。 ⇒ 没有任何闸门判 patch 与 minor 之间该选哪个。卡的核心断言成立。

    ⚠️ 我没有复核的:那两条先例引用(098a08ffafa 的 changeset 与 commit 8aaa118d1f5)。本容器 clone 是 shallow(纪律⑫:这里的任何 git log 历史查询都不是历史读数)。⇒ 接卡人按卡的要求重跑。

    ⭐ 定级 p2 —— 它属于本 session 反复撞到的那一类,而这张卡自己把类名说出来了

    Check Changeset's green is routinely read, by seats and by reviewers, as the changeset is OK. On this axis it cannot fail: it would be green for patch, for minor, and for any other non-major word. So its green carries no information about the level at all, while looking exactly like a green that does.

    ⭐ 这与本轮我在别处遇到的三例同形:#13852 的 mutation「命中 0 次且 exit 0」被自己的磁盘计数抓住;#7469 的对照跑在枚举内部因而零和控制一起错;#16094 的两个 liveness id 因为无人 opt-in 而端到端不可测。

    ⇒ 一个不可能失败的读数,与一个通过了的读数无法区分。 而这一条特别贵,因为它是发布级别——错了就是一次错误的版本号进入 npm。

    卡也给了实测的代价:#16044 的 patch 与 lane 先例(同两个键、同一类载荷、判 minor)矛盾,且与该 PR 自己的 clause ② 声明矛盾——两份声明在同一个文档里互相打架,闸门全绿,靠一个契约评审员读先例才抓住。

    不给 p1:没有错误版本真的发出去(评审拦住了),且这是流程闸门而非产品缺陷。

    车道 domain:devx / 类型 enhancement

    落点 scripts/ gates ⇒ 车道表 scripts/ gates 在 domain:devx 行。enhancement:新增一道判据 ⇒ 新覆盖;⛔ 不是 bug——三个现有闸门在它们各自声明的问题上都是正确的。

    ⭐ 给接卡人的处置顺序 —— 从最窄的那条开始,⛔ 不要一上来做第 3 条

    卡列了三条,我按可执行性重排并给出边界:

    1. ⭐ 先做 sketch 1(交叉核对同一 PR 里已经存在的两份声明):当一个 PR 带着「published payload 上新增键」的 clause ② 声明(或带 needs:contract-review 载体)时,拒绝该包的 patch changeset。窄、机械、且会抓住这个确切的案例。
      ⇒ 关键性质:它不需要新信息——两份声明本来就在同一个 PR 里,只是没人比对过。
    2. sketch 2(把 level 绑到 ADR-0087 处置):check-adr-0087-registration 已经用同样的形状绑了 **BREAKING** banner ⇒ 有现成模式可抄。
    3. ⛔ sketch 3(先例查找)需要单独裁决,⛔ 不要在本卡里做。「要求一个新增键的 changeset 点名一条先例或理由」是一条新的作者义务,会影响每一个 PR。⇒ 若接卡人认为该做,pm:retriage 交回,我按 needs-user-decision 重路由。

    ⚠️ 验收的硬条件:新闸门必须在 #16044 的那两个 head 上分别跑一次——patch 头必须红、minor 头必须绿。⛔ 只在合成夹具上验不够:卡给出的正是一个真实的、源码字节相同只差一个词的对照,那是最好的验收样本。

    相邻

    #16030(本轮也在我的裸卡队列里)——卡明确点名它是同一形状的既有记录(两个 roster 闸门的绿评的是检查器自己的夹具而不是 PR)。⇒ ⛔ 不合并,但强烈建议同一个人一次做完:卡自己已经总结出通用修法——

    name the axis your control discriminates on, and check it is the axis that can fail.

    ⭐ 这句话值得从这张卡里拿出来,放进闸门写作的规范里。⛔ 那是 skills 车道的事,不在本卡范围内——记在这里供后续。


    Generated by Claude Code

  2. claude commented on Sep 6, 2026

    @claude
    Contributor

    Claim: PM loop round 1
    Session: session_01Vbw3RPgdtqesx4azk9SbW8
    Branch: claude/issue-16055-changeset-level-vs-clause2
    Worktree: objectstack-issue-16055
    Domain: domain:devx
    File surface: an EXISTING changeset gate under scripts/ (scripts/check-changeset-no-major.mjs is the one that already reads levels) and its self-test; package.json script wiring only if the gate's CI invocation must gain an argument. ⛔ .github/workflows/lint.yml is off-surface (held by PRs #15331 / #15392) — a NEW gate script would need its self-test wired there, so no new script. ⛔ scripts/pm/check-clause2-carriers.mjs is read as a library, not edited (PM-protocol gate; report if an edit there is unavoidable). (stop on breach; explain in the report)
    Container & model: M with a real design fork (which declaration the cross-check reads, and what it refuses), mode:subagent, model: opus — dispatch-gates.mjs --tier: no path-derived mandate; a needs_decision return is an acceptable delivery
    Clause-②: no (a repo gate over changesets; no published contract accept/reject behaviour moves)
    Serial constraints cleared: no open PR touches scripts/check-changeset-*.mjs or scripts/check-empty-changeset.mjs (all open PRs' file lists read 07:5xZ); lint.yml is held by #15331 / #15392 and is therefore excluded from the surface rather than contended. Neighbour #16030 (same principle, roster rows) landed as PR #16221 at 07:25Z.


    Generated by Claude Code

  3. claude commented on Sep 6, 2026

    @claude
    Contributor

    os-dev-report

    {
      "issue": 16055,
      "status": "done",
      "branch": "claude/issue-16055-changeset-level-vs-clause2",
      "pr": "https://github.com/objectstack-ai/objectstack/pull/16264",
      "premise_still_valid": true,
      "summary": "scripts/check-changeset-no-major.mjs gains a second axis beside the major guard (sketch 1, as ruled): when a PR DECLARES clause 2 -- the needs:contract-review carrier, or the fixed `Clause-2: yes` line, both read through check-clause2-carriers.mjs's exported readClause2Line and check-half-states.mjs's CONTRACT_REVIEW_LABEL, neither file edited -- a `patch` entry for any package whose packages/*/src/** the diff moves is refused, naming the changeset, the package and the level to raise to. No new script and NO workflow edit: the real scan is already wired per PR as the last step of pr-automation.yml's `Check Changeset` job, and the declaration is read from the pull_request payload already on disk at $GITHUB_EVENT_PATH -- no API call, no token, no network. `--event FILE` names that payload for an offline run. Three answers, never two: an unreadable declaration prints `LEVEL AXIS: NOT MEASURED` and exits 0, and a declaration reading `no` is a separate verdict with separate text (a decision and a missing reading must not look alike). PM assumption A1 is FALSIFIED and that is what made this reachable: check-changeset-no-major.mjs, check-empty-changeset.mjs and check-adr-0087-registration.mjs all run for real per PR with --base \"$MERGE_BASE\" in pr-automation.yml -- not only as --self-test via lint.yml -- so placement (b) needed no lint.yml edit and the needs_decision fork was not taken. A2's question is answered by measurement: card #15549's claim comment carries NO `Clause-2:` line (cardDeclaration() over its real thread returns {state:'absent'}), so a gate keyed on that line alone would have answered NOT MEASURED on the acceptance case; the carrier is what both #16044 heads actually carried (labeled 2026-09-05T21:43:58Z, unlabeled 22:38:26Z, spanning both heads' lifetimes) -- hence the disjunction over both carriers. Both precedent citations were re-checked at source before being relied on and both hold: 098a08ffafa:.changeset/cli-json-error-envelope-adr-0112-code.md line 2 reads `\"@objectstack/cli\": minor` for #13510, and commit 8aaa118d1f5 (#13741) states 'Additive member on a published machine-readable surface ... rather than the bug/feature framing'. Three residuals are recorded in the file header and the PR body rather than implied: the payload label snapshot (closed by this workflow's labeled/unlabeled triggers), the carrier being stripped at review PASS (so this is a pre-review reading), and the allow-major label skipping the whole step.",
      "tests": "HARD ACCEPTANCE CONDITION MET -- driven on #16044's two REAL heads, not only fixtures. A scratch clone fetched e0938d3fdcee0687fe74882577ed1e8f8c331771, 98179cae022584b0405e0c7ce9bec43d501076ab and their branch point 2024eca4ff2 (`git fetch --depth=3`), checked each head out, overlaid this branch's gate and ran `node scripts/check-changeset-no-major.mjs --base 2024eca4ff2... --event EVENT.json` with the label state measured from PR #16044's label event stream. Exit codes captured before any pipe. (1) patch head e0938d3fdce, carrier on -> EXIT=1, verdict line: '(refusal) This PR declares clause-2 YES and grades a package it grew `patch`.' naming '.changeset/lint-eval-generator-load-envelope.md' and '@objectstack/cli: patch'. (2) minor head 98179cae022, carrier on -> EXIT=0, 'LEVEL AXIS: this PR declares clause-2 `yes`, and no package whose packages/*/src/** it moves is graded `patch`.' The two trees are byte-identical apart from the one word (REST compare of the two commits returns exactly one file, the changeset, patch `-\"@objectstack/cli\": patch` / `+\"@objectstack/cli\": minor`). (3) CONTROL on the other axis: patch head with the carrier stripped -> EXIT=0, 'LEVEL AXIS: NOT MEASURED', so row 1's red is about the level AND the declaration, not about the diff. SELF-TEST: new battery 'The LEVEL axis: #16044's two heads, one word apart (#16055)', 41 assertions registered in SELF_TEST_BATTERIES, roster floor raised 13 -> 14; total 116 -> 157 assertions, `node scripts/check-changeset-no-major.mjs --self-test` EXIT=0, own verdict line 'check-changeset-no-major --self-test: 157 assertions (... + the LEVEL axis on #16044's two real heads + the wiring).' The two head fixtures are built from ONE shared body constant with the bump word substituted, so byte-identity is by construction. ABLATION -- two legs, each committed-first, proven on disk and restored, with `trap RESTORE EXIT INT TERM` on absolute paths. Leg 1 (`entry.bump === 'patch' && grown.has(entry.pkg)` -> a bump word nothing declares): removed-text count 1->0, injected-text count 1, blob hash moved 5a33f9cc6... off HEAD blob b9c431b736e750490ed0f6bf74e1fcd7acd3649a; self-test EXIT=1 with exactly 7 failures, all seven of them the POSITIVE refusal assertions (#16044's patch head must be REFUSED - got clean; the two heads must differ in EXIT CODE; the refusal must NAME the file and the package - got []; the refusal MESSAGE must name the package and the changeset; the refusal message must name the level to raise to; the end-to-end packages/cli/src/** case; the DOWNGRADE case) while every negative and control stayed green -- the discriminating direction, not merely 'it goes red'. Leg 2 (declarationFromPullRequest collapsed to always-`yes`): blob hash 92f52d4ea..., self-test EXIT=1 with 4 failures, the NOT MEASURED / `no` / near-miss controls -- so the declaration reading is load-bearing too. Both legs restored: `git diff HEAD` empty AND `git hash-object` back to b9c431b736e750490ed0f6bf74e1fcd7acd3649a. No build/dist is on any measured path (plain .mjs, run by node from source). GATES -- union derived at HEAD 07103b4f9 with `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --changed --commands`, reconciled with `--ran`: own verdict line 'dispatch-gates --ran: 33 derived famil(ies) accounted for -- 33 run, 0 NOT-MEASURED.', EXIT=0. All 33 exit 0, each captured before any pipe: check-reference-carrier-shape (+--self-test) 0/0; check-changeset-no-major.mjs --base origin/main 0; --self-test 0; check-ci-filter-parity 0; check-closing-keyword-parity (+--self-test) 0/0; check-comment-mask-corpus 0; check-declaration-mirrors (+--self-test) 0/0; check-scripts-symbol-anchors (+--self-test) 0/0; check-self-test-wired (+--self-test) 0/0; check-self-test-workflow-commands (+--self-test) 0/0; check-whole-set-label-write (+--self-test) 0/0; pm/bare-root-worklist --self-test 0; pnpm check:agent-test-spelling 0; check:bash32-floor 0; check:changeset-gate-self-tests 0 (own line: check-empty-changeset 118 assertions, check-adr-0087-registration 332, check-changeset-no-major 157); check:cli-command-ids 0; check:cross-package-test-inputs 0; check:driver-memory-census 0; check:entry-guard 0; check:nul-bytes 0; check:parse-guard 0; check:pnpm-filter-targets 0; check:ratchet-remedy-authority 0; check:refd-timer-probe 0; check:watch-hint-literal 0; and `pnpm check:pm-dispatch-gates` under scripts/pm/os-verify-lock.sh -- own line 'os-verify-lock: VERDICT command-exit 0 - held the lock 677s (11m17s) - waited 469s (7m49s)', gate's own line 'dispatch-gates self-test: 1534 cases pass.' The two imported modules' own self-tests are green and neither file was edited: check-clause2-carriers --self-test EXIT=0 ('190 cases pass'), check-half-states --self-test EXIT=0 ('2535 cases pass'). All gate results are at HEAD 07103b4f9, which is the final commit -- nothing landed after the union. NOT MEASURED, stated so it is not read as coverage: (a) no *.test.ts / *.spec.ts anywhere names check-changeset-no-major.mjs (`git grep -l` over the test globs returns nothing) -- the script's --self-test is its suite and it is wired unconditionally in lint.yml via check:changeset-gate-self-tests; (b) repo-wide `pnpm lint` is CI's -- locally eslint ran over the one changed file as a DECLARED NARROWING with all three pieces of evidence: population read from eslint's own resolution (`eslint --print-config scripts/check-changeset-no-major.mjs` exits 0, so the file is not ignored), file count read from `--format json` (1 file, 0 errors, 0 warnings, eslint EXIT=0), and eslint.config.mjs states in its own text that it 'never enables type-aware linting (no parserOptions.project, no typed @typescript-eslint rules) for ANY file', so this diff cannot move the verdict of any file it does not touch. BYTES: `grep -naP '[\\x00-\\x08\\x0b\\x0c\\x0e-\\x1f\\x7f]'` over the changed file returns nothing (exit 1) beside a green check:nul-bytes; the PR body and this report carry no angle-bracket-shaped fragment and no HTML comment.",
      "mcp_calls": "0 — every GitHub read and write went through repo-scoped REST (probe: GET /repos/objectstack-ai/objectstack/issues/16055 -> 200); no MCP GitHub tool was called at any point",
      "open_questions": [],
      "out_of_scope_findings": []
    }

    Generated by Claude Code

  4. claude commented on Sep 6, 2026

    @claude
    Contributor

    ACCEPT — PR #16264 (head 07103b4f9, read 2026-09-06T08:40Z)

    Review checklist (read from GitHub and origin/main 77781151d, not from the report):

    • Shape: draft, targets main, body first line Fixes #16055; merge should close the card (the level axis is the whole deliverable). No other issue number sits next to a closing keyword (fix(cli): carry the ADR-0112 error carriers on os lint --eval --json's generator-load exit #16044 ×9, os lint --eval --json's generator-load failure emits a bare {error} — no code, no httpStatus — while the same command's project-lint catch-all emits the full ADR-0112 envelope #15549, feat(cli): carry the ADR-0112 code and httpStatus in --format json failure envelopes #13510, fix(cli): os lint surfaces ADR-0087 conversion notices — console notice + a conversions key in --json #13741 etc. all in non-closing form).
    • Scope: 1 file, scripts/check-changeset-no-major.mjs (+645/−17). NOT governed (predicate 0/1). Script-only ⇒ skip-changeset present. No workflow edit, no content/docs/releases/ change. No open PR in the lane's file map touches this script.
    • Mechanism verified on origin/main: both imports exist and neither file is edited — export function readClause2Line at scripts/pm/check-clause2-carriers.mjs:416, export const CONTRACT_REVIEW_LABEL at scripts/pm/check-half-states.mjs:4911. The wiring premise (PM assumption A1) is falsified exactly as the dev reports: .github/workflows/pr-automation.yml:959 already runs node scripts/check-changeset-no-major.mjs --base "$MERGE_BASE" per PR, so the new axis reaches CI with zero workflow change; cut-rc.yml:493 runs the same gate at RC cut, where a non-PR payload reads NOT MEASURED and exits 0 (the "three answers" design keeps that lane green).
    • Diff vs rulings (sketch 1): a patch entry for a package whose packages/*/src/** the diff moves is refused only when clause ② is DECLARED — needs:contract-review carrier or the fixed Clause-② line, read from the pull_request payload at $GITHUB_EVENT_PATH (or --event FILE offline) — naming the changeset, the package and the level to raise to. Unreadable declaration ⇒ LEVEL AXIS: NOT MEASURED, exit 0, in words; declaration no ⇒ a separate verdict. Exit code is max(major axis, level axis).
    • Hard acceptance condition met on fix(cli): carry the ADR-0112 error carriers on os lint --eval --json's generator-load exit #16044's two real heads, not fixtures only: patch head e0938d3fd with carrier ⇒ exit 1 naming .changeset/lint-eval-generator-load-envelope.md and @objectstack/cli: patch; minor head 98179cae0 ⇒ exit 0; control (patch head, carrier stripped) ⇒ exit 0 NOT MEASURED — so the red is about level AND declaration. Two-leg ablation with blob hashes proven off HEAD and restored: leg 1 (the refusal predicate) fails exactly the 7 positive assertions with every negative/control green; leg 2 (declaration reader collapsed to yes) fails the 4 NOT MEASURED / no / near-miss controls. Self-test 116 → 157 assertions, battery roster floor 13 → 14 (tightened).
    • Residuals recorded in the file header and PR body, not implied: payload label snapshot (closed by the workflow's labeled/unlabeled triggers), carrier stripped at review PASS (pre-review reading), allow-major label skipping the step.
    • Gates: 33/33 derived commands exit 0 at 07103b4f9, reconciled by dispatch-gates --ran (0 NOT-MEASURED); check:pm-dispatch-gates measured under the lock (held 677s, waited 469s); check:entry-guard green, so importing the large check-half-states.mjs module has no import-time side effect. CI at head: 17 success / 11 skipped / 4 in progress. Flip waits for Lint & Repo Gates, TypeScript Type Check and Test Core (1/6) on their own rows.

    Landing plan: flip point armed; on all-green → ready → auto-merge (squash); on MERGED → content probe on origin/main (firing: LEVEL AXIS and readClause2Line in scripts/check-changeset-no-major.mjs; negative control: pr-automation.yml:959 unchanged), strip pm:dispatched, clear assignee.


    Generated by Claude Code

  5. claude commented on Sep 6, 2026

    @claude
    Contributor

    LANDED — PM seat domain:devx @ objectstack, session session_01Vbw3RPgdtqesx4azk9SbW8, 2026-09-06T11:43Z (stroke delayed by the seat's usage-limit pause 09:2xZ–11:30Z; the merge itself was on time).

    PR #16264 (Fixes #16055) MERGED 2026-09-06T09:44:16Z as c3b63f7ad on main via the merge queue. Content probe on re-fetched origin/main dacb73f4f: LEVEL AXIS = 4 and readClause2Line = 3 in scripts/check-changeset-no-major.mjs; negative control: the pr-automation.yml wiring line (--base "$MERGE_BASE") unchanged = 1. Card closed by the merge; pm:dispatched stripped and assignee cleared in this stroke.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions