test(semantics): mutate the formal-model signature; separate stage, evidence and blocking claims (B0) - #4661
Conversation
`check_formal_model` is the only guard between a hand-written `formal_model` block and a reader who takes it for a machine-checked proof, and exactly one test touched it: `test_candidate_decisions_are_exhaustive_and_default_to_unknown` reads two fields of `candidate_decisions`. The key set, the five roles, the consumer hierarchy, the six invariant ids, the per-invariant shape and the four-lane partition were unmutated, so a regression deleting any of them would have merged green. Adds 26 single-mutation regressions in a new file, each asserting the checker fails closed *naming its own rule* -- a mutation that fails for an unrelated reason proves nothing. New file rather than an append to test_semantic_vocabulary_drift.py, which three open PRs already collide at. One mutation escaped and is fixed here rather than asserted away: an exactly duplicated invariant entry. The id set and the lane partition are both sets, so a repeat leaves them unchanged, and every dict `check_formal_model` builds by id keeps only the last occurrence. A second `F1_producer_closedness` carrying a weaker statement validated, and nothing said which of the two the smoke had walked. The list is now required to state each id exactly once. `test_metadata_shape_is_not_an_executed_proof` pins the B0 exit condition in executable form: a validated block asserts only that each obligation declares a stage, a non-empty evidence boundary and a domain the smoke recomputes from the registry -- never that the obligation holds. Refs loopx-project#4447 (B0) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: song <22676124+songoow@users.noreply.github.com>
I2, I11 to I14 and the four enforcement lanes wrote schema validation, implementation stage, evidence status and actual blocking behaviour interchangeably. That is how a validated metadata block comes to read as an executed proof, and it had already produced one false sentence. The Section 11 lane table glossed `blocking_next` as "planned blocking checks after M0.5; not claimed by M0". F1, F2 and F4 sit in that lane and all three fail closed today, which was measured rather than assumed: dropping a registered value that `executor.py::_run_turn` writes raises `producer writes unregistered values`; adding a kernel value nobody produces raises `decoder does not produce registered input`; removing one context from the `SOURCE_SURFACES` declaration raises `contexts must name every defining module exactly once`. Each exits the smoke non-zero, and by I10 the smoke is on the pull-request path. The RFC text is what changed: the lane name still records the milestone that owns the check, and the blocking claim moved to a column of its own. Section 5 gains "Four separate readings of one obligation row", stating what each reading can and cannot say, and the domain table now carries stage and blocking beside the verified/registered counts. The `formal_model` schema rows say schema validation only. Appendix A and Appendix B both carry a dated 2026-09-17 entry in each mirror. The residue is stated, not hidden: moving an invariant's `enforcement` and its policy lane together is still internally consistent, so a coordinated two-field edit can downgrade a check with no test failing. B0 narrows the gap; closing it needs the lane derived from the code that runs. Refs loopx-project#4447 (B0) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: song <22676124+songoow@users.noreply.github.com>
huangruiteng
left a comment
There was a problem hiding this comment.
Reviewed exact head: 6671bc3c22a915652a5ea5b918caa2b33e920a9d (base main @ 440b002fb, merge base unchanged since 2026-09-17T14:27:33Z).
动机
这是 #4447 Track B 的 B0 切片,审阅框架是仓库自己的 RFC docs/architecture/rfcs/semantic-vocabulary-convergence-v0.md(第 5 节、第 11 节、附录 A)。
作者指出的是一个可测量的文档不一致:第 11 节的层级表把 blocking_next 注解为"planned blocking checks after M0.5; not claimed by M0"。这句话在该层级被命名时成立,但在 M0.5b 交付之后就变成错的——check_producers 与 check_scope_declarations 都由 smoke 的 main()(经 check_inventory)调用,按 I10,这个 smoke 在每个跑 Python 测试的 PR 上失败即关闭。于是"实施阶段"和"今天是否阻断合并"这两件事被同一张表的同一句散文混在一起。
第二个问题是同类风险的另一面:check_formal_model 是唯一守着 formal_model 这块规范性元数据的检查器,但此前只有一个测试触及它,而且只读了 candidate_decisions 的两个字段。键集合、五个角色、consumer 层级、七种关系边、六个不变量 ID、逐条不变量的形状、四层划分——全部删掉也能绿。元数据被校验过,就容易被读成"已被证明";而校验器本身无人守护。
改动思路
作者没有重命名 blocking_next(那会丢掉"哪个里程碑拥有这项检查"的信息),也没有给 formal_model 加 blocks_today 布尔字段(那只会再多一个可被误读为"测量值"的声明字段)。他选择把一条义务行的四种读法拆开陈述——schema 校验、实施阶段、证据状态、实际阻断行为——并明确"只有 main() 里的代码决定什么会拦住合并"。
对第 4 种读法,他补的是测试而不是宣称:26 条单点突变,每条只改一处,断言检查器失败即关闭并指名那一条规则;再加 4 条把关键结构钉住的直接断言。并且他真的做了一次突变实验,找到了一个逃逸案例:重复的不变量条目(id 集合与层级划分都是集合,重复项不会改变它们,而检查器按 id 建的 dict 只保留最后一次出现)能被接受,于是新增了一条 fail-closed 规则。
具体改动
examples/semantic-vocabulary-drift-smoke.py(+8/-3):唯一的行为改动。原来的 {item.get("id") for item in invariants} == FORMAL_INVARIANTS 被拆成三步:先要求是 list,再要求 id 集合相等,再要求 len(invariant_ids) == len(set(invariant_ids))。这是一条真实的收紧,不是措辞调整。
tests/architecture/test_semantic_formal_model.py(新增 311 行):MUTATIONS 表是 (mutation, match) 对,match 字符串直接绑定检查器的报错文本;另有基线、重复条目、"层级与阶段一致"、"形状不等于已执行的证明"四条直接断言。文件用 runpy.run_path 加载 smoke 并深拷贝 registry,不重写任何规则。
两份 RFC 镜像(+138/-23 与 +111/-24):I2、I11–I14 各加上实施阶段、证据边界、今天是否阻断三件事;第 5 节新增"同一条义务行的四种读法"表格;第 11 节层级表拆成"实施阶段"与"今天是否阻断 PR"两列,并在正文里说明为什么保留旧层级名;附录 A 新增执行账本条目,明确写出未解决的部分(把 enforcement 与层级一起改动仍能降级检查,B0 只把缺口收窄到一次协同编辑,没有关闭它)。
对主干的风险
风险面很窄且方向正确:产品运行时、CLI、持久化、权限边界都不涉及,唯一的运行时行为变化是让一个此前会静默通过的畸形 registry 变成失败。
我独立做了两件事,而不是只读 CI:
- 验证突变测试确实绑住规则,而不只是"能通过"。把
require(set(model) == FORMAL_MODEL_KEYS, ...)放宽为>=,test_formal_model_mutation_fails_closed[extra_key]立刻变红;把新增的唯一性require换成True,[duplicated_invariant_id]与test_duplicate_invariant_entry_cannot_restate_an_obligation立刻变红。两次实验都在审阅用 worktree 内完成,随后git checkout还原,git status干净。 - 验证文档里"今天即阻断"的事实主张。直接在 registry 层面删掉一个
SOURCE_SURFACES声明上下文,check_scope_declarations抛出SOURCE_SURFACES: contexts must name every defining module exactly once——与 RFC 引用的报错逐字一致;check_scope_declarations确实由check_inventory调用,而check_inventory在main()里。I10 也成立:tests/architecture/test_semantic_vocabulary_drift.py以子进程运行 smoke 并断言returncode == 0,该文件由python-tests.yml的pytest -q收集。
验证结果:突变测试文件 31 passed;tests/architecture 全量 366 passed(91.20s);smoke 打印 ok,formal_domain=F1:6/26,F2:6/26,F3:0/26,F4:4/4,F5:1/1,F6:0/0,与 RFC 表格逐行吻合。
非阻塞的两点(均已在 PR 正文中披露,不构成阻断):
- P3:
match字符串与检查器报错文本耦合。改一条报错文案会同时让对应突变用例以"未匹配"失败。失败方向是响的(红而非绿),代价是改动时的连带修改成本;文件 docstring 已经把match定义为契约的一部分。 - P3:把某条不变量的
enforcement阶段与它所在的 policy lane 一起改,仍然可以降级检查而不触发任何测试——这正是作者在附录 A "Not addressed here" 里主动写下的残留。关闭它需要层级由实际运行的代码推导,属于另一片工作,B0 只把缺口收窄到一次协同编辑。
我的整体评价
APPROVE。这是一个"用测量代替宣称"的切片:它先证明旧注释已经为假,再证明校验器几乎无人守护,然后只用一条 fail-closed 规则和一组单点突变把两件事钉住,并且把没关掉的缺口写在账本里而不是留在正文的暗示里。文档层面的改动方向与我读到的实际代码一致,我没有找到把声明写强于代码的地方。
我另外确认这不是 PR farming:songoow 在同一条 #4447 链上已有 8 个已合并切片,本次三个 PR(4661 / 4662 / 4663)是 B0、Track A、B5 三个不同切片,不是同一形状的重复提交。
English verdict: APPROVE - PR #4661 at head 6671bc3; the lane table's blocking claim is corrected and the formal-model signature is now guarded by 26 single-mutation regressions plus one real fail-closed rule (duplicate invariant ids), validated locally (31 passed for the new file, 366 passed for tests/architecture, smoke ok) and independently reproduced by weakening two checker rules and by dropping a declared scope context.
Merge-readiness qualification — #4661 @
|
| 阻塞项 | 触发规则 | 需要的动作 |
|---|---|---|
merge_state_requires_update(BEHIND) |
strict_required_status_checks_policy: true —— head 必须包含 main 的最新提交 |
更新本分支(作者 rebase 或合并 main,维护者也可用 “Update branch”) |
该 exact head 的评审结论(APPROVE)与全部必需检查都已就绪,所以更新分支之后需要重新做的是对新的 head 重新评审并重跑门,而不是重新讨论本 PR 的设计。
更新分支时请注意同一 ruleset 的另外两条规则
dismiss_stale_reviews_on_push: true—— 推入新 commit 会废止旧审批,所以更新分支后必须在新 head 上重新评审。require_last_push_approval: true—— 推入更新的那个人不能提供计入的审批。因此这次更新最好由作者本人完成:如果由维护者推入,则本轮维护者的审批将无法计入,PR 反而会被卡在REVIEW_REQUIRED(本仓库的 test(manager-context): guard receiver isolation and successive requests #4640 正是这种情形)。
本轮做了什么、没做什么
- 读取了该 exact head 的审批状态、检查结论与 review thread 摘要,并运行了上面的合并门命令;结论与 PR 正文的证据一致。
- 没有合并,也没有使用 admin bypass(门返回
admin_bypass_overrides_this_gate=false)。 - 门必须在合并前对未变化的 head 重新运行;本 head 一旦变动即
restart_review_and_rerun_gate。
English verdict: APPROVE - PR #4661 at head 6671bc3; the review conclusion and all required checks are green at this exact head, and merge readiness is blocked only by merge_state_requires_update (strict required status checks, ruleset protect main), which the author should resolve by updating the branch before the gate is re-run.
Delivers slice B0 of #4447, the declared head of the Track B chain and the last slice still unstarted. Per the task brief, PR #4631 turned out to be merged into
mainalready (440b002fb), so the domain fixtures are present in my base rather than pending — I did not duplicate them and did not depend on them.What I found
check_formal_model()was thorough on shape and touched by exactly one test (test_candidate_decisions_are_exhaustive_and_default_to_unknown, which reads two fields ofcandidate_decisions). The exact key set, the five roles, the consumer hierarchy, the seven relation kinds, the six invariant ids, the per-invariant shape and the four-lane partition were entirely unmutated. The regressions B0 asks for did not exist.Two real defects fell out of doing the work:
1. A duplicate invariant entry was accepted. The id check and the lane-partition check are both set comparisons, so a repeated entry leaves them unchanged, and every dict
check_formal_modelbuilds by id keeps only the last occurrence. A secondF1_producer_closednesscarrying a weaker statement validated, and nothing recorded which of the two the smoke had walked. Fixed by requiring each id exactly once — a one-line tightening, not a weakened check.2. The RFC's
blocking_nextgloss was false. Section 11 described that lane as "Planned blocking checks after M0.5; not claimed by M0". F1, F2 and F4 sit in that lane and all three fail closed today. Measured, not assumed:executor.py::_run_turnwritesturn_result_kind: producer writes unregistered values ['repair_required']turn_result_kind: decoder does not produce registered input zzz_never_producedSOURCE_SURFACESdeclarationSOURCE_SURFACES: contexts must name every defining module exactly onceEach exits the smoke non-zero, and by I10 the smoke runs on the pull-request path. The RFC text is what changed here — the lane name still records the milestone that owns the check, and the blocking claim moved to its own column, so the two readings stop being one sentence.
What each regression proves
tests/architecture/test_semantic_formal_model.py— new file, not an append totest_semantic_vocabulary_drift.py, which three open PRs already collide at. 26 single mutations plus 5 structural tests. Every mutation asserts the checker fails closed naming its own rule; a mutation that fails for an unrelated reason proves nothing and would rot into a test that only asserts "something raised".role_hierarchy, emptied hierarchy, hierarchy reparented toownerpersists, which F6 quantifies over), extra relation kindstatement, emptyevidence, missingdomain, extra fieldenforcement, dropped laneunprovedclaims verified members,advisoryclaims verified members, widened domainunknownPlus
test_metadata_shape_is_not_an_executed_proof, which pins the B0 exit condition in executable form: a validated block asserts only that each obligation declares a stage, a non-empty evidence boundary and a domain the smoke recomputes from the registry — never that the obligation holds.What RFC text changed and why
Both mirrors, same rows (
docs-governance-smoke.pypasses; verified 45 headings, 2 ledger entries and 5 decision rows in each).verified / registeredcounts. I2 and I11–I14 each gained an explicit stage / evidence / blocking sentence.formal_modelis now described as schema validation only;enforcement_policyis described as a milestone label, not a blocking claim.blocks_todayfield — rejected, one more declared field a reader could mistake for a measurement). Both mirrors.Deliberately left undone
The grounding gap the 2026-09-17 domain entry named is still open, and I have said so in Appendix A rather than implying B0 closed it: moving an invariant's
enforcementand its policy lane together stays internally self-consistent, so a coordinated two-field edit can still downgrade a check with no test failing. Closing it needs the lane derived from the code that runs rather than declared beside it. B0 narrows the gap to a coordinated edit and documents the residue; it does not close it.Validation
Note for the reviewer:
origin/mainadvanced to001c6daf2while I worked. The three PRs merged there (#4659,#4657,#4653) touchsupport_control*.py, the interaction-pattern catalog and the change-window fixture — disjoint from every path in this diff, so no rebase was needed and none of the four gates above is affected.Worktree:
/home/ubuntu/loopx-b0, branchcodex/b0-formal-model-reconciliation, two commits split by reviewer logic (tests / docs), both DCO signed off.Refs #4447 (B0)
🤖 Generated with Claude Code