feat(semantics): bind one re-export hop in the Python producer scanner (B2) - #4573
Conversation
…r (B2) After M2 moved the three Turn owners into turn_contract_generated.py, every production site that still imported an owner through the transaction.py / driver.py compatibility re-exports became `unknown_producer` (43 -> 52 unresolved sites) with no code change in those modules and no failing check: `_qualified_bindings` bound an owner only when it was imported from the owner's own module. Bind one unrenamed re-export hop through a tracked module. A second hop, a renamed re-export (`as Other`), a same-name class or assignment, or a later plain `import` of the name leaves the consumer unknown; `import X as X` counts as unrenamed. Only names that are owner symbols are followed and parsed trees are cached, so the full drift smoke keeps its runtime (32.6 s before and after). Select the two executable input witnesses from one code-owned table keyed by the registered `input_producer` site instead of a name branch in the smoke and a literal branch in collect_production; the smoke keeps a single INPUT_PRODUCER_ANCHOR and requires each anchored site to have a witness. Registry data still cannot import a callable. Measured with examples/semantic-vocabulary-drift-smoke.py --report on main vs this change: unresolved_producer_sites 52 -> 41 (executor.py x10, loop_controller.py::_envelope_route:118); no site becomes newly visible or unregistered; every other summary line is identical. Refs loopx-project#4447 (Track B, B2 producer pilot for the Turn kernel vocabularies). Signed-off-by: song <22676124+songoow@users.noreply.github.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
huangruiteng
left a comment
There was a problem hiding this comment.
我感觉基于这个rfc,可以打磨一套capability在仓库里,甚至可以for其它项目使用了,比如用loopx开发别的项目时,也注重词表复杂度这块
huangruiteng
left a comment
There was a problem hiding this comment.
动机
评审 head:56c4a9f4b2fa2bc64f67a48e080622fe8e46d440;base:main(merge-base f1166e81e,当前 main 已含 #4571)。
M2 把三个 Turn owner 搬进 turn_contract_generated.py 后,凡是仍通过 transaction.py / driver.py 兼容 re-export 导入 owner 的消费方,都失去了 producer attribution——而且没有任何检查会失败。上一轮的 #4571 用一行 import 修好了 executor.py 这一个消费方,但那只是"消费方记得改",依赖本身还在:下一个重构又会静默丢掉归因。
本 PR 直接改规则:_qualified_bindings 允许一跳未改名 re-export(经由 tracked module),从机制上取消"每个消费方都要记得"的依赖;同时把 witness 选择从"smoke 里的名字分支 + collect_production 里的字面分支"收敛成一张按注册 input_producer 取用的表。
更小的修法我评估过三条:继续逐个修消费方(不可强制、会反复回归)、跟随任意跳数(归因无边界的误报风险)、把 re-export 关系写进注册表(等于在 import graph 之外再造一个权威)。一跳 + tracked module + owner symbol 是能覆盖真实场景的最小、且可证伪的规则。
改动思路
权威输入仍是注册表派生的 owner 表、tracked module 集合与各模块 AST;判定权在 python_production._qualified_bindings,一跳规则由新的 _reexported 实现。判定的边界写得很克制:只有名字本身是 owner symbol(alias.name in symbols)才会去解析目标模块,目标必须是 tracked module,import 必须未改名(import X as X 视为未改名),且该名字之后不能被重新赋值/定义/再 import。查找使用 owner-qualified 的 imports 表,因此第二跳不会被链式跟随——这正是"一跳"的可执行定义。
Witness 侧是收敛而不是扩张:INPUT_WITNESSES 成为唯一入口(注册表数据无法 import callable),smoke 保留一个 INPUT_PRODUCER_ANCHOR 并要求每个被锚定的 site 都有 witness。
具体改动
7 个文件、+199/-29:python_production.py 约 50 行(一跳解析 + (path, text hash) 解析树缓存)、production.py 的 witness 表收敛约 35 行、smoke 的锚点合并、双语 RFC §5 记录一跳规则与附录 A/B 行、其余是正负用例。
关键代码讲解
python_production.py::_reexported(约 46 行):按语句顺序扫描目标模块,遇到未改名的 ImportFrom 就用 owner-qualified imports 表取 binding;遇到改名导入、同名 ClassDef/FunctionDef、同名赋值或后续 Import 就清空该 binding。因为查的是 owner-qualified 表,目标模块若自己也是从第三个 re-export 导入,就取不到值——第二跳天然失败,而不是靠注释约定。
python_production.py::_qualified_bindings(约 183 行):在原有"直接 owner import"之外,仅当 alias.name in symbols 且目标是 tracked module 且不是自身时,才调用 _reexported;_tracked_module 同时支持 x.py 与 x/__init__.py。这样非 owner 名字不会触发任何额外解析。
production.py::INPUT_WITNESSES(约 231 行):按注册的 input_producer site 选 witness,注册表不承载 callable;smoke 第 329 行的断言让"被锚定的 producer 必须有 witness"成为可失败的门禁。
对主干的风险
最强回归是过度归因:把只是改名、遮蔽或转手的名字记成生产点,那比 unresolved 更糟,因为它会掩盖真实缺口。我逐条读到实现里对应的拒绝分支,并跑了负例矩阵;实现只扫描顶层语句,因此像 try/except 里的条件式 re-export 这类非常规形态会"失败到 unknown",不会误判。
实测(当前 main 已含 #4571,所以增量只剩一跳的效果):unresolved_producer_sites 42 → 41,唯一被解决的条目是 loop_controller.py::_envelope_route:118(经 driver.py 的 re-export 绑定),unknown 列表 diff 只有这一行删除、无新增;两次 smoke 均 ok。测试:tests/architecture/test_semantic_python_production.py + test_semantic_production.py + test_turn_contract_generation.py 共 123 通过。我还对新门禁做了变异验证:清空 INPUT_WITNESSES 后 smoke 直接 FAIL: turn_result_kind: anchored input producer has no executable witness(exit 1),随后已还原文件。
P3(非阻塞):_TREES 是模块级无淘汰缓存,按 (path, text hash) 累积。smoke 这类短进程无碍,但长驻调用方扫多版本会持续增长;建议限定生命周期或改成有界缓存。
P3(非阻塞):unresolved_producer_sites 这个数字仍未被锚定(本 PR 锚定的是 witness,而非计数)。也就是说未来的绑定回归仍可能"绿着退化"。既然 B2 的目标就是证据,给计数或目标词表的行加一个锚点会让下一次回归立刻可见。
我的整体评价
baseline(当前 main,42)与 head(41)对比只有预期的单点变化,其余行逐字一致;负例矩阵、witness 锚点与解析树缓存让这条规则既可证伪又与运行期开销无关。这不是"为了让自己的模块通过而放宽检查",而是把一条真实存在的隐式依赖用有界规则表达出来——与仓库"每个规则一个 owner、避免第二权威"的约定一致。
范围与收益匹配(change_proportionality: proportionate):一处扫描器规则 + 一处 witness 收敛,没有新增注册表字段、CLI 或权限面;repository_reuse: reused、typed_state_rule 为类型化 AST 规则(非子串启发式)、authority_semantics 不适用。
结论 APPROVE,两条 P3 非阻塞。复评只需在 head 变化时重跑 main/head 两次 smoke 对比与上述测试文件。
English verdict: APPROVE - exact head 56c4a9f; binding one unrenamed re-export hop through tracked modules removes the per-consumer import dependency while keeping a falsifiable, owner-qualified rule, and consolidates witness selection into one code-owned table. Verified independently on the current main (which already contains #4571): unresolved_producer_sites 42 -> 41 with exactly one resolved entry (loop_controller.py::_envelope_route:118) and no new unknown sites, smoke ok at both revisions, 123 focused tests pass, and emptying INPUT_WITNESSES makes the smoke fail with 'anchored input producer has no executable witness'. Two non-blocking P3s: the parsed-tree cache has no eviction, and the unresolved-site count itself is still not pinned by an anchor.
Rebaselines this branch on main after loopx-project#4571 merged the executor import fix, so the producer-site measurement in the PR body is taken against the real base rather than the pre-loopx-project#4571 tree. Signed-off-by: song <22676124+songoow@users.noreply.github.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
huangruiteng
left a comment
There was a problem hiding this comment.
动机
问题是真的,而且是我自己在真实 smoke 上复现的:M2 把三个 Turn owner 搬进 turn_contract_generated.py 之后,凡是仍从 transaction.py / driver.py 这两个兼容垫片导入 owner 的生产点,都静默变成 unknown_producer——这些模块一行没改,也没有任何检查报错。根因正如正文所说:python_production._qualified_bindings 只在"从 owner 自己的模块导入"时才建立绑定。我在 262923de1(M2)上实跑 drift smoke,unresolved_producer_sites=52。
这个失效形状恰好是 scanner 存在的意义所在:生产点悄悄丢失归因,而没有任何门变红。所以本 PR 的动机不是"让数字更好看",而是把这条静默丢失通道关掉。
改动思路
方向选得对:只绑一跳 unrenamed re-export,并且保持 fail-closed——重命名 re-export(as Other)、第二跳、同名 class/assignment、后续普通 import 都仍然让 consumer 保持 unknown,而不是伪造出一个值;只有本身就是 owner symbol 的名字才会被跟进。这比"多跳都绑"更安全:unknown 变成 evidence 比 unresolved 更糟,这一点我认同。
改动里还包含一处 witness 收敛:把 collect_production 里的名字分支(if vocabulary.get('input_producer') == '...loop_controller.py::decide_loop_disposition')改成一张 code-owned 表 INPUT_WITNESSES,smoke 侧四份字面量收敛成一个 INPUT_PRODUCER_ANCHOR。这是压缩而不是追加,而且它加了约束:smoke 现在为每个 anchor 断言 site in INPUT_WITNESSES,所以 anchor 掉了 witness 会立刻失败,而不是静默通过。注册表数据依然无法导入 callable。
具体改动
loopx/semantics/python_production.py:新增_reexported(一跳解析)与_tracked_module,把modules贯穿_qualified_bindings/scan_python_production,docstring 明确写出"一跳、其余保持 unknown"。loopx/semantics/production.py:collect_production传入modules=by_path;名字分支替换为INPUT_WITNESSES表(新增_probe_controller_domain)。examples/semantic-vocabulary-drift-smoke.py:单一INPUT_PRODUCER_ANCHOR,每个 anchor 必须有 witness 的断言。tests/architecture/test_semantic_python_production.py(+45)与test_semantic_production.py(+27):正向(unrenamed hop、import X as X、consumer alias)与反向(重命名 re-export、本地同名 class、重绑定、后续普通 import、错误来源、第二跳、未跟踪模块)fixture,以及真实仓库里loop_controller.py::_envelope_route经driver.py的归因。- 两个语言版本的 RFC §5 与 Appendix A/B。
我在这个 exact head 上自己跑的验证:
python examples/semantic-vocabulary-drift-smoke.py:M2262923de1→ 52,merge basecd1b32f4b→ 42,head → 41;其余汇总行一致,ok。pytest tests/architecture -q→ 280 passed(75.5s)。- 直接探针(构造 owner/shim/consumer 三个
SourceFile调scan_python_production):普通顶层 re-export 解析出values={'run'};顶层Action = None正确变成unresolved=True。
对主干的风险
P2(非阻塞,latent):one-hop 规则看不见"模块级控制流里的重绑定"。_reexported 只遍历 _parsed(source).body 的直接语句,只有直接出现的 Import / Assign / AnnAssign / FunctionDef / ClassDef 才清除绑定;嵌在模块级 try / if / for / with 里的同名绑定不是 body 的直接语句,因此不可见。我用五种形式对 scan_python_production 做了探针——try: Action = factory() / except ImportError: pass、if True: Action = None、for Action in (None,): pass、with open('x') as Action: pass、del Action——全部仍然绑定到 owner(values={'run'},unresolved=False),而顶层 Action = None 会正确变成 unresolved=True。try/except 正是这类兼容模块最标准的可选依赖写法,而同一函数的 docstring 明确承诺相反("a local class or assignment of the same name ... leaves the symbol unbound, so the consumer stays unknown")。今天它是 latent 的——仓库里两个垫片(loopx/control_plane/turn_driver/transaction.py、driver.py)都是顶层简单 re-export——所以不是阻塞项;但一旦命中,后果正是 scanner 要避免的那一种。最小修复:把模块体内任意嵌套层级的同名绑定都当作 unbind(walk Name/ExceptHandler/For/With 的 target 与 Delete,或复用本模块已有的 shadowing 逻辑),并补嵌套形式的负例。建议 P2 优先于 P3。
P3(非阻塞):Appendix A ledger 那条 "Unresolved sites: 52 → 41" 的审计证据不可复现。可复现的写法是分两段:M2 262923de1 → merge base cd1b32f4b 是 52 → 42(其中 10 个是 #4571 已经解决掉的 executor 站点),merge base → head 才是本 PR 自身在落地树上的 42 → 41。PR body 其实已经把这层说清楚了("Against this branch's original pre-#4571 base ... 52 → 41"),但 ledger 是那一条的审计依据,而它把两段基线合并成了一个数字;同一句话只在英文版出现一次,中文版随共享句一起改即可。
两点观察(不作为 finding):
_TREES这个 AST 缓存以(path, hash(text))为键、无上限,长扫描的内存曲线没有测过;目前 semantics 包之外没有调用者,所以只是记录。- anchor 站点字符串在 smoke 的
INPUT_PRODUCER_ANCHOR和生产的INPUT_WITNESSES里各有一份字面量。两份被 smoke 的site in INPUT_WITNESSES断言绑在一起,所以漂移会红,不是本次引入的新分歧(原实现同样是==字面量比较);只是记录一下这两处的收敛可能。
我的整体评价
这是一次正确且方向明确的收口:把一个已经被证实的静默归因丢失(M2 之后 52 个站点无声变成 unknown)用最小、fail-closed 的规则修掉,并且没有把守卫放宽——反向 fixture 覆盖了重命名、第二跳、本地同名、后续 import、未跟踪模块,smoke 还新增了 "anchor 必须有 witness" 这条真正承重的断言。我独立复现了它落在的树上的数字(52 / 42 / 41),head 上 280 个 architecture 测试全绿,探针也确认顶层重绑定会正确 unbind。
两条记录项(P2 嵌套重绑定失明、P3 ledger 基线混用)都不构成这次 post-merge audit 的阻塞;建议后续按最小修复补上,其中 P2 优先,因为它直接违背同一函数 docstring 给出的保证,且 try/except 属于常见写法而非边角。
English verdict: APPROVE (exact head c7a79c8)
Summary
turn_contract_generated.py, every production site that still imported an owner through thetransaction.py/driver.pycompatibility re-exports becameunknown_producer(43 → 52 unresolved sites) with no code change in those modules and no failing check.python_production._qualified_bindingsbound an owner only when it was imported from the owner's own module.executor.py's import by hand, which resolves 10 of those sites. Against current main this change is 42 → 41: it additionally resolvesloop_controller.py::_envelope_route:118(which fix(turn-driver): import the Turn result owner from its generated module #4571 cannot, becauseloop_controllerimportsLoopXTurnRoutefromdriver.py's re-export), and — the actual point — it removes the dependency on every consumer remembering to import the owner module, which is what failed silently in M2.as Other), a same-name class or assignment, or a later plainimportleaves the consumer unknown;import X as Xcounts as unrenamed. Only names that are owner symbols are followed and parsed trees are cached, so the full drift smoke keeps its runtime (32.6 s before and after).INPUT_WITNESSES, keyed by the registeredinput_producersite) instead of a name branch in the smoke plus a literal branch incollect_production. The smoke keeps a singleINPUT_PRODUCER_ANCHOR(was four literal copies) and requires each anchored site to have a witness. Registry data still cannot import a callable.Issue Or Task
Validation
unitpassed(replayed)tests/architecture/test_semantic_python_production.py: 4 new tests — positive: unrenamed hop,import X as X, consumer alias; negative: renamed re-export, local twin class, rebinding, later plain import, wrong source, second hop, untracked module.tests/architecture/test_semantic_production.py: real-repo attribution ofloop_controller.py::_envelope_routethroughdriver.py's re-export; witness runs only for the anchored site. pytest is not installable in the authoring sandbox, so each case was replayed by calling the same functions directly; CItest-shardis authoritative.regression_paritypassed(re-measured)python examples/semantic-vocabulary-drift-smoke.py --report. Against current maincd1b32f4b(which already contains #4571): 42 → 41, the remaining gain beingloop_controller.py::_envelope_route:118. Against this branch's original pre-#4571 base the same change measured 52 → 41 (executor.py×10 plus_envelope_route), i.e. the 10 executor sites overlap with #4571. In both runs no site becomes newly visible or unregistered, every other summary line is identical, F1 did not fire, and wall time is unchanged (32.56 s → 32.60 s).staticpassedpy_compileon all changed Python files.integrationpassedloopx canary premerge --from-git-diff --git-diff-base origin/main(tierstandard, run from the checkout):ok: true, 0 failures; the selected catalog canaries include the drift smoke itself.project_turn_route/_ControllerInputs.render(covered by the executable controller witness),driver.py::build_loopx_turn_plan/_envelope_route:119pass-through of a producer call,executor.pyparameter pass-through (3), and 34effective_actionsites for the next slice.settlement.py::turn_settlement_failure_outcomeis a second TypeScript-outcome decoder without an input witness; it is outside the three kernel vocabularies' scan by registry design (literal_scan: null) and is recorded as a gap rather than silently registered.Frontend / Visual Evidence
N/A — no user-visible change.
🤖 Generated with Claude Code