Skip to content

feat(semantics): bind one re-export hop in the Python producer scanner (B2) - #4573

Merged
huangruiteng merged 2 commits into
loopx-project:mainfrom
songoow:codex/b2-turn-kernel-producers
Sep 16, 2026
Merged

huangruiteng merged 2 commits into
loopx-project:mainfrom
songoow:codex/b2-turn-kernel-producers

Conversation

@songoow

@songoow songoow commented Sep 16, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • B2 (Refs [Task]: Semantic vocabulary convergence follow-ups after RFC PR #4433 / 语义词表收敛后续任务 #4447): after M2 moved the three Turn owners into turn_contract_generated.py, every production site that still imported an owner through the transaction.py / driver.py compatibility re-exports became unknown_producer (43 → 52 unresolved sites) with no code change in those modules and no failing check. python_production._qualified_bindings bound an owner only when it was imported from the owner's own module.
  • Rebaselined after fix(turn-driver): import the Turn result owner from its generated module #4571 merged. fix(turn-driver): import the Turn result owner from its generated module #4571 fixed executor.py's import by hand, which resolves 10 of those sites. Against current main this change is 42 → 41: it additionally resolves loop_controller.py::_envelope_route:118 (which fix(turn-driver): import the Turn result owner from its generated module #4571 cannot, because loop_controller imports LoopXTurnRoute from driver.py's re-export), and — the actual point — it removes the dependency on every consumer remembering to import the owner module, which is what failed silently in M2.
  • Bind one unrenamed re-export hop through a tracked module. A second hop, a renamed re-export (as Other), a same-name class or assignment, or a later plain import leaves the consumer unknown; import X as X counts as unrenamed. Only names that are owner symbols are followed and parsed trees are cached, so the full drift smoke keeps its runtime (32.6 s before and after).
  • Simplification, still explicit: the two executable input witnesses are now selected from one code-owned table (INPUT_WITNESSES, keyed by the registered input_producer site) instead of a name branch in the smoke plus a literal branch in collect_production. The smoke keeps a single INPUT_PRODUCER_ANCHOR (was four literal copies) and requires each anchored site to have a witness. Registry data still cannot import a callable.
  • RFC §5 (both mirrors) states the one-hop rule; Appendix A ledger entry and Appendix B decision row added.
  • Relationship to fix(turn-driver): import the Turn result owner from its generated module #4571: that one-line import fix remains good dependency hygiene, but attribution no longer depends on every consumer remembering it.

Issue Or Task

Validation

  • Tested revision: 56c4a9f
  • Run state: finished
  • Input classes: synthetic, public_fixture
Check kind Result Public-safe evidence / limitation
unit passed (replayed) tests/architecture/test_semantic_python_production.py: 4 new tests — positive: unrenamed hop, import X as X, consumer alias; negative: renamed re-export, local twin class, rebinding, later plain import, wrong source, second hop, untracked module. tests/architecture/test_semantic_production.py: real-repo attribution of loop_controller.py::_envelope_route through driver.py's re-export; witness runs only for the anchored site. pytest is not installable in the authoring sandbox, so each case was replayed by calling the same functions directly; CI test-shard is authoritative.
regression_parity passed (re-measured) python examples/semantic-vocabulary-drift-smoke.py --report. Against current main cd1b32f4b (which already contains #4571): 42 → 41, the remaining gain being loop_controller.py::_envelope_route:118. Against this branch's original pre-#4571 base the same change measured 52 → 41 (executor.py ×10 plus _envelope_route), i.e. the 10 executor sites overlap with #4571. In both runs no site becomes newly visible or unregistered, every other summary line is identical, F1 did not fire, and wall time is unchanged (32.56 s → 32.60 s).
static passed py_compile on all changed Python files.
integration passed loopx canary premerge --from-git-diff --git-diff-base origin/main (tier standard, run from the checkout): ok: true, 0 failures; the selected catalog canaries include the drift smoke itself.
  • Coverage and gaps: the remaining 41 unresolved sites are parameter/table-driven paths that need B2 slot analysis, not binding: controller project_turn_route / _ControllerInputs.render (covered by the executable controller witness), driver.py::build_loopx_turn_plan / _envelope_route:119 pass-through of a producer call, executor.py parameter pass-through (3), and 34 effective_action sites for the next slice. settlement.py::turn_settlement_failure_outcome is a second TypeScript-outcome decoder without an input witness; it is outside the three kernel vocabularies' scan by registry design (literal_scan: null) and is recorded as a gap rather than silently registered.

Frontend / Visual Evidence

N/A — no user-visible change.

🤖 Generated with Claude Code

…r (B2)

After M2 moved the three Turn owners into turn_contract_generated.py, every
production site that still imported an owner through the transaction.py /
driver.py compatibility re-exports became `unknown_producer` (43 -> 52
unresolved sites) with no code change in those modules and no failing
check: `_qualified_bindings` bound an owner only when it was imported from
the owner's own module.

Bind one unrenamed re-export hop through a tracked module. A second hop, a
renamed re-export (`as Other`), a same-name class or assignment, or a later
plain `import` of the name leaves the consumer unknown; `import X as X`
counts as unrenamed. Only names that are owner symbols are followed and
parsed trees are cached, so the full drift smoke keeps its runtime
(32.6 s before and after).

Select the two executable input witnesses from one code-owned table keyed
by the registered `input_producer` site instead of a name branch in the
smoke and a literal branch in collect_production; the smoke keeps a single
INPUT_PRODUCER_ANCHOR and requires each anchored site to have a witness.
Registry data still cannot import a callable.

Measured with examples/semantic-vocabulary-drift-smoke.py --report on main
vs this change: unresolved_producer_sites 52 -> 41 (executor.py x10,
loop_controller.py::_envelope_route:118); no site becomes newly visible or
unregistered; every other summary line is identical.

Refs loopx-project#4447 (Track B, B2 producer pilot for the Turn kernel vocabularies).

Signed-off-by: song <22676124+songoow@users.noreply.github.com>

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我感觉基于这个rfc,可以打磨一套capability在仓库里,甚至可以for其它项目使用了,比如用loopx开发别的项目时,也注重词表复杂度这块

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

动机

评审 head:56c4a9f4b2fa2bc64f67a48e080622fe8e46d440;base:main(merge-base f1166e81e,当前 main 已含 #4571)。

M2 把三个 Turn owner 搬进 turn_contract_generated.py 后,凡是仍通过 transaction.py / driver.py 兼容 re-export 导入 owner 的消费方,都失去了 producer attribution——而且没有任何检查会失败。上一轮的 #4571 用一行 import 修好了 executor.py 这一个消费方,但那只是"消费方记得改",依赖本身还在:下一个重构又会静默丢掉归因。

本 PR 直接改规则:_qualified_bindings 允许一跳未改名 re-export(经由 tracked module),从机制上取消"每个消费方都要记得"的依赖;同时把 witness 选择从"smoke 里的名字分支 + collect_production 里的字面分支"收敛成一张按注册 input_producer 取用的表。

更小的修法我评估过三条:继续逐个修消费方(不可强制、会反复回归)、跟随任意跳数(归因无边界的误报风险)、把 re-export 关系写进注册表(等于在 import graph 之外再造一个权威)。一跳 + tracked module + owner symbol 是能覆盖真实场景的最小、且可证伪的规则。

改动思路

权威输入仍是注册表派生的 owner 表、tracked module 集合与各模块 AST;判定权在 python_production._qualified_bindings,一跳规则由新的 _reexported 实现。判定的边界写得很克制:只有名字本身是 owner symbol(alias.name in symbols)才会去解析目标模块,目标必须是 tracked module,import 必须未改名(import X as X 视为未改名),且该名字之后不能被重新赋值/定义/再 import。查找使用 owner-qualified 的 imports 表,因此第二跳不会被链式跟随——这正是"一跳"的可执行定义。

Witness 侧是收敛而不是扩张:INPUT_WITNESSES 成为唯一入口(注册表数据无法 import callable),smoke 保留一个 INPUT_PRODUCER_ANCHOR 并要求每个被锚定的 site 都有 witness。

具体改动

7 个文件、+199/-29:python_production.py 约 50 行(一跳解析 + (path, text hash) 解析树缓存)、production.py 的 witness 表收敛约 35 行、smoke 的锚点合并、双语 RFC §5 记录一跳规则与附录 A/B 行、其余是正负用例。

关键代码讲解

python_production.py::_reexported(约 46 行):按语句顺序扫描目标模块,遇到未改名的 ImportFrom 就用 owner-qualified imports 表取 binding;遇到改名导入、同名 ClassDef/FunctionDef、同名赋值或后续 Import 就清空该 binding。因为查的是 owner-qualified 表,目标模块若自己也是从第三个 re-export 导入,就取不到值——第二跳天然失败,而不是靠注释约定。

python_production.py::_qualified_bindings(约 183 行):在原有"直接 owner import"之外,仅当 alias.name in symbols 且目标是 tracked module 且不是自身时,才调用 _reexported;_tracked_module 同时支持 x.py 与 x/__init__.py。这样非 owner 名字不会触发任何额外解析。

production.py::INPUT_WITNESSES(约 231 行):按注册的 input_producer site 选 witness,注册表不承载 callable;smoke 第 329 行的断言让"被锚定的 producer 必须有 witness"成为可失败的门禁。

对主干的风险

最强回归是过度归因:把只是改名、遮蔽或转手的名字记成生产点,那比 unresolved 更糟,因为它会掩盖真实缺口。我逐条读到实现里对应的拒绝分支,并跑了负例矩阵;实现只扫描顶层语句,因此像 try/except 里的条件式 re-export 这类非常规形态会"失败到 unknown",不会误判。

实测(当前 main 已含 #4571,所以增量只剩一跳的效果):unresolved_producer_sites 42 → 41,唯一被解决的条目是 loop_controller.py::_envelope_route:118(经 driver.py 的 re-export 绑定),unknown 列表 diff 只有这一行删除、无新增;两次 smoke 均 ok。测试:tests/architecture/test_semantic_python_production.py + test_semantic_production.py + test_turn_contract_generation.py 共 123 通过。我还对新门禁做了变异验证:清空 INPUT_WITNESSES 后 smoke 直接 FAIL: turn_result_kind: anchored input producer has no executable witness(exit 1),随后已还原文件。

P3(非阻塞):_TREES 是模块级无淘汰缓存,按 (path, text hash) 累积。smoke 这类短进程无碍,但长驻调用方扫多版本会持续增长;建议限定生命周期或改成有界缓存。

P3(非阻塞):unresolved_producer_sites 这个数字仍未被锚定(本 PR 锚定的是 witness,而非计数)。也就是说未来的绑定回归仍可能"绿着退化"。既然 B2 的目标就是证据,给计数或目标词表的行加一个锚点会让下一次回归立刻可见。

我的整体评价

baseline(当前 main,42)与 head(41)对比只有预期的单点变化,其余行逐字一致;负例矩阵、witness 锚点与解析树缓存让这条规则既可证伪又与运行期开销无关。这不是"为了让自己的模块通过而放宽检查",而是把一条真实存在的隐式依赖用有界规则表达出来——与仓库"每个规则一个 owner、避免第二权威"的约定一致。

范围与收益匹配(change_proportionality: proportionate):一处扫描器规则 + 一处 witness 收敛,没有新增注册表字段、CLI 或权限面;repository_reuse: reused、typed_state_rule 为类型化 AST 规则(非子串启发式)、authority_semantics 不适用。

结论 APPROVE,两条 P3 非阻塞。复评只需在 head 变化时重跑 main/head 两次 smoke 对比与上述测试文件。

English verdict: APPROVE - exact head 56c4a9f; binding one unrenamed re-export hop through tracked modules removes the per-consumer import dependency while keeping a falsifiable, owner-qualified rule, and consolidates witness selection into one code-owned table. Verified independently on the current main (which already contains #4571): unresolved_producer_sites 42 -> 41 with exactly one resolved entry (loop_controller.py::_envelope_route:118) and no new unknown sites, smoke ok at both revisions, 123 focused tests pass, and emptying INPUT_WITNESSES makes the smoke fail with 'anchored input producer has no executable witness'. Two non-blocking P3s: the parsed-tree cache has no eviction, and the unresolved-site count itself is still not pinned by an anchor.

Rebaselines this branch on main after loopx-project#4571 merged the executor import fix,
so the producer-site measurement in the PR body is taken against the real
base rather than the pre-loopx-project#4571 tree.

Signed-off-by: song <22676124+songoow@users.noreply.github.com>

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

动机

问题是真的,而且是我自己在真实 smoke 上复现的:M2 把三个 Turn owner 搬进 turn_contract_generated.py 之后,凡是仍从 transaction.py / driver.py 这两个兼容垫片导入 owner 的生产点,都静默变成 unknown_producer——这些模块一行没改,也没有任何检查报错。根因正如正文所说:python_production._qualified_bindings 只在"从 owner 自己的模块导入"时才建立绑定。我在 262923de1(M2)上实跑 drift smoke,unresolved_producer_sites=52。

这个失效形状恰好是 scanner 存在的意义所在:生产点悄悄丢失归因,而没有任何门变红。所以本 PR 的动机不是"让数字更好看",而是把这条静默丢失通道关掉。

改动思路

方向选得对:只绑一跳 unrenamed re-export,并且保持 fail-closed——重命名 re-export(as Other)、第二跳、同名 class/assignment、后续普通 import 都仍然让 consumer 保持 unknown,而不是伪造出一个值;只有本身就是 owner symbol 的名字才会被跟进。这比"多跳都绑"更安全:unknown 变成 evidence 比 unresolved 更糟,这一点我认同。

改动里还包含一处 witness 收敛:把 collect_production 里的名字分支(if vocabulary.get('input_producer') == '...loop_controller.py::decide_loop_disposition')改成一张 code-owned 表 INPUT_WITNESSES,smoke 侧四份字面量收敛成一个 INPUT_PRODUCER_ANCHOR。这是压缩而不是追加,而且它加了约束:smoke 现在为每个 anchor 断言 site in INPUT_WITNESSES,所以 anchor 掉了 witness 会立刻失败,而不是静默通过。注册表数据依然无法导入 callable。

具体改动

  • loopx/semantics/python_production.py:新增 _reexported(一跳解析)与 _tracked_module,把 modules 贯穿 _qualified_bindings / scan_python_production,docstring 明确写出"一跳、其余保持 unknown"。
  • loopx/semantics/production.py:collect_production 传入 modules=by_path;名字分支替换为 INPUT_WITNESSES 表(新增 _probe_controller_domain)。
  • examples/semantic-vocabulary-drift-smoke.py:单一 INPUT_PRODUCER_ANCHOR,每个 anchor 必须有 witness 的断言。
  • tests/architecture/test_semantic_python_production.py(+45)与 test_semantic_production.py(+27):正向(unrenamed hop、import X as X、consumer alias)与反向(重命名 re-export、本地同名 class、重绑定、后续普通 import、错误来源、第二跳、未跟踪模块)fixture,以及真实仓库里 loop_controller.py::_envelope_route 经 driver.py 的归因。
  • 两个语言版本的 RFC §5 与 Appendix A/B。

我在这个 exact head 上自己跑的验证:

  • python examples/semantic-vocabulary-drift-smoke.py:M2 262923de1 → 52,merge base cd1b32f4b → 42,head → 41;其余汇总行一致,ok。
  • pytest tests/architecture -q → 280 passed(75.5s)。
  • 直接探针(构造 owner/shim/consumer 三个 SourceFile 调 scan_python_production):普通顶层 re-export 解析出 values={'run'};顶层 Action = None 正确变成 unresolved=True。

对主干的风险

P2(非阻塞,latent):one-hop 规则看不见"模块级控制流里的重绑定"。_reexported 只遍历 _parsed(source).body 的直接语句,只有直接出现的 Import / Assign / AnnAssign / FunctionDef / ClassDef 才清除绑定;嵌在模块级 try / if / for / with 里的同名绑定不是 body 的直接语句,因此不可见。我用五种形式对 scan_python_production 做了探针——try: Action = factory() / except ImportError: pass、if True: Action = None、for Action in (None,): pass、with open('x') as Action: pass、del Action——全部仍然绑定到 owner(values={'run'},unresolved=False),而顶层 Action = None 会正确变成 unresolved=True。try/except 正是这类兼容模块最标准的可选依赖写法,而同一函数的 docstring 明确承诺相反("a local class or assignment of the same name ... leaves the symbol unbound, so the consumer stays unknown")。今天它是 latent 的——仓库里两个垫片(loopx/control_plane/turn_driver/transaction.py、driver.py)都是顶层简单 re-export——所以不是阻塞项;但一旦命中,后果正是 scanner 要避免的那一种。最小修复:把模块体内任意嵌套层级的同名绑定都当作 unbind(walk Name/ExceptHandler/For/With 的 target 与 Delete,或复用本模块已有的 shadowing 逻辑),并补嵌套形式的负例。建议 P2 优先于 P3。

P3(非阻塞):Appendix A ledger 那条 "Unresolved sites: 52 → 41" 的审计证据不可复现。可复现的写法是分两段:M2 262923de1 → merge base cd1b32f4b 是 52 → 42(其中 10 个是 #4571 已经解决掉的 executor 站点),merge base → head 才是本 PR 自身在落地树上的 42 → 41。PR body 其实已经把这层说清楚了("Against this branch's original pre-#4571 base ... 52 → 41"),但 ledger 是那一条的审计依据,而它把两段基线合并成了一个数字;同一句话只在英文版出现一次,中文版随共享句一起改即可。

两点观察(不作为 finding):

  • _TREES 这个 AST 缓存以 (path, hash(text)) 为键、无上限,长扫描的内存曲线没有测过;目前 semantics 包之外没有调用者,所以只是记录。
  • anchor 站点字符串在 smoke 的 INPUT_PRODUCER_ANCHOR 和生产的 INPUT_WITNESSES 里各有一份字面量。两份被 smoke 的 site in INPUT_WITNESSES 断言绑在一起,所以漂移会红,不是本次引入的新分歧(原实现同样是 == 字面量比较);只是记录一下这两处的收敛可能。

我的整体评价

这是一次正确且方向明确的收口:把一个已经被证实的静默归因丢失(M2 之后 52 个站点无声变成 unknown)用最小、fail-closed 的规则修掉,并且没有把守卫放宽——反向 fixture 覆盖了重命名、第二跳、本地同名、后续 import、未跟踪模块,smoke 还新增了 "anchor 必须有 witness" 这条真正承重的断言。我独立复现了它落在的树上的数字(52 / 42 / 41),head 上 280 个 architecture 测试全绿,探针也确认顶层重绑定会正确 unbind。

两条记录项(P2 嵌套重绑定失明、P3 ledger 基线混用)都不构成这次 post-merge audit 的阻塞;建议后续按最小修复补上,其中 P2 优先,因为它直接违背同一函数 docstring 给出的保证,且 try/except 属于常见写法而非边角。

English verdict: APPROVE (exact head c7a79c8)

@songoow
songoow deleted the codex/b2-turn-kernel-producers branch September 28, 2026 03:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants