test(authority): add the shared-goal-authority E2E stage ladder - #3869
Conversation
|
CI note on |
huangruiteng
left a comment
huangruiteng
left a comment
There was a problem hiding this comment.
动机
这个 PR 想把 Shared Goal Authority RFC 已完成阶段的证据从分散的 unit/E2E、示例脚本和人工说明，收敛成一个可增量执行的 stage ladder。价值在于：每个阶段声明都有稳定 row id、明确 product path、环境 gate、机器可读结果和退出策略；尚未完成的 Stage 2A/2C2 项也应显式显示为 unverified 或 pending，而不是被绿色测试吞掉。旧状态下，file/NoKV/PostgreSQL、CLI observation、迁移、runtime-root 等证据分散，无法用一份报告回答“这个 exact checkout 实际证明了哪些阶段”。这个 PR 的方向与 RFC 晋升流程吻合，而且坚持 test-only：writer 由真实 python -m loopx.cli 驱动，candidate 通过生产 TypeScript FileAuthorityStore 只读回查，没有新增第二套生产写入口。
改动思路
authority_e2e_ladder.py 作为 registry/runner/report owner：LadderRow 定义 stage、path、gate 和执行函数，PendingRow 声明未来义务，run_row() 把 assertion、环境缺失和异常收敛为 pass/fail/unverified，build_report() 绑定 commit/probe digest/经过哈希的环境事实并做 privacy scan。authority_e2e_fixtures.py 创建隔离 registry、runtime、repo 和 HOME，所有行为写入都经 child CLI；authority_store_readback_probe.ts 只调用 load/scan/readReceipt，不调用会创建 identity 的 storeIdentity()。
正向路径是选择 stage/row → 检查环境 gate → 每行在独立临时 workspace 执行真实 CLI 或 store probe → 收集 public-safe evidence → 汇总 pending/bindings → privacy scan → 根据 fail/unverified 决定退出码。负向路径包括缺 NoKV/PostgreSQL 环境时显式 unverified、POSIX-only 在 Windows 上 unverified/skip、CLI/探针失败时 redacted fail、报告中出现 root/HOME/URL/config value 时改写为 privacy_violation。pytest projection 为 CI 提供逐行可见性，standalone example 保留完整 11-row 执行；五条已有相同 CLI E2E 覆盖的慢行默认 skip，并可用 LOOPX_LADDER_FULL=1 恢复。
具体改动
authority_e2e_fixtures.py：定义GoalWorkspace、migration fixture、隔离 child environment、CLI spawn/kill、observation lock、candidate document 与 TAP/readback helpers。authority_e2e_ladder.py：实现 Stage 0 file/NoKV matrix、Stage 1 CLI→TS store readback、Stage 2B live PostgreSQL，以及 Stage 2C1 configure、writer family、default-off、candidate failure、SIGKILL gap、dual runtime root、migration rows；并提供 row filtering、bindings、redaction、JSON report 和 CLI。authority_store_readback_probe.ts：对真实FileAuthorityStore做 bounded load/scan/receipt projection，只输出 cursor、revision、ids 与计数等有限字段。test_shared_goal_authority_e2e.py：把 row 投影为参数化 pytest，并钉住 vocabulary、pending registry、unverified exit policy、privacy violation、list/filter 和 report schema。examples/shared-goal-authority-e2e/：提供可直接复跑的入口、row/gate/报告说明，以及后续 2C2 PR 必须暴露的测试 seams。- 英中 RFC 与 evidence note：登记 2026-09-03 的 9 pass / 2 unverified / 10 pending 边界；pyproject/tsconfig 纳入 strict mypy 与 probe typecheck。
关键代码讲解
run_row()是 typed failure boundary：先处理 OS/environment gate，再把 runner exception 变成经过 forbidden-token redaction 的fail，不会让异常绕过报告。build_report()/assert_public_safe()是公开证据边界：rows 中的泄漏降级为privacy_violation，bindings 泄漏则整体清空并标记，之后重新计算 summary/exit code。_row_every_writer_family_captures()和_row_dual_runtime_root_consistency()是最有产品价值的 Stage 2C1 行：它们从 CLI action 贯穿到 candidate cursor/head/store identity，并断言 provider 不参与 local decision/writeback。_row_crash_gap_loses_observation()通过持有 observation lock、等待 primary Todo 可见、SIGKILL writer，再验证下一次 snapshot 恢复，准确描述 #3818 当前“无 durable outbox”的边界。exit_code_for()是 harness 是否可用于 gate 的最终权威，目前只检查 fail/unverified，未处理 selected pending，这也是下述 blocker 之一。
对主干的风险
当前 exact head 有两个会让 stage report 产生错误绿色/错误完备性声明的 blocker。
-
已完成且已在本 head 中的 Stage 2A 仍被声明为等待合并。
PENDING_ROWS把s2a.nokv_live_qualification写成pending_until="PR #3819 merge"，RFC/README 也说等待 #3819；但 #3819 已于2026-09-02T15:59:33Z合并，而 #3869 创建于次日，并且 merge commited00e671a…已确认是 exact head7a663939…的 ancestor。代码树中也已经包含examples/nokv-authority-store/live-qualification.ts与相应 harness tests。于是本 PR “exercise every completed stage claim”的核心声明不成立。应把它升级为真正的 env-gated Stage 2A row（调用已合并的 live qualification，缺独立写入参数时报告具体 unverified），或收窄 PR/RFC 的完备性声明并给出新的、真实的 pending condition；不能继续以已经满足的 merge 条件标 pending。 -
选择 pending-only row 会零执行却 exit 0。 我在 exact head 运行
ladder.py --row s2c2.parity_equal，得到pass=0, fail=0, unverified=0, pending=1且进程退出0。这与模块文档“green only when every selected row passed”冲突，也允许 CI 选择一个尚未实现的 obligation 并拿到成功。exit_code_for()完全忽略summary.pending。最小修复是：当用户显式通过--row/--stage选择到 pending 项且没有可执行通过证据时非零退出（或新增显式--allow-pending，默认非零），并给 pending-only、mixed executable+pending、list-only 三条路径加测试。全量增量报告可以继续展示 pending，但必须把“报告生成成功”和“所选资格项验证成功”区分清楚。
独立验证：默认 pytest projection 8 passed, 7 skipped；standalone 完整 ladder 9 pass, 2 unverified, 10 pending，与作者结果一致；未传 --allow-unverified 的 exit policy 已由测试覆盖。TypeScript typecheck、两个新 Python module 的 configured mypy、Ruff、diff-check 和 GitHub required checks通过。NoKV/PostgreSQL live rows因本环境缺对应服务未执行；这部分正确显示为 unverified。默认 CI 跳过五条已有 CLI E2E 同形覆盖的慢行是公开且可恢复的，没有被我当作隐式通过。
我的整体评价
实现的 test-only 隔离、真实 CLI 路径、TS readback、public-safe report 和未验证状态表达都很扎实，9 条 deterministic 结果也确实可复现；约 2.7k 行主要是可维护的场景 fixture 和声明式证据，而非生产复杂度。不过这个 PR 的核心产品就是“证据是否诚实、完备、可作 gate”，因此 stale Stage 2A pending 与 pending-only 绿色退出不是文案小问题，而是 capability 语义错误。整 PR 结论为 REQUEST_CHANGES。修复两项并更新英中 RFC/README/测试后，可在新 exact head 重新评审。
English verdict: REQUEST_CHANGES at exact head 7a6639390cb5c00c068895acd43e3cff9b6fcc97. Stage 2A is still marked pending on “PR #3819 merge” even though #3819 is already merged and its merge commit is in this head, so the ladder does not cover every completed stage as claimed. Also, selecting a pending-only row executes zero tests but exits 0. The deterministic ladder otherwise reproduced at 9 pass / 2 unverified / 10 pending; pytest, typecheck, mypy, Ruff, diff-check, and GitHub checks pass.
|
@huangruiteng Both blockers are addressed on exact head
The new row pushed the ladder module over the 1500-line ratchet ceiling, so the row vocabulary and the Stage 2C rows now live in two sibling modules (strict mypy, probe digests updated). README, RFC (en, zh-CN), and the evidence note no longer reference #3819 as pending. Premerge on this head: 4/4 direct checks, 18/18 catalog canaries, boundary clean, 0 holds. |
|
The honesty machinery here is the best part of the ladder. Three points (fine to defer any):
|
huangruiteng
left a comment
huangruiteng
left a comment
There was a problem hiding this comment.
Findings
[P1] 默认测试投影仍然把必需的 pytest 检查推过 15 分钟上限
tests/control_plane/test_shared_goal_authority_e2e.py:77 会把未显式跳过的 ladder 行加入默认 pytest；虽然本轮已经排除了五个重复的 CLI E2E 行，但精确 head bdded14f477b11afd5d6fe85e128256e4e7a956c 的必需检查仍未完成：Python Tests / pytest 在 99% 时被 job 的 15 分钟上限取消（13:27:33 到达 99%，13:27:56 收到 The operation was canceled），因此没有 pytest 汇总、coverage 结论或绿色 required check。这不是 teardown 后才发生的取消；测试主体尚未完成。这个 PR 的目标是增加可复跑的 CI 证据，但当前默认投影使主验证通道无法给出结论。请继续削减默认 CI 的新增耗时，或调整/拆分 workflow 的预算，并在当前 head 上取得一次完整绿色的必需检查后再合并。
Open questions / assumptions
- 我把 CI 上未完成的必需检查视为合并 blocker；本地窄测通过不能替代完整 required check。
- 非阻塞建议：
CLI_E2E_COVERED_ROW_IDS依赖一组硬编码 row id 和“另一个测试文件已经覆盖”的约定，但没有守卫对应底层测试节点仍存在。后续可以把共享断言或节点映射做成可验证的单一来源，避免原测试被改名/删除后这里继续静默 skip。 - 当前
candidate_failure_preserves_primary覆盖的是候选目录创建失败；如果损坏的既有候选文档也属于本阶段承诺，建议补一条独立负例，避免把两种失败路径混为一谈。
动机
这组改动试图把 shared-goal authority RFC 中已经完成的 Stage 0/1/2A/2B/2C1 声明，收敛成一条逐行、可机读、默认 fail-closed 的 E2E ladder。它解决的核心问题是：RFC、既有测试和 live probe 分散，操作者难以从一次运行中判断哪些阶段真正执行、哪些只是环境缺失、哪些仍为 pending。
改动思路
整体设计方向是对的：用统一 row registry 描述 stage、gate、product path 与 runner；真实 CLI 路径负责写入，生产 TypeScript FileAuthorityStore 只读回放；报告把 pass、unverified、pending 分开，并把 commit、tree dirtiness 与 probe digest 绑定到证据。Stage 2C2 尚未实现的部分只声明为 pending，不冒充完成。
具体改动
- 新增 Python ladder、fixture 和 Stage 2C1 row support，覆盖 file matrix、CLI/TS 回读、NoKV/PostgreSQL 环境门控、shadow writer family、default-off、candidate failure、crash gap、双 runtime root 与 migration。
- 新增只读 TypeScript readback probe，并纳入 control-plane typecheck。
- 新增 pytest 投影、standalone example、README，以及英中 RFC/证据台账更新。
- 本轮已经修复旧 head 的两个关键问题：Stage 2A 现在是真实 env-gated 执行行；pending-only 或 mixed selection 在未传
--allow-pending时会非零退出。
关键代码讲解
run_row 负责把 gate 与 runner 的结果归一成 typed row result；build_report 再统一计算计数、binding 和 privacy scan，最后由 exit_code_for 执行 fail-closed 策略。CLI fixture 通过真实 python -m loopx.cli 驱动主写，TS probe 仅调用生产 store 的 loadAuthority、scanCommitted、readReceipt 做候选回读。Stage 2C1 的 writer-family 行验证 observation 与 primary writeback 的关系，而 crash-gap 行明确只证明“主写成功但该次 observation 可丢”，没有越界宣称 outbox 或 parity。
对主干的风险
生产运行路径没有被修改，主要风险集中在 CI 预算、证据准确性和未来维护漂移。隐私扫描、显式 live gate、pending/unverified 的非绿色退出策略都降低了公共证据误报风险；但当前 required pytest timeout 是实际的主干合并风险，必须先消除。Stage 2C2 仍全部 pending，因此这次改动不能被解读为 provider 晋升或完整 parity 已交付。
我的整体评价
从产品和架构上看，这是一套比散落脚本更可复用的证据合同，尤其是 typed row、单一报告 schema、真实 CLI 驱动和 fail-closed 退出语义值得保留。当前实现已经解决前两轮的主要语义问题，本地窄测也支持实现正确性；但完整必需检查没有在 CI 预算内完成，所以现在仍应 hold / request changes。
验证：
pytest -q -n 2 tests/control_plane/test_shared_goal_authority_e2e.py tests/control_plane/test_local_authority_shadow_config.py tests/control_plane/test_local_authority_shadow_cli_e2e.py：18 passed，8 skipped。- standalone ladder 使用显式
--allow-unverified --allow-pending：9 pass，3 unverified，9 pending；pending-only 未放宽时退出 1，放宽后退出 0。 ruff check：通过。mypy（四个新增 control-plane testing 模块）：通过。npm run typecheck:control-plane：通过。git diff --check与公共边界扫描：通过。- NoKV authority、NoKV legacy 与 PostgreSQL live 栈本地不可用，因此没有把 unverified 当作通过。
English verdict: Request changes — the exact head's required pytest job still times out before producing a complete test and coverage result; make the default CI projection fit the budget (or adjust/split that budget) and obtain a green required check.
Add one incremental end-to-end ladder that exercises every completed stage of the shared-goal-authority RFC through the real `python -m loopx.cli`, with an exit policy that never reports green while a selected row is unverified. - loopx/control_plane/testing/authority_e2e_ladder.py: row registry for Stage 0/1/2B/2C1, runners, the loopx_shared_goal_authority_e2e_report_v0 JSON report, bindings, privacy scan, and the exit policy `exit 0 iff fail == 0 and (unverified == 0 or --allow-unverified)`; Stage 2A and Stage 2C parity rows are declared pending, not claimed. - loopx/control_plane/testing/authority_e2e_fixtures.py: goal workspaces, CLI runners, observation-lock window, candidate read-back, TAP summary, and the lifted legacy migration source. - tests/control_plane_ts/authority_store_readback_probe.ts: read-only FileAuthorityStore probe (loadAuthority, paged scanCommitted, readReceipt), included in tsconfig.control-plane.json. - tests/control_plane/test_shared_goal_authority_e2e.py: one test per row (env-gated rows skip as unverified, POSIX-only rows skip on Windows) plus pins for the exit policy, privacy scan, listing, and registry. - examples/shared-goal-authority-e2e: thin runner and README documenting rows, gates, environment variables, exit policy, and the test seams the Stage 2C parity PRs must provide. - docs: Stage-ladder evidence subsection in RFC section 11 (en, zh-CN) and section 8 of the evidence note. Test-only: no production entry point constructs any store. Signed-off-by: wchwawa <wch19961116@gmail.com>
…erride The Stage 2C observation PR now routes every writer hook through one effective runtime root, so the dual-root scenario is no longer a pending parity row: it is a deterministic Stage 2C1 row. - fixtures: a `cli_override_divergent` binding registers a different `common_runtime_root` than the `--runtime-root` override. - ladder: `s2c1.dual_runtime_root_consistency` drives todo add, task-lease acquire, todo update, capture-followups, and a leased completion through the real CLI and requires one store identity, a head with both todos and the released lease, lease state under the override root, and neither a candidate lineage nor lease state under the registry root. - registry, pytest projection, README, RFC (en, zh-CN), and the evidence note drop the row from the pending list and count seven `s2c1.*` rows. Signed-off-by: wchwawa <wch19961116@gmail.com>
…budget The pytest job runs within seconds of its 15-minute timeout on current main (14m38s to 14m44s), and the full ladder projection pushed this branch over it twice. Five s2c1 rows repeat assertions that tests/control_plane/test_local_authority_shadow_cli_e2e.py already pins through the same product CLI path (configure round trip, default-off isolation, candidate failure, crash gap, dual runtime root). The default pytest projection now skips exactly those rows with an explicit reason; the rows that only the ladder exercises (file matrix, TypeScript read-back, every writer family, migration) stay in CI, LOOPX_LADDER_FULL=1 runs every row in pytest, and the example runner is unchanged. Signed-off-by: wchwawa <wch19961116@gmail.com>
…s from exiting green Review findings on 7a66393: 1. Stage 2A was still declared pending on "PR #3819 merge" although that PR is merged and in this head. `s2a.nokv_live_qualification` is now an env-gated `store_direct` row: it runs the merged `examples/nokv-authority-store/live-qualification.ts --execute-live` against an existing workbench with a fresh tenant/goal pair and requires `ok=true`, the single-node store-conformance scope, every check `passed`, NoKV SDK 0.11.0 / API 1, and no promotion or availability claim. The gate `env:nokv_authority` needs `LOOPX_NOKV_AUTHORITY_LIVE=1` plus the ignored client configuration path, the Python executable that resolves the SDK, and the workbench name; each missing input reports its own unverified reason. Every string leaf of the configuration and every input path is a forbidden token, and the evidence carries only check ids, counts, and config and workbench digest prefixes. The bindings digest now prefers the authority configuration file. 2. Selecting only pending rows executed nothing but exited 0. Pending rows are unmet obligations: `exit 0 iff fail == 0 and (unverified == 0 or --allow-unverified) and (pending == 0 or --allow-pending)`. The report records `allow_pending` and `summary.executed`, the summary line names the pending rows, and tests pin pending-only, pending-stage, mixed, and list-only selections. The ladder module crossed the 1500-line maintainability ceiling with the new row, so the shared row vocabulary moves to `authority_e2e_row_support.py` and the Stage 2C rows to `authority_e2e_rows_stage2c.py`; both join the strict mypy list and the probe digests. README, RFC (en, zh-CN), and the evidence note record the Stage 2A row, the new gate, the pending exit rule, and a live run of both NoKV rows on a local single-node stack. Signed-off-by: wchwawa <wch19961116@gmail.com>
bdded14 to
579681f
Compare
huangruiteng
left a comment
huangruiteng
left a comment
There was a problem hiding this comment.
Findings
[P1] binding-only privacy violations still return a green exit
loopx/control_plane/testing/authority_e2e_ladder.py:931-959 handles a forbidden token in bindings by replacing every binding with null and setting bindings.privacy_violation=true, but it then calls _finalize_report() with the original rows. Because exit_code_for() only reads row counts, a report with one passing row and a leak confined to bindings.probe_sha256[].path is finalized as fail=0 and exit_code=0. I reproduced this on exact head 579681fb9ba394b60110dbc9cfb15efaf65d2d6a by building a clean one-row report with authority_e2e_ladder.py as the forbidden token: the binding block was redacted and marked privacy_violation, while the summary remained pass=1, fail=0 and the exit policy remained green.
This breaks the ladder's stated fail-closed contract: a privacy violation in report metadata is still a failed evidence run, even if the leaked value is successfully removed from the serialized output. Please make a binding leak participate in the failure/exit decision (for example, by adding a typed synthetic fail/privacy_violation result or by making the exit policy explicitly consume a report-level privacy failure), and add a regression test for a leak that appears only in bindings.
动机
这个 PR 想把 shared-goal authority RFC 中散落在 Stage 0、1、2A、2B 与 2C1 的证明收敛成一个可增量运行、可机读且默认 fail-closed 的 stage ladder。此前真实 CLI 写入、file/NoKV/PostgreSQL store 探针、local shadow 的异常路径以及 RFC 证据分别存在，操作者很难从一次运行里区分“已经通过”“环境不具备”“尚未实现”。这次精确 head 还针对前轮评审补齐了两个关键缺口：Stage 2A 不再是过期的 pending 声明，而是显式 opt-in 的 live qualification；pending-only 选择也不再因为零执行而退出 0。
改动思路
入口 examples/shared-goal-authority-e2e/ladder.py 只负责定位 checkout 并调用统一 main()；authority_e2e_ladder.py 持有 row registry、环境 gate、Stage 0/1/2A/2B runner、typed report 与退出策略；fixture/support/Stage 2C 模块负责隔离 HOME、构造临时 goal、通过真实 python -m loopx.cli 写 primary state，再从候选字节或只读 TypeScript FileAuthorityStore 取证。正向路径是“选择 row -> gate -> runner -> typed RowResult -> bindings/privacy scan -> JSON 与 exit code”；负向路径把环境缺失记为 unverified、实现缺口记为 pending、断言/异常记为 fail，并要求显式 --allow-unverified/--allow-pending 才能放宽。架构边界总体清楚：candidate 不参与决策、不反写 local authority，Stage 2C2 也没有被冒充为已完成。
具体改动
- 三份 RFC/证据文档新增 stage-ladder 证据、边界和未验证项；example README 描述 12 个可执行 rows、9 个 pending rows、环境变量、退出策略与后续 test seam。
authority_e2e_fixtures.py提供隔离 registry/runtime/HOME、真实 CLI 子进程、SIGKILL 窗口、migration fixture、candidate document 和 TAP 解析；authority_e2e_row_support.py集中 row outcome、写入 helper 与 observation 不变量。authority_e2e_ladder.py注册 file matrix、CLI-to-TS readback、NoKV qualification、PostgreSQL conformance 和报告生成；authority_e2e_rows_stage2c.py覆盖配置往返、writer family、default-off、候选失败、crash gap、双 runtime root 与迁移新 lineage。- TypeScript probe 只调用
loadAuthority、分页scanCommitted和readReceipt，不调用会创建 identity 的storeIdentity()；pytest 投影、strict mypy 列表与 control-plane tsconfig 同步纳入新表面。
关键代码讲解
run_row() 先应用显式 gate，再把 runner 的异常收敛为经过脱敏的 typed failure；_row_nokv_live_qualification() 要求独立 opt-in、绝对配置/解释器路径、全量 passed checks、固定 SDK/API 版本，并明确拒绝 promotion/HA 语义扩张；row_every_writer_family_captures() 与 row_default_off_isolation() 分别验证所有本地 writer 的单向 observation 不变量以及关闭时的响应/存储等价；TypeScript summarizeScan() 对生产 store 做分页只读回放。assert_public_safe() 是最终证据边界，但也正是在这里出现本轮 blocker：row leak 会变成 fail/privacy_violation，binding-only leak 却只清空 metadata，没有影响退出码。
对主干的风险
本 PR 没有新增生产调用者，rg 只发现 example/tests/config 引用这些模块；因此默认关闭路径和 authority source 都未改变。我独立运行了完整 standalone ladder：9 pass、3 unverified、9 pending、0 fail；default-off 无候选目录，候选失败保留 primary，crash gap 不声称 outbox，双 runtime root 只有一个 candidate identity。变更体量仍很大（13 files，+3275/-0），但已经按 fixture、support、Stage 2C rows 与主 registry 拆分，维护成本与其跨阶段证据目标基本相称。
主要回归风险是报告可信度而非生产写路径。当前 binding-only privacy case 能在明确记录 privacy_violation=true 的同时退出 0，会让自动化把不合格的证据包误判为绿色，必须先修。远端 required pytest 已经跑完整套件，但因一个未触及文件里的随机 scheduler blob 命中 credential-like heuristic 而红；该精确节点本地复跑通过，属于独立的 CI 波动，但合并前仍应取得一次绿色 required check。NoKV 与 PostgreSQL live 栈本轮不可用，因此我保留其 unverified 状态，没有从作者描述或历史 CI 推断为 verified。
验证：
- focused Python suites：18 passed，8 skipped；单文件 ladder suite：10 passed，8 skipped。
- standalone ladder：9 pass，3 unverified，9 pending，0 fail；显式放宽两类未满足义务后退出 0。
- 新增四个 Python 模块 strict mypy 通过；Ruff 通过。
- control-plane TypeScript typecheck 通过。
- premerge canary：18/18（3 direct checks、1 Python compile、9 catalog、8 risk-profile、1 public-boundary），0 failures。
git diff --check与 exact-head/readback 检查通过。
我的整体评价
这套 ladder 的整体方向值得保留：它用真实 CLI、生产只读 store、typed 状态和清晰的 Stage 2C2 pending 边界，显著提升了 RFC 证据的可复跑性；本轮也确实修复了 Stage 2A 与 pending exit 的前序问题。当前仍需 request changes，因为最终 privacy gate 存在一个可稳定复现的“已检测违规但仍返回绿色”分支。修复 binding-only leak 的失败传播、补上对应回归测试，并让 required pytest 在当前 head 上转绿后，这个 PR 才满足它自己声明的 fail-closed 验收标准。
English verdict: Request changes on exact head 579681fb9ba394b60110dbc9cfb15efaf65d2d6a — the ladder now correctly covers Stage 2A and pending-only selections, but a privacy violation confined to report bindings is redacted and marked while the run still exits 0; make that condition fail the evidence run and add a binding-only regression test. Focused suites, standalone ladder, strict mypy, Ruff, TypeScript typecheck, diff checks, and 18 premerge canaries passed; the required pytest check is currently red on an unrelated credential-heuristic flake and should be green before merge.
…port bindings assert_public_safe() nulled the bindings and marked bindings.privacy_violation when a forbidden token appeared only there, but _finalize_report() derived the exit code from row counts alone, so a report with one passing row and a leak confined to the bindings block finalized as fail=0 and exit 0 while recording the violation. The summary now carries privacy_violations (leaking rows plus the bindings block), the exit policy consumes it ahead of every relaxation flag, and the standalone runner prints the condition on stderr. A regression pins the bindings-only case. The pytest projection's CI skip list now names each row's product-CLI twin test and a guard fails when a twin is renamed or removed, so the skip cannot outlive its coverage. README, RFC (en and zh), and the evidence note state the rule. Signed-off-by: wchwawa <wch19961116@gmail.com>
|
@huangruiteng Round 3 is on exact head The bindings-only leak reproduced exactly as you described (one passing row, Your non-blocking note on The red required check on Deferred to keep this round to the blocker, both worth a follow-up: a negative row for a corrupt existing candidate document (your note and @now-ing's point 3) and a stage tag for the four recoverable-execution verbs inside the Stage 0 matrix row (@now-ing's point 1). If you want either in this PR instead, say so. Validation on |
huangruiteng
left a comment
huangruiteng
left a comment
There was a problem hiding this comment.
评审 exact head：33889b0c11b868afe94e16427c044c8b47c308ee
动机
这个 PR 为 shared-goal authority RFC 建立从文件 provider、真实 CLI 写入与 TypeScript store 读回，到 Stage 2A/2B live gate 和 Stage 2C1 shadow observation 的一条可执行 E2E ladder。它的核心价值是让阶段声明对应到机器可判定证据，并确保未执行、待实现或环境缺失的行不会被误报为已验证。
改动思路
ladder 把每个已声明能力建模为 typed row，区分 deterministic、live env gate 与 pending；执行结果统一进入公开安全 report，再由 exit policy 决定整轮是否 green。pytest 提供 CI 投影，standalone runner 保留完整执行面；五条昂贵的 Stage 2C1 行只有在默认 pytest 投影中由已有 product-CLI E2E twin 覆盖，完整模式与 standalone 仍会真实执行。
具体改动
本轮新 head 修复了上一轮 blocker：summary.privacy_violations 同时统计 row 泄露和仅发生在 bindings 的泄露；bindings-only 情况会清空所有绑定、标记 bindings.privacy_violation=true，并且 --allow-unverified 与 --allow-pending 都不能放宽退出码。新增回归构造只在 probe manifest bindings 中出现的 token，证明 row 保持 pass、敏感 token 从最终 JSON 消失、summary 计数为 1、exit code 为 1。CI projection 也从裸 row 列表升级为 row→product-test 显式映射，并用 AST guard 防止 twin 被重命名或删除后继续静默跳过。
对主干的风险
没有发现新的 actionable blocker。隐私 fail-closed、pending-only 非绿色、Stage 2A 真实 gate、CI skip 映射和完整 Stage 2C1 行均有直接回归。完整 standalone 本地结果为 12 个执行行中 9 pass、3 个 live 环境行 unverified、9 个 Stage 2C2 声明 pending；在显式允许这两类未执行状态时退出 0，且 privacy violations 为 0。残余风险主要是 live NoKV/PostgreSQL 依赖外部环境，本地未执行，但远端 required checks 全绿，当前 head mergeable。
我的整体评价
结论为 APPROVE。上一轮仅 bindings 泄露仍可能退出 0 的问题已在 report schema、exit policy、回归测试与双语文档四层闭环；实现与 RFC 的公开证据边界一致。独立验证通过：默认 pytest 12 passed, 8 skipped，完整投影 17 passed, 3 skipped，standalone deterministic rows 全部通过，privacy/projection focused tests 通过，scoped Ruff、mypy、TypeScript typecheck、git diff --check 均通过；远端 required checks 全绿。
English verdict: APPROVE — exact head 33889b0c11b868afe94e16427c044c8b47c308ee. The bindings-only privacy leak now redacts every binding, increments summary.privacy_violations, and fails regardless of relaxation flags. Focused, default, full-mode, standalone, lint, type-check, and remote required checks pass; no blocker remains.
Signed-off-by: huangruiteng <huangrt01@163.com>
b8dd89d to
7d52029
Compare
3 participants
Summary
python -m loopx.cli(real_cli) or the retained store-level probes (store_direct): 12 rows for Stage 0, Stage 1, Stage 2A, Stage 2B, and the Stage 2C observation foundation; the 9 rows of the Stage 2C parity half are declaredpending, never counted as pass.exit 0ifffail == 0and (unverified == 0or--allow-unverified) and (pending == 0or--allow-pending). An environment-gated row without its stack reportsunverified; a selected pending row is an unmet obligation and blocks a green exit; the pytest projection skips gated rows visibly asunverified: <reason>. A privacy scan rewrites any leaked temporary root, home directory, connection URL, configuration value, or NoKV authority input path intofail/privacy_violation.FileAuthorityStorevia a read-only probe.@huangruiteng This PR is stacked on #3818 (base branch
codex/local-authority-shadow-product-path). The rows2c1.dual_runtime_root_consistencyis the ladder form of the mismatched-root regression gate from #3818 round 3; retarget tomainonce #3818 merges.Issue Or Task
docs/architecture/rfcs/shared-goal-authority-state-provider-v0.md(Stage 0, 1, 2B, 2C observation foundation), with the Stage 2A and Stage 2C parity rows declared pending.stage-2c-e2e-stage-ladderWhat is in the ladder
s0.file_matrix_twelve_rowss0.nokv_live_matrixs1.cli_document_decodes_through_ts_stores2a.nokv_live_qualifications2b.postgresql_conformance_lives2c1.configure_enable_disable_roundtrips2c1.every_writer_family_capturess2c1.default_off_isolations2c1.candidate_failure_preserves_primarys2c1.crash_gap_loses_observations2c1.dual_runtime_root_consistencys2c1.migration_seeds_new_lineagePending (declared, not claimed, and blocking a green exit unless
--allow-pending): nines2c2.*rows (outbox entries, idempotent drain, SIGKILL before and during drain, rollback with pending entries, parity equal and divergent, migration seed-and-drain, growth measurement).examples/shared-goal-authority-e2e/README.mdlists the test seams the Stage 2C parity PRs must expose so those rows can be implemented.Files:
loopx/control_plane/testing/authority_e2e_ladder.py(row registry, Stage 0/1/2A/2B runners,loopx_shared_goal_authority_e2e_report_v0, bindings, privacy scan, exit policy),authority_e2e_row_support.py(row vocabulary and CLI helpers),authority_e2e_rows_stage2c.py(Stage 2C rows),loopx/control_plane/testing/authority_e2e_fixtures.py,tests/control_plane_ts/authority_store_readback_probe.ts,tests/control_plane/test_shared_goal_authority_e2e.py,examples/shared-goal-authority-e2e/, and a Stage-ladder evidence subsection in the RFC (en, zh-CN) plus section 8 of the evidence note.Behavior disclosure: child CLI processes run with
HOME(andUSERPROFILEon Windows) pointed at a per-workspace directory so the ladder never touches the operator's real home.Round 2 (exact head
bdded14f4)s2a.nokv_live_qualificationruns the mergedexamples/nokv-authority-store/live-qualification.ts --execute-liveagainst an existing workbench with a fresh tenant/goal pair (gateenv:nokv_authority:LOOPX_NOKV_AUTHORITY_LIVE=1,LOOPX_NOKV_AUTHORITY_CONFIG_JSON,LOOPX_NOKV_AUTHORITY_PYTHON,LOOPX_NOKV_AUTHORITY_WORKBENCH; each missing input has its own unverified reason). It requiresok=true, the single-node store-conformance scope, every checkpassed, SDK0.11.0/ API1, and no promotion or availability claim; evidence carries check ids, counts, and config and workbench digest prefixes only, and every configuration value and input path is a forbidden token.exit_code_fortreatspending != 0likeunverified != 0unless--allow-pending;--row s2c2.parity_equalexits 1 withexecuted=0,--stage 2c2exits 1, a mixed selection exits 1 even when its executable row passes, and--listprints the registry without an exit policy.summary.executedandexit_policy.allow_pendingmake "report generated" and "selection verified" distinguishable.--stage 0 --stage 2a:s0.file_matrix_twelve_rowspass,s0.nokv_live_matrixpass (13 rows, 12 parity rows, restored lineage fails closed),s2a.nokv_live_qualificationpass (13 checks, final generation 3); the report contains no endpoint, credential, path, or workbench name.Round 3 (exact head
33889b0c1)Fixes the blocker found on
579681fb9: a forbidden token that appeared only inbindings(for example a probe path) was redacted and markedbindings.privacy_violation=true, butexit_code_for()read row counts alone, so the run finalized asfail=0/ exit 0.summary.privacy_violationscounts leaking rows plus the bindings block; the exit policy consumes it ahead of every relaxation flag (exit 0 iff fail == 0 and privacy_violations == 0 and (unverified == 0 or allow_unverified) and (pending == 0 or allow_pending)), and the standalone runner prints the condition on stderr. A report-level count was chosen over a synthetic row soexecutedand the row registry keep meaning what they say.bindings.privacy_violation=true,summary.privacy_violations=1,fail=0, exit 1, andexit_code_for(..., allow_unverified=True, allow_pending=True) == 1.Validation
Round 3 on
33889b0c1: pytest projection 12 passed / 8 skipped,LOOPX_LADDER_FULL=117 passed / 3 skipped; standalone ladder 9 pass / 3 unverified / 9 pending,privacy_violations=0, exit 0 with both relaxation flags; configured mypy (19 files), Ruff,git diff --check, docs governance and asset-integrity smokes, premerge againstorigin/mainall clean.pytest -n 2 tests/control_plane/test_shared_goal_authority_e2e.py tests/control_plane/test_local_authority_shadow_config.py tests/control_plane/test_local_authority_shadow_cli_e2e.py: 18 passed, 8 skipped (3 gated rowsunverified: ..., 5 rows pinned by the CLI E2E suite);LOOPX_LADDER_FULL=1: 15 passed, 3 skipped; maintainability ratchet green after the module splitpython examples/shared-goal-authority-e2e/ladder.py: pass 9, fail 0, unverified 3, pending 9, executed 12, exit 1;--allow-unverifiedalone still exits 1 (pending);--allow-unverified --allow-pendingexits 0;--row s2c2.parity_equalexits 1 (executed 0) and 0 only with--allow-pending;--stage 2c2exits 1; mixed--row s0.file_matrix_twelve_rows --row s2c2.parity_equalexits 1; the JSON report contains no temporary root, home, repository path, or configuration value--stage 0 --stage 2a --allow-pending: 3 pass (s0.file_matrix_twelve_rows,s0.nokv_live_matrix13 rows,s2a.nokv_live_qualification13 checks, SDK 0.11.0 / API 1), exit 0, no leaked tokens2c1.every_writer_family_captures: 12 writer families captured, candidate cursor12;s2c1.dual_runtime_root_consistency: 5 observations, 1 store identity, candidate cursor5, registry root without lineage or lease statenpm run typecheck:control-plane(the read-only probe is in the include list)tests,loopx/control_plane,examples/shared-goal-authority-e2e; configured mypy with both new modules in the strict listloopx canary premerge --from-git-diff --git-diff-base <#3818 head>on this head: standard tier, 4/4 direct checks, 18/18 catalog canaries with 0 failures and 0 warnings,full-publicrisk profile ok,premerge-public-boundaryclean, 0 manual holdss2b.postgresql_conformance_livewas not run in this environment (no PostgreSQL); it reportsunverifiedby designPremerge comment fields: changed_surfaces =
loopx/control_plane/testing/*(new, test-only),tests/control_plane_ts/authority_store_readback_probe.ts,tests/control_plane/test_shared_goal_authority_e2e.py,examples/shared-goal-authority-e2e/*, RFC and evidence docs,pyproject.tomlmypy list,tsconfig.control-plane.json; direct_checks = 4/4; catalog_canaries = 18/18; risk_profile_smokes = full-public ok; public_private_boundary = clean; failures_or_skips = 3 env-gated rows unverified without their stacks (two NoKV rows verified live locally); manual_holds = none; merge_decision = ready for review after #3818.Type of Change
LoopX Area
Technical Direction
Shared Goal Authority and cross-host coordination
Target base branch:
codex/local-authority-shadow-product-path(stacked on feat(authority): add local post-commit observation capture #3818; retarget tomainafter it merges)Direction tracker or promotion unit: Stage 2C stage-ladder evidence
Boundary Checklist
.loopx/,.codex/goals/, liveACTIVE_GOAL_STATE.md, credentials, private benchmark traces, verifier output, raw agent sessions, internal document links, or local machine paths.Signed-off-bytrailer (git commit -s).