feat(capability): add auditable external evidence lifecycle - #4813
huangruiteng wants to merge 12 commits into
Conversation
|
Windows CI root cause was the managed Effect runtime package boundary, not the contract logic: the runtime staging test copies only |
|
Added the missing typed discovery step at exact head What changed:
Validation:
Product surfaces: CLI and managed Turn changed in this slice. Frontend/Lark remain a companion because there is still no persisted external-evidence configuration/readback owner for those surfaces; adding form state now would create a second source of truth. They should consume this same discovery/plan/admission projection once that owner exists. |
ce7d782 to
a99a8db
Compare
|
Rebased onto current Post-rebase validation:
The contract remains unchanged: discovery reports inventory/readiness only and does not claim provider execution or evidence coverage. Fresh CI is running. |
|
Added the provider-execution observation boundary at exact head This keeps execution with the existing method/connector owner while adding the shared TypeScript Validation:
This consumes the parent calibration constraint for |
huangruiteng
left a comment
There was a problem hiding this comment.
Request changes conclusion (author-owned PR; GitHub blocks formal self-review)
Reviewed exact head: 7cd3dc5ad373adbc4fc19b3e04ccd3b08b2956ee
动机
这个 PR 要把分散的 host research method、connector inventory 和下游证据消费统一成一个可审计的 discover → plan → execute → receipt → admit/reject → retire 生命周期。方向是对的:它明确区分“注册表里存在”“当前 ready”“已经执行”“证据覆盖”“被父调用方接纳”这些过去容易混在一起的语义,并且不把 provider 执行权、凭据或后续动作权限收进 Core。
当前 blocker 不在方向,而在这个生命周期最核心的承诺——receipt/admission 必须绑定到“原先的 exact ready plan”。现有实现只比较调用方重新提交的可变 plan 对象里的几个字符串,没有证明它仍然是最初生成的 plan。
改动思路
TypeScript 的 external_evidence.ts 负责 request/provider 规范化、ready provider 选择、receipt/admission 校验和 retirement 投影;Python external_research/cli.py 只读取有界 JSON 并通过 Effect runtime 调用同一 typed owner。connector registry 被保留为 inventory,provider 真正执行仍由原 method/connector owner 完成;这符合现有所有权边界。
正向路径是完整且内聚的。但负向路径有断层:planExternalEvidenceRequest 会为规范化 request 计算 request_id,之后 normalizeExecutionReceipt 却不重新规范化 request、也不校验完整 plan digest。调用方可以修改 objective/decision/constraints 或 provider 状态,只要保留旧 request_id 和 provider 字段,receipt 仍会被认为“绑定 exact plan”。最小修复应是对完整规范化 ready plan 生成并校验 plan_id,或在后续 reducer 中完整重建并核验所有 plan 不变量。
具体改动
- 新增 7 个 v0 schema 与五个 CLI/Effect 方法,覆盖 discovery、planning、execution receipt、admission、retirement。
- 新增 built-in capability catalog、双语 RFC/README 和 roadmap 登记。
- 新增 10 个 TypeScript 单测和 Python CLI/runtime 包装测试;现有 stale receipt identity、file provenance、subset admission 与 retirement coverage 都有覆盖。
关键代码讲解
planExternalEvidenceRequest正确地把规范化 request hash 成request_id,并只选择declared && installed && enabled && ready的 provider。normalizeExecutionReceipt是后续审计边界,但它直接信任传入plan.request.request_id与plan.selected_provider,没有验证 plan 自身是否被改写。recordExternalEvidenceExecution在这个弱校验后输出provider_execution_observed=true,因此问题不是少一个防御性字段,而是公开 truth contract 可能说错话。handle_external_evidence_command把文件里的 plan/receipt 直接交给 reducer;CLI 文件边界使篡改场景成为真实公共入口,而不只是内部函数误用。
对主干的风险
P1:可变 plan 可以绕过 exact-plan 绑定
我在 exact head 上先生成合法 plan,随后只把 plan.request.objective 从原值改为另一语义,保留旧 request_id,再提交匹配该 id 的 succeeded receipt。实际输出仍是:
status=succeeded, provider_execution_observed=true
这会让后续 admission/retirement 产生看似一致的 digest/id,但这些 id 证明的是调用方当下递交的对象组合,不是原始 ready plan 未被篡改。现有测试只改 receipt 的 request/provider id,因而全部通过但没有命中这个反例。
最低修复:
- 对完整规范化 ready plan(request、selected provider、execution envelope 等)生成不可歧义的
plan_id; - receipt/admission 必须重新验证该 plan 或校验
plan_id,不能只比较对象内自报字段; - 增加 objective、decision、constraints、provider readiness、execution envelope 的逐字段 mutation regression。
验证结果:control-plane typecheck 通过;10 个 TS 测试通过;7 个 Python CLI/runtime 测试通过;Ruff check/format 与 git diff --check 通过。按 capability packet 的 wait_for_ci=false,未轮询 GitHub CI。失败证据来自真实 exact-head TypeScript reducer,不依赖 mock。
语义与 CI 对齐
RFC 的“exact ready plan binding”和运行时当前能保证的“与传入 plan 对象字段相等”不一致。新增 mutation regression 后,应让旧实现失败、修复 head 通过,才能把这个 v0 audit vocabulary 视为成立。
我的整体评价
能力归属、provider 权限边界、inventory/readiness 分离以及 Python→TypeScript owner 的架构选择都合理,1694 行也围绕一个完整 preview 生命周期展开,不是无关拼装;future-facing pass 最值得做的是抽出一个唯一 canonical plan decoder/digest verifier,让 plan、receipt、admission 共用同一不变量,而不是再增加第二套校验。
但 exact-plan 绑定是这项能力存在的核心价值,当前反例会直接让审计语义失真,所以本 head 需要修改后复审。修复后仍应把一个真实 host method 与一个 connector 的端到端 acceptance 保留为明确 companion evidence,不能用 reducer 单测替代。
English verdict: REQUEST_CHANGES - head 7cd3dc5 accepts a semantically mutated plan while still reporting succeeded observed execution; typecheck, 10 TS tests, 7 Python tests, Ruff and diff checks pass, but canonical exact-plan integrity needs a plan digest plus mutation coverage.
7cd3dc5 to
8b016ba
Compare
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
258255e to
b34f14a
Compare
Summary
external_evidence_research_v0request, discovery, plan, receipt-observation, admission, and retirement lifecycleexternal-researchand connector providers behind one auditable capability boundaryloopx external-evidence discover|plan|receipt|admit|retire, plus capability catalog, bilingual RFC, and roadmap registrationplan_id, and bind retirement to the complete normalized admission identityProduct delivery
Validation
npm run typecheck:control-planenode --no-warnings --experimental-strip-types --test tests/control_plane_ts/external_evidence_research.test.ts(13 passed)uv run --extra test python -m pytest -q tests/capabilities/test_external_evidence_cli.py tests/control_plane/test_effect_runtime_integration.py::test_runtime_decode_change_rotates_identity_and_starts_replacement(7 passed)uv run --extra test npm run test:control-plane(2202 passed, 18 skipped, 0 failed)uv run --extra test ruff check loopx/capabilities/external_research tests/capabilities/test_external_evidence_cli.pyuv run --extra test ruff format --check loopx/capabilities/external_research tests/capabilities/test_external_evidence_cli.pyuv run --extra test loopx canary premerge --from-git-diff(19 selected checks passed, 0 manual holds)external_evidence.tsis present, the Effect semantic probe passed, anddoctor --deepreportedtypescript_effect_runtime_ready=readygit diff --check origin/main...HEADRunning the control-plane suite outside the source-checkout
uvenvironment resolves subprocesses to the system Python 3.9 and fails on the repository's existing@dataclass(slots=True)usage. The requireduv run --extra testsource environment uses Python 3.13 and passes the full suite.