diff --git a/docs/architecture/rfcs/ledger/shared-goal-authority-state-provider-v0/2026-09-24-default-cutover-reconciliation.md b/docs/architecture/rfcs/ledger/shared-goal-authority-state-provider-v0/2026-09-24-default-cutover-reconciliation.md index 7ef0be0b3e..5d7bbc2a10 100644 --- a/docs/architecture/rfcs/ledger/shared-goal-authority-state-provider-v0/2026-09-24-default-cutover-reconciliation.md +++ b/docs/architecture/rfcs/ledger/shared-goal-authority-state-provider-v0/2026-09-24-default-cutover-reconciliation.md @@ -20,7 +20,7 @@ them as though they were interchangeable PRs. | #4870 claim-preserving writes; #4888 reviewed cutover; #4920 drain planning | Implemented. Exercise their combined head; do not commission replacements. | | #4922 complete canonical snapshot pagination; #4960 qualified SQLite runtime admission; #4961 display refresh recovery; #4964 shared source summaries | Implemented. Consumer and packaged-client acceptance still needs integration evidence; a whole new pagination/recovery implementation is not pending. | | #4967 typed complete-source assembly; #4968 native outbox delivery/recovery | Implemented. Complete-source transport is also merged in #5013; capture assembly is not missing. | -| #5003 atomic event-owned completion | Merged. Solves batch publication/retry, **not** the event writer's shadow-capture binding. | +| #5003 atomic event-owned completion | Historical implementation; retired with its Todo event source in #5054. Do not rebuild capture for it. | | #4994 explicit leased Agent handoff; #4995 generated Monitor proof; #4991 rejected poll reservation; #4992 deferred receipt-bound Turn | Merged. Audit the integrated callers before deciding what remains; do not recreate them under a new caller-refactor PR. | | #4931 retained SQLite proof encoding, contributor #4224 | Open optimization plus incomplete D2 qualification. A speedup is not capacity/recovery/soak acceptance. | | #4915 default `.loopx` filesystem placement | Separate configuration migration; does not select File/SQLite authority. | @@ -41,7 +41,7 @@ not implement another store or close the whole migration package or D2 gate. | Proposed PR | Observable result and owner | Exit | | --- | --- | --- | | 1. External-effect execution fencing | Lease/effect owners protect the actual execution interval, takeover, timeout, exit and uncertain completion. Reuse merged #4994/#4995. | Stale executors cannot continue or settle; real executor and receipt recovery matrix passes. A point-in-time proof check is insufficient. | -| 2. Event-writer binding and whole-Goal migration/rollback | Bind event writer locks/atomic publication to existing outbox; integrate Markdown/event/lease capture, drain, saved cutover, consumers and fenced export/rollback; delete Python decisions replaced by TS. | Reuse #5003. Retain `event_log_writer_not_bound` until binding passes; close D1, command inventory and D3 cohort. One Goal without an event overlay does not prove this package. | +| 2. Whole-Goal migration/rollback | Qualify existing Markdown/lease capture, drain, saved cutover, provider consumers and fenced export/rollback. Retire reachable Python decisions after TS adoption. | Close D1, command inventory and D3 cohort. #5054 removes the experimental event source; no new event-writer binding is planned. | | 3. Default entrypoints and bounded Python retirement | New Goals, settings, installation and packaged frontend/Lark/CLI select a qualified profile consistently; existing Goals have explicit migration/disable flows. | 1/2 and applicable D1–D3 pass; user entrypoints work; delete business writers only after their last callers migrate. Retain rendering, host IO and lawful import/export. | **Plan three named future implementation PRs, plus existing #4931 and outstanding diff --git a/docs/architecture/rfcs/ledger/shared-goal-authority-state-provider-v0/2026-09-24-default-cutover-reconciliation.zh-CN.md b/docs/architecture/rfcs/ledger/shared-goal-authority-state-provider-v0/2026-09-24-default-cutover-reconciliation.zh-CN.md index dd7bb82427..47cd57a57c 100644 --- a/docs/architecture/rfcs/ledger/shared-goal-authority-state-provider-v0/2026-09-24-default-cutover-reconciliation.zh-CN.md +++ b/docs/architecture/rfcs/ledger/shared-goal-authority-state-provider-v0/2026-09-24-default-cutover-reconciliation.zh-CN.md @@ -17,7 +17,7 @@ | #4870 保留 claim 的写入、#4888 reviewed cutover、#4920 drain 规划 | 已实现。验收组合 head，不再重新安排一套替代实现。 | | #4922 完整 canonical 快照分页、#4960 SQLite runtime 准入、#4961 显示刷新恢复、#4964 共享来源摘要 | 已实现。消费者和打包客户端仍需组合验收，不等于还缺一个全新的分页/恢复实现。 | | #4967 TS 完整来源组装、#4968 原生 outbox 交付/恢复 | 已实现。大型来源传输亦已通过 #5013 合入；不能再称为 capture 未做。 | -| #5003 event-owned completion 原子提交 | 已合入。解决整批发布/重试，不负责 event writer 与 shadow capture 的绑定。 | +| #5003 event-owned completion 原子提交 | 历史实现；在 #5054 中随旧 Todo 事件来源一起退役，不再补其 capture。 | | #4994 带 lease 的显式 Agent 交接、#4995 Monitor 命令 proof、#4991 拒绝 poll 后释放预约、#4992 延期且绑定 receipt 的 Turn | 已合入。组合现有实现盘点 caller，不能再开一个 caller 重构 PR 重做它们。 | | #4931 SQLite retained proof 编码、contributor #4224 | 优化 PR 开放，D2 资格未闭合。提速不等于容量、恢复和 soak 验收通过。 | | #4915 默认 `.loopx` 目录 | 独立的配置迁移，不会选择 File/SQLite authority。 | @@ -37,7 +37,7 @@ scan 100 p95 801.81 ms / 250 ms），#4931 尚未提供精确 head 的正式复 | 拟议 PR | 可观察结果与 owner | 退出条件 | | --- | --- | --- | | 1. 外部动作执行区间保护 | lease/effect owner 将执行身份验证覆盖到实际外部动作、接管、超时、退出及不确定完成。复用已合入 #4994/#4995。 | 过期 executor 不能继续执行/结算；真实执行器及 receipt 恢复矩阵通过。执行前查一次 proof 不够。 | -| 2. 事件 writer 绑定与整 Goal 迁移/回退闭环 | 将 event writer 锁和原子发布接入现有 outbox；组合 Markdown/event/lease writer、drain、saved cutover、消费者和 fenced export/rollback，删除被 TS 替代的 Python 决策。 | 复用 #5003，绑定通过前保留 `event_log_writer_not_bound`；闭合 D1、命令清单与 D3 cohort。单个无 event overlay 的 Goal 晋升不证明本项。 | +| 2. 整 Goal 迁移／回退闭环 | 验证现有 Markdown／lease capture、drain、saved cutover、provider 消费者及 fenced export／rollback；TS 接管后删除仍可达的 Python 决策。 | 闭合 D1、命令清单和 D3 cohort。#5054 删除实验事件来源，不再规划新的 event writer 绑定。 | | 3. 默认入口与有界 Python 退役 | 新 Goal、settings、安装及 packaged frontend/Lark/CLI 一致选择合格 profile；存量有显式迁移与停用流程。 | 1/2 及适用 D1–D3 通过，验证用户入口，删除最后 caller 已转走的业务 writer；保留 renderer、host IO、合法导入导出。 | **计划是三个可命名的后续实现 PR，加已有 #4931 和未闭合证据；不是保证总计四个 diff --git a/docs/architecture/rfcs/ledger/shared-goal-authority-state-provider-v0/2026-09-24-event-completion-transaction.md b/docs/architecture/rfcs/ledger/shared-goal-authority-state-provider-v0/2026-09-24-event-completion-transaction.md index b869451d2c..35035e7af3 100644 --- a/docs/architecture/rfcs/ledger/shared-goal-authority-state-provider-v0/2026-09-24-event-completion-transaction.md +++ b/docs/architecture/rfcs/ledger/shared-goal-authority-state-provider-v0/2026-09-24-event-completion-transaction.md @@ -1,5 +1,7 @@ # Event-owned completion: one publication before capture integration +> Superseded by [Todo event retirement](2026-09-25-todo-event-retirement.md): this historical implementation is removed in #5054. + Baseline: `90f21a5299188d54f984a5313e774c9ac48d6595`. This advances overall roadmap R5/G2, shared-authority L2/L7 and TS T1/T2. It closes an existing event writer correctness gap; it does not qualify that writer for shadow capture. diff --git a/docs/architecture/rfcs/ledger/shared-goal-authority-state-provider-v0/2026-09-24-event-completion-transaction.zh-CN.md b/docs/architecture/rfcs/ledger/shared-goal-authority-state-provider-v0/2026-09-24-event-completion-transaction.zh-CN.md index 88b76bea57..15a81314b7 100644 --- a/docs/architecture/rfcs/ledger/shared-goal-authority-state-provider-v0/2026-09-24-event-completion-transaction.zh-CN.md +++ b/docs/architecture/rfcs/ledger/shared-goal-authority-state-provider-v0/2026-09-24-event-completion-transaction.zh-CN.md @@ -1,5 +1,7 @@ # 事件源完成事务：先建立完整提交，再接入捕获 +> Superseded by [Todo event retirement](2026-09-25-todo-event-retirement.md): this historical implementation is removed in #5054. + 基线：`90f21a5299188d54f984a5313e774c9ac48d6595`。对应总路线 R5/G2、 shared-authority L2/L7 与 TS T1/T2。本批修复已有事件写入者的正确性， 不授予该写入者 shadow capture 资格。 diff --git a/docs/architecture/rfcs/ledger/shared-goal-authority-state-provider-v0/2026-09-25-todo-event-retirement.md b/docs/architecture/rfcs/ledger/shared-goal-authority-state-provider-v0/2026-09-25-todo-event-retirement.md new file mode 100644 index 0000000000..88c2915102 --- /dev/null +++ b/docs/architecture/rfcs/ledger/shared-goal-authority-state-provider-v0/2026-09-25-todo-event-retirement.md @@ -0,0 +1,46 @@ +# Retire the experimental Todo event source + +## Delivery brief + +- Goal/source: roadmap R5/S12, TypeScript T4 and shared-authority D1–D3; + maintainer direction on #5054 retires the old Todo event route. +- Gap: a second Todo projection/writeback pipeline was being extended solely to + capture a retired source. Supervisor proposal/receipt logging was its remaining + independent product caller. +- Result: delete Todo replay/overlay/backfill/completion and the unused migration + bridge. Keep Markdown compatibility and provider authority; detect nonempty + retired sources rather than silently omit their records. +- Owners: Todo source admission; existing TS completion/authority owners; + experimental supervisor log under `control_plane/agents`. +- Acceptance: source selectors refuse without writes or validation effects; + canonical reads ignore stale legacy files; normal completion/successors and + downstream status/quota/review-packet work; supervisor concurrent receipts, + preview, conflicting identity and uncertain-publication replay are covered. + +## Compatibility and scope + +A nonempty `events.jsonl`, `state_event_log`, `state_events_file` or `event_log` +source refuses legacy Todo reads/writes and shadow qualification. The operator +must preserve it and export/inspect its Todos with a compatible older release +before deliberately removing the binding/file from the active source location. +There is no automatic replay, deletion, Markdown fallback or new migration API. +Absent and zero-byte files have no event-owned Todos. Previously prepared +unsupported event outbox records remain rejected; this PR does not certify them. + +The supervisor is experimental/default-off. Its `supervisor_log_event_v0` +envelope accepts only local-private proposals/receipts. Old experimental log +formats require manual archival before a fresh log; unknown formats fail without +rewriting. Admission and publication share the log lock; execution replay checks +full semantic identity while allowing a new observation timestamp. Preview never +publishes or syncs the log. A durable executed receipt prevents a second executed +receipt; this does not close the crash interval between an external host effect +and its receipt. External-effect fencing retains its own roadmap acceptance. + +## Remaining boundary + +This supersedes older entries calling for event-writer binding/capture. It does +not recount already delivered transaction capture, nor subtract a PR from a +fixed total. Remaining exits are executor-effect fencing, qualified whole-Goal +migration/rollback, default onboarding and bounded deletion of reachable Python +writers. D2 backend/capacity/soak and #4931 remain independent evidence. PostgreSQL +continues using the same authority contract; no new provider is added here. diff --git a/docs/architecture/rfcs/ledger/shared-goal-authority-state-provider-v0/2026-09-25-todo-event-retirement.zh-CN.md b/docs/architecture/rfcs/ledger/shared-goal-authority-state-provider-v0/2026-09-25-todo-event-retirement.zh-CN.md new file mode 100644 index 0000000000..83e31c282a --- /dev/null +++ b/docs/architecture/rfcs/ledger/shared-goal-authority-state-provider-v0/2026-09-25-todo-event-retirement.zh-CN.md @@ -0,0 +1,37 @@ +# 退役实验性 Todo 事件来源 + +## 交付摘要 + +- 目标／来源：roadmap R5/S12、TypeScript T4 与 shared-authority D1–D3， + 以及维护者在 #5054 中删除旧 Todo 事件路径的明确方向。 +- 缺口：为捕获一个退役来源而继续扩展第二套 Todo 投影和写回；其中仍有独立用途的 + caller 是 supervisor 提案／回执日志。 +- 结果：删除 Todo replay、overlay、backfill、completion 与未被产品调用的迁移桥。 + 保留 Markdown 兼容及 provider 权威；拒绝非空旧来源，避免无声丢失记录。 +- 归属：Todo 来源准入、现有 TS completion／authority owner，以及 + `control_plane/agents` 下的实验 supervisor 日志。 +- 验收：各来源别名拒绝且不写数据／不执行验收命令；canonical 读取忽略遗留文件； + 普通 completion／successor 与 status／quota／review-packet 链路正常； + supervisor 覆盖并发回执、预览、身份冲突和提交结果未知后的重放。 + +## 兼容与边界 + +非空 `events.jsonl`、`state_event_log`、`state_events_file` 或 `event_log` 来源 +会阻止旧 Todo 读写和 shadow 资格验证。操作者须保留原文件，先用兼容旧版本检查／ +导出其中 Todo，再有意移除活动来源位置的绑定或文件。没有自动回放、删除、 +Markdown 降级或新增迁移 API。缺失及零字节文件不承载事件 Todo。 +以前准备但不受支持的事件 outbox 记录仍会被拒绝，本 PR 不为其提供资格认证。 + +Supervisor 保持实验性、默认关闭；新 `supervisor_log_event_v0` 只接收本地私有的 +提案和回执。旧实验格式须人工归档后启用新日志；未知格式会明确拒绝，不自动改写。 +准入与发布共用日志锁；重试比较完整语义身份，允许观察时间变化。 +预览不发布或同步日志。持久化 executed 回执阻止第二份 executed 回执，但不解决 +“宿主外部动作执行后、回执落盘前崩溃”的区间；外部动作保护继续遵循独立验收。 + +## 剩余边界 + +本决策替代旧交付记录中“补事件 writer 绑定／捕获”的计划，不重复计数已交付的 +事务捕获，也不机械地从固定 PR 数量中减一。剩余退出条件仍是执行器外部动作保护、 +整 Goal 迁移／回退资格、默认启用和可达 Python writer 的有界删除。 +D2 后端／容量／持续运行证据与 #4931 单列。PostgreSQL 继续复用原 authority 合同； +本 PR 不添加 provider。 diff --git a/docs/architecture/rfcs/shared-goal-authority-state-provider-v0.md b/docs/architecture/rfcs/shared-goal-authority-state-provider-v0.md index 3276631f87..2283b213e1 100644 --- a/docs/architecture/rfcs/shared-goal-authority-state-provider-v0.md +++ b/docs/architecture/rfcs/shared-goal-authority-state-provider-v0.md @@ -23,16 +23,20 @@ [Chinese version](./shared-goal-authority-state-provider-v0.zh-CN.md) and this English version are semantic mirrors. A difference between them is a defect. -## Current delivery frontier (2026-09-25) +## Todo event retirement (2026-09-25) -Audit `37bbaec79` and current PR states: complete-source transport, transaction -capture, source assembly and the five previously open caller/event fixes are -merged, not future implementation. After the current promotion-admission repair, -three named code boundaries remain planned: external-effect execution fencing; -event-writer binding plus whole-Goal migration/rollback; default onboarding plus -bounded Python retirement. #4931 and outstanding D2 evidence are tracked -separately. Three is a delivery plan, not a guaranteed total PR count. -[Current inventory and exits](ledger/shared-goal-authority-state-provider-v0/2026-09-24-default-cutover-reconciliation.md). +PR #5054 replaces its original event-writer capture proposal with removal of +that experimental Todo source. `events.jsonl` is no longer projected, overlaid, +backfilled or used for completion. Nonempty default/aliased sources are refused +without changing their bytes; empty/absent sources permit the Markdown path. +Promoted Goals continue reading their selected provider even if stale legacy +files remain. The supervisor uses its own local-private experimental log. + +This closes the *retired source* branch by deletion, not by claiming event-writer +capture was qualified. Whole-Goal migration/rollback and default onboarding +still require their existing acceptance; this change does not make all Python +writers or PostgreSQL deployment ready. Do not add a replacement event capture +PR to the remaining work. [Decision and exits](ledger/shared-goal-authority-state-provider-v0/2026-09-25-todo-event-retirement.md). ## Persistence route for steward scale (2026-09-16) @@ -64,8 +68,7 @@ integration defects; it neither recounts shipped capture nor flips global defaul [Operation and boundaries](../../reference/reviewed-coordination-promotion.md). Handoff-mode changes now share one TS ownership-fact classifier before and -after promotion. Legacy event-only claims reject rather than disappear at a -Markdown boundary; event append locks protect the observation through writeback. +after promotion. Legacy nonempty event sources are refused before any Markdown substitution. Canonical changes reuse durable command receipt recovery. This is an L2/L3 compatibility correction with Python decision deletion, not cohort migration, SQLite D2 completion or a default flip. Remaining work depends on executor/consumer closure, qualification, @@ -1194,7 +1197,7 @@ Qualify an **embedded transactional store, with SQLite as the first candidate**, behind the existing TypeScript `AuthorityStore` owner. A local goal must not require a PostgreSQL service. The file-v0 provider remains a conformance/import baseline; no general-purpose ten-day promotion may rely on its full-history -rewrite. [PR #4121](https://github.com/huangruiteng/loopx/pull/4121) supplies an +rewrite. [PR #4121](https://github.com/loopx-project/loopx/pull/4121) supplies an opt-in SQLite conformance candidate behind that owner; it does not by itself qualify long-goal durability or change the default. Dependency/package, Windows/macOS/Linux and supported Node-profile evidence remain explicit gates. diff --git a/docs/architecture/rfcs/shared-goal-authority-state-provider-v0.zh-CN.md b/docs/architecture/rfcs/shared-goal-authority-state-provider-v0.zh-CN.md index a51de6138d..949b52e5ca 100644 --- a/docs/architecture/rfcs/shared-goal-authority-state-provider-v0.zh-CN.md +++ b/docs/architecture/rfcs/shared-goal-authority-state-provider-v0.zh-CN.md @@ -20,13 +20,18 @@ - 语言说明：[英文版](./shared-goal-authority-state-provider-v0.md)与本中文版互为 语义镜像；两者不一致属于缺陷 -## 当前交付边界（2026-09-25） +## Todo 事件路径退役（2026-09-25） -按 `37bbaec79` 与当前 PR 状态核对：完整来源传输、事务捕获、来源组装及此前五个 -在途 caller/event 修复都已合入，不再计入待开发。当前晋升准入修复之后，规划三个 -明确代码边界：外部动作执行区间保护、事件 writer 绑定与整 Goal 迁移/回退闭环、 -默认启用与最后一批有界 Python 退役。#4931 与 D2 的剩余资格证据单列；三个是 -可命名的开发批次，不是保证总 PR 数。[唯一当前清单与退出条件](ledger/shared-goal-authority-state-provider-v0/2026-09-24-default-cutover-reconciliation.zh-CN.md)。 +PR #5054 将原先的事件 writer 捕获方案改为删除这条实验性 Todo 来源。 +`events.jsonl` 不再参与投影、叠加、回填或 completion；默认路径及显式别名指向的 +非空文件会被明确拒绝使用，原字节保留。空文件或缺失文件允许 Markdown 路径继续。 +已晋升 Goal 仍以选定 provider 为准，不受遗留文件影响。Supervisor 改用自己拥有的 +本地私有实验日志。 + +这是通过删除关闭旧来源分支，不是宣称事件 writer 已通过捕获资格验证。 +整 Goal 迁移／回退、默认启用仍须满足现有验收；不能据此宣称所有 Python writer +已经退役或 PostgreSQL 已可部署。剩余工作不再新增“补事件捕获”PR。 +[决策与退出条件](ledger/shared-goal-authority-state-provider-v0/2026-09-25-todo-event-retirement.zh-CN.md)。 ## 旧观测退役检查点（2026-09-24） @@ -933,7 +938,7 @@ projection 后的累计重写。这是 payload 解析估算，不是 SSD 物理 在既有 TypeScript `AuthorityStore` owner 后资格化**嵌入式事务存储，首选候选为 SQLite**。本地 goal 不应依赖 PostgreSQL 服务。file-v0 保留作 conformance/import -基线，通用十天 goal 晋升不能依靠其全历史重写。[PR #4121](https://github.com/huangruiteng/loopx/pull/4121) +基线，通用十天 goal 晋升不能依靠其全历史重写。[PR #4121](https://github.com/loopx-project/loopx/pull/4121) 在该 owner 后提供显式 opt-in 的 SQLite conformance 候选；它本身不证明长程耐久性， 也不切换默认值。依赖／打包、Windows/macOS/Linux 与受支持 Node profile 的证据仍是 显式门禁。分段文件日志作为比较候选；PostgreSQL 继续走独立的共享服务路线。 diff --git a/docs/architecture/rfcs/typescript-control-plane-migration-v0.md b/docs/architecture/rfcs/typescript-control-plane-migration-v0.md index e0d1d9fecf..d60886ddc5 100644 --- a/docs/architecture/rfcs/typescript-control-plane-migration-v0.md +++ b/docs/architecture/rfcs/typescript-control-plane-migration-v0.md @@ -7,23 +7,27 @@ - Scope: an incremental, replacement-first migration of the LoopX control-plane core from Python to TypeScript without maintaining two semantic implementations -- Tracking issue: [#3225](https://github.com/huangruiteng/loopx/issues/3225) +- Tracking issue: [#3225](https://github.com/loopx-project/loopx/issues/3225) - Language note: the [Chinese version](./typescript-control-plane-migration-v0.zh-CN.md) and this English version are semantic mirrors. A difference between them is a defect. --- -## Current delivery frontier (2026-09-25) +## Todo event retirement (2026-09-25) -Audit `37bbaec79` and current PR states: complete-source transport, transaction -capture, source assembly and the five previously open caller/event fixes are -merged, not future implementation. After the current promotion-admission repair, -three named code boundaries remain planned: external-effect execution fencing; -event-writer binding plus whole-Goal migration/rollback; default onboarding plus -bounded Python retirement. #4931 and outstanding D2 evidence are tracked -separately. Three is a delivery plan, not a guaranteed total PR count. -[Current inventory and exits](ledger/shared-goal-authority-state-provider-v0/2026-09-24-default-cutover-reconciliation.md). +PR #5054 replaces its original event-writer capture proposal with removal of +that experimental Todo source. `events.jsonl` is no longer projected, overlaid, +backfilled or used for completion. Nonempty default/aliased sources are refused +without changing their bytes; empty/absent sources permit the Markdown path. +Promoted Goals continue reading their selected provider even if stale legacy +files remain. The supervisor uses its own local-private experimental log. + +This closes the *retired source* branch by deletion, not by claiming event-writer +capture was qualified. Whole-Goal migration/rollback and default onboarding +still require their existing acceptance; this change does not make all Python +writers or PostgreSQL deployment ready. Do not add a replacement event capture +PR to the remaining work. [Decision and exits](ledger/shared-goal-authority-state-provider-v0/2026-09-25-todo-event-retirement.md). ## Observation writer retirement (2026-09-24) @@ -83,7 +87,7 @@ Handoff-mode transition now shares typed ownership facts and an explicit valid/invalid previous-mode state across legacy and canonical paths. Python's blocker classification, artificial previous mode and whole-text rewrite are removed; its retained boundary is source projection/locking and capture IO. -The legacy scan includes event-only claims, and canonical mode receipts reuse +The legacy scan refuses retired nonempty event sources, and canonical mode receipts reuse command recovery with strict historical decisions. Full-source snapshot and real-provider validation guard this T1/T2 replacement. This closes a rule and caller discrepancy, not a whole default-cutover package; use the current @@ -1407,15 +1411,15 @@ choice is now implemented rather than hypothetical. | Slice | Canonical TypeScript ownership now shipped | Remaining migration debt | | --- | --- | --- | -| Effect runtime and Turn journal ([#3416](https://github.com/huangruiteng/loopx/pull/3416)) | Effect algebra, settlement rules, runtime lifecycle, typed Turn-journal interpretation, and durable checkpoint effects | Python settlement facades still expose fine-grained calls and duplicate DTO/enum shapes | -| Todo, quota, and scheduler proof slices ([#3431](https://github.com/huangruiteng/loopx/pull/3431)–[#3434](https://github.com/huangruiteng/loopx/pull/3434)) | Completion fence/state, workspace causality, and scheduler transitions each have one TS rule owner | The cuts are mostly leaf-shaped; Python still composes several product transactions | -| Scheduler durable state ([#3440](https://github.com/huangruiteng/loopx/pull/3440)) | State normalization, persistence, replay, and one coarse transition are TS-owned | The Python compatibility path still pays a cross-runtime transport tax | +| Effect runtime and Turn journal ([#3416](https://github.com/loopx-project/loopx/pull/3416)) | Effect algebra, settlement rules, runtime lifecycle, typed Turn-journal interpretation, and durable checkpoint effects | Python settlement facades still expose fine-grained calls and duplicate DTO/enum shapes | +| Todo, quota, and scheduler proof slices ([#3431](https://github.com/loopx-project/loopx/pull/3431)–[#3434](https://github.com/loopx-project/loopx/pull/3434)) | Completion fence/state, workspace causality, and scheduler transitions each have one TS rule owner | The cuts are mostly leaf-shaped; Python still composes several product transactions | +| Scheduler durable state ([#3440](https://github.com/loopx-project/loopx/pull/3440)) | State normalization, persistence, replay, and one coarse transition are TS-owned | The Python compatibility path still pays a cross-runtime transport tax | | Scheduler heartbeat/state transaction | TypeScript owns receipt freshness, ACK and host-failure validation, state construction, failure-cache transitions, replay/CAS fencing, atomic writes, and the public JSON/Markdown projection | Generated, receipt-bound host follow-up runs through the native TS CLI; Python remains only for unbound/manual compatibility calls and external host mutation | | Quota spend commit transaction | TypeScript owns final spend-transition validation, typed event construction, effect replay/CAS fencing, crash repair, and the JSON/Markdown/index write set | Python still projects `should-run` and settlement readback facts, and holds the legacy cross-writer index lock until the CLI/index writers move in-process | | Quota void commit transaction | TypeScript owns spend-target resolution, before/after reduction, canonical correction construction, effect replay/index CAS, prepared-receipt repair, and the JSON/Markdown/index write set | Python retains `should-run` facts, clock/effect identity, the legacy cross-writer index lock, one transport call, and compatibility entry points | | Quota monitor-poll commit transaction | TypeScript owns monitor admission revalidation, target/event/result construction, effect replay/index CAS, provider intent, and repairable JSON/Markdown/index persistence | Python projects compact `should-run` facts, invokes the real Todo provider between at most two reductions, reloads legacy status, and holds the cross-writer index lock | -| Runtime decoders ([#3443](https://github.com/huangruiteng/loopx/pull/3443)) | Stable primitive decoding has one small shared module; domain decoders remain local | No larger schema framework is justified | -| Transaction payoff ([#3464](https://github.com/huangruiteng/loopx/pull/3464), [#3481](https://github.com/huangruiteng/loopx/pull/3481), and Todo completion) | Turn settlement, quota delivery routing, and Todo completion each cross one coarse TS boundary; the Todo transaction owns identity, replay fencing, validation planning/result reduction, continuation/recovery, and completion metadata | Python still executes explicitly external providers and materializes legacy Markdown/event results; other domains still need their own bounded cutovers | +| Runtime decoders ([#3443](https://github.com/loopx-project/loopx/pull/3443)) | Stable primitive decoding has one small shared module; domain decoders remain local | No larger schema framework is justified | +| Transaction payoff ([#3464](https://github.com/loopx-project/loopx/pull/3464), [#3481](https://github.com/loopx-project/loopx/pull/3481), and Todo completion) | Turn settlement, quota delivery routing, and Todo completion each cross one coarse TS boundary; the Todo transaction owns identity, replay fencing, validation planning/result reduction, continuation/recovery, and completion metadata | Python still executes explicitly external providers and materializes legacy Markdown/event results; other domains still need their own bounded cutovers | | Promoted-authority Todo claim | TypeScript owns the provider-head read, lifecycle validation, complete-record update, hard-lease check, CAS, receipt, and readback-safe result for claims after authority promotion | Default local Markdown mode remains on the legacy writer; other Todo mutations and Markdown regeneration remain bounded follow-ups | The scheduler facade exit now includes its first bounded Stage 3 route. A diff --git a/docs/architecture/rfcs/typescript-control-plane-migration-v0.zh-CN.md b/docs/architecture/rfcs/typescript-control-plane-migration-v0.zh-CN.md index f84b005a69..65a003df5f 100644 --- a/docs/architecture/rfcs/typescript-control-plane-migration-v0.zh-CN.md +++ b/docs/architecture/rfcs/typescript-control-plane-migration-v0.zh-CN.md @@ -6,7 +6,7 @@ - Last revised：2026-09-13 - Scope：LoopX 控制面核心从 Python 到 TypeScript 的增量、replacement-first 迁移；不长期维护两份语义实现 -- Tracking issue：[#3225](https://github.com/huangruiteng/loopx/issues/3225) +- Tracking issue：[#3225](https://github.com/loopx-project/loopx/issues/3225) - Language note：本中文版与 [英文版](./typescript-control-plane-migration-v0.md) 为语义镜像； 两者不一致视为缺陷。 @@ -14,13 +14,18 @@ --- -## 当前交付边界（2026-09-25） +## Todo 事件路径退役（2026-09-25） -按 `37bbaec79` 与当前 PR 状态核对：完整来源传输、事务捕获、来源组装及此前五个 -在途 caller/event 修复都已合入，不再计入待开发。当前晋升准入修复之后，规划三个 -明确代码边界：外部动作执行区间保护、事件 writer 绑定与整 Goal 迁移/回退闭环、 -默认启用与最后一批有界 Python 退役。#4931 与 D2 的剩余资格证据单列；三个是 -可命名的开发批次，不是保证总 PR 数。[唯一当前清单与退出条件](ledger/shared-goal-authority-state-provider-v0/2026-09-24-default-cutover-reconciliation.zh-CN.md)。 +PR #5054 将原先的事件 writer 捕获方案改为删除这条实验性 Todo 来源。 +`events.jsonl` 不再参与投影、叠加、回填或 completion；默认路径及显式别名指向的 +非空文件会被明确拒绝使用，原字节保留。空文件或缺失文件允许 Markdown 路径继续。 +已晋升 Goal 仍以选定 provider 为准，不受遗留文件影响。Supervisor 改用自己拥有的 +本地私有实验日志。 + +这是通过删除关闭旧来源分支，不是宣称事件 writer 已通过捕获资格验证。 +整 Goal 迁移／回退、默认启用仍须满足现有验收；不能据此宣称所有 Python writer +已经退役或 PostgreSQL 已可部署。剩余工作不再新增“补事件捕获”PR。 +[决策与退出条件](ledger/shared-goal-authority-state-provider-v0/2026-09-25-todo-event-retirement.zh-CN.md)。 ## 旧观测写入退役（2026-09-24） @@ -1058,15 +1063,15 @@ replay、receipt 与 settlement。这个架构选择已经落地，不再是假 | 切片 | 已交付的 TypeScript 权威能力 | 剩余迁移债务 | | --- | --- | --- | -| Effect runtime 与 Turn journal（[#3416](https://github.com/huangruiteng/loopx/pull/3416)） | Effect algebra、settlement rule、runtime lifecycle、typed Turn-journal interpretation 与 durable checkpoint effect | Python settlement facade 仍暴露细粒度调用，并重复 DTO/enum shape | -| Todo、quota 与 scheduler 证明切片（[#3431](https://github.com/huangruiteng/loopx/pull/3431)–[#3434](https://github.com/huangruiteng/loopx/pull/3434)） | Completion fence/state、workspace causality 与 scheduler transition 各有一个 TS rule owner | 切口大多仍是 leaf-shaped；Python 继续组合多个产品 transaction | -| Scheduler durable state（[#3440](https://github.com/huangruiteng/loopx/pull/3440)） | State normalization、persistence、replay 与一笔粗粒度 transition 由 TS 拥有 | Python compatibility path 仍承担跨 runtime transport 税 | +| Effect runtime 与 Turn journal（[#3416](https://github.com/loopx-project/loopx/pull/3416)） | Effect algebra、settlement rule、runtime lifecycle、typed Turn-journal interpretation 与 durable checkpoint effect | Python settlement facade 仍暴露细粒度调用，并重复 DTO/enum shape | +| Todo、quota 与 scheduler 证明切片（[#3431](https://github.com/loopx-project/loopx/pull/3431)–[#3434](https://github.com/loopx-project/loopx/pull/3434)） | Completion fence/state、workspace causality 与 scheduler transition 各有一个 TS rule owner | 切口大多仍是 leaf-shaped；Python 继续组合多个产品 transaction | +| Scheduler durable state（[#3440](https://github.com/loopx-project/loopx/pull/3440)） | State normalization、persistence、replay 与一笔粗粒度 transition 由 TS 拥有 | Python compatibility path 仍承担跨 runtime transport 税 | | Scheduler heartbeat/state transaction | TypeScript 拥有 receipt freshness、ACK 与 host-failure validation、state construction、failure-cache transition、replay/CAS fencing、atomic write，以及 public JSON/Markdown projection | 生成的 receipt-bound host follow-up 直接进入 native TS CLI；Python 只处理 unbound/manual compatibility call 与 external host mutation | | Quota spend commit transaction | TypeScript 拥有最终 spend transition 校验、typed event 构造、effect replay/CAS fencing、crash repair，以及 JSON/Markdown/index write set | Python 仍投影 `should-run` 与 settlement readback facts，并在 CLI/index writer 进程内迁移前持有 legacy cross-writer index lock | | Quota void commit transaction | TypeScript 拥有 spend-target resolution、before/after reduction、canonical correction 构造、effect replay/index CAS、prepared-receipt repair，以及 JSON/Markdown/index write set | Python 保留 `should-run` facts、clock/effect identity、legacy cross-writer index lock、一次 transport 与 compatibility entrypoint | | Quota monitor-poll commit transaction | TypeScript 拥有 monitor admission 复核、target/event/result 构造、effect replay/index CAS、provider intent，以及可修复的 JSON/Markdown/index persistence | Python 投影 compact `should-run` facts，在最多两次 reduction 之间调用真实 Todo provider，刷新 legacy status，并持有 cross-writer index lock | -| Runtime decoder（[#3443](https://github.com/huangruiteng/loopx/pull/3443)） | 稳定 primitive decoding 进入一个很小的共享模块；domain decoder 仍留在本地 | 没有理由建设更大的 schema framework | -| Transaction 兑现（[#3464](https://github.com/huangruiteng/loopx/pull/3464)、[#3481](https://github.com/huangruiteng/loopx/pull/3481) 与 Todo completion） | Turn settlement、quota delivery routing 与 Todo completion 均只跨一个粗粒度 TS boundary；Todo transaction 拥有 identity、replay fence、validation planning/result reduction、continuation/recovery 与 completion metadata | Python 仍执行显式 external provider，并物化 legacy Markdown/event result；其他 domain 仍需各自的 bounded cutover | +| Runtime decoder（[#3443](https://github.com/loopx-project/loopx/pull/3443)） | 稳定 primitive decoding 进入一个很小的共享模块；domain decoder 仍留在本地 | 没有理由建设更大的 schema framework | +| Transaction 兑现（[#3464](https://github.com/loopx-project/loopx/pull/3464)、[#3481](https://github.com/loopx-project/loopx/pull/3481) 与 Todo completion） | Turn settlement、quota delivery routing 与 Todo completion 均只跨一个粗粒度 TS boundary；Todo transaction 拥有 identity、replay fence、validation planning/result reduction、continuation/recovery 与 completion metadata | Python 仍执行显式 external provider，并物化 legacy Markdown/event result；其他 domain 仍需各自的 bounded cutover | Scheduler facade exit 已交付第一段有边界的 Stage 3 路径。带版本的 `heartbeat_followup_cli.ts` 从生成的 ACK/failure hint 接收有大小上限的 compact host diff --git a/docs/book/chapters/state-substrate.md b/docs/book/chapters/state-substrate.md index 7f6d88bc25..b47f6f11ff 100644 --- a/docs/book/chapters/state-substrate.md +++ b/docs/book/chapters/state-substrate.md @@ -1,5 +1,11 @@ # 持久状态与只读投影 +> Update (2026-09-25): the Todo `events.jsonl` API, replay, backfill and completion +> examples below describe a retired experiment. Current Todos use legacy Markdown +> or the selected File/SQLite authority. See the +> [retirement contract](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/event-sourced-state-contract-v0.md). + + 长程任务能跨 session 恢复，不是因为系统保存了更多聊天记录，而是因为关键事实有稳定归属， 并且能够被重新投影成当前决策。本章建立 LoopX 的状态底座：哪些表面保存事实，哪些表面只负责 阅读，以及为什么“看起来像当前状态”的页面或 Markdown 不能自动成为写入入口。 @@ -87,7 +93,7 @@ Registry 不证明某个 Host 已经成功启动，也不保存每一轮 Agent ### 2. Event ledger：发生过什么 -[`event_sourced_state_contract_v0`](https://github.com/huangruiteng/loopx/blob/main/docs/reference/protocols/event-sourced-state-contract-v0.md) +[`event_sourced_state_contract_v0`](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/event-sourced-state-contract-v0.md) 把 Todo、Gate、run、evidence、projection 和 quota 变化表达为 append-only events。 事件至少需要满足四个不变量： @@ -111,7 +117,7 @@ Progress。它是重要的工作台，但不能笼统地理解为“所有真相 在迁移或兼容阶段，Markdown 可能仍参与 Todo 读取；规范写入仍应通过 LoopX lifecycle commands 形成事件或受控 writeback。直接编辑一个被投影出来的段落，不等于完成状态转换。 -[`active_state_structured_projection_v0`](https://github.com/huangruiteng/loopx/blob/main/docs/reference/protocols/active-state-structured-projection-v0.md) +[`active_state_structured_projection_v0`](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/active-state-structured-projection-v0.md) 定义了如何从这个工作台生成 typed、read-only 的 Todo、Gate 与 Next Action 视图。协议明确： - projection 可以重算； @@ -155,7 +161,7 @@ Goal lifecycle 仍由 Todo、Gate、events 与 acceptance 组合决定。 - 通过修改卡片或图节点绕过 write API； - 把 stale external observation 当成当前事实。 -[`task_graph_projection_v0`](https://github.com/huangruiteng/loopx/blob/main/docs/reference/protocols/task-graph-projection-v0.md) +[`task_graph_projection_v0`](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/task-graph-projection-v0.md) 尤其强调：图中的 `blocks`、`validates`、`continues` 和 `hands_off_to` 是派生关系，不是新的调度 命令。 @@ -208,7 +214,7 @@ LoopX 当前是 **本地优先** 的控制面：项目 registry、active-state w runtime state 位于项目或用户本地。这个事实不意味着“Markdown 文件本身就是 authority”，也不 意味着把目录换成数据库就自动获得正确的并发与恢复语义。 -[`event_sourced_state_contract_v0`](https://github.com/huangruiteng/loopx/blob/main/docs/reference/protocols/event-sourced-state-contract-v0.md) +[`event_sourced_state_contract_v0`](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/event-sourced-state-contract-v0.md) 明确允许 JSONL、SQLite 或其他 local-first append-only 实现，只要它们保持： - stable event id 与 ordered replay； @@ -217,7 +223,7 @@ runtime state 位于项目或用户本地。这个事实不意味着“Markdown - public-safe、local-private 与 private-pointer 分区； - Markdown 继续作为 workbench/projection，而不是任意写入口。 -[`local_state_write_correctness_v0`](https://github.com/huangruiteng/loopx/blob/main/docs/reference/protocols/local-state-write-correctness-v0.md) +[`local_state_write_correctness_v0`](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/local-state-write-correctness-v0.md) 当前标记为 public-safe protocol draft。它把更强的写入正确性目标分成 `prepare -> preview -> apply -> record -> project`： @@ -284,7 +290,7 @@ LoopX 可以让研究、验证和决策产物不被静默改写，但这不等 变化后重新验证 applicability。无法确认时，把它标为 historical observation 或 stale evidence， 不要删除历史，也不要继续把它当作 current authority。 -[`agent_scoped_evidence_ledger_v0`](https://github.com/huangruiteng/loopx/blob/main/docs/reference/protocols/agent-scoped-evidence-ledger-v0.md) +[`agent_scoped_evidence_ledger_v0`](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/agent-scoped-evidence-ledger-v0.md) 提供 bounded、read-only 的 Agent chronology，适合 replan 和 handoff；它不替代 current status、 quota decision 或外部系统 readback。 @@ -347,17 +353,17 @@ run_recorded(R1, tests_passed_at=commit-a) 本章拥有概念顺序，不复制完整 schema。需要修改 LoopX 状态行为时，优先阅读： -- [`event_sourced_state_contract_v0`](https://github.com/huangruiteng/loopx/blob/main/docs/reference/protocols/event-sourced-state-contract-v0.md)： +- [`event_sourced_state_contract_v0`](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/event-sourced-state-contract-v0.md)： event、replay、ordering、privacy； -- [`active_state_structured_projection_v0`](https://github.com/huangruiteng/loopx/blob/main/docs/reference/protocols/active-state-structured-projection-v0.md)： +- [`active_state_structured_projection_v0`](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/active-state-structured-projection-v0.md)： Markdown workbench 的 typed read model； -- [`task_graph_projection_v0`](https://github.com/huangruiteng/loopx/blob/main/docs/reference/protocols/task-graph-projection-v0.md)： +- [`task_graph_projection_v0`](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/task-graph-projection-v0.md)： Todo、Gate、evidence 与 handoff 的只读关系图； -- [`long_horizon_agent_state_protocol_v0`](https://github.com/huangruiteng/loopx/blob/main/docs/reference/protocols/long-horizon-agent-state-protocol-v0.md)： +- [`long_horizon_agent_state_protocol_v0`](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/long-horizon-agent-state-protocol-v0.md)： 长程工作中的 source/projection、并发 Agent 与 lifecycle； -- [`agent_scoped_evidence_ledger_v0`](https://github.com/huangruiteng/loopx/blob/main/docs/reference/protocols/agent-scoped-evidence-ledger-v0.md)： +- [`agent_scoped_evidence_ledger_v0`](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/agent-scoped-evidence-ledger-v0.md)： replan/handoff 前的 Agent-scoped chronological read model； -- [Status Data Contract](https://github.com/huangruiteng/loopx/blob/main/docs/status-data-contract.md)： +- [Status Data Contract](https://github.com/loopx-project/loopx/blob/main/docs/status-data-contract.md)： operator 与 Agent 读取的聚合表面。 如果你准备修改 registry、event、Domain State、replay 或 projection builder，继续阅读 diff --git a/docs/book/en/chapters/state-substrate.md b/docs/book/en/chapters/state-substrate.md index f96f78e584..ffd3c63fc9 100644 --- a/docs/book/en/chapters/state-substrate.md +++ b/docs/book/en/chapters/state-substrate.md @@ -1,5 +1,11 @@ # Durable State and Read-Only Projections +> Update (2026-09-25): the Todo `events.jsonl` API, replay, backfill and completion +> examples below describe a retired experiment. Current Todos use legacy Markdown +> or the selected File/SQLite authority. See the +> [retirement contract](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/event-sourced-state-contract-v0.md). + + Long-running work does not recover because a system stores more conversation. It recovers because each important fact has a stable owner and can be projected into a fresh decision. This chapter establishes the LoopX state substrate: which surfaces own facts, which surfaces only help readers, and why a page or @@ -93,7 +99,7 @@ It owns connection and policy facts, not execution receipts. ### 2. Event ledger: what happened -[`event_sourced_state_contract_v0`](https://github.com/huangruiteng/loopx/blob/main/docs/reference/protocols/event-sourced-state-contract-v0.md) +[`event_sourced_state_contract_v0`](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/event-sourced-state-contract-v0.md) represents Todo, Gate, run, evidence, projection, and quota changes as append-only events. At least four invariants matter: @@ -118,7 +124,7 @@ During migration or compatibility windows, Markdown may still participate in Tod should still pass through LoopX lifecycle commands and controlled writeback so they form governed events. Editing a projected paragraph does not automatically perform a lifecycle transition. -[`active_state_structured_projection_v0`](https://github.com/huangruiteng/loopx/blob/main/docs/reference/protocols/active-state-structured-projection-v0.md) +[`active_state_structured_projection_v0`](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/active-state-structured-projection-v0.md) defines a typed, read-only view of Todos, Gates, and Next Action from that workbench. The contract preserves several boundaries: @@ -163,7 +169,7 @@ They must not: - let card or graph edits bypass the write API; - treat a stale external observation as a current fact. -[`task_graph_projection_v0`](https://github.com/huangruiteng/loopx/blob/main/docs/reference/protocols/task-graph-projection-v0.md) +[`task_graph_projection_v0`](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/task-graph-projection-v0.md) is explicit about this boundary. Relationships such as `blocks`, `validates`, `continues`, and `hands_off_to` are derived graph edges, not new scheduling commands. @@ -223,7 +229,7 @@ run history, and runtime state live in project-local or user-local storage. This file the authority by itself, and replacing files with a database does not automatically create correct concurrency or recovery semantics. -[`event_sourced_state_contract_v0`](https://github.com/huangruiteng/loopx/blob/main/docs/reference/protocols/event-sourced-state-contract-v0.md) +[`event_sourced_state_contract_v0`](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/event-sourced-state-contract-v0.md) allows JSONL, SQLite, or another local-first append-only implementation when it preserves: - stable event ids and ordered replay; @@ -232,7 +238,7 @@ allows JSONL, SQLite, or another local-first append-only implementation when it - public-safe, local-private, and private-pointer partitions; - Markdown as a workbench or projection rather than an arbitrary write API. -[`local_state_write_correctness_v0`](https://github.com/huangruiteng/loopx/blob/main/docs/reference/protocols/local-state-write-correctness-v0.md) +[`local_state_write_correctness_v0`](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/local-state-write-correctness-v0.md) is currently marked as a public-safe protocol draft. Its stronger write-correctness target separates `prepare -> preview -> apply -> record -> project`: @@ -302,7 +308,7 @@ fresh. Research notes, test results, and PR readbacks need stable join keys and after material inputs change. When applicability is unknown, retain the artifact as a historical observation or stale evidence instead of deleting it or treating it as current authority. -[`agent_scoped_evidence_ledger_v0`](https://github.com/huangruiteng/loopx/blob/main/docs/reference/protocols/agent-scoped-evidence-ledger-v0.md) +[`agent_scoped_evidence_ledger_v0`](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/agent-scoped-evidence-ledger-v0.md) provides a bounded, read-only Agent chronology for replan and handoff. It does not replace current status, a quota decision, or external-system readback. @@ -367,17 +373,17 @@ boundary probably needs to be split first. This chapter owns the learning sequence, not the complete schemas. For state changes, start with: -- [`event_sourced_state_contract_v0`](https://github.com/huangruiteng/loopx/blob/main/docs/reference/protocols/event-sourced-state-contract-v0.md) +- [`event_sourced_state_contract_v0`](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/event-sourced-state-contract-v0.md) for events, replay, ordering, and privacy; -- [`active_state_structured_projection_v0`](https://github.com/huangruiteng/loopx/blob/main/docs/reference/protocols/active-state-structured-projection-v0.md) +- [`active_state_structured_projection_v0`](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/active-state-structured-projection-v0.md) for the typed read model over the Markdown workbench; -- [`task_graph_projection_v0`](https://github.com/huangruiteng/loopx/blob/main/docs/reference/protocols/task-graph-projection-v0.md) +- [`task_graph_projection_v0`](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/task-graph-projection-v0.md) for the read-only relation graph; -- [`long_horizon_agent_state_protocol_v0`](https://github.com/huangruiteng/loopx/blob/main/docs/reference/protocols/long-horizon-agent-state-protocol-v0.md) +- [`long_horizon_agent_state_protocol_v0`](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/long-horizon-agent-state-protocol-v0.md) for source and projection ownership, concurrent Agents, and lifecycle in long-running work; -- [`agent_scoped_evidence_ledger_v0`](https://github.com/huangruiteng/loopx/blob/main/docs/reference/protocols/agent-scoped-evidence-ledger-v0.md) +- [`agent_scoped_evidence_ledger_v0`](https://github.com/loopx-project/loopx/blob/main/docs/reference/protocols/agent-scoped-evidence-ledger-v0.md) for the Agent-scoped chronological read model used before replan and handoff; -- the [Status Data Contract](https://github.com/huangruiteng/loopx/blob/main/docs/status-data-contract.md) +- the [Status Data Contract](https://github.com/loopx-project/loopx/blob/main/docs/status-data-contract.md) for Agent and operator-facing aggregation. If you plan to change registry, event, Domain State, replay, or projection builders, continue to diff --git a/docs/development/control-plane-course/04-state-substrate.md b/docs/development/control-plane-course/04-state-substrate.md index 52e22019a5..4ac438cd7c 100644 --- a/docs/development/control-plane-course/04-state-substrate.md +++ b/docs/development/control-plane-course/04-state-substrate.md @@ -1,5 +1,11 @@ # 第 4 讲：状态底座与可重放事实 +> Update (2026-09-25): the Todo `events.jsonl` API, replay, backfill and completion +> examples below describe a retired experiment. Current Todos use legacy Markdown +> or the selected File/SQLite authority. See the +> [retirement contract](../../reference/protocols/event-sourced-state-contract-v0.md). + + > **本讲结论：** Canonical event/state contract 拥有长期事实；active state、status 和 > dashboard 是可重建 read model；session context 不能替代 project memory。 @@ -193,7 +199,7 @@ Registry 适合保存： ### 2. Event ledger：追加式事实 -事件流保存“发生过什么”。协议见 `docs/reference/protocols/event-sourced-state-contract-v0.md`。 +事件流保存“发生过什么”。协议见 `docs../../reference/protocols/event-sourced-state-contract-v0.md`。 代表性事件包括： @@ -686,7 +692,7 @@ loopx --format json quota should-run --goal-id --agent-id | --- | --- | --- | | 1 | `examples/registry.example.json` | 注册配置的边界 | | 2 | `examples/active-goal-state.example.md` | 人可读工作台 | -| 3 | `docs/reference/protocols/event-sourced-state-contract-v0.md` | 事件与 replay | +| 3 | `docs../../reference/protocols/event-sourced-state-contract-v0.md` | 事件与 replay | | 4 | `docs/reference/protocols/active-state-structured-projection-v0.md` | projection 非写 API | | 5 | `docs/status-data-contract.md` | status read model | | 6 | `loopx/control_plane/todos/`、`loopx/control_plane/work_items/lifecycle.py` | todo contract 与 transition lifecycle | @@ -694,7 +700,7 @@ loopx --format json quota should-run --goal-id --agent-id ## 代表性 Smoke -- `examples/control_plane/event-sourced-state-api-smoke.py` +- `tests/control_plane/test_retired_todo_event_source.py` (retirement regression) - `examples/control_plane/todo-lifecycle-cli-smoke.py` - `examples/state-projection-gap-smoke.py` - `examples/control_plane/todo-projection-shared-helper-smoke.py` diff --git a/docs/reference/protocols/event-sourced-state-contract-v0.md b/docs/reference/protocols/event-sourced-state-contract-v0.md index c9340779af..1f018c3400 100644 --- a/docs/reference/protocols/event-sourced-state-contract-v0.md +++ b/docs/reference/protocols/event-sourced-state-contract-v0.md @@ -1,192 +1,19 @@ -# event_sourced_state_contract_v0 +# Retired: experimental Todo event source -`event_sourced_state_contract_v0` defines how LoopX can keep -`ACTIVE_GOAL_STATE.md` as the human/agent workbench while moving canonical -todo and history truth to an append-only event stream. +This experimental protocol is retired by PR #5054. There is no current +`loopx.event_sourced_state` API, Todo event replay, Markdown backfill or event +completion route. Historical references to this page describe the former design. -This is a product/control-plane contract, not an implementation mandate for a -specific database. Implementations may store the stream as JSONL, SQLite rows, -or another local-first append-only format as long as replay, ordering, -privacy, and idempotency behave the same way. +Use [shared Goal authority and providers](../../architecture/rfcs/shared-goal-authority-state-provider-v0.md) +for canonical Todo storage and migration. Legacy Markdown remains supported; +nonempty old event sources fail explicitly instead of disappearing into fallback. +Preserve their bytes and inspect/export with a compatible older release before +removing them from the active source location. This release never deletes them. -## Role Split +The default filename is `events.jsonl`; old `state_event_log`, `state_events_file` +and `event_log` aliases are checked too. Empty files are allowed. Promoted Goals +use their provider even when unrelated old files remain. Source qualification +continues checking captured source bytes; no event writer or event outbox was added. -`ACTIVE_GOAL_STATE.md` remains the human/agent workbench. It is the readable -surface where agents and users can inspect current goals, progress, todos, -gates, validation surfaces, and next action. It may keep private or -project-local context when the project has explicitly ignored local state. - -Canonical todo/history truth belongs to the event stream: - -- Markdown edits are not canonical state changes unless they are converted into - events by a LoopX command or a migration/backfill tool. -- Markdown renderers are projections. They may be regenerated from events and - may compact old detail for prompt and review budgets. -- During migration, the Markdown parser may remain a compatibility fallback, - but the event projection should become the preferred source for status, - quota, review packets, todo CLI reads, and dashboard exports. - -## Canonical Event Stream - -Each goal has an ordered event stream. Every event must include: - -- `schema_version`: event schema version, starting with - `loopx_state_event_v0`; -- `event_id`: stable unique id for idempotent append and audit; -- `goal_id`: owning goal id; -- `event_type`: one of the allowed lifecycle event types; -- `recorded_at`: producer timestamp in UTC or offset-aware ISO-8601; -- `append_sequence`: monotonic sequence assigned by the local event store; -- `producer`: compact source such as `loopx.todo`, `loopx.refresh_state`, or - `agent.codex-product-capability`; -- `privacy`: `public_safe`, `local_private`, or `private_pointer`; -- `projection_version`: projection contract version expected by the writer; -- `refs`: compact ids for todo, gate, run, quota, PR, evidence, or parent - events; -- `payload`: event-specific compact data. - -`append_sequence` is the final same-priority tie-breaker. If a planner creates -multiple P0 todos, the planner emits `planner_order`, and the store preserves -that order through append sequence. UI and prompt projections sort by priority -first, then `planner_order` when present, then `append_sequence`. - -## Event Types - -First supported todo/history event types: - -| Event type | Purpose | -| --- | --- | -| `todo_added` | Add a new todo with role, priority, title, metadata, and planner order. | -| `todo_claimed` | Record or renew ownership, lease, or `claimed_by`. | -| `todo_updated` | Update compact metadata that does not rewrite event history. | -| `todo_blocked` | Mark a todo blocked with a public-safe blocker reason and optional gate refs. | -| `todo_deferred` | Mark a todo deferred with resume conditions. | -| `todo_completed` | Close a todo with validation/evidence refs and completion rationale. | -| `gate_added` | Add a user, owner, operator, or controller gate. | -| `gate_resolved` | Record approve, reject, or defer for a specific gate. | -| `run_recorded` | Attach compact run-history status, classification, and delivery outcome. | -| `refresh_recorded` | Record a state-only or progress refresh summary. | -| `quota_spent` | Record accounting for automatic compute spend. | -| `evidence_attached` | Attach compact public-safe evidence refs to a todo, gate, or run. | -| `projection_rendered` | Record a generated Markdown/status/dashboard projection checksum. | -| `snapshot_compacted` | Declare a derived snapshot checkpoint without replacing the underlying event lineage. | - -Forbidden event styles: - -- no event may mutate or delete a prior event; -- no event may embed raw chat transcripts, raw logs, credentials, or private - source bodies in a public-safe stream; -- no projection may become a write API by accepting state that lacks a matching - canonical event. - -## Ordering And Idempotency - -Replay order is: - -1. `append_sequence`; -2. `recorded_at`; -3. `event_id` as a deterministic final tie-breaker. - -Append is idempotent on `event_id`: re-appending the same event id with the -same normalized body is a no-op; re-appending it with a different body is a -conflict. Consumers should ignore duplicate identical events and fail closed on -conflicting duplicates. - -Todo ids, gate ids, and evidence ids are stable references. Events may point to -parent events through `refs.parent_event_id`, but a child event must not rewrite -parent payload. - -## Projection Rules - -The event projection renders: - -- current active todos grouped by role and priority; -- completed todo summaries and archive candidates; -- user and controller gate inboxes; -- run-history and refresh timeline summaries; -- quota spend summaries; -- review-packet evidence refs; -- Markdown-compatible `ACTIVE_GOAL_STATE.md` sections. - -Projection outputs must carry: - -- `schema_version`; -- `goal_id`; -- `generated_at`; -- `source_event_count`; -- `last_event_id`; -- `last_append_sequence`; -- `projection_version`; -- `source_checksum` or equivalent integrity marker. - -A projection may be stale after any lifecycle event. Writers should append the -event first, then render projection output. Readers should prefer the latest -projection only when its `last_append_sequence` matches the event store head. - -## Privacy Boundary - -LoopX should support separate streams or partitioned records: - -- `public_safe`: compact state that can be committed or shown in public docs; -- `local_private`: local state such as project-private active Markdown, - private todo details, or local-only evidence notes; -- `private_pointer`: a compact pointer to private material without copying the - material into public state. - -`ACTIVE_GOAL_STATE.md` can carry private details when the project keeps it out -of git. Public docs, fixtures, dashboards, and PR packets must not copy those -details. Public projections should include only compact labels, ids, redacted -summaries, omission notes, and validation refs. - -Tracked outputs require explicit redaction or compact pointers before -projecting information from private streams. A project-level LoopX config may -set defaults such as: - -```json -{ - "state_privacy": { - "active_state": "local_private", - "public_projection": "public_safe", - "allow_private_links_in_ignored_state": true, - "require_redaction_for_tracked_outputs": true - } -} -``` - -## Migration And Compatibility - -The migration should be staged: - -1. Define this contract and smoke-test replay/privacy invariants. -2. Add a minimal event store and projection API for todo/history events. -3. Dual-write `loopx todo`, `refresh-state`, quota spend, and gate commands. -4. Compare event projection against current Markdown parsing through - `event_store_migration_bridge_v0`. -5. Prefer event projection for status, quota, review packets, dashboard, and - slash-command help. -6. Keep Markdown rendering as the workbench and compatibility export. -7. Retire Markdown-as-canonical only after replay and idempotency checks are - clean on real local goals. - -Migration tools may backfill events from existing Markdown, but each backfilled -event should mark `producer=loopx.backfill` and include enough source refs to -explain provenance without copying private raw material into public streams. - -## Acceptance Checks - -A valid implementation or fixture must prove: - -- Markdown remains a workbench/projection, not canonical todo/history truth; -- `todo_added`, `todo_claimed`, `todo_updated`, `todo_blocked`, - `todo_deferred`, and `todo_completed` replay into a deterministic todo - projection; -- same-priority todos preserve planner order and append order; -- duplicate identical `event_id` append is idempotent; -- duplicate conflicting `event_id` append fails closed; -- prior events are never mutated or deleted; -- projections expose `last_event_id`, `last_append_sequence`, and - `projection_version`; -- public projections do not include local absolute paths, credentials, raw - transcripts, raw logs, or private source bodies; -- ignored/private active state may reference private links, while tracked - outputs require explicit redaction or compact pointers. +[Supervisor](peer-supervisor-v0.md) logs are independent, experimental, local-private +proposal/receipt records. They neither own Todos nor migrate Goal state. diff --git a/docs/reference/protocols/event-store-migration-bridge-v0.md b/docs/reference/protocols/event-store-migration-bridge-v0.md index ca09b38444..1f018c3400 100644 --- a/docs/reference/protocols/event-store-migration-bridge-v0.md +++ b/docs/reference/protocols/event-store-migration-bridge-v0.md @@ -1,87 +1,19 @@ -# event_store_migration_bridge_v0 +# Retired: experimental Todo event source -`event_store_migration_bridge_v0` is the fail-closed bridge between the -Markdown active-state read model and a future event projection read model. +This experimental protocol is retired by PR #5054. There is no current +`loopx.event_sourced_state` API, Todo event replay, Markdown backfill or event +completion route. Historical references to this page describe the former design. -It does not make the event projection canonical. It records the gates that must -be clean before a reviewed runtime change may prefer event projection for -status, quota, review packets, dashboards, or slash-command reads. +Use [shared Goal authority and providers](../../architecture/rfcs/shared-goal-authority-state-provider-v0.md) +for canonical Todo storage and migration. Legacy Markdown remains supported; +nonempty old event sources fail explicitly instead of disappearing into fallback. +Preserve their bytes and inspect/export with a compatible older release before +removing them from the active source location. This release never deletes them. -## Contract +The default filename is `events.jsonl`; old `state_event_log`, `state_events_file` +and `event_log` aliases are checked too. Empty files are allowed. Promoted Goals +use their provider even when unrelated old files remain. Source qualification +continues checking captured source bytes; no event writer or event outbox was added. -The bridge packet is built by -`loopx.control_plane.runtime.event_store_migration_bridge.build_event_store_migration_bridge` -and carries: - -- `source_of_truth`: currently `markdown_active_state`; -- `candidate_source`: currently `event_projection`; -- `stage`: one of `wait_for_event_read_path`, `dual_read_shadow`, - `bounded_canary`, or `promotion_candidate`; -- `promotion_allowed`: always `false` in this bridge contract; -- `promotion_candidate`: true only when all pre-promotion checks are clean; -- `checks`: compact booleans for read-path, parity, rollback, canary, - idempotency, projection-head, and public-boundary readiness; -- `missing_for_shadow`, `missing_for_canary`, and `missing_for_promotion`; -- `dual_read`, `rollback`, and `canary` subcontracts. - -## Stages - -`wait_for_event_read_path` means the event read path or structured -active-state projection is not ready. The only safe action is to finish those -prerequisites. - -`dual_read_shadow` means both read models may be compared, but Markdown remains -the source of truth. Any parity delta must prefer Markdown and record a compact -delta rather than silently promoting event projection. - -`bounded_canary` means parity, rollback, idempotency, projection-head, and -public-boundary checks are clean enough to run a small read-only canary. The -canary uses a limited goal set and duration, with event write preference still -disabled. - -`promotion_candidate` means the bridge has enough evidence to propose a -separate reviewed runtime PR. It is not an automatic promotion state. - -## Required Gates - -Promotion requires all of these to be clean: - -- event read path ready; -- active-state structured projection ready; -- dual-read parity clean for todo ids, status, priority/planner order, - `claimed_by`, gate refs, and projection head sequence; -- event projection head matches the event store head; -- rollback plan recorded; -- bounded canary passed; -- idempotency conflicts clean; -- public boundary clean. - -## Rollback - -Rollback is mandatory. The fallback source is always the Markdown active-state -parser until a later reviewed write-path change changes the source of truth. - -Rollback triggers include: - -- parity delta; -- projection head mismatch; -- event append conflict; -- public boundary warning; -- canary regression. - -The rollback action is to disable event projection preference and keep the -Markdown parser as canonical read fallback. - -## Canary - -The bounded canary is read-only: - -- small goal limit, default 1; -- short duration, default 30 minutes; -- event write path disabled; -- read preference remains Markdown; -- observe status todo summaries, quota selected todo, review packet todo refs, - dashboard/frontstage projection, and event projection head sequence. - -Success requires no parity delta, no idempotency conflict, no private-boundary -warning, and a one-command-safe rollback. +[Supervisor](peer-supervisor-v0.md) logs are independent, experimental, local-private +proposal/receipt records. They neither own Todos nor migrate Goal state. diff --git a/docs/reference/protocols/peer-supervisor-v0.md b/docs/reference/protocols/peer-supervisor-v0.md index 2df84c93b0..074df22860 100644 --- a/docs/reference/protocols/peer-supervisor-v0.md +++ b/docs/reference/protocols/peer-supervisor-v0.md @@ -1,5 +1,19 @@ # Peer Supervisor v0 +## Experimental log boundary + +Supervisor is default-off and may change without historical log compatibility. +The log now uses `supervisor_log_event_v0`, with only local-private proposals and +host receipts. It does not use the retired Todo event store or produce Todo +projections. Archive any older experimental log before starting a fresh one; +an unknown schema is refused without modifying the file. + +Proposal/receipt admission and publication hold one log lock. Preview does not +write or sync the log. Retry of the same semantic identity returns the original +record; conflicting content is rejected. Competing executed receipts for one +decision are serialized. These guarantees cover the log, not exactly-once host +execution across a crash before receipt publication. + ## Status Experimental and default-off. This protocol adds an observation and proposal diff --git a/docs/state-interaction-model.md b/docs/state-interaction-model.md index 24927c33dc..a014c7cde9 100644 --- a/docs/state-interaction-model.md +++ b/docs/state-interaction-model.md @@ -1,5 +1,11 @@ # State Interaction Model +> Update (2026-09-25): the Todo `events.jsonl` API, replay, backfill and completion +> examples below describe a retired experiment. Current Todos use legacy Markdown +> or the selected File/SQLite authority. See the +> [retirement contract](reference/protocols/event-sourced-state-contract-v0.md). + + LoopX should not grow by adding commands one at a time. New capabilities must fit a clear state model between the goal, the Codex App executor, the human operator, and the dashboard. diff --git a/examples/canary/catalog-planner-smoke.py b/examples/canary/catalog-planner-smoke.py index e83aa22c55..4aec465a60 100644 --- a/examples/canary/catalog-planner-smoke.py +++ b/examples/canary/catalog-planner-smoke.py @@ -53,7 +53,6 @@ def assert_profiles_come_from_catalog_matrix() -> None: "status-read-path", "status-projection-cache", "review-packet-read-path", - "event-sourced-read-path", "cli-command-contract", "todo-lifecycle", "monitor-scheduler", @@ -508,23 +507,6 @@ def assert_pr_release_and_refactor_profiles_select() -> None: assert review_packet_profile["deep_checks_available"] is True, review_packet_profile assert review_packet_profile["deep_checks_included"] is False, review_packet_profile - event_read_payload = build_catalog_canary_plan( - changed_files=["loopx/event_sourced_state.py", "loopx/rollout_event_log.py"], - surfaces=["event projection downstream read event-store read-path"], - ) - event_read_profiles = { - profile["id"]: profile for profile in event_read_payload["domain_profiles"] - } - assert "event-sourced-read-path" in event_read_profiles, event_read_payload - event_read_profile = event_read_profiles["event-sourced-read-path"] - event_read_commands = [check["command"] for check in event_read_profile["checks"]] - assert "python3 examples/control_plane/event-sourced-state-api-smoke.py" in event_read_commands, event_read_profile - assert "python3 examples/control_plane/event-sourced-status-read-path-smoke.py" in event_read_commands, event_read_profile - assert "python3 examples/control_plane/event-sourced-downstream-read-path-smoke.py" in event_read_commands, event_read_profile - assert all(check["tier"] == "default" for check in event_read_profile["checks"]), event_read_profile - assert event_read_profile["deep_checks_available"] is True, event_read_profile - assert event_read_profile["deep_checks_included"] is False, event_read_profile - cli_payload = build_catalog_canary_plan( changed_files=["loopx/cli.py", "loopx/cli_commands/version.py"], surfaces=["cli command modularization"], diff --git a/examples/control_plane/control-plane-integrated-canary-smoke.py b/examples/control_plane/control-plane-integrated-canary-smoke.py index 2b002d3270..63049aa45b 100644 --- a/examples/control_plane/control-plane-integrated-canary-smoke.py +++ b/examples/control_plane/control-plane-integrated-canary-smoke.py @@ -1,5 +1,5 @@ #!/usr/bin/env python3 -"""Bounded canary for the status -> quota -> review-packet event read path.""" +"""Bounded canary for the status -> quota -> review-packet Todo read path.""" from __future__ import annotations @@ -16,13 +16,6 @@ if str(REPO_ROOT) not in sys.path: sys.path.insert(0, str(REPO_ROOT)) -from loopx.event_sourced_state import ( # noqa: E402 - AppendOnlyStateEventStore, - TODO_ADDED, - TODO_CLAIMED, - TODO_COMPLETED, - make_state_event, -) from loopx.control_plane.testing.canary_harness import ( # noqa: E402 read_run_index, run_json_cli, @@ -36,8 +29,8 @@ AGENT_ID = "codex-product-capability" PRIMARY_AGENT_ID = "codex-main-control" CANARY_TODO_ID = "todo_integrated_canary" -CANARY_TODO_TITLE = "Design bounded status/quota/review-packet/event/read-path canary" -SUCCESSOR_TODO_TITLE = "Continue integrated event-sourced successor routing canary" +CANARY_TODO_TITLE = "Design bounded status/quota/review-packet/Todo/read-path canary" +SUCCESSOR_TODO_TITLE = "Continue integrated Markdown successor routing canary" MARKDOWN_CONTINUATION_TITLE = "Continue same-agent markdown continuation through status and quota." MONITOR_TODO_ID = "todo_integrated_due_monitor" MONITOR_TARGET_KEY = "integrated-due-monitor-watch" @@ -50,7 +43,6 @@ def write_fixture(root: Path) -> tuple[Path, Path, Path]: project = root / "project" runtime = root / "runtime" state_file = project / ".codex" / "goals" / GOAL_ID / "ACTIVE_GOAL_STATE.md" - event_log = state_file.with_name("events.jsonl") registry_path = project / ".loopx" / "registry.json" state_file.parent.mkdir(parents=True) state_file.write_text( @@ -74,12 +66,11 @@ def write_fixture(root: Path) -> tuple[Path, Path, Path]: goal_id=GOAL_ID, domain="control-plane-canary", adapter_kind="generic_project_goal_v0", - state_event_log=f".codex/goals/{GOAL_ID}/events.jsonl", registered_agents=[PRIMARY_AGENT_ID, AGENT_ID], quota_allowed_slots=10, peer_independent_worktree_required=False, ) - return registry_path, state_file, event_log + return registry_path, state_file, runtime def write_monitor_fixture(root: Path) -> tuple[Path, Path]: @@ -149,64 +140,13 @@ def write_markdown_continuation_fixture(root: Path) -> tuple[Path, Path, Path]: return registry_path, runtime, project -def append_event_todos(event_log: Path) -> None: - store = AppendOnlyStateEventStore(event_log) - - def append(event_id: str, event_type: str, todo_id: str, payload: dict[str, Any], seq: int) -> None: - store.append( - make_state_event( - event_id=event_id, - goal_id=GOAL_ID, - event_type=event_type, - refs={"todo_id": todo_id}, - payload=payload, - recorded_at=f"2026-06-27T00:00:{seq:02d}Z", - producer="control-plane-integrated-canary-smoke", - ) - ) - - append( - "evt-integrated-canary-add", - TODO_ADDED, - CANARY_TODO_ID, - { - "role": "agent", - "priority": "P1", - "title": CANARY_TODO_TITLE, - "planner_order": 1, - "task_class": "advancement_task", - "action_kind": "integrated_canary_design", - "target_capabilities": ["status_quota_review_packet_event_read_path_canary"], - }, - 1, - ) - append( - "evt-integrated-canary-claim", - TODO_CLAIMED, - CANARY_TODO_ID, - {"claimed_by": AGENT_ID}, - 2, - ) - append( - "evt-user-prior-approval-add", - TODO_ADDED, - "todo_user_prior_approval", - { - "role": "user", - "priority": "P2", - "title": "Prior canary scope approval", - "planner_order": 1, - "task_class": "user_gate", - }, - 3, - ) - append( - "evt-user-prior-approval-complete", - TODO_COMPLETED, - "todo_user_prior_approval", - {"evidence": "fixture gate already cleared"}, - 4, - ) +def write_canary_todos(state_file: Path) -> None: + state_file.write_text(state_file.read_text().split("## Agent Todo")[0] + + "## Agent Todo\n\n" + f"- [ ] [P1] {CANARY_TODO_TITLE}\n" + + f" \n" + + "\n## User Todo\n\n- [x] [P2] Prior canary scope approval\n" + " \n") def run_cli(registry_path: Path, runtime_root: Path, *args: str) -> dict[str, Any]: @@ -237,7 +177,7 @@ def find_queue_item(status_payload: dict[str, Any], *, goal_id: str = GOAL_ID) - raise AssertionError(f"{goal_id} missing from attention queue: {status_payload}") -def assert_event_projected_agent_todo( +def assert_selected_agent_todo( summary: dict[str, Any], *, compact_quota: bool = False, @@ -371,7 +311,7 @@ def assert_scheduler_ack_state_machine( return steady_payload -def assert_event_todo_completion_successor_state_machine( +def assert_todo_completion_successor_state_machine( registry_path: Path, runtime_root: Path, ) -> str: @@ -392,7 +332,7 @@ def assert_event_todo_completion_successor_state_machine( AGENT_ID, "--self-merged", "--evidence", - "fixture event-projected todo completion passed", + "fixture Todo completion passed", "--next-agent-todo", SUCCESSOR_TODO_TITLE, "--next-claimed-by", @@ -403,7 +343,6 @@ def assert_event_todo_completion_successor_state_machine( "state_machine_canary_refactor", ) assert completed["ok"] is True, completed - assert completed["source"] == "event_log", completed assert completed["completed"] is True, completed assert completed["self_merged"] is True, completed assert completed["todo_id"] == CANARY_TODO_ID, completed @@ -412,7 +351,6 @@ def assert_event_todo_completion_successor_state_machine( successor = completed["next_todos"][0] successor_id = successor["todo_id"] assert completed["successor_todo_ids"] == [successor_id], completed - assert successor["source"] == "event_log", completed assert successor["claimed_by"] == AGENT_ID, completed assert successor["task_class"] == "advancement_task", completed assert successor["action_kind"] == "state_machine_canary_refactor", completed @@ -429,13 +367,10 @@ def assert_event_todo_completion_successor_state_machine( "--agent-id", AGENT_ID, ) - assert listed["source"] in { - "event_projection", - "event_projection_with_markdown_overlay", - }, listed + assert listed["source"] == "markdown_active_state", listed by_id = {item["todo_id"]: item for item in listed["todos"]} assert by_id[CANARY_TODO_ID]["status"] == "done", listed - assert by_id[CANARY_TODO_ID]["evidence"] == "fixture event-projected todo completion passed", listed + assert by_id[CANARY_TODO_ID]["evidence"] == "fixture Todo completion passed", listed assert by_id[CANARY_TODO_ID]["successor_todo_ids"] == [successor_id], listed assert by_id[successor_id]["status"] == "open", listed assert by_id[successor_id]["claimed_by"] == AGENT_ID, listed @@ -801,9 +736,9 @@ def assert_markdown_same_agent_continuation_read_path(root: Path) -> None: def run_fixture_canary(root: Path) -> None: - registry_path, _, event_log = write_fixture(root) + registry_path, state_file, runtime_root = write_fixture(root) runtime_root = root / "runtime" - append_event_todos(event_log) + write_canary_todos(state_file) status_payload = run_cli( registry_path, @@ -822,8 +757,7 @@ def run_fixture_canary(root: Path) -> None: assert queue_item["status"] in allowed_status, queue_item assert queue_item["waiting_on"] == "codex", queue_item assert CANARY_TODO_TITLE in queue_item["recommended_action"], queue_item - assert queue_item["state_event_projection"]["source"] == "event_log", queue_item - assert_event_projected_agent_todo(queue_item["agent_todos"]) + assert_selected_agent_todo(queue_item["agent_todos"]) assert queue_item["project_asset"]["agent_todos"]["payload_reference"][ "canonical_path" ] == "attention_queue.items[].agent_todos", queue_item @@ -844,13 +778,13 @@ def run_fixture_canary(root: Path) -> None: assert quota_payload["effective_action"] == "normal_run", quota_payload assert quota_payload["interaction_contract"]["agent_channel"]["must_attempt"] is True, quota_payload assert CANARY_TODO_TITLE in quota_payload["recommended_action"], quota_payload - assert_event_projected_agent_todo( + assert_selected_agent_todo( quota_payload["agent_todo_summary"], compact_quota=True, ) assert_bounded_delivery_state_machine_bundle(quota_payload) assert_scheduler_ack_state_machine(registry_path, runtime_root, quota_payload) - successor_id = assert_event_todo_completion_successor_state_machine(registry_path, runtime_root) + successor_id = assert_todo_completion_successor_state_machine(registry_path, runtime_root) assert_refresh_and_spend_state_machine(registry_path, runtime_root, successor_id) packet_payload = run_cli( diff --git a/examples/control_plane/event-completion-rehearsal.py b/examples/control_plane/event-completion-rehearsal.py deleted file mode 100644 index 6b4cce4f1b..0000000000 --- a/examples/control_plane/event-completion-rehearsal.py +++ /dev/null @@ -1,183 +0,0 @@ -#!/usr/bin/env python3 -"""Exercise event-owned completion on a detached real Markdown snapshot. - -Only synthetic Todos are completed. Source Goal configuration, validators and -runtime are never activated. All writes target a temporary registry and event -log; output contains counts/timings only. This is event-adapter qualification, -not shadow-capture, provider promotion or SQLite soak evidence. -""" - -from __future__ import annotations - -import argparse -import hashlib -import json -from pathlib import Path -import subprocess -import sys -import tempfile -import time - -REPOSITORY = Path(__file__).resolve().parents[2] -if str(REPOSITORY) not in sys.path: - sys.path.insert(0, str(REPOSITORY)) - -from loopx.event_sourced_state import ( # noqa: E402 - TODO_ADDED, - AppendOnlyStateEventStore, - backfill_todo_events_from_markdown, - build_state_projection, - make_state_event, -) -from loopx.history import load_registry # noqa: E402 -from loopx.state_refresh import resolve_goal_state # noqa: E402 - - -def main() -> None: - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--registry", type=Path, required=True) - parser.add_argument("--goal-id", required=True) - args = parser.parse_args() - registry_path = args.registry.expanduser().resolve() - registry = load_registry(registry_path) - _, _, source = resolve_goal_state( - registry=registry, - goal_id=args.goal_id, - project_override=None, - state_file_override=None, - ) - source_bytes = source.read_bytes() - source_digest = hashlib.sha256(source_bytes).digest() - # Preserve real record variety and volume, without inheriting a live owner. - events = backfill_todo_events_from_markdown( - source_bytes.decode("utf-8"), - goal_id="event-rehearsal", - recorded_at="2026-09-24T00:00:00Z", - ) - if not events: - raise SystemExit("selected source has no Markdown Todos to rehearse") - with tempfile.TemporaryDirectory(prefix="loopx-event-rehearsal-") as directory: - root = Path(directory) - state = root / "ACTIVE_GOAL_STATE.md" - state.write_text( - "---\ngoal_id: event-rehearsal\n---\n\n## Agent Todo\n", encoding="utf-8" - ) - clone_registry = root / "registry.json" - clone_registry.write_text( - json.dumps( - { - "common_runtime_root": str(root / "runtime"), - "goals": [ - { - "id": "event-rehearsal", - "status": "active", - "repo": str(root), - "state_file": state.name, - "domain": "harness_self_improvement", - "adapter": {"kind": "harness_self_improvement"}, - "coordination": { - "agent_model": "peer_v1", - "registered_agents": ["rehearsal-worker"], - }, - } - ], - } - ), - encoding="utf-8", - ) - store = AppendOnlyStateEventStore(root / "events.jsonl") - store.append_many(events) - initial_count = len(store.load()) - todo_id = "todo_rehearsal_atomic_completion" - assert not any(row.get("refs", {}).get("todo_id") == todo_id for row in events) - store.append( - make_state_event( - event_id="rehearsal-parent", - goal_id="event-rehearsal", - event_type=TODO_ADDED, - refs={"todo_id": todo_id}, - payload={ - "role": "agent", - "title": "Qualify one detached event transaction.", - "task_class": "advancement_task", - "claimed_by": "rehearsal-worker", - }, - recorded_at="2026-09-24T01:00:00Z", - ) - ) - before = store.path.read_bytes() - command = [ - sys.executable, - "-c", - "from loopx.cli import main; raise SystemExit(main())", - "--registry", - str(clone_registry), - "--format", - "json", - "todo", - "complete", - "--goal-id", - "event-rehearsal", - "--todo-id", - todo_id, - "--claimed-by", - "rehearsal-worker", - "--evidence", - "Detached validation passed.", - "--next-agent-todo", - "Independently review the detached delivery.", - "--next-task-class", - "advancement_task", - "--next-claimed-by", - "rehearsal-worker", - "--execute", - ] - started = time.perf_counter() - first = subprocess.run( - command, cwd=REPOSITORY, capture_output=True, text=True, timeout=120 - ) - # Do not include subprocess output: it can contain copied private titles. - if first.returncode: - raise RuntimeError( - f"detached completion CLI failed (exit {first.returncode})" - ) - result = json.loads(first.stdout) - assert result["ok"] and result["source"] == "event_log" - elapsed_ms = round((time.perf_counter() - started) * 1000) - landed = store.path.read_bytes() - assert landed.startswith(before) - projection = build_state_projection(store.load()) - parent = next( - row - for row in projection["agent_todos"]["items"] - if row["todo_id"] == todo_id - ) - assert parent["status"] == "done" and len(parent["successor_todo_ids"]) == 1 - replay = subprocess.run( - command, cwd=REPOSITORY, capture_output=True, text=True, timeout=120 - ) - if replay.returncode: - raise RuntimeError(f"detached replay CLI failed (exit {replay.returncode})") - assert json.loads(replay.stdout)["idempotent_replay"] is True - assert store.path.read_bytes() == landed - assert hashlib.sha256(source.read_bytes()).digest() == source_digest, ( - "live source changed during rehearsal" - ) - print( - json.dumps( - { - "ok": True, - "source_unchanged": True, - "source_bytes": len(source_bytes), - "backfilled_events": initial_count, - "detached_log_bytes": len(landed), - "completion_events_added": len(store.load()) - initial_count - 1, - "cli_completion_ms": elapsed_ms, - "replay_unchanged": True, - } - ) - ) - - -if __name__ == "__main__": - main() diff --git a/examples/control_plane/event-sourced-downstream-read-path-smoke.py b/examples/control_plane/event-sourced-downstream-read-path-smoke.py deleted file mode 100644 index 21bc3e572b..0000000000 --- a/examples/control_plane/event-sourced-downstream-read-path-smoke.py +++ /dev/null @@ -1,464 +0,0 @@ -#!/usr/bin/env python3 -"""Validate event-projected todos flow into quota and review-packet reads.""" - -from __future__ import annotations - -import sys -import tempfile -from pathlib import Path - - -REPO_ROOT = Path(__file__).resolve().parents[2] -if str(REPO_ROOT) not in sys.path: - sys.path.insert(0, str(REPO_ROOT)) - -from loopx.event_sourced_state import ( # noqa: E402 - AppendOnlyStateEventStore, - TODO_ADDED, - TODO_CLAIMED, - TODO_UPDATED, - make_state_event, -) -from loopx.control_plane.todos.quota_selection import project_quota_planning # noqa: E402 -from loopx.quota import build_quota_should_run, render_quota_should_run_markdown # noqa: E402 -from loopx.review_packet import build_review_packet # noqa: E402 -from loopx.status import active_state_todo_fields, project_asset_todo_summary # noqa: E402 - - -GOAL_ID = "event-sourced-downstream-read-fixture" -EVENT_TODO_ID = "todo_event_downstream_read" -EVENT_TODO = "Use event projection for downstream read surfaces" -EVENT_MONITOR_ID = "todo_event_monitor_readonly" -EVENT_MONITOR = "Poll event-projected monitor metadata without writeback" -MARKDOWN_TODO_ID = "todo_markdown_fallback" -MARKDOWN_TODO = "Fallback Markdown todo for corrupted event logs" -LEGACY_REVIEW_ID = "todo_event_legacy_review" -LEGACY_FALLBACK_ID = "todo_event_legacy_fallback" -AUTHOR_AGENT = "codex-main-control" -REVIEWER_AGENT = "codex-product-capability" - - -def write_active_state(state_path: Path) -> None: - state_path.parent.mkdir(parents=True, exist_ok=True) - state_path.write_text( - "\n".join( - [ - "# Event Read Fixture", - "", - "## Next Action", - "", - "- Keep downstream reads public-safe.", - "", - "## Agent Todo", - "", - f"- [ ] [P2] {MARKDOWN_TODO}.", - f" ", - "", - ] - ), - encoding="utf-8", - ) - - -def append_event_todos(event_log: Path) -> None: - store = AppendOnlyStateEventStore(event_log) - store.append( - make_state_event( - event_id="evt-add-downstream-read", - goal_id=GOAL_ID, - event_type=TODO_ADDED, - refs={"todo_id": EVENT_TODO_ID}, - payload={ - "role": "agent", - "priority": "P0", - "title": EVENT_TODO, - "planner_order": 1, - "task_class": "advancement_task", - "action_kind": "event_projection_read_path", - }, - producer="event-sourced-downstream-read-path-smoke", - recorded_at="2026-06-27T02:00:01Z", - ) - ) - store.append( - make_state_event( - event_id="evt-claim-downstream-read", - goal_id=GOAL_ID, - event_type=TODO_CLAIMED, - refs={"todo_id": EVENT_TODO_ID}, - payload={"claimed_by": "codex-product-capability"}, - producer="event-sourced-downstream-read-path-smoke", - recorded_at="2026-06-27T02:00:02Z", - ) - ) - - -def append_event_todo_and_due_monitor(event_log: Path) -> None: - append_event_todos(event_log) - store = AppendOnlyStateEventStore(event_log) - store.append( - make_state_event( - event_id="evt-add-readonly-monitor", - goal_id=GOAL_ID, - event_type=TODO_ADDED, - refs={"todo_id": EVENT_MONITOR_ID}, - payload={ - "role": "agent", - "priority": "P0", - "title": EVENT_MONITOR, - "planner_order": 2, - "task_class": "continuous_monitor", - "action_kind": "poll", - "target_key": "event-projected-watch", - "cadence": "15m", - "next_due_at": "2026-01-01T00:00:00+00:00", - }, - producer="event-sourced-downstream-read-path-smoke", - recorded_at="2026-06-27T02:00:03Z", - ) - ) - - -def append_legacy_review_events(event_log: Path, *, policy: str) -> None: - store = AppendOnlyStateEventStore(event_log) - added_payload = { - "role": "agent", - "priority": "P0", - "title": f"Legacy {policy} event must remain blocked", - "planner_order": 1, - "task_class": "advancement_task", - "action_kind": "review", - } - if policy == "review_handoff": - added_payload["continuation_policy"] = policy - else: - added_payload["continuation_policy"] = "independent_handoff" - store.append( - make_state_event( - event_id=f"evt-add-legacy-{policy}", - goal_id=GOAL_ID, - event_type=TODO_ADDED, - refs={"todo_id": LEGACY_REVIEW_ID}, - payload=added_payload, - producer="event-sourced-downstream-read-path-smoke", - recorded_at="2026-07-11T00:00:01Z", - ) - ) - if policy == "primary_review": - store.append( - make_state_event( - event_id=f"evt-update-legacy-{policy}", - goal_id=GOAL_ID, - event_type=TODO_UPDATED, - refs={"todo_id": LEGACY_REVIEW_ID}, - payload={"continuation_policy": policy}, - producer="event-sourced-downstream-read-path-smoke", - recorded_at="2026-07-11T00:00:02Z", - ) - ) - store.append( - make_state_event( - event_id=f"evt-update-incidental-{policy}", - goal_id=GOAL_ID, - event_type=TODO_UPDATED, - refs={"todo_id": LEGACY_REVIEW_ID}, - payload={"title": f"Legacy {policy} survives incidental update"}, - producer="event-sourced-downstream-read-path-smoke", - recorded_at="2026-07-11T00:00:03Z", - ) - ) - store.append( - make_state_event( - event_id=f"evt-add-fallback-{policy}", - goal_id=GOAL_ID, - event_type=TODO_ADDED, - refs={"todo_id": LEGACY_FALLBACK_ID}, - payload={ - "role": "agent", - "priority": "P1", - "title": "Repair the legacy event projection", - "planner_order": 2, - "task_class": "advancement_task", - "action_kind": "repair", - }, - producer="event-sourced-downstream-read-path-smoke", - recorded_at="2026-07-11T00:00:04Z", - ) - ) - store.append( - make_state_event( - event_id=f"evt-claim-fallback-{policy}", - goal_id=GOAL_ID, - event_type=TODO_CLAIMED, - refs={"todo_id": LEGACY_FALLBACK_ID}, - payload={"claimed_by": AUTHOR_AGENT}, - producer="event-sourced-downstream-read-path-smoke", - recorded_at="2026-07-11T00:00:05Z", - ) - ) - - -def status_payload_from_fields(project: Path, fields: dict) -> dict: - agent_todos = fields.get("agent_todos") if isinstance(fields.get("agent_todos"), dict) else {} - asset_agent_todos = project_asset_todo_summary(agent_todos, role="agent") - item = { - "goal_id": GOAL_ID, - "status": "event_projection_downstream_read_fixture", - "waiting_on": "codex", - "severity": "action", - "source": "project_asset", - "recommended_action": "Advance the event-projected downstream read todo.", - "quota": { - "compute": 1.0, - "slot_minutes": 1, - "allowed_slots": 1440, - "spent_slots": 0, - "state": "eligible", - "reason": "eligible fixture", - }, - "agent_todos": agent_todos, - "state_event_projection": fields.get("state_event_projection"), - "project_asset": { - "owner": "codex", - "next_action": "Advance the event-projected downstream read todo.", - "stop_condition": "stop on fixture boundary", - "agent_todos": asset_agent_todos, - "state_event_projection": fields.get("state_event_projection"), - "quota": { - "compute": 1.0, - "slot_minutes": 1, - "allowed_slots": 1440, - "spent_slots": 0, - "state": "eligible", - "reason": "eligible fixture", - }, - }, - } - return { - "ok": True, - "attention_queue": {"items": [item]}, - "run_history": { - "goals": [ - { - "id": GOAL_ID, - "repo": str(project), - "registry_member": True, - "status": "active", - "quota": {"compute": 1.0, "window_hours": 24}, - "coordination": { - "agent_model": "peer_v1", - "registered_agents": [ - "codex-main-control", - "codex-product-capability", - ], - }, - "latest_runs": [], - } - ] - }, - } - - -def agent_todo_ids(fields: dict) -> list[str]: - agent_todos = fields.get("agent_todos") if isinstance(fields.get("agent_todos"), dict) else {} - return [str(item.get("todo_id") or "") for item in agent_todos.get("items") or []] - - -def agent_todo_by_id(fields: dict, todo_id: str) -> dict: - agent_todos = fields.get("agent_todos") if isinstance(fields.get("agent_todos"), dict) else {} - for item in agent_todos.get("items") or []: - if isinstance(item, dict) and item.get("todo_id") == todo_id: - return item - raise AssertionError(f"missing todo {todo_id}: {fields}") - - -def test_event_projection_feeds_quota_and_review_packet() -> None: - with tempfile.TemporaryDirectory(prefix="loopx-event-downstream-") as tmp: - project = Path(tmp) - state_path = project / ".codex" / "goals" / GOAL_ID / "ACTIVE_GOAL_STATE.md" - write_active_state(state_path) - append_event_todos(state_path.with_name("events.jsonl")) - - goal = { - "id": GOAL_ID, - "repo": str(project), - "state_file": f".codex/goals/{GOAL_ID}/ACTIVE_GOAL_STATE.md", - } - fields = active_state_todo_fields(goal) - assert fields["state_event_projection"]["source"] == "event_log", fields - assert agent_todo_ids(fields) == [EVENT_TODO_ID], fields - assert MARKDOWN_TODO_ID not in agent_todo_ids(fields), fields - - status = status_payload_from_fields(project, fields) - guard = build_quota_should_run( - status, - goal_id=GOAL_ID, - agent_id="codex-product-capability", - ) - assert guard["should_run"] is True, guard - assert guard["agent_todo_summary"]["first_open_items"][0]["todo_id"] == EVENT_TODO_ID, guard - assert guard["agent_lane_next_action"]["todo_id"] == EVENT_TODO_ID, guard - markdown = render_quota_should_run_markdown(guard) - assert EVENT_TODO in markdown, markdown - assert MARKDOWN_TODO not in markdown, markdown - - packet = build_review_packet(status, goal_id=GOAL_ID, action_kind="codex") - assert packet["agent_todo_items"] == [ - f"[P0] {EVENT_TODO} claimed_by=codex-product-capability" - ], packet - assert EVENT_TODO in packet["project_agent_handoff"], packet - assert MARKDOWN_TODO not in packet["project_agent_handoff"], packet - - -def test_event_projected_due_monitor_is_read_only_for_writeback() -> None: - with tempfile.TemporaryDirectory(prefix="loopx-event-readonly-monitor-") as tmp: - project = Path(tmp) - state_path = project / ".codex" / "goals" / GOAL_ID / "ACTIVE_GOAL_STATE.md" - write_active_state(state_path) - append_event_todo_and_due_monitor(state_path.with_name("events.jsonl")) - - goal = { - "id": GOAL_ID, - "repo": str(project), - "state_file": f".codex/goals/{GOAL_ID}/ACTIVE_GOAL_STATE.md", - } - fields = active_state_todo_fields(goal) - assert agent_todo_ids(fields) == [EVENT_TODO_ID, EVENT_MONITOR_ID], fields - monitor = agent_todo_by_id(fields, EVENT_MONITOR_ID) - assert monitor["target_key"] == "event-projected-watch", monitor - assert monitor["cadence"] == "15m", monitor - assert monitor["next_due_at"] == "2026-01-01T00:00:00+00:00", monitor - monitor_writeback = fields["agent_todos"]["monitor_writeback"] - assert monitor_writeback == { - "schema_version": "monitor_writeback_contract_v0", - "supported": False, - "source": "event_projection_read_model", - }, fields - - status = status_payload_from_fields(project, fields) - guard = build_quota_should_run( - status, - goal_id=GOAL_ID, - agent_id="codex-product-capability", - ) - assert guard["decision"] == "run", guard - assert guard["agent_lane_next_action"]["todo_id"] == EVENT_TODO_ID, guard - lane = guard["work_lane_contract"] - assert lane["lane"] == "advancement_task", lane - assert "open_agent_todo" in lane["reason_codes"], lane - assert "external_monitor_context" in lane["reason_codes"], lane - assert "due_monitor_context" not in lane["reason_codes"], lane - assert lane.get("obligation") != "attempt_due_monitor", lane - summary = guard["agent_todo_summary"] - assert summary["monitor_due_count"] == 0, summary - assert summary["monitor_open_items"][0]["todo_id"] == EVENT_MONITOR_ID, summary - assert summary["monitor_writeback"] == { - "supported": False, - "source": "event_projection_read_model", - }, summary - - -def test_corrupted_event_log_falls_back_to_markdown() -> None: - with tempfile.TemporaryDirectory(prefix="loopx-event-downstream-fallback-") as tmp: - project = Path(tmp) - state_path = project / ".codex" / "goals" / GOAL_ID / "ACTIVE_GOAL_STATE.md" - write_active_state(state_path) - state_path.with_name("events.jsonl").write_text("{not json\n", encoding="utf-8") - - goal = { - "id": GOAL_ID, - "repo": str(project), - "state_file": f".codex/goals/{GOAL_ID}/ACTIVE_GOAL_STATE.md", - } - fields = active_state_todo_fields(goal) - assert fields["state_event_projection_warning"]["fallback"] == "markdown_active_state", fields - assert agent_todo_ids(fields) == [MARKDOWN_TODO_ID], fields - - status = status_payload_from_fields(project, fields) - guard = build_quota_should_run(status, goal_id=GOAL_ID, agent_id="codex-product-capability") - assert guard["agent_todo_summary"]["first_open_items"][0]["todo_id"] == MARKDOWN_TODO_ID, guard - packet = build_review_packet(status, goal_id=GOAL_ID, action_kind="codex") - assert MARKDOWN_TODO in packet["project_agent_handoff"], packet - - -def test_legacy_event_review_handoffs_fail_closed_until_explicit_repair() -> None: - for policy in ("review_handoff", "primary_review"): - with tempfile.TemporaryDirectory(prefix=f"loopx-event-legacy-{policy}-") as tmp: - project = Path(tmp) - state_path = project / ".codex" / "goals" / GOAL_ID / "ACTIVE_GOAL_STATE.md" - write_active_state(state_path) - event_log = state_path.with_name("events.jsonl") - append_legacy_review_events(event_log, policy=policy) - goal = { - "id": GOAL_ID, - "repo": str(project), - "state_file": f".codex/goals/{GOAL_ID}/ACTIVE_GOAL_STATE.md", - } - - fields = active_state_todo_fields(goal) - legacy = agent_todo_by_id(fields, LEGACY_REVIEW_ID) - assert legacy["removed_continuation_policy"] == policy, fields - assert legacy.get("continuation_policy") is None, fields - items = fields["agent_todos"]["items"] - planning = project_quota_planning( - {}, all_open_items=items, source_open_count=len(items), - agent_identity={"agent_id": AUTHOR_AGENT, "agent_model": "peer_v1"}, - filter_user_gate_blocks_agent=False, available_capabilities=None, - ) - author_selectable = planning["lanes"]["open_items"] - author_scope = planning["lanes"]["claim_scope"] - assert [item["todo_id"] for item in author_selectable] == [LEGACY_FALLBACK_ID] - assert author_scope["removed_continuation_blocked_count"] == 1, author_scope - planning = project_quota_planning( - {}, all_open_items=items, source_open_count=len(items), - agent_identity={"agent_id": REVIEWER_AGENT, "agent_model": "peer_v1"}, - filter_user_gate_blocks_agent=False, available_capabilities=None, - ) - reviewer_selectable = planning["lanes"]["open_items"] - reviewer_scope = planning["lanes"]["claim_scope"] - assert reviewer_selectable == [], reviewer_selectable - assert reviewer_scope["removed_continuation_blocked_count"] == 1, reviewer_scope - - status = status_payload_from_fields(project, fields) - author_guard = build_quota_should_run(status, goal_id=GOAL_ID, agent_id=AUTHOR_AGENT) - assert author_guard["agent_lane_next_action"]["todo_id"] == LEGACY_FALLBACK_ID, author_guard - assert LEGACY_REVIEW_ID not in author_guard["recommended_action"], author_guard - - AppendOnlyStateEventStore(event_log).append( - make_state_event( - event_id=f"evt-repair-legacy-{policy}", - goal_id=GOAL_ID, - event_type=TODO_UPDATED, - refs={"todo_id": LEGACY_REVIEW_ID}, - payload={ - "continuation_policy": "independent_handoff", - "excluded_agents": [AUTHOR_AGENT], - }, - producer="event-sourced-downstream-read-path-smoke", - recorded_at="2026-07-11T00:00:06Z", - ) - ) - repaired_fields = active_state_todo_fields(goal) - repaired = agent_todo_by_id(repaired_fields, LEGACY_REVIEW_ID) - assert repaired.get("removed_continuation_policy") is None, repaired - assert repaired["continuation_policy"] == "independent_handoff", repaired - assert repaired["excluded_agents"] == [AUTHOR_AGENT], repaired - reviewer_guard = build_quota_should_run( - status_payload_from_fields(project, repaired_fields), - goal_id=GOAL_ID, - agent_id=REVIEWER_AGENT, - ) - assert reviewer_guard["agent_lane_next_action"]["todo_id"] == LEGACY_REVIEW_ID, reviewer_guard - - -def main() -> int: - test_event_projection_feeds_quota_and_review_packet() - test_event_projected_due_monitor_is_read_only_for_writeback() - test_corrupted_event_log_falls_back_to_markdown() - test_legacy_event_review_handoffs_fail_closed_until_explicit_repair() - print("event-sourced-downstream-read-path-smoke ok") - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/examples/control_plane/event-sourced-markdown-backfill-smoke.py b/examples/control_plane/event-sourced-markdown-backfill-smoke.py deleted file mode 100644 index dc05ab5365..0000000000 --- a/examples/control_plane/event-sourced-markdown-backfill-smoke.py +++ /dev/null @@ -1,158 +0,0 @@ -#!/usr/bin/env python3 -"""Validate Markdown active-state backfill into append-only LoopX events.""" - -from __future__ import annotations - -import json -import sys -import tempfile -from pathlib import Path - - -REPO_ROOT = Path(__file__).resolve().parents[2] -if str(REPO_ROOT) not in sys.path: - sys.path.insert(0, str(REPO_ROOT)) - -from loopx.event_sourced_state import ( # noqa: E402 - AppendOnlyStateEventStore, - LOCAL_PRIVATE_PRIVACY, - MARKDOWN_BACKFILL_PRODUCER, - PUBLIC_BACKFILL_REDACTION, - PUBLIC_PRIVACY, - backfill_todo_events_from_markdown, - build_state_projection, -) - - -GOAL_ID = "event-sourced-markdown-backfill-fixture" - - -def fixture_markdown() -> str: - return "\n".join( - [ - "# Fixture Active State", - "", - "## User Todo / Owner Review Reading Queue", - "", - "- [ ] [P0] Approve private launch note https://private.example/doc", - " ", - "", - "## Agent Todo", - "", - "- [ ] [P0] First same-priority migration check", - " ", - "- [ ] [P0] Second same-priority migration check", - " ", - "- [x] [P1] Completed migration proof", - " ", - "", - ] - ) - - -def event_blob(events: list[dict]) -> str: - return json.dumps(events, sort_keys=True, ensure_ascii=False) - - -def test_public_backfill_redacts_private_state() -> None: - markdown = fixture_markdown() - original = str(markdown) - events = backfill_todo_events_from_markdown( - markdown, - goal_id=GOAL_ID, - source_ref="/Users/example/project/.local/goals/example/ACTIVE_GOAL_STATE.md", - recorded_at="2026-06-27T00:00:00Z", - privacy=PUBLIC_PRIVACY, - ) - - assert markdown == original, "backfill must not mutate Markdown workbench" - assert events, events - assert all(event["producer"] == MARKDOWN_BACKFILL_PRODUCER for event in events), events - assert all(event["privacy"] == PUBLIC_PRIVACY for event in events), events - serialized = event_blob(events) - assert "private.example" not in serialized, serialized - assert "/Users/example" not in serialized, serialized - assert "https://" not in serialized, serialized - assert PUBLIC_BACKFILL_REDACTION in serialized, serialized - assert all(event["refs"].get("source_ref") == "ACTIVE_GOAL_STATE.md" for event in events), events - - projection = build_state_projection(events, goal_id=GOAL_ID, generated_at="2026-06-27T00:01:00Z") - assert [item["todo_id"] for item in projection["agent_todos"]["items"]] == [ - "todo_first", - "todo_second", - "todo_done", - ], projection - first = projection["agent_todos"]["items"][0] - assert first["claimed_by"] == "codex-product-capability", projection - assert projection["agent_todos"]["items"][2]["status"] == "done", projection - assert projection["agent_todos"]["items"][2]["evidence"] == "validation_packet", projection - assert projection["user_todos"]["items"][0]["task_class"] == "user_gate", projection - assert projection["user_todos"]["items"][0]["title"] == PUBLIC_BACKFILL_REDACTION, projection - - -def test_local_private_backfill_preserves_private_workbench_text() -> None: - events = backfill_todo_events_from_markdown( - fixture_markdown(), - goal_id=GOAL_ID, - source_ref="ACTIVE_GOAL_STATE.md", - recorded_at="2026-06-27T00:00:00Z", - privacy=LOCAL_PRIVATE_PRIVACY, - ) - serialized = event_blob(events) - assert "https://private.example/doc" in serialized, serialized - assert all(event["privacy"] == LOCAL_PRIVATE_PRIVACY for event in events), events - - -def test_append_many_is_idempotent_for_backfilled_events() -> None: - with tempfile.TemporaryDirectory(prefix="loopx-markdown-backfill-") as tmp: - store = AppendOnlyStateEventStore(Path(tmp) / "events.jsonl") - events = backfill_todo_events_from_markdown( - fixture_markdown(), - goal_id=GOAL_ID, - source_ref="ACTIVE_GOAL_STATE.md", - recorded_at="2026-06-27T00:00:00Z", - privacy=PUBLIC_PRIVACY, - ) - store.append_many(events) - store.append_many(events) - projection = build_state_projection(store.load(), goal_id=GOAL_ID) - assert projection["source_event_count"] == len(events), projection - assert projection["last_append_sequence"] == len(events), projection - - -def test_backfill_preserves_removed_continuation_diagnostic() -> None: - for policy in ("review_handoff", "primary_review"): - events = backfill_todo_events_from_markdown( - "\n".join( - [ - "## Agent Todo", - "", - "- [ ] [P0] Legacy review handoff must remain blocked", - " ", - ] - ), - goal_id=GOAL_ID, - source_ref="ACTIVE_GOAL_STATE.md", - recorded_at="2026-07-11T00:00:00Z", - privacy=PUBLIC_PRIVACY, - ) - assert events[0]["payload"]["removed_continuation_policy"] == policy, events - projection = build_state_projection(events, goal_id=GOAL_ID) - item = projection["agent_todos"]["items"][0] - assert item["removed_continuation_policy"] == policy, projection - assert item.get("continuation_policy") is None, projection - - -def main() -> int: - test_public_backfill_redacts_private_state() - test_local_private_backfill_preserves_private_workbench_text() - test_append_many_is_idempotent_for_backfilled_events() - test_backfill_preserves_removed_continuation_diagnostic() - print("event-sourced-markdown-backfill-smoke ok") - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/examples/control_plane/event-sourced-replay-compaction-smoke.py b/examples/control_plane/event-sourced-replay-compaction-smoke.py deleted file mode 100644 index 78c8e6e370..0000000000 --- a/examples/control_plane/event-sourced-replay-compaction-smoke.py +++ /dev/null @@ -1,244 +0,0 @@ -#!/usr/bin/env python3 -"""Validate event replay, compact workbench projection, and canonical edit guards.""" - -from __future__ import annotations - -import sys -import tempfile -from pathlib import Path - - -REPO_ROOT = Path(__file__).resolve().parents[2] -if str(REPO_ROOT) not in sys.path: - sys.path.insert(0, str(REPO_ROOT)) - -from loopx.event_sourced_state import ( # noqa: E402 - AppendOnlyStateEventStore, - REFRESH_RECORDED, - RUN_RECORDED, - StateEventError, - TODO_ADDED, - TODO_CLAIMED, - TODO_COMPLETED, - TODO_UPDATED, - build_state_projection, - make_state_event, - render_active_state_sections, -) -from loopx.status import active_state_todo_fields, parse_active_state_todos # noqa: E402 - - -GOAL_ID = "event-sourced-replay-compaction-fixture" - - -def fixture_event(event_id: str, event_type: str, *, refs: dict | None = None, payload: dict | None = None) -> dict: - return make_state_event( - event_id=event_id, - goal_id=GOAL_ID, - event_type=event_type, - refs=refs or {}, - payload=payload or {}, - recorded_at=f"2026-06-27T01:00:{len(event_id):02d}Z", - producer="event-sourced-replay-compaction-smoke", - ) - - -def append_fixture_events(event_log: Path) -> AppendOnlyStateEventStore: - store = AppendOnlyStateEventStore(event_log) - store.append( - fixture_event( - "evt-add-plan", - TODO_ADDED, - refs={"todo_id": "todo_event_plan"}, - payload={ - "role": "agent", - "priority": "P0", - "title": "Plan event replay migration", - "planner_order": 1, - "task_class": "advancement_task", - "action_kind": "plan", - }, - ) - ) - store.append( - fixture_event( - "evt-add-render", - TODO_ADDED, - refs={"todo_id": "todo_event_render"}, - payload={ - "role": "agent", - "priority": "P0", - "title": "Render compact workbench from events", - "planner_order": 2, - "task_class": "advancement_task", - "action_kind": "validate", - }, - ) - ) - store.append( - fixture_event( - "evt-add-gate", - TODO_ADDED, - refs={"todo_id": "todo_event_gate"}, - payload={ - "role": "user", - "priority": "P1", - "title": "Approve promoting event projection reads", - "planner_order": 1, - "task_class": "user_gate", - }, - ) - ) - store.append( - fixture_event( - "evt-claim-render", - TODO_CLAIMED, - refs={"todo_id": "todo_event_render"}, - payload={"claimed_by": "codex-product-capability"}, - ) - ) - store.append( - fixture_event( - "evt-update-render", - TODO_UPDATED, - refs={"todo_id": "todo_event_render"}, - payload={"title": "Render compact Markdown workbench from events"}, - ) - ) - store.append( - fixture_event( - "evt-complete-plan", - TODO_COMPLETED, - refs={"todo_id": "todo_event_plan"}, - payload={"evidence": "replay smoke"}, - ) - ) - store.append( - fixture_event( - "evt-refresh", - REFRESH_RECORDED, - payload={"summary": "event projection rendered into Markdown workbench"}, - ) - ) - store.append( - fixture_event( - "evt-run", - RUN_RECORDED, - payload={"summary": "compact workbench stayed readable after replay"}, - ) - ) - return store - - -def todo_ids(fields: dict, role: str) -> list[str]: - return [str(item.get("todo_id") or "") for item in (fields.get(f"{role}_todos") or {}).get("items") or []] - - -def todo_statuses(fields: dict, role: str) -> dict[str, str]: - return { - str(item.get("todo_id") or ""): str(item.get("status") or "") - for item in (fields.get(f"{role}_todos") or {}).get("items") or [] - } - - -def test_replay_regenerates_equivalent_projection() -> None: - with tempfile.TemporaryDirectory(prefix="loopx-event-replay-") as tmp: - project = Path(tmp) - state_path = project / ".codex" / "goals" / GOAL_ID / "ACTIVE_GOAL_STATE.md" - event_log = state_path.with_name("events.jsonl") - store = append_fixture_events(event_log) - - projection = build_state_projection(store.load(), generated_at="2026-06-27T01:01:00Z") - replayed = build_state_projection(reversed(store.load()), generated_at="2026-06-27T01:01:00Z") - assert projection["source_checksum"].startswith("sha256:"), projection - assert projection["source_checksum"] == replayed["source_checksum"], replayed - assert projection["last_append_sequence"] == 8, projection - assert todo_ids(projection, "agent") == ["todo_event_plan", "todo_event_render"], projection - assert projection["agent_todos"]["items"][0]["status"] == "done", projection - assert projection["agent_todos"]["items"][1]["claimed_by"] == "codex-product-capability", projection - assert todo_ids(projection, "user") == ["todo_event_gate"], projection - - state_path.parent.mkdir(parents=True, exist_ok=True) - state_path.write_text(render_active_state_sections(projection), encoding="utf-8") - parsed_workbench = parse_active_state_todos(state_path.read_text(encoding="utf-8")) - assert set(todo_ids(parsed_workbench, "agent")) == set(todo_ids(projection, "agent")), parsed_workbench - assert set(todo_ids(parsed_workbench, "user")) == set(todo_ids(projection, "user")), parsed_workbench - assert todo_statuses(parsed_workbench, "agent") == todo_statuses(projection, "agent"), parsed_workbench - - -def test_compact_markdown_cannot_override_canonical_events() -> None: - with tempfile.TemporaryDirectory(prefix="loopx-event-compact-") as tmp: - project = Path(tmp) - state_path = project / ".codex" / "goals" / GOAL_ID / "ACTIVE_GOAL_STATE.md" - event_log = state_path.with_name("events.jsonl") - store = append_fixture_events(event_log) - projection = build_state_projection(store.load(), generated_at="2026-06-27T01:02:00Z") - - state_path.parent.mkdir(parents=True, exist_ok=True) - state_path.write_text( - "\n".join( - [ - "# Compact Workbench", - "", - "## Next Action", - "", - "- Continue from the compact event projection.", - "", - "## Agent Todo", - "", - "- [x] [P0] Locally edited workbench item that must not become truth", - " ", - "", - "## Progress Ledger", - "", - "- compact projection; full lineage stays in events.jsonl", - "", - ] - ), - encoding="utf-8", - ) - markdown_only = parse_active_state_todos(state_path.read_text(encoding="utf-8")) - assert markdown_only["agent_todos"]["items"][0]["status"] == "done", markdown_only - - goal = { - "id": GOAL_ID, - "repo": str(project), - "state_file": f".codex/goals/{GOAL_ID}/ACTIVE_GOAL_STATE.md", - } - fields = active_state_todo_fields(goal) - assert fields["state_event_projection"]["source"] == "event_log", fields - assert fields["state_event_projection"]["source_checksum"] == projection["source_checksum"], fields - assert set(todo_ids(fields, "agent")) == {"todo_event_plan", "todo_event_render"}, fields - assert todo_statuses(fields, "agent") == { - "todo_event_plan": "done", - "todo_event_render": "open", - }, fields - assert todo_ids(fields, "user") == ["todo_event_gate"], fields - render_item = next(item for item in fields["agent_todos"]["items"] if item["todo_id"] == "todo_event_render") - assert render_item["todo_id"] == "todo_event_render", fields - assert render_item["status"] == "open", fields - assert render_item["claimed_by"] == "codex-product-capability", fields - - try: - mutation = fixture_event( - "evt-mutate-prior", - TODO_UPDATED, - refs={"todo_id": "todo_event_render", "mutates_prior_event_id": "evt-add-render"}, - payload={"title": "Rewrite prior canonical event"}, - ) - store.append(mutation) - except StateEventError as exc: - assert "must not mutate prior events" in str(exc), exc - else: - raise AssertionError("mutable canonical event edit was accepted") - - -def main() -> int: - test_replay_regenerates_equivalent_projection() - test_compact_markdown_cannot_override_canonical_events() - print("event-sourced-replay-compaction-smoke ok") - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/examples/control_plane/event-sourced-state-api-smoke.py b/examples/control_plane/event-sourced-state-api-smoke.py deleted file mode 100644 index b03571b298..0000000000 --- a/examples/control_plane/event-sourced-state-api-smoke.py +++ /dev/null @@ -1,256 +0,0 @@ -#!/usr/bin/env python3 -"""Smoke-test the minimal append-only LoopX state event API.""" - -from __future__ import annotations - -import tempfile -from pathlib import Path - -import sys - - -REPO_ROOT = Path(__file__).resolve().parents[2] -if str(REPO_ROOT) not in sys.path: - sys.path.insert(0, str(REPO_ROOT)) - -from loopx.event_sourced_state import ( # noqa: E402 - AppendOnlyStateEventStore, - StateEventConflictError, - StateEventError, - TODO_ADDED, - TODO_BLOCKED, - TODO_CLAIMED, - TODO_COMPLETED, - TODO_UPDATED, - REFRESH_RECORDED, - build_state_projection, - make_state_event, - render_active_state_sections, -) -from loopx.status import parse_active_state_todos # noqa: E402 - - -GOAL_ID = "event-sourced-state-api-fixture" - - -def todo_event(event_id: str, event_type: str, todo_id: str, payload: dict) -> dict: - return make_state_event( - event_id=event_id, - goal_id=GOAL_ID, - event_type=event_type, - refs={"todo_id": todo_id}, - payload=payload, - recorded_at=f"2026-06-27T00:00:{len(event_id):02d}Z", - producer="event-sourced-state-api-smoke", - ) - - -def test_store_projection_and_markdown() -> None: - with tempfile.TemporaryDirectory(prefix="loopx-event-store-") as tmp: - store = AppendOnlyStateEventStore(Path(tmp) / "events.jsonl") - todo_a = todo_event( - "evt-todo-a", - TODO_ADDED, - "todo_event_a", - { - "role": "agent", - "priority": "P0", - "title": "Implement append-only event store", - "planner_order": 1, - "task_class": "advancement_task", - "action_kind": "implement", - "capability_binding_ref": "issue-fix:feasibility-a1b2c3d4", - "continuation_policy": "independent_handoff", - "required_write_scopes": ["loopx/**"], - "blocks_agent": "codex-main-control", - }, - ) - todo_b = todo_event( - "evt-todo-b", - TODO_ADDED, - "todo_event_b", - { - "role": "agent", - "priority": "P0", - "title": "Render Markdown-compatible active-state sections", - "planner_order": 2, - "task_class": "advancement_task", - "action_kind": "validate", - }, - ) - user_gate = todo_event( - "evt-user-gate", - TODO_ADDED, - "todo_user_gate", - { - "role": "user", - "priority": "P1", - "title": "Approve promoting event projection to status reads", - "planner_order": 1, - "task_class": "user_gate", - }, - ) - store.append(todo_b) - appended_a = store.append(todo_a) - assert appended_a["append_sequence"] == 2, appended_a - assert store.append(todo_a)["append_sequence"] == 2 - store.append(user_gate) - store.append( - todo_event( - "evt-claim-a", - TODO_CLAIMED, - "todo_event_a", - {"claimed_by": "codex-product-capability"}, - ) - ) - store.append( - todo_event( - "evt-update-b", - TODO_UPDATED, - "todo_event_b", - {"title": "Render Markdown-compatible sections from projection"}, - ) - ) - store.append( - todo_event( - "evt-block-b", - TODO_BLOCKED, - "todo_event_b", - {"reason": "blocked pending projection review"}, - ) - ) - store.append( - todo_event( - "evt-complete-a", - TODO_COMPLETED, - "todo_event_a", - {"evidence": "event store smoke"}, - ) - ) - store.append( - make_state_event( - event_id="evt-refresh", - goal_id=GOAL_ID, - event_type=REFRESH_RECORDED, - payload={"summary": "projection rendered from append-only events"}, - recorded_at="2026-06-27T00:01:00Z", - ) - ) - - projection = build_state_projection(store.load(), generated_at="2026-06-27T00:02:00Z") - assert projection["schema_version"] == "event_sourced_state_projection_v0", projection - assert projection["source_event_count"] == 8, projection - assert projection["last_append_sequence"] == 8, projection - assert [item["todo_id"] for item in projection["agent_todos"]["items"]] == [ - "todo_event_a", - "todo_event_b", - ], projection - assert projection["agent_todos"]["items"][0]["status"] == "done", projection - assert ( - projection["agent_todos"]["items"][0]["continuation_policy"] - == "independent_handoff" - ), projection - assert projection["agent_todos"]["items"][0]["capability_binding_ref"] == ( - "issue-fix:feasibility-a1b2c3d4" - ), projection - assert projection["agent_todos"]["items"][0]["required_write_scopes"] == [ - "loopx/**" - ], projection - assert ( - projection["agent_todos"]["items"][0]["blocks_agent"] - == "codex-main-control" - ), projection - assert projection["agent_todos"]["items"][1]["status"] == "blocked", projection - assert projection["user_todos"]["first_open_items"][0]["todo_id"] == "todo_user_gate", projection - - markdown = render_active_state_sections(projection) - assert "## User Todo / Owner Review Reading Queue" in markdown, markdown - assert "## Agent Todo" in markdown, markdown - assert "todo_id=todo_event_a" in markdown, markdown - assert "todo_id=todo_user_gate" in markdown, markdown - assert "claimed_by=codex-product-capability" in markdown, markdown - assert ( - "capability_binding_ref=issue-fix:feasibility-a1b2c3d4" in markdown - ), markdown - assert "continuation_policy=independent_handoff" in markdown, markdown - assert "required_write_scopes=loopx%2F%2A%2A" in markdown, markdown - assert "blocks_agent=codex-main-control" in markdown, markdown - assert "## Progress Ledger" in markdown, markdown - - parsed = parse_active_state_todos(markdown) - assert parsed["agent_todos"]["total_count"] == 2, parsed - assert parsed["agent_todos"]["done_count"] == 1, parsed - parsed_agent_todos = { - item["todo_id"]: item for item in parsed["agent_todos"]["items"] - } - assert parsed_agent_todos["todo_event_a"]["capability_binding_ref"] == ( - "issue-fix:feasibility-a1b2c3d4" - ), parsed - assert parsed["user_todos"]["open_count"] == 1, parsed - - -def test_conflicts_and_mutations_fail_closed() -> None: - with tempfile.TemporaryDirectory(prefix="loopx-event-store-conflict-") as tmp: - store = AppendOnlyStateEventStore(Path(tmp) / "events.jsonl") - event = todo_event( - "evt-conflict", - TODO_ADDED, - "todo_conflict", - {"role": "agent", "priority": "P1", "title": "Stable event"}, - ) - store.append(event) - conflict = dict(event) - conflict["payload"] = dict(event["payload"]) - conflict["payload"]["title"] = "Different event body" - try: - store.append(conflict) - except StateEventConflictError: - pass - else: - raise AssertionError("conflicting duplicate event_id was accepted") - - mutation = todo_event( - "evt-mutation", - TODO_UPDATED, - "todo_conflict", - {"title": "Attempt mutation"}, - ) - mutation["refs"]["mutates_prior_event_id"] = "evt-conflict" - try: - store.append(mutation) - except StateEventError as exc: - assert "must not mutate prior events" in str(exc), exc - else: - raise AssertionError("prior event mutation was accepted") - - rebound = todo_event( - "evt-rebind", - TODO_UPDATED, - "todo_conflict", - {"capability_binding_ref": "issue-fix:feasibility-second"}, - ) - bound = dict(event) - bound["event_id"] = "evt-bound" - bound["append_sequence"] = 1 - bound["payload"] = { - **event["payload"], - "capability_binding_ref": "issue-fix:feasibility-first", - } - rebound["append_sequence"] = 2 - try: - build_state_projection([bound, rebound]) - except StateEventError as exc: - assert "capability_binding_ref is immutable once set" in str(exc), exc - else: - raise AssertionError("event replay accepted capability binding re-assignment") - - -def main() -> int: - test_store_projection_and_markdown() - test_conflicts_and_mutations_fail_closed() - print("event-sourced-state-api-smoke ok") - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/examples/control_plane/event-sourced-state-contract-smoke.py b/examples/control_plane/event-sourced-state-contract-smoke.py deleted file mode 100644 index 2b0410d705..0000000000 --- a/examples/control_plane/event-sourced-state-contract-smoke.py +++ /dev/null @@ -1,255 +0,0 @@ -#!/usr/bin/env python3 -"""Smoke-test event-sourced todo/history contract invariants.""" - -from __future__ import annotations - -from dataclasses import dataclass, field -from pathlib import Path -from typing import Any - - -REPO_ROOT = Path(__file__).resolve().parents[2] -CONTRACT = REPO_ROOT / "docs/reference/protocols/event-sourced-state-contract-v0.md" -PROTOCOL_INDEX = REPO_ROOT / "docs/reference/protocols/README.md" -STATE_MODEL = REPO_ROOT / "docs/state-interaction-model.md" - - -PUBLIC_FORBIDDEN = ( - "/Users/", - "/home/", - "/private/tmp/", - "Bearer ", - "AKIA", -) - - -@dataclass -class EventStore: - events_by_id: dict[str, dict[str, Any]] = field(default_factory=dict) - events: list[dict[str, Any]] = field(default_factory=list) - - def append(self, event: dict[str, Any]) -> None: - event_id = event["event_id"] - existing = self.events_by_id.get(event_id) - normalized = dict(sorted(event.items())) - if existing is not None: - if dict(sorted(existing.items())) != normalized: - raise ValueError(f"conflicting event_id: {event_id}") - return - if event.get("event_type") == "event_mutated": - raise ValueError("events must not mutate prior events") - if "mutates_prior_event_id" in event.get("refs", {}): - raise ValueError("events must not mutate prior events") - self.events_by_id[event_id] = normalized - self.events.append(normalized) - - def replay_todos(self) -> dict[str, Any]: - todos: dict[str, dict[str, Any]] = {} - completed: list[str] = [] - for event in sorted( - self.events, - key=lambda item: ( - item["append_sequence"], - item["recorded_at"], - item["event_id"], - ), - ): - event_type = event["event_type"] - todo_id = event.get("refs", {}).get("todo_id") - if event_type == "todo_added": - todos[todo_id] = { - "todo_id": todo_id, - "priority": event["payload"]["priority"], - "role": event["payload"]["role"], - "title": event["payload"]["title"], - "planner_order": event["payload"].get("planner_order"), - "append_sequence": event["append_sequence"], - "status": "open", - } - elif event_type == "todo_claimed": - todos[todo_id]["claimed_by"] = event["payload"]["claimed_by"] - elif event_type == "todo_blocked": - todos[todo_id]["status"] = "blocked" - todos[todo_id]["blocker"] = event["payload"]["reason"] - elif event_type == "todo_completed": - todos[todo_id]["status"] = "done" - completed.append(todo_id) - - open_todos = [ - todo for todo in todos.values() if todo.get("status") in {"open", "blocked"} - ] - open_todos.sort( - key=lambda todo: ( - todo["priority"], - todo.get("planner_order") or 9999, - todo["append_sequence"], - ) - ) - return { - "schema_version": "event_sourced_state_projection_v0", - "goal_id": "loopx-meta", - "source_event_count": len(self.events), - "last_event_id": self.events[-1]["event_id"], - "last_append_sequence": self.events[-1]["append_sequence"], - "projection_version": "event_sourced_state_contract_v0", - "open_todos": open_todos, - "completed_todo_ids": completed, - } - - -def read(path: Path) -> str: - return path.read_text(encoding="utf-8") - - -def test_contract_text() -> None: - contract = read(CONTRACT) - compact = " ".join(contract.split()) - for required in [ - "# event_sourced_state_contract_v0", - "ACTIVE_GOAL_STATE.md` remains the human/agent workbench", - "Markdown edits are not canonical state changes", - "append_sequence` is the final same-priority tie-breaker", - "public_safe", - "local_private", - "private_pointer", - "last_append_sequence", - "projection_version", - "raw chat transcripts", - "raw logs", - "duplicate conflicting `event_id` append fails closed", - "Tracked outputs require explicit redaction or compact pointers", - ]: - assert required in contract, required - - for event_type in [ - "todo_added", - "todo_claimed", - "todo_updated", - "todo_blocked", - "todo_deferred", - "todo_completed", - "gate_added", - "gate_resolved", - "run_recorded", - "refresh_recorded", - "quota_spent", - "evidence_attached", - "projection_rendered", - "snapshot_compacted", - ]: - assert f"`{event_type}`" in contract, event_type - - for forbidden in PUBLIC_FORBIDDEN: - assert forbidden not in compact, forbidden - - -def test_docs_are_linked() -> None: - index = read(PROTOCOL_INDEX) - state_model = read(STATE_MODEL) - assert "event_sourced_state_contract_v0" in index, index - assert "event-sourced-state-contract-v0.md" in index, index - assert "event_sourced_state_contract_v0" in state_model, state_model - assert "ACTIVE_GOAL_STATE.md` stays the human/agent workbench" in state_model - - -def event( - event_id: str, - event_type: str, - append_sequence: int, - todo_id: str, - payload: dict[str, Any], -) -> dict[str, Any]: - return { - "schema_version": "loopx_state_event_v0", - "event_id": event_id, - "goal_id": "loopx-meta", - "event_type": event_type, - "recorded_at": f"2026-06-27T00:00:{append_sequence:02d}Z", - "append_sequence": append_sequence, - "producer": "loopx.todo", - "privacy": "public_safe", - "projection_version": "event_sourced_state_contract_v0", - "refs": {"todo_id": todo_id}, - "payload": payload, - } - - -def test_replay_order_and_idempotency() -> None: - store = EventStore() - todo_a = event( - "evt-a", - "todo_added", - 1, - "todo_a", - { - "role": "agent", - "priority": "P0", - "title": "Define event source contract", - "planner_order": 1, - }, - ) - todo_b = event( - "evt-b", - "todo_added", - 2, - "todo_b", - { - "role": "agent", - "priority": "P0", - "title": "Implement event store", - "planner_order": 2, - }, - ) - store.append(todo_b) - store.append(todo_a) - store.append(todo_a) - store.append( - event( - "evt-claim-a", - "todo_claimed", - 3, - "todo_a", - {"claimed_by": "codex-product-capability"}, - ) - ) - - projection = store.replay_todos() - assert [todo["todo_id"] for todo in projection["open_todos"]] == [ - "todo_a", - "todo_b", - ], projection - assert projection["source_event_count"] == 3, projection - assert projection["last_event_id"] == "evt-claim-a", projection - assert projection["last_append_sequence"] == 3, projection - assert projection["projection_version"] == "event_sourced_state_contract_v0" - - conflicting = dict(todo_a) - conflicting["payload"] = dict(todo_a["payload"]) - conflicting["payload"]["title"] = "Rewrite prior event" - try: - store.append(conflicting) - except ValueError as exc: - assert "conflicting event_id" in str(exc), exc - else: - raise AssertionError("conflicting duplicate event id was accepted") - - mutation = event("evt-mutate", "todo_updated", 4, "todo_a", {"title": "mutate"}) - mutation["refs"]["mutates_prior_event_id"] = "evt-a" - try: - store.append(mutation) - except ValueError as exc: - assert "must not mutate prior events" in str(exc), exc - else: - raise AssertionError("prior-event mutation was accepted") - - -def main() -> int: - test_contract_text() - test_docs_are_linked() - test_replay_order_and_idempotency() - print("event-sourced-state-contract-smoke ok") - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/examples/control_plane/event-sourced-status-read-path-smoke.py b/examples/control_plane/event-sourced-status-read-path-smoke.py deleted file mode 100644 index e44c670717..0000000000 --- a/examples/control_plane/event-sourced-status-read-path-smoke.py +++ /dev/null @@ -1,192 +0,0 @@ -#!/usr/bin/env python3 -"""Validate that status todo reads prefer event projection with Markdown fallback.""" - -from __future__ import annotations - -import sys -import tempfile -from pathlib import Path - - -REPO_ROOT = Path(__file__).resolve().parents[2] -if str(REPO_ROOT) not in sys.path: - sys.path.insert(0, str(REPO_ROOT)) - -from loopx.event_sourced_state import ( # noqa: E402 - AppendOnlyStateEventStore, - TODO_ADDED, - TODO_CLAIMED, - make_state_event, -) -from loopx.control_plane.goals.active_state_event_projection import ( # noqa: E402 - active_state_event_projection_fields as active_state_event_projection_fields_read_model, -) -from loopx.control_plane.todos import active_state_todos as active_state_todos_read_model # noqa: E402 -from loopx.status import active_state_todo_fields # noqa: E402 -from loopx import status as status_module # noqa: E402 - - -GOAL_ID = "event-sourced-status-read-fixture" - - -def write_active_state(state_path: Path) -> None: - state_path.parent.mkdir(parents=True, exist_ok=True) - state_path.write_text( - "\n".join( - [ - "# Fixture State", - "", - "## Next Action", - "", - "- Keep the current status read path stable.", - "", - "## Agent Todo", - "", - "- [ ] [P2] Stale Markdown todo that should lose to events.", - " ", - "", - ] - ), - encoding="utf-8", - ) - - -def append_event_todos(event_log: Path) -> None: - store = AppendOnlyStateEventStore(event_log) - store.append( - make_state_event( - event_id="evt-add-agent-status-read", - goal_id=GOAL_ID, - event_type=TODO_ADDED, - refs={"todo_id": "todo_event_status_read"}, - payload={ - "role": "agent", - "priority": "P0", - "title": "Prefer event projection for status todo reads", - "planner_order": 1, - "task_class": "advancement_task", - "action_kind": "event_projection_read_path", - }, - producer="event-sourced-status-read-path-smoke", - recorded_at="2026-06-27T00:00:01Z", - ) - ) - store.append( - make_state_event( - event_id="evt-claim-agent-status-read", - goal_id=GOAL_ID, - event_type=TODO_CLAIMED, - refs={"todo_id": "todo_event_status_read"}, - payload={"claimed_by": "codex-product-capability"}, - producer="event-sourced-status-read-path-smoke", - recorded_at="2026-06-27T00:00:02Z", - ) - ) - - -def event_todo_ids(fields: dict) -> list[str]: - agent_todos = fields.get("agent_todos") or {} - return [str(item.get("todo_id") or "") for item in agent_todos.get("items") or []] - - -def direct_active_state_todo_fields(goal: dict, *, runtime_root: Path | None = None) -> dict: - return active_state_todos_read_model.active_state_todo_fields( - goal, - runtime_root=runtime_root, - resolve_goal_local_path=status_module.resolve_goal_local_path, - active_state_next_action_entries=status_module.active_state_next_action_entries, - active_next_action_todo_ids=status_module.active_next_action_todo_ids, - load_rollout_events=status_module.load_rollout_events, - rollout_event_log_path=status_module.rollout_event_log_path, - max_todo_index_rollout_events_per_goal=status_module.MAX_TODO_INDEX_ROLLOUT_EVENTS_PER_GOAL, - active_state_event_projection_fields=status_module.active_state_event_projection_fields, - parse_active_state_todos=status_module.parse_active_state_todos, - parse_issue_meta_surface=status_module.parse_issue_meta_surface, - backlog_hygiene_warning=status_module.backlog_hygiene_warning, - completed_todo_archive_warning=status_module.completed_todo_archive_warning, - state_projection_gap_warning=status_module.state_projection_gap_warning, - ) - - -def direct_active_state_event_projection_fields(goal: dict, *, state_path: Path) -> dict: - return active_state_event_projection_fields_read_model( - goal, - state_path=state_path, - resolve_goal_local_path=status_module.resolve_goal_local_path, - parse_active_state_todos=status_module.parse_active_state_todos, - item_limit=status_module.MAX_STATUS_TODOS_PER_ROLE, - event_log_basename=status_module.STATE_EVENT_LOG_BASENAME, - ) - - -def test_event_projection_preferred() -> None: - with tempfile.TemporaryDirectory(prefix="loopx-event-status-") as tmp: - project = Path(tmp) - state_path = project / ".codex" / "goals" / GOAL_ID / "ACTIVE_GOAL_STATE.md" - write_active_state(state_path) - append_event_todos(state_path.with_name("events.jsonl")) - - goal = { - "id": GOAL_ID, - "repo": str(project), - "state_file": f".codex/goals/{GOAL_ID}/ACTIVE_GOAL_STATE.md", - } - projection_fields = status_module.active_state_event_projection_fields(goal, state_path=state_path) - assert projection_fields == direct_active_state_event_projection_fields(goal, state_path=state_path), ( - projection_fields - ) - fields = active_state_todo_fields(goal) - assert fields == direct_active_state_todo_fields(goal), fields - assert fields["state_event_projection"]["source"] == "event_log", fields - assert fields["state_event_projection"]["last_append_sequence"] == 2, fields - assert event_todo_ids(fields) == ["todo_event_status_read"], fields - assert "todo_markdown_stale" not in event_todo_ids(fields), fields - assert fields["agent_todos"]["items"][0]["claimed_by"] == "codex-product-capability", fields - - -def test_markdown_fallback_without_valid_event_log() -> None: - with tempfile.TemporaryDirectory(prefix="loopx-event-status-fallback-") as tmp: - project = Path(tmp) - state_path = project / ".codex" / "goals" / GOAL_ID / "ACTIVE_GOAL_STATE.md" - write_active_state(state_path) - goal = { - "id": GOAL_ID, - "repo": str(project), - "state_file": f".codex/goals/{GOAL_ID}/ACTIVE_GOAL_STATE.md", - } - - projection_fields = status_module.active_state_event_projection_fields(goal, state_path=state_path) - assert projection_fields == direct_active_state_event_projection_fields(goal, state_path=state_path), ( - projection_fields - ) - fields = active_state_todo_fields(goal) - assert fields == direct_active_state_todo_fields(goal), fields - assert event_todo_ids(fields) == ["todo_markdown_stale"], fields - assert "state_event_projection" not in fields, fields - - state_path.with_name("events.jsonl").write_text("{not json\n", encoding="utf-8") - corrupted_projection_fields = status_module.active_state_event_projection_fields( - goal, - state_path=state_path, - ) - assert corrupted_projection_fields == direct_active_state_event_projection_fields( - goal, - state_path=state_path, - ), corrupted_projection_fields - corrupted_fields = active_state_todo_fields(goal) - assert corrupted_fields == direct_active_state_todo_fields(goal), corrupted_fields - assert event_todo_ids(corrupted_fields) == ["todo_markdown_stale"], corrupted_fields - assert corrupted_fields["state_event_projection_warning"]["fallback"] == "markdown_active_state", ( - corrupted_fields - ) - - -def main() -> int: - test_event_projection_preferred() - test_markdown_fallback_without_valid_event_log() - print("event-sourced-status-read-path-smoke ok") - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/examples/control_plane/event-store-migration-bridge-smoke.py b/examples/control_plane/event-store-migration-bridge-smoke.py deleted file mode 100644 index 493805187d..0000000000 --- a/examples/control_plane/event-store-migration-bridge-smoke.py +++ /dev/null @@ -1,44 +0,0 @@ -#!/usr/bin/env python3 -"""Smoke-test the fail-closed event-store promotion boundary.""" - -from __future__ import annotations - -from pathlib import Path -import sys - - -REPO_ROOT = Path(__file__).resolve().parents[2] -if str(REPO_ROOT) not in sys.path: - sys.path.insert(0, str(REPO_ROOT)) - -from loopx.control_plane.runtime.event_store_migration_bridge import ( # noqa: E402 - EVENT_PROJECTION_SOURCE, - MARKDOWN_ACTIVE_STATE_SOURCE, - build_event_store_migration_bridge, -) - - -def main() -> int: - bridge = build_event_store_migration_bridge( - goal_id="event-store-migration-bridge-smoke", - event_read_path_ready=True, - active_state_projection_ready=True, - dual_read_parity_clean=True, - event_projection_head_matches_store=True, - rollback_plan_recorded=True, - idempotency_conflicts_clean=True, - public_boundary_clean=True, - bounded_canary_passed=True, - ) - assert bridge["source_of_truth"] == MARKDOWN_ACTIVE_STATE_SOURCE, bridge - assert bridge["candidate_source"] == EVENT_PROJECTION_SOURCE, bridge - assert bridge["stage"] == "promotion_candidate", bridge - assert bridge["promotion_candidate"] is True, bridge - assert bridge["promotion_allowed"] is False, bridge - assert bridge["rollback"]["fallback_source"] == MARKDOWN_ACTIVE_STATE_SOURCE, bridge - print("event-store-migration-bridge-smoke ok") - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/examples/control_plane/task-graph-projection-fixture-smoke.py b/examples/control_plane/task-graph-projection-fixture-smoke.py index f94df4388f..5e1cd7097b 100644 --- a/examples/control_plane/task-graph-projection-fixture-smoke.py +++ b/examples/control_plane/task-graph-projection-fixture-smoke.py @@ -17,12 +17,7 @@ from loopx.control_plane.todos.handoff_note import ( # noqa: E402 attach_todo_handoff_note, ) -from loopx.event_sourced_state import ( # noqa: E402 - TODO_ADDED, - TODO_UPDATED, - build_state_projection, - make_state_event, -) + from loopx.review_packet import build_review_packet # noqa: E402 from loopx.status import build_task_graph_projection # noqa: E402 @@ -365,48 +360,11 @@ def assert_predecessor_budget_and_actor_contract() -> None: assert source_limits["predecessor_truncated"] is False, source_limits assert source_limits["source_truncated"] is True, source_limits - def state_event( - event_id: str, - event_type: str, - payload: dict[str, object], - *, - actor_agent_id: str | None = None, - ) -> dict[str, object]: - return make_state_event( - event_id=event_id, - goal_id="task-graph-actor-clearing", - event_type=event_type, - refs={"todo_id": "todo_actor_audit"}, - payload=payload, - actor_agent_id=actor_agent_id, - recorded_at=f"2026-08-06T00:00:{len(event_id):02d}Z", - ) - - actor_projection = build_state_projection( - [ - state_event( - "evt-add-actor", - TODO_ADDED, - {"role": "agent", "text": "Audit actor projection"}, - actor_agent_id="creator-agent", - ), - state_event( - "evt-update-actor", - TODO_UPDATED, - {"title": "Audit actor projection after mutation"}, - actor_agent_id="mutator-agent", - ), - state_event( - "evt-update-no-actor", - TODO_UPDATED, - {"title": "Audit actor projection after cleared actor"}, - ), - ], - goal_id="task-graph-actor-clearing", - ) - actor_todo = actor_projection["agent_todos"]["items"][0] - assert actor_todo["created_by"] == "creator-agent", actor_todo - assert "last_actor_agent_id" not in actor_todo, actor_todo + actor_todo = {"todo_id": "todo_actor_audit", "role": "agent", "status": "open", + "title": "Audit actor projection", "text": "Audit actor projection", + "created_by": "creator-agent", "task_class": "advancement_task"} + actor_projection = {"agent_todos": {"items": [actor_todo], "total_count": 1, "open_count": 1}, + "user_todos": {"items": [], "total_count": 0, "open_count": 0}} actor_graph = build_task_graph_projection( { "goal_id": "task-graph-actor-clearing", diff --git a/examples/control_plane/todo-list-event-projection-smoke.py b/examples/control_plane/todo-list-event-projection-smoke.py deleted file mode 100644 index 6f9391259f..0000000000 --- a/examples/control_plane/todo-list-event-projection-smoke.py +++ /dev/null @@ -1,364 +0,0 @@ -#!/usr/bin/env python3 -"""Smoke-test `loopx todo list` event projection with Markdown late-todo overlay.""" - -from __future__ import annotations - -import sys -import tempfile -from pathlib import Path - - -REPO_ROOT = Path(__file__).resolve().parents[2] -if str(REPO_ROOT) not in sys.path: - sys.path.insert(0, str(REPO_ROOT)) - -from loopx.control_plane.testing.canary_harness import ( # noqa: E402 - run_json_cli, - write_fixture_registry, -) -from loopx.event_sourced_state import ( # noqa: E402 - AppendOnlyStateEventStore, - TODO_ADDED, - TODO_COMPLETED, - make_state_event, -) - - -GOAL_ID = "todo-list-event-projection-fixture" -PRIMARY_AGENT = "codex-main-control" -SIDE_AGENT = "codex-side-bypass" -GATE_TODO_ID = "todo_markdown_gate_done" -DEPENDENT_TODO_ID = "todo_event_dependent" -EVENT_GATE_TODO_ID = "todo_event_non_delivery_gate" -EVENT_SUCCESSOR_TODO_ID = "todo_event_same_agent_successor" -EVENT_ATOMIC_SOURCE_TODO_ID = "todo_event_atomic_handoff_source" -EVENT_ATOMIC_SUCCESSOR_TEXT = "Independently review the event-projected delivery" -SUCCESSOR_REPOSITORY = "git:github.com/huangruiteng/loopx" -SUCCESSOR_CAPABILITIES = ["network", "external_evidence_poll"] - - -def write_fixture(root: Path) -> tuple[Path, Path, Path]: - project = root / "project" - runtime = root / "runtime" - state_file = project / ".codex" / "goals" / GOAL_ID / "ACTIVE_GOAL_STATE.md" - event_log = state_file.with_name("events.jsonl") - registry_path = project / ".loopx" / "registry.json" - state_file.parent.mkdir(parents=True) - state_file.write_text( - "---\n" - "status: active\n" - "updated_at: 2026-01-01T00:00:00+00:00\n" - "---\n\n" - "# Active Goal State\n\n" - "## User Todo\n\n" - "- [x] [P0] Completed Markdown gate\n" - f" \n\n" - "## Agent Todo\n\n" - "- [ ] [P1] Markdown fallback todo\n" - " \n" - "- [-] [P1] Continue after the Markdown gate\n" - f" \n", - encoding="utf-8", - ) - write_fixture_registry( - project=project, - runtime_root=runtime, - registry_path=registry_path, - goal_id=GOAL_ID, - domain="todo-list-fixture", - adapter_kind="generic_project_goal_v0", - state_event_log=f".codex/goals/{GOAL_ID}/events.jsonl", - registered_agents=(PRIMARY_AGENT, SIDE_AGENT), - quota_allowed_slots=None, - extra_goal_fields={ - "coordination": { - "registered_agents": [PRIMARY_AGENT, SIDE_AGENT], - "agent_model": "peer_v1", - "side_agent_handoff_agent": SIDE_AGENT, - } - }, - ) - return registry_path, state_file, event_log - - -def event(event_id: str, event_type: str, todo_id: str, payload: dict) -> dict: - return make_state_event( - event_id=event_id, - goal_id=GOAL_ID, - event_type=event_type, - refs={"todo_id": todo_id}, - payload=payload, - recorded_at=f"2026-06-27T00:00:{len(event_id):02d}Z", - producer="todo-list-event-projection-smoke", - ) - - -def run_cli(registry_path: Path, *args: str) -> dict: - return run_json_cli( - *args, - registry_path=registry_path, - include_returncode=False, - ) - - -def write_events(event_log: Path) -> None: - store = AppendOnlyStateEventStore(event_log) - store.append( - event( - "evt-open", - TODO_ADDED, - "todo_event_open", - { - "role": "agent", - "priority": "P0", - "title": "Projected open todo", - "planner_order": 1, - "task_class": "advancement_task", - "action_kind": "implement", - "continuation_policy": "independent_handoff", - }, - ) - ) - store.append( - event( - "evt-done", - TODO_ADDED, - "todo_event_done", - { - "role": "agent", - "priority": "P1", - "title": "Projected completed todo", - "planner_order": 2, - "task_class": "advancement_task", - }, - ) - ) - store.append( - event( - "evt-complete", - TODO_COMPLETED, - "todo_event_done", - {"evidence": "projection smoke"}, - ) - ) - - -def main() -> int: - with tempfile.TemporaryDirectory(prefix="loopx-todo-list-projection-") as tmp: - registry_path, _, event_log = write_fixture(Path(tmp)) - write_events(event_log) - - projected = run_cli(registry_path, "todo", "list", "--goal-id", GOAL_ID, "--role", "agent") - assert projected["ok"] is True, projected - assert projected["read_only"] is True, projected - assert projected["source"] == "event_projection_with_markdown_overlay", projected - assert projected["todo_count"] == 4, projected - assert [item["todo_id"] for item in projected["todos"]] == [ - "todo_event_open", - "todo_markdown", - DEPENDENT_TODO_ID, - "todo_event_done", - ], projected - dependent = next( - item for item in projected["todos"] if item["todo_id"] == DEPENDENT_TODO_ID - ) - assert dependent["resume_ready"] is True, dependent - assert dependent["resume_condition"]["target_status"] == "done", dependent - projected_open = next( - item for item in projected["todos"] if item["todo_id"] == "todo_event_open" - ) - assert projected_open["continuation_policy"] == "independent_handoff", projected_open - assert projected["projection_overlay"]["markdown_only_todo_ids"] == [ - GATE_TODO_ID, - DEPENDENT_TODO_ID, - "todo_markdown", - ], projected - assert projected["projection_overlay"]["event_only_todo_ids"] == [ - "todo_event_open", - "todo_event_done", - ], projected - assert projected["state_event_projection"]["source_event_count"] == 3, projected - - dependent_only = run_cli( - registry_path, - "todo", - "list", - "--goal-id", - GOAL_ID, - "--todo-id", - DEPENDENT_TODO_ID, - ) - assert dependent_only["todo_count"] == 1, dependent_only - assert dependent_only["relations"]["resume_ready"] is True, dependent_only - assert ( - dependent_only["relations"]["resume_condition"]["target_status"] == "done" - ), dependent_only - - done_only = run_cli( - registry_path, - "todo", - "list", - "--goal-id", - GOAL_ID, - "--role", - "agent", - "--status", - "done", - ) - assert done_only["source"] == "event_projection_with_markdown_overlay", done_only - assert done_only["todo_count"] == 1, done_only - assert done_only["todos"][0]["todo_id"] == "todo_event_done", done_only - - store = AppendOnlyStateEventStore(event_log) - store.append( - event( - "evt-non-delivery-gate", - TODO_ADDED, - EVENT_GATE_TODO_ID, - { - "role": "agent", - "priority": "P1", - "title": "Complete an event-projected readiness gate", - "task_class": "advancement_task", - "action_kind": "readiness_check", - "continuation_policy": "same_agent_non_delivery", - "claimed_by": SIDE_AGENT, - "blocks_agent": PRIMARY_AGENT, - }, - ) - ) - store.append( - event( - "evt-same-agent-successor", - TODO_ADDED, - EVENT_SUCCESSOR_TODO_ID, - { - "role": "agent", - "priority": "P1", - "title": "Continue the event-projected lane", - "task_class": "advancement_task", - "action_kind": "continue_lane", - "continuation_policy": "independent_handoff", - "claimed_by": SIDE_AGENT, - }, - ) - ) - store.append( - event( - "evt-atomic-handoff-source", - TODO_ADDED, - EVENT_ATOMIC_SOURCE_TODO_ID, - { - "role": "agent", - "priority": "P1", - "title": "Deliver an event-projected implementation", - "task_class": "advancement_task", - "action_kind": "implement", - "claimed_by": SIDE_AGENT, - }, - ) - ) - completed = run_cli( - registry_path, - "todo", - "complete", - "--goal-id", - GOAL_ID, - "--role", - "agent", - "--todo-id", - EVENT_GATE_TODO_ID, - "--agent-id", - SIDE_AGENT, - "--claimed-by", - SIDE_AGENT, - "--evidence", - "public-safe event-projected readiness evidence", - "--successor-todo-id", - EVENT_SUCCESSOR_TODO_ID, - ) - assert completed["linked_successor_id"] == EVENT_SUCCESSOR_TODO_ID, completed - assert completed["self_merged"] is False, completed - - atomic_handoff = run_cli( - registry_path, - "todo", - "complete", - "--goal-id", - GOAL_ID, - "--role", - "agent", - "--todo-id", - EVENT_ATOMIC_SOURCE_TODO_ID, - "--agent-id", - SIDE_AGENT, - "--claimed-by", - SIDE_AGENT, - "--evidence", - "public-safe event-projected implementation evidence", - "--next-agent-todo", - EVENT_ATOMIC_SUCCESSOR_TEXT, - "--next-claimed-by", - PRIMARY_AGENT, - "--next-action-kind", - "review", - "--next-task-repository", - SUCCESSOR_REPOSITORY, - "--next-required-capability", - "network", - "--next-required-capability", - "external_evidence_poll", - "--next-excluded-agent", - SIDE_AGENT, - ) - atomic_successor = atomic_handoff["next_todos"][0] - assert atomic_successor["claimed_by"] == PRIMARY_AGENT, atomic_handoff - assert atomic_successor["excluded_agents"] == [SIDE_AGENT], atomic_handoff - assert atomic_successor["task_repository"] == SUCCESSOR_REPOSITORY, atomic_handoff - assert atomic_successor["required_capabilities"] == SUCCESSOR_CAPABILITIES, ( - atomic_handoff - ) - projected_atomic = run_cli( - registry_path, - "todo", - "list", - "--goal-id", - GOAL_ID, - "--todo-id", - atomic_successor["todo_id"], - ) - atomic_item = projected_atomic["todos"][0] - assert atomic_item["claimed_by"] == PRIMARY_AGENT, atomic_item - assert atomic_item["excluded_agents"] == [SIDE_AGENT], atomic_item - assert atomic_item["task_repository"] == SUCCESSOR_REPOSITORY, atomic_item - assert atomic_item["required_capabilities"] == SUCCESSOR_CAPABILITIES, atomic_item - - event_log.unlink() - fallback = run_cli(registry_path, "todo", "list", "--goal-id", GOAL_ID, "--role", "agent") - assert fallback["source"] == "markdown_active_state", fallback - assert fallback["todo_count"] == 2, fallback - assert [item["todo_id"] for item in fallback["todos"]] == [ - "todo_markdown", - DEPENDENT_TODO_ID, - ], fallback - fallback_dependent = run_cli( - registry_path, - "todo", - "list", - "--goal-id", - GOAL_ID, - "--todo-id", - DEPENDENT_TODO_ID, - ) - assert fallback_dependent["relations"]["resume_ready"] is True, fallback_dependent - assert ( - fallback_dependent["relations"]["resume_condition"]["target_status"] == "done" - ), fallback_dependent - - print("todo-list-event-projection-smoke ok") - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/loopx/canary/planner.py b/loopx/canary/planner.py index 5011b2f53f..fd2d4fd48c 100644 --- a/loopx/canary/planner.py +++ b/loopx/canary/planner.py @@ -314,7 +314,6 @@ "loopx/control_plane/work_items/delivery_history.py", "loopx/control_plane/work_items/delivery_history.ts", "loopx/control_plane/work_items/work_lane.py", - "loopx/control_plane/runtime/event_store_migration_bridge.py", ), "checks": [ { @@ -520,7 +519,7 @@ { "command": "python3 examples/control_plane/control-plane-integrated-canary-smoke.py", "tier": "deep", - "reason": "samples the bounded status -> quota -> review-packet event read path", + "reason": "samples the bounded status -> quota -> review-packet Todo read path", }, { "command": "python3 examples/control_plane/hot-path-interface-budget-smoke.py", @@ -529,51 +528,6 @@ }, ], }, - { - "id": "event-sourced-read-path", - "title": "Event-sourced read-path contract", - "purpose": "Check event projection, status read path, downstream read surfaces, and migration gates before event-store/read-path changes ship.", - "catalog_families": ["Work Routing", "State And Boundary", "Planning Governance"], - "trigger_hints": ( - "event-sourced", - "event sourced", - "event projection", - "event read-path", - "downstream read", - "event-store", - "event store", - "loopx/event_sourced_state.py", - "loopx/rollout_event_log.py", - "docs/reference/protocols/event-store-migration-bridge-v0.md", - ), - "checks": [ - { - "command": "python3 examples/control_plane/event-sourced-state-api-smoke.py", - "tier": "default", - "reason": "guards event append/replay API behavior used by read-path projections", - }, - { - "command": "python3 examples/control_plane/event-sourced-status-read-path-smoke.py", - "tier": "default", - "reason": "checks status consumption of event projection with Markdown fallback", - }, - { - "command": "python3 examples/control_plane/event-sourced-downstream-read-path-smoke.py", - "tier": "default", - "reason": "checks downstream read surfaces consume event projection without private state", - }, - { - "command": "python3 examples/control_plane/event-store-migration-bridge-smoke.py", - "tier": "deep", - "reason": "samples the migration bridge gates before bounded event read-path canaries", - }, - { - "command": "python3 examples/control_plane/event-sourced-replay-compaction-smoke.py", - "tier": "deep", - "reason": "checks replay compaction when broader event-store changes are promoted", - }, - ], - }, { "id": "cli-command-contract", "title": "CLI command module contract", @@ -638,11 +592,6 @@ "tier": "default", "reason": "guards deferred writes, fail-closed resume kinds, and runtime capacity resume routing", }, - { - "command": "python3 examples/control_plane/todo-list-event-projection-smoke.py", - "tier": "default", - "reason": "guards event-sourced todo list projection with Markdown fallback", - }, { "command": "python3 examples/control_plane/todo-concurrent-write-lock-smoke.py", "tier": "deep", diff --git a/loopx/canary/qualification_profiles.py b/loopx/canary/qualification_profiles.py index 764284f87a..50a9ad641d 100644 --- a/loopx/canary/qualification_profiles.py +++ b/loopx/canary/qualification_profiles.py @@ -109,7 +109,6 @@ "loopx/control_plane/work_items/delivery_history.py", "loopx/control_plane/work_items/delivery_history.ts", "loopx/control_plane/work_items/work_lane.py", - "loopx/control_plane/runtime/event_store_migration_bridge.py", "control-plane-integrated-canary-smoke.py", "interaction-contract-state-machine-smoke.py", "interaction-scheduler-authority-smoke.py", @@ -120,7 +119,7 @@ "command": "python3 examples/control_plane/control-plane-integrated-canary-smoke.py", "tier": "deep", "reason": ( - "samples the full event-sourced todo projection, status, quota interaction contract, " + "samples the Markdown Todo projection, status, quota interaction contract, " "work-lane contract, scheduler ack, refresh-state, spend-slot, and review-packet handoff path; " "kept deep because it is a slow end-to-end fixture" ), diff --git a/loopx/capabilities/manager_context/tracking.py b/loopx/capabilities/manager_context/tracking.py index ef282cabd1..004f82908b 100644 --- a/loopx/capabilities/manager_context/tracking.py +++ b/loopx/capabilities/manager_context/tracking.py @@ -20,7 +20,7 @@ def _core_todos(registry_path, root, goal_id): result = list_goal_todos( registry_path=registry_path, runtime_root_arg=str(root), goal_id=goal_id ) - if result.get("ok") is not True or result.get("state_event_projection_warning"): + if result.get("ok") is not True: raise ValueError("Core Todo authority unavailable") return {r["todo_id"]: r for r in result.get("todos", []) if r.get("todo_id")} diff --git a/loopx/chat_manager_details.py b/loopx/chat_manager_details.py index 0a0fe12e6a..6e4546516f 100644 --- a/loopx/chat_manager_details.py +++ b/loopx/chat_manager_details.py @@ -28,7 +28,7 @@ def read_manager_goal_details( registry_path=registry_path, runtime_root_arg=str(runtime_root), goal_id=goal_id, ) - if result.get("ok") is not True or result.get("state_event_projection_warning"): + if result.get("ok") is not True: raise ValueError("Todo authority unavailable or conflicting") records = result.get("todos", []) active = [r for r in records if r.get("status") in {"open", "blocked", "deferred"}] diff --git a/loopx/control_plane/agents/supervisor_event_append.ts b/loopx/control_plane/agents/supervisor_event_append.ts new file mode 100644 index 0000000000..87da241e37 --- /dev/null +++ b/loopx/control_plane/agents/supervisor_event_append.ts @@ -0,0 +1,37 @@ +/** Admission for one experimental supervisor log append under its file lock. */ +import type {JsonObject} from "../effect_program.ts"; +import {requireJsonObject, requireNonEmptyString} from "../runtime_decode.ts"; +import {EffectRuntimeRequestError} from "../effect_runtime_errors.ts"; + +function sequence(value: unknown, minimum: number): number { + if (typeof value !== "number" || !Number.isSafeInteger(value) || value < minimum) { + throw new EffectRuntimeRequestError("supervisor sequence must be a safe integer"); + } + return value; +} +function identity(value: unknown) { + const row = requireJsonObject(value, "supervisor event identity"); + const fingerprint = requireNonEmptyString(row.fingerprint, "event fingerprint"); + if (!/^[a-f0-9]{64}$/.test(fingerprint)) throw new EffectRuntimeRequestError("invalid event fingerprint"); + return {event_id: requireNonEmptyString(row.event_id, "event_id"), fingerprint}; +} +export function planSupervisorEventAppend(value: unknown): JsonObject { + const request = requireJsonObject(value, "supervisor append plan"); + if (request.schema_version !== "loopx_supervisor_event_append_plan_v0") { + throw new EffectRuntimeRequestError("invalid supervisor append schema"); + } + const last = sequence(request.last_sequence, 0), event = identity(request.event); + if (request.existing !== null) { + const row = requireJsonObject(request.existing, "stored supervisor identity"); + const prior = identity(row), index = sequence(row.append_sequence, 1); + if (index > last || prior.event_id !== event.event_id) { + throw new EffectRuntimeRequestError("inconsistent stored supervisor identity"); + } + return prior.fingerprint === event.fingerprint + ? {status: "planned", kind: "replay", append_sequence: index} + : {status: "rejected", reason_code: "event_id_conflict"}; + } + return last === Number.MAX_SAFE_INTEGER + ? {status: "rejected", reason_code: "event_sequence_exhausted"} + : {status: "planned", kind: "append", append_sequence: last + 1}; +} diff --git a/loopx/control_plane/agents/supervisor_event_log.py b/loopx/control_plane/agents/supervisor_event_log.py new file mode 100644 index 0000000000..e7c276a3bf --- /dev/null +++ b/loopx/control_plane/agents/supervisor_event_log.py @@ -0,0 +1,126 @@ +"""Private experimental supervisor log, independent of Todo state authority. + +Python owns locked JSONL publication using the existing durable file primitives. +TypeScript owns append identity and sequence admission. Only proposals and host +receipts are records here; there is no replay into Goal state or Markdown. +""" +from __future__ import annotations + +import hashlib +import json +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +from ..effect_runtime import effect_runtime_result +from ..runtime.time import now_utc_iso +from ..todos.active_state_editing import atomic_write_state_text, verify_state_text_durable + +SUPERVISOR_EVENT_SCHEMA = "supervisor_log_event_v0" +LOCAL_PRIVATE_PRIVACY = "local_private" +SUPERVISOR_PROPOSED = "supervisor_proposed" +SUPERVISOR_RECEIPT_RECORDED = "supervisor_receipt_recorded" + + +class SupervisorEventError(ValueError): + """Invalid supervisor log; never treat unreadable history as empty.""" + + +class SupervisorEventConflictError(SupervisorEventError): + """An existing identity names different semantic content.""" + + +class SupervisorEventCommitUnknownError(SupervisorEventError): + """Publication may have landed; inspect before repeating business work.""" + + +def event_identity(event: dict[str, Any]) -> dict[str, str]: + semantic = {key: value for key, value in event.items() + if key not in {"recorded_at", "append_sequence"}} + encoded = json.dumps(semantic, sort_keys=True, separators=(",", ":"), ensure_ascii=False) + return {"event_id": event["event_id"], "fingerprint": hashlib.sha256(encoded.encode()).hexdigest()} + + +def make_supervisor_event(*, event_id: str, goal_id: str, event_type: str, + refs: dict[str, Any], payload: dict[str, Any], recorded_at: str | None = None, +) -> dict[str, Any]: + return {"schema_version": SUPERVISOR_EVENT_SCHEMA, "event_id": event_id, + "goal_id": goal_id, "event_type": event_type, "recorded_at": recorded_at or now_utc_iso(), + "privacy": LOCAL_PRIVATE_PRIVACY, "refs": refs, "payload": payload} + + +def _validate_event(event: Any, *, stored: bool) -> dict[str, Any]: + if not isinstance(event, dict) or event.get("schema_version") != SUPERVISOR_EVENT_SCHEMA: + raise SupervisorEventError("unsupported experimental supervisor log schema") + if event.get("event_type") not in {SUPERVISOR_PROPOSED, SUPERVISOR_RECEIPT_RECORDED}: + raise SupervisorEventError("supervisor log accepts only proposals and receipts") + if event.get("privacy") != LOCAL_PRIVATE_PRIVACY: + raise SupervisorEventError("supervisor log must remain local_private") + for field in ("event_id", "goal_id", "recorded_at"): + if not isinstance(event.get(field), str) or not event[field].strip(): + raise SupervisorEventError(f"supervisor event requires {field}") + for field in ("refs", "payload"): + if not isinstance(event.get(field), dict): + raise SupervisorEventError(f"supervisor event {field} must be an object") + if stored: + seq = event.get("append_sequence") + if type(seq) is not int or not 0 < seq <= 2**53 - 1: + raise SupervisorEventError("supervisor sequence must be a positive safe integer") + return dict(event) + + +@dataclass +class SupervisorEventStore: + path: Path + + def load(self) -> list[dict[str, Any]]: + if not self.path.exists(): + return [] + events: list[dict[str, Any]] = [] + ids: set[str] = set() + for line_number, line in enumerate(self.path.read_text(encoding="utf-8").splitlines(), 1): + if not line.strip(): + continue + try: + event = _validate_event(json.loads(line), stored=True) + except (ValueError, TypeError) as exc: + raise SupervisorEventError(f"invalid supervisor log row {line_number}: {exc}") from exc + if event["event_id"] in ids or event["append_sequence"] != len(events) + 1: + raise SupervisorEventError("supervisor log identities/sequences are inconsistent") + if events and event["goal_id"] != events[0]["goal_id"]: + raise SupervisorEventError("supervisor log mixes Goals") + ids.add(event["event_id"]) + events.append(event) + return events + + def record_locked(self, event: dict[str, Any], *, execute: bool, + events: list[dict[str, Any]], + ) -> tuple[dict[str, Any], bool]: + """Caller holds the log lock across read, decision checks and this write.""" + requested = _validate_event(event, stored=False) + if events and requested["goal_id"] != events[0]["goal_id"]: + raise SupervisorEventConflictError("supervisor log belongs to another Goal") + prior = next((item for item in events if item["event_id"] == requested["event_id"]), None) + plan = effect_runtime_result("agent.supervisor.plan_append", { + "schema_version": "loopx_supervisor_event_append_plan_v0", + "last_sequence": len(events), + "existing": {**event_identity(prior), "append_sequence": prior["append_sequence"]} if prior else None, + "event": event_identity(requested), + }) + if plan.get("status") != "planned": + raise SupervisorEventConflictError(str(plan.get("reason_code") or "invalid supervisor append plan")) + requested["append_sequence"] = plan["append_sequence"] + if not execute: + return prior or requested, False + try: + if prior is not None: + verify_state_text_durable(self.path, self.path.read_bytes().decode("utf-8")) + return prior, False + text = "".join(json.dumps(item, sort_keys=True, ensure_ascii=False) + "\n" + for item in [*events, requested]) + atomic_write_state_text(self.path, text) + except OSError as error: + raise SupervisorEventCommitUnknownError( + "supervisor publication uncertain; inspect the original decision/receipt before retrying" + ) from error + return requested, True diff --git a/loopx/control_plane/agents/supervisor_events.py b/loopx/control_plane/agents/supervisor_events.py index 8478a46795..26e33ab232 100644 --- a/loopx/control_plane/agents/supervisor_events.py +++ b/loopx/control_plane/agents/supervisor_events.py @@ -6,13 +6,12 @@ from pathlib import Path from typing import Any -from ...event_sourced_state import ( +from .supervisor_event_log import ( LOCAL_PRIVATE_PRIVACY, SUPERVISOR_PROPOSED, SUPERVISOR_RECEIPT_RECORDED, - AppendOnlyStateEventStore, - StateEventConflictError, - make_state_event, + SupervisorEventStore, + make_supervisor_event, ) from ..todos.contract import normalize_todo_claimed_by from .supervisor import normalize_supervisor_decision @@ -91,7 +90,7 @@ def build_supervisor_proposal_event( _reject_inline_secrets(normalized, field="decision") decision_id = str(normalized["decision_id"]) supervisor_agent_id = str(supervisor.get("agent_id") or "") - return make_state_event( + return make_supervisor_event( event_id=f"supervisor-proposal-{decision_id}", goal_id=safe_goal_id, event_type=SUPERVISOR_PROPOSED, @@ -101,8 +100,6 @@ def build_supervisor_proposal_event( }, payload={"decision": normalized}, recorded_at=recorded_at, - producer="loopx.supervisor", - privacy=LOCAL_PRIVATE_PRIVACY, ) @@ -209,7 +206,7 @@ def build_supervisor_receipt_event( proposal=proposal, host_capabilities=host_capabilities, ) - return make_state_event( + return make_supervisor_event( event_id=f"supervisor-receipt-{normalized['receipt_id']}", goal_id=safe_goal_id, event_type=SUPERVISOR_RECEIPT_RECORDED, @@ -219,8 +216,6 @@ def build_supervisor_receipt_event( }, payload={"receipt": normalized}, recorded_at=recorded_at, - producer="loopx.supervisor", - privacy=LOCAL_PRIVATE_PRIVACY, ) @@ -242,38 +237,6 @@ def _matching_event( ) -def _append_idempotent( - store: AppendOnlyStateEventStore, - event: dict[str, Any], -) -> tuple[dict[str, Any], bool]: - prior = next( - (item for item in store.load() if item.get("event_id") == event.get("event_id")), - None, - ) - if prior is not None: - if prior.get("refs") != event.get("refs") or prior.get("payload") != event.get("payload"): - raise StateEventConflictError(f"conflicting event_id: {event.get('event_id')}") - return prior, False - try: - return store.append(event), True - except StateEventConflictError: - concurrent = next( - ( - item - for item in store.load() - if item.get("event_id") == event.get("event_id") - ), - None, - ) - if ( - concurrent is not None - and concurrent.get("refs") == event.get("refs") - and concurrent.get("payload") == event.get("payload") - ): - return concurrent, False - raise - - def record_supervisor_proposal( *, log_path: Path, @@ -282,35 +245,30 @@ def record_supervisor_proposal( decision: Mapping[str, Any], execute: bool, ) -> dict[str, Any]: - store = AppendOnlyStateEventStore(log_path) - events = store.load() - event = build_supervisor_proposal_event( - goal_id=goal_id, - supervisor=supervisor, - decision=decision, - ) - prior = next( - (item for item in events if item.get("event_id") == event.get("event_id")), - None, - ) - if execute: - appended, created = _append_idempotent(store, event) - projection_events = store.load() - elif prior is not None: - appended, created = _append_idempotent(store, event) - projection_events = events - else: - appended, created = event, False - projection_events = [*events, event] - return { - "ok": True, - "mode": "supervisor_proposal", - "dry_run": not execute, - "appended": created, - "would_append": prior is None, - "event": appended, - "projection": build_supervisor_event_projection(projection_events, goal_id=goal_id), - } + from ...file_lock import exclusive_file_lock + with exclusive_file_lock(log_path, operation="supervisor_event"): + store = SupervisorEventStore(log_path) + events = store.load() + event = build_supervisor_proposal_event( + goal_id=goal_id, + supervisor=supervisor, + decision=decision, + ) + prior = next( + (item for item in events if item.get("event_id") == event.get("event_id")), + None, + ) + appended, created = store.record_locked(event, execute=execute, events=events) + projection_events = events if prior is not None else [*events, appended] + return { + "ok": True, + "mode": "supervisor_proposal", + "dry_run": not execute, + "appended": created, + "would_append": prior is None, + "event": appended, + "projection": build_supervisor_event_projection(projection_events, goal_id=goal_id), + } def record_supervisor_receipt( @@ -321,58 +279,53 @@ def record_supervisor_receipt( execute: bool, host_capabilities: list[str] | tuple[str, ...] | None = None, ) -> dict[str, Any]: - store = AppendOnlyStateEventStore(log_path) - events = store.load() - decision_id = _required_token(receipt, "decision_id") - proposal = _matching_event( - events, - event_type=SUPERVISOR_PROPOSED, - ref_name="decision_id", - ref_value=decision_id, - ) - if proposal is None: - raise ValueError(f"no recorded supervisor proposal for decision_id={decision_id}") - prior_executed = next( - ( - event - for event in events - if event.get("event_type") == SUPERVISOR_RECEIPT_RECORDED - and (event.get("refs") or {}).get("decision_id") == decision_id - and ((event.get("payload") or {}).get("receipt") or {}).get("outcome") - == SupervisorReceiptOutcome.EXECUTED.value - ), - None, - ) - event = build_supervisor_receipt_event( - goal_id=goal_id, - proposal_event=proposal, - receipt=receipt, - host_capabilities=host_capabilities, - ) - if prior_executed is not None and prior_executed.get("event_id") != event.get("event_id"): - raise ValueError(f"decision_id={decision_id} already has an executed receipt") - prior = next( - (item for item in events if item.get("event_id") == event.get("event_id")), - None, - ) - if execute: - appended, created = _append_idempotent(store, event) - projection_events = store.load() - elif prior is not None: - appended, created = _append_idempotent(store, event) - projection_events = events - else: - appended, created = event, False - projection_events = [*events, event] - return { - "ok": True, - "mode": "supervisor_receipt", - "dry_run": not execute, - "appended": created, - "would_append": prior is None, - "event": appended, - "projection": build_supervisor_event_projection(projection_events, goal_id=goal_id), - } + from ...file_lock import exclusive_file_lock + with exclusive_file_lock(log_path, operation="supervisor_event"): + store = SupervisorEventStore(log_path) + events = store.load() + decision_id = _required_token(receipt, "decision_id") + proposal = _matching_event( + events, + event_type=SUPERVISOR_PROPOSED, + ref_name="decision_id", + ref_value=decision_id, + ) + if proposal is None: + raise ValueError(f"no recorded supervisor proposal for decision_id={decision_id}") + prior_executed = next( + ( + event + for event in events + if event.get("event_type") == SUPERVISOR_RECEIPT_RECORDED + and (event.get("refs") or {}).get("decision_id") == decision_id + and ((event.get("payload") or {}).get("receipt") or {}).get("outcome") + == SupervisorReceiptOutcome.EXECUTED.value + ), + None, + ) + event = build_supervisor_receipt_event( + goal_id=goal_id, + proposal_event=proposal, + receipt=receipt, + host_capabilities=host_capabilities, + ) + if prior_executed is not None and prior_executed.get("event_id") != event.get("event_id"): + raise ValueError(f"decision_id={decision_id} already has an executed receipt") + prior = next( + (item for item in events if item.get("event_id") == event.get("event_id")), + None, + ) + appended, created = store.record_locked(event, execute=execute, events=events) + projection_events = events if prior is not None else [*events, appended] + return { + "ok": True, + "mode": "supervisor_receipt", + "dry_run": not execute, + "appended": created, + "would_append": prior is None, + "event": appended, + "projection": build_supervisor_event_projection(projection_events, goal_id=goal_id), + } def build_supervisor_event_projection( @@ -442,7 +395,7 @@ def build_supervisor_event_projection( def load_supervisor_event_projection(log_path: Path, *, goal_id: str) -> dict[str, Any]: return build_supervisor_event_projection( - AppendOnlyStateEventStore(log_path).load(), + SupervisorEventStore(log_path).load(), goal_id=goal_id, ) diff --git a/loopx/control_plane/agents/supervisor_inject.py b/loopx/control_plane/agents/supervisor_inject.py index 9b321bf11b..862e0e5ac3 100644 --- a/loopx/control_plane/agents/supervisor_inject.py +++ b/loopx/control_plane/agents/supervisor_inject.py @@ -4,7 +4,7 @@ from pathlib import Path from typing import Protocol -from ...event_sourced_state import SUPERVISOR_PROPOSED, AppendOnlyStateEventStore +from .supervisor_event_log import SUPERVISOR_PROPOSED, SupervisorEventStore from ..todos.contract import normalize_todo_claimed_by from .supervisor_events import ( SupervisorReceiptOutcome, @@ -69,7 +69,7 @@ def _capabilities(adapter: SupervisorInjectHostAdapter) -> list[str]: def _proposal(log_path: Path, *, goal_id: str, decision_id: str) -> dict: - for event in AppendOnlyStateEventStore(log_path).load(): + for event in SupervisorEventStore(log_path).load(): if ( event.get("event_type") == SUPERVISOR_PROPOSED and event.get("goal_id") == goal_id diff --git a/loopx/control_plane/coordination/legacy_writer_fence.py b/loopx/control_plane/coordination/legacy_writer_fence.py index 8fbc549e06..c1f420f26a 100644 --- a/loopx/control_plane/coordination/legacy_writer_fence.py +++ b/loopx/control_plane/coordination/legacy_writer_fence.py @@ -159,6 +159,11 @@ def require_registry_source_write_allowed( "the state file is not the source established by the active capture binding", ) registry = load_registry(registry_path) + if canonical_mutation: + from ..goals.legacy_event_source import require_no_legacy_todo_events + goal = next((g for g in registry.get("goals", []) if g.get("id") == goal_id), {"id": goal_id}) + require_no_legacy_todo_events(goal, state_path=state_file) + registered_root = resolve_runtime_root(registry, None, registry_path=registry_path) _require_other_goal_source_write_allowed( registry=registry, diff --git a/loopx/control_plane/coordination/local_authority_shadow_outbox.py b/loopx/control_plane/coordination/local_authority_shadow_outbox.py index 7e42b710da..40e5eebd9b 100644 --- a/loopx/control_plane/coordination/local_authority_shadow_outbox.py +++ b/loopx/control_plane/coordination/local_authority_shadow_outbox.py @@ -769,7 +769,6 @@ def __init__( ) self._seq: int | None = None self._entry_id: str | None = None - self._event_id: str | None = None self._lineage_id: str | None = None self.outcome = CaptureOutcome(partition=TODO_PARTITION if enabled else None) @@ -829,12 +828,8 @@ def _fail(self, reason_code: str, error: BaseException) -> None: "error_class": error.__class__.__name__, } - def prepare(self, new_text: str, *, event_id: str | None = None) -> None: - """Record the prepared entry for the bytes about to be written. - - Event-only writers have no source-owned outbox transaction and return - an explicit hold without creating an entry. - """ + def prepare(self, new_text: str) -> None: + """Prepare capture for the exact Markdown bytes about to be written.""" if not self.enabled or self._directory is None or self._runtime_root is None: self.outcome.skipped_reason = "shadow_disabled" @@ -848,9 +843,6 @@ def prepare(self, new_text: str, *, event_id: str | None = None) -> None: binding = binding_view["binding"] self._lineage_id = str(binding["capture_lineage_id"]) source_root_digest = str(binding["source_root_digest"]) - if event_id is not None: - self.outcome.skipped_reason = "event_log_writer_not_bound" - return try: projection = self._project(new_text) digest = partition_digest(projection) @@ -880,7 +872,7 @@ def prepare(self, new_text: str, *, event_id: str | None = None) -> None: writer=_writer( self._write_class, runtime=WRITER_RUNTIME_PYTHON, - operation_id=event_id, + operation_id=None, ), source={ "kind": source_kind, @@ -888,7 +880,7 @@ def prepare(self, new_text: str, *, event_id: str | None = None) -> None: "previous_partition_digest": previous_digest, "bytes_digest": bytes_digest, "lease": None, - "event_id": event_id, + "event_id": None, }, source_root_digest=source_root_digest, capture_lineage_id=self._lineage_id, @@ -904,7 +896,6 @@ def prepare(self, new_text: str, *, event_id: str | None = None) -> None: return self._seq = seq self._entry_id = entry_id - self._event_id = event_id self.outcome.entry_id = entry_id self.outcome.seq = seq self.outcome.partition_digest = digest diff --git a/loopx/control_plane/coordination/runtime_shadow.py b/loopx/control_plane/coordination/runtime_shadow.py index 231f33e5b6..a30fbe3af6 100644 --- a/loopx/control_plane/coordination/runtime_shadow.py +++ b/loopx/control_plane/coordination/runtime_shadow.py @@ -321,11 +321,10 @@ def _build_runtime_shadow_source_snapshot( TS takes the shared source locks and verifies every byte/inventory before publishing a baseline or a bounded qualification result. """ - from ...event_sourced_state import build_state_projection, normalize_state_event, render_active_state_sections from ...rollout_event_log import ROLLOUT_EVENT_SCHEMA_VERSION, rollout_event_log_path from ...paths import resolve_runtime_root from ...state_refresh import resolve_goal_state - from ..status.active_state_projection import state_event_log_candidates + from ..goals.legacy_event_source import state_event_log_candidates from ..todos.active_state_todo_parser import parse_active_state_todos from ..todos.goal_todo_projection import todo_summaries_from_fields from ..todos.handoff_mode import goal_handoff_mode @@ -357,21 +356,16 @@ def read_evidence(path: Path) -> bytes | None: if isinstance(value, dict) and value.get("schema_version") == ROLLOUT_EVENT_SCHEMA_VERSION: rollout_events.append(value) - # Use the production candidate selection and projection semantics. A log - # with no Todo projection is harmless; an unbound Todo overlay is a hold. + # Preserve absent/empty source witnesses; reject retired sources before publishing. for path in state_event_log_candidates(dict(goal), state_path=state_path): data = read_evidence(path) if not data: continue - events = [normalize_state_event(json.loads(line)) for line in data.decode("utf-8").splitlines() if line.strip()] - rendered = render_active_state_sections(build_state_projection(events, goal_id=goal_id)) - fields = parse_active_state_todos(rendered, goal=dict(goal), state_path=state_path, item_limit=None, rollout_events=rollout_events) - if any(fields.get(f"{role}_todos") for role in ("user", "agent")): - raise ShadowManagementError("event_log_writer_not_bound") + raise ShadowManagementError("legacy_todo_event_source_retired", + "legacy_todo_event_source_retired: preserve and export legacy Todo events with a compatible older release before migration") fields = parse_active_state_todos(state_text, goal=dict(goal), state_path=state_path, item_limit=None, rollout_events=rollout_events) - todos = todo_summaries_from_fields(fields=fields, source="markdown_active_state", projection_fields={}, - projection_overlay=None, rollout_events=rollout_events, roles=["user", "agent"], status=None, + todos = todo_summaries_from_fields(fields=fields, source="markdown_active_state", rollout_events=rollout_events, roles=["user", "agent"], status=None, todo_id=None, agent_id=None, limit=None).todos todos = capture_todo_archive_dependencies(todos, state_text) leases: list[dict[str, Any]] = [] diff --git a/loopx/control_plane/effect_runtime_handlers.ts b/loopx/control_plane/effect_runtime_handlers.ts index 730c3b9c8a..03fa888e32 100644 --- a/loopx/control_plane/effect_runtime_handlers.ts +++ b/loopx/control_plane/effect_runtime_handlers.ts @@ -1,4 +1,3 @@ -import {planStateEventReplay} from "./goals/state_event_replay.ts"; import {projectTodoSummary} from "./todos/summary_projection.ts"; import {admitAutomationStart, confirmAutomationStart, manageAutomationCadence, projectCadenceSchedule} from "./quota/automation_cadence.ts"; import {deliverShadowEntry} from "./coordination/shadow_entry_delivery.ts"; @@ -197,7 +196,7 @@ import {evaluateCapabilityGate} from "./agents/capability_gate.ts"; import {projectCoordinationSource} from "./coordination/source_projection.ts"; import {withCoordinationSourceTransfer} from "./coordination/source_transfer.ts"; import {captureArchivedTodoDependencies} from "./todos/archive_capture.ts"; -import {planStateEventAppend} from "./goals/state_event_append.ts"; +import {planSupervisorEventAppend} from "./agents/supervisor_event_append.ts"; import {projectAdvancementFrontier, evaluateLongTodoChain} from "./todos/frontier_revision.ts"; import { evaluateCoordinationTodoSuccessorDerivation } from "./coordination/todo_successor_derivation.ts"; import { @@ -439,7 +438,6 @@ export function createEffectRuntimeHandlers( (params) => interpretTurnJournal(turnJournalInspectionRequest(params)), ], ["turn_journal.write", commitTurnJournal], - ["goal.state_event.plan_replay", planStateEventReplay], ["todo.completion_fence.evaluate", evaluateTodoCompletionFence], ["todo.completion_state.normalize", normalizeTodoCompletionValue], ["todo.completion_state.require_metadata", requireTodoCompletionMetadataValue], @@ -458,7 +456,7 @@ export function createEffectRuntimeHandlers( ["agent.capability_gate.evaluate", evaluateCapabilityGate], ["agent.capability_memory", agentCapabilityMemory], ["todo.archive.capture_dependencies", withCoordinationSourceTransfer("todo.archive.capture_dependencies", captureArchivedTodoDependencies)], - ["goal.state_event.plan_append", planStateEventAppend], + ["agent.supervisor.plan_append", planSupervisorEventAppend], ["coordination.source.project", withCoordinationSourceTransfer("coordination.source.project", projectCoordinationSource)], ["todo.monitor_metadata.plan", planMonitorMetadata], ["todo.authoring_scope.plan", planTodoAuthoringScope], diff --git a/loopx/control_plane/goals/active_state_event_projection.py b/loopx/control_plane/goals/active_state_event_projection.py deleted file mode 100644 index f842da0ba3..0000000000 --- a/loopx/control_plane/goals/active_state_event_projection.py +++ /dev/null @@ -1,106 +0,0 @@ -from __future__ import annotations - -from pathlib import Path -from typing import Any, Callable, Optional - -from loopx.event_sourced_state import ( - AppendOnlyStateEventStore, - StateEventError, - build_state_projection, - render_active_state_sections, -) - - -DEFAULT_STATE_EVENT_LOG_BASENAME = "events.jsonl" -STATE_EVENT_PROJECTION_SCHEMA_VERSION = "event_sourced_state_status_projection_v0" -STATE_EVENT_READ_WARNING_SCHEMA_VERSION = "event_sourced_state_read_warning_v0" - -ResolveGoalLocalPath = Callable[..., Optional[Path]] -ParseActiveStateTodos = Callable[..., dict[str, Any]] - - -def state_event_log_candidates( - goal: dict[str, Any], - *, - state_path: Path, - resolve_goal_local_path: ResolveGoalLocalPath, - event_log_basename: str = DEFAULT_STATE_EVENT_LOG_BASENAME, -) -> list[Path]: - candidates: list[Path] = [] - for key in ("state_event_log", "state_events_file", "event_log"): - resolved = resolve_goal_local_path(goal.get(key), goal, fallback_base=state_path.parent) - if resolved is not None: - candidates.append(resolved) - candidates.append(state_path.with_name(event_log_basename)) - - unique: list[Path] = [] - seen: set[str] = set() - for path in candidates: - key = str(path.expanduser()) - if key in seen: - continue - seen.add(key) - unique.append(path) - return unique - - -def active_state_event_projection_fields( - goal: dict[str, Any], - *, - state_path: Path, - resolve_goal_local_path: ResolveGoalLocalPath, - parse_active_state_todos: ParseActiveStateTodos, - preferred_todo_ids: set[str] | None = None, - rollout_events: list[dict[str, Any]] | None = None, - item_limit: int | None = None, - event_log_basename: str = DEFAULT_STATE_EVENT_LOG_BASENAME, -) -> dict[str, Any]: - goal_id = str(goal.get("id") or "").strip() - first_warning: dict[str, Any] | None = None - for event_log_path in state_event_log_candidates( - goal, - state_path=state_path, - resolve_goal_local_path=resolve_goal_local_path, - event_log_basename=event_log_basename, - ): - if not event_log_path.exists(): - continue - try: - events = AppendOnlyStateEventStore(event_log_path).load() - if not events: - continue - projection = build_state_projection(events, goal_id=goal_id or None) - projection_markdown = render_active_state_sections(projection) - fields = parse_active_state_todos( - projection_markdown, - goal=goal, - state_path=state_path, - preferred_todo_ids=preferred_todo_ids, - rollout_events=rollout_events, - item_limit=item_limit, - ) - except (OSError, StateEventError) as exc: - if first_warning is None: - first_warning = { - "state_event_projection_warning": { - "schema_version": STATE_EVENT_READ_WARNING_SCHEMA_VERSION, - "source": "event_log", - "event_log": event_log_path.name, - "fallback": "markdown_active_state", - "reason": type(exc).__name__, - } - } - continue - if fields: - fields["state_event_projection"] = { - "schema_version": STATE_EVENT_PROJECTION_SCHEMA_VERSION, - "source": "event_log", - "event_log": event_log_path.name, - "source_event_count": projection.get("source_event_count"), - "source_checksum": projection.get("source_checksum"), - "last_event_id": projection.get("last_event_id"), - "last_append_sequence": projection.get("last_append_sequence"), - "projection_version": projection.get("projection_version"), - } - return fields - return first_warning or {} diff --git a/loopx/control_plane/goals/goal_amendment_proposal.py b/loopx/control_plane/goals/goal_amendment_proposal.py index 6b15d05afe..29f28cad0c 100644 --- a/loopx/control_plane/goals/goal_amendment_proposal.py +++ b/loopx/control_plane/goals/goal_amendment_proposal.py @@ -63,7 +63,7 @@ from typing import Any from ...agent_registry import registered_agent_ids_for_goal -from ...event_sourced_state import now_utc_iso +from ..runtime.time import now_utc_iso from ...file_lock import exclusive_file_lock from ...history import load_index, load_registry from ...runtime import validate_goal_id_path_segment diff --git a/loopx/control_plane/goals/legacy_event_source.py b/loopx/control_plane/goals/legacy_event_source.py new file mode 100644 index 0000000000..13bcc6c2b7 --- /dev/null +++ b/loopx/control_plane/goals/legacy_event_source.py @@ -0,0 +1,41 @@ +"""Admission for retired Todo event sources; no event projection or writer.""" +from __future__ import annotations +from pathlib import Path +from typing import Any +from .path_resolution import resolve_goal_local_path + +def state_event_log_candidates( + goal: dict[str, Any], + *, + state_path: Path, +) -> list[Path]: + candidates: list[Path] = [] + for key in ("state_event_log", "state_events_file", "event_log"): + resolved = resolve_goal_local_path(goal.get(key), goal, fallback_base=state_path.parent) + if resolved is not None: + candidates.append(resolved) + candidates.append(state_path.with_name("events.jsonl")) + + unique: list[Path] = [] + seen: set[str] = set() + for path in candidates: + key = str(path.expanduser()) + if key in seen: + continue + seen.add(key) + unique.append(path) + return unique + + +class RetiredTodoEventSourceError(ValueError): + reason_code = "legacy_todo_event_source_retired" + + +def require_no_legacy_todo_events(goal: dict[str, Any], *, state_path: Path) -> None: + for path in state_event_log_candidates(goal, state_path=state_path): + if path.exists() and path.stat().st_size: + raise RetiredTodoEventSourceError( + "legacy_todo_event_source_retired: preserve the legacy event file and " + "export its Todo records with a compatible older release before migration; " + "this release refuses to substitute Markdown for event-owned Todos" + ) diff --git a/loopx/control_plane/goals/shared_goal_alignment.py b/loopx/control_plane/goals/shared_goal_alignment.py index 25e5f5bcfa..c2e31acc13 100644 --- a/loopx/control_plane/goals/shared_goal_alignment.py +++ b/loopx/control_plane/goals/shared_goal_alignment.py @@ -1,32 +1,9 @@ -"""Read-only shared goal alignment projection adapter (RFC Stage 1). +"""Read-only shared Goal alignment over the selected Todo/lease authority. -This adapter collects typed facts for one registered Agent around one shared -Goal — registry identity, the selected Todo/lease source, the append-only state -event log, Todo claim/lease fields, and recorded replan obligations — and -asks the TypeScript-owned reducer (``goal.shared_goal_alignment.project``) -to project ``shared_goal_alignment_v0``. - -Derivation invariants (RFC shared-goal-alignment-and-governed-amendment-v0 -§3.3): every projected field is derived from typed facts only. Shared -``Next Action`` prose, agent vision prose, and chat prose are never inputs. - -The projection is strictly read-only: no writer path is touched, and no -approval or escalation semantics exist here. ``source_basis_digest`` is a -typed source-facts basis summary (goal status, registered agents, and -event-log basis facts, and canonical Todo revision when promoted), not a canonical intent-envelope digest — the full -RFC §3.1 envelope (objective, non-goals, acceptance, permission scope, -terminal conditions) has no typed storage yet, so nothing here claims -canonical intent identity. - -Basis semantics: the only goal-level monotonic sequence carrier on this -codebase is the state event log's ``append_sequence``, so -``state_event_basis_sequence`` reports that event projection basis — it is -NOT a canonical goal/intent revision. Goals without a parsable -``events.jsonl`` use sequence 0 and an unbound Agent frontier. Before promotion -this is ``markdown_active_state``; after promotion it is ``canonical_todo_snapshot`` -with a separate ``todo_basis`` token, never a fabricated event sequence; -drift is then reported as ``frontier_basis_unverifiable`` instead of a -fabricated behind fact. +Promoted Goals carry a provider revision; legacy Markdown has no monotonic +revision. Neither source fabricates an event sequence or an Agent frontier. +The historical sequence fields remain zero/unbound for persisted proposal +compatibility; typed alignment and amendment admission decide what is provable. """ from __future__ import annotations @@ -38,31 +15,21 @@ from typing import Any from ...agent_registry import registered_agent_ids_for_goal -from ...event_sourced_state import ( - AppendOnlyStateEventStore, - StateEventError, - build_state_projection, - event_sort_key, -) from ...history import load_registry from ...registry import registry_goals from ..effect_runtime import EffectRuntimeRejected, effect_runtime_result from ..todos.contract import normalize_todo_claimed_by from .shared_goal_work_source import SharedGoalWorkSource, read_shared_goal_work_source -from .active_state_event_projection import state_event_log_candidates from .active_state_metadata import parse_state_frontmatter from .goal_frontier import ( autonomous_replan_is_required, select_autonomous_replan_obligation, ) -from .path_resolution import resolve_goal_local_path SHARED_GOAL_ALIGNMENT_EFFECT_METHOD = "goal.shared_goal_alignment.project" SHARED_GOAL_ALIGNMENT_REQUEST_SCHEMA_VERSION = "shared_goal_alignment_request_v0" SHARED_GOAL_ALIGNMENT_SCHEMA_VERSION = "shared_goal_alignment_v0" -REVISION_BASIS_STATE_EVENT_LOG = "state_event_log" REVISION_BASIS_MARKDOWN_ACTIVE_STATE = "markdown_active_state" -BASIS_SOURCE_STATE_EVENT_LOG = "state_event_log" BASIS_SOURCE_UNBOUND = "unbound" DEFAULT_REGISTRY_RELATIVE_PATH = Path(".loopx") / "registry.json" @@ -87,82 +54,8 @@ def _registered_goal( raise ValueError(f"goal is not registered: {goal_id}") -def _load_state_event_facts( - goal: Mapping[str, Any], - *, - state_path: Path, -) -> dict[str, Any] | None: - """Load the first parsable state event log for the goal, read-only.""" - - for event_log_path in state_event_log_candidates( - dict(goal), - state_path=state_path, - resolve_goal_local_path=resolve_goal_local_path, - ): - if not event_log_path.exists(): - continue - try: - events = AppendOnlyStateEventStore(event_log_path).load() - if not events: - continue - projection = build_state_projection( - events, - goal_id=str(goal.get("id") or "") or None, - ) - except (OSError, StateEventError): - continue - return {"events": events, "projection": projection} - return None -def _agent_frontier_basis( - event_facts: Mapping[str, Any] | None, - *, - agent_id: str, -) -> dict[str, Any]: - """Derive the Agent's frontier basis from its own attributed events. - - ``based_on_state_event_sequence`` is the highest append sequence among - events whose ``actor_agent_id`` belongs to this Agent. Events attributed - to peers never advance another Agent's basis. - """ - - events = event_facts.get("events") if event_facts else None - if not isinstance(events, list): - return { - "based_on_state_event_sequence": None, - "basis_source": BASIS_SOURCE_UNBOUND, - "last_agent_event_id": None, - } - based_on: int | None = None - last_agent_event_id: str | None = None - for event in sorted( - (item for item in events if isinstance(item, dict)), - key=event_sort_key, - ): - actor = normalize_todo_claimed_by(event.get("actor_agent_id")) - if actor != agent_id: - continue - try: - sequence = int(event.get("append_sequence") or 0) - except (TypeError, ValueError): - continue - if sequence < 1: - continue - based_on = sequence - event_id = str(event.get("event_id") or "").strip() - last_agent_event_id = event_id or None - if based_on is None: - return { - "based_on_state_event_sequence": None, - "basis_source": BASIS_SOURCE_UNBOUND, - "last_agent_event_id": None, - } - return { - "based_on_state_event_sequence": based_on, - "basis_source": BASIS_SOURCE_STATE_EVENT_LOG, - "last_agent_event_id": last_agent_event_id, - } def _source_basis_facts_envelope( @@ -245,30 +138,18 @@ def _project_shared_goal_alignment( ) if source.goal_id != normalized_goal_id: raise ValueError("shared work snapshot belongs to another Goal") - state_file, state_text = source.state_path, source.state_text + state_text = source.state_text - event_facts = _load_state_event_facts(goal, state_path=state_file) frontmatter = parse_state_frontmatter(state_text) state_updated_at = str(frontmatter.get("updated_at") or "").strip() or None goal_status = str(goal.get("status") or "").strip() or ( str(frontmatter.get("status") or "").strip() or None ) - if event_facts is not None: - projection = event_facts["projection"] - revision_basis = REVISION_BASIS_STATE_EVENT_LOG - try: - basis_sequence = int(projection.get("last_append_sequence") or 0) - except (TypeError, ValueError): - basis_sequence = 0 - source_checksum = ( - str(projection.get("source_checksum") or "").strip() or None - ) - else: - revision_basis = ("canonical_todo_snapshot" if source.canonical_basis is not None - else REVISION_BASIS_MARKDOWN_ACTIVE_STATE) - basis_sequence = 0 - source_checksum = None + revision_basis = ("canonical_todo_snapshot" if source.canonical_basis is not None + else REVISION_BASIS_MARKDOWN_ACTIVE_STATE) + basis_sequence = 0 + source_checksum = None source_basis_digest = _canonical_digest( _source_basis_facts_envelope( @@ -276,11 +157,7 @@ def _project_shared_goal_alignment( goal_status=goal_status, registered_agents=registered_agents, revision_basis=revision_basis, - last_append_sequence=( - basis_sequence - if revision_basis == REVISION_BASIS_STATE_EVENT_LOG - else None - ), + last_append_sequence=None, source_checksum=source_checksum, state_updated_at=state_updated_at, todo_basis=source.canonical_basis, @@ -294,10 +171,8 @@ def _project_shared_goal_alignment( **({"todo_basis": source.canonical_basis} if source.canonical_basis is not None else {}), } - frontier_basis = _agent_frontier_basis( - event_facts, - agent_id=normalized_agent_id, - ) + frontier_basis = {"based_on_state_event_sequence": None, + "basis_source": BASIS_SOURCE_UNBOUND, "last_agent_event_id": None} replan_obligation = select_autonomous_replan_obligation( dict(status_item) if isinstance(status_item, Mapping) else {}, diff --git a/loopx/control_plane/goals/shared_goal_work_source.py b/loopx/control_plane/goals/shared_goal_work_source.py index 6d17ddf3c9..4f7ae1692f 100644 --- a/loopx/control_plane/goals/shared_goal_work_source.py +++ b/loopx/control_plane/goals/shared_goal_work_source.py @@ -36,6 +36,8 @@ def read_shared_goal_work_source(*, goal: dict[str, Any], project: Path, canonical = read_canonical_todos_if_promoted(runtime_root=runtime_root, goal_id=goal_id, include_leases=True) if runtime_root is not None else None if canonical is None: + from .legacy_event_source import require_no_legacy_todo_events + require_no_legacy_todo_events(goal, state_path=state_path) if not state_path.is_file(): raise ValueError(f"goal state file is missing for {goal_id}") state_text = state_path.read_text(encoding="utf-8") diff --git a/loopx/control_plane/goals/state_event_append.ts b/loopx/control_plane/goals/state_event_append.ts deleted file mode 100644 index 9e5bbfbabb..0000000000 --- a/loopx/control_plane/goals/state_event_append.ts +++ /dev/null @@ -1,68 +0,0 @@ -/** Legacy event-log append admission. Python owns the locked bytes and codec; - * this owner allocates sequences and rejects a whole conflicting/stale batch. */ -import type {JsonObject} from "../effect_program.ts"; -import {requireJsonObject, requireNonEmptyString} from "../runtime_decode.ts"; -import {EffectRuntimeRequestError} from "../effect_runtime_errors.ts"; - -type EventIdentity = {event_id: string; fingerprint: string}; -type StoredIdentity = EventIdentity & {append_sequence: number}; -type AppendChoice = {kind: "replay" | "append"; event_id: string; append_sequence: number}; - -function sequence(value: unknown, minimum: number): number { - if (typeof value !== "number" || !Number.isSafeInteger(value) || value < minimum) { - throw new EffectRuntimeRequestError("event append sequence must be a safe integer"); - } - return value; -} - -function identity(value: unknown): EventIdentity { - const row = requireJsonObject(value, "event identity"); - const fingerprint = requireNonEmptyString(row.fingerprint, "event fingerprint"); - if (!/^[a-f0-9]{64}$/.test(fingerprint)) throw new EffectRuntimeRequestError("invalid event fingerprint"); - return {event_id: requireNonEmptyString(row.event_id, "event_id"), fingerprint}; -} - -export function planStateEventAppend(value: unknown): JsonObject { - const request = requireJsonObject(value, "state event append plan"); - if (request.schema_version !== "loopx_state_event_append_plan_v0" || - !Array.isArray(request.existing) || !Array.isArray(request.events)) { - throw new EffectRuntimeRequestError("invalid state event append plan"); - } - requireNonEmptyString(request.source_checksum, "source_checksum"); - if (request.expected_checksum !== null) { - requireNonEmptyString(request.expected_checksum, "expected_checksum"); - } - const result = (status: "planned" | "rejected", fields: JsonObject): JsonObject => - ({schema_version: "loopx_state_event_append_result_v0", status, ...fields}); - if (request.expected_checksum !== null && request.expected_checksum !== request.source_checksum) { - return result("rejected", {reason_code: "event_source_changed"}); - } - let last = sequence(request.last_sequence, 0); - const known = new Map(); - for (const raw of request.existing) { - const row = requireJsonObject(raw, "stored event identity"); - const record = {...identity(row), append_sequence: sequence(row.append_sequence, 1)}; - if (record.append_sequence > last || known.has(record.event_id)) { - throw new EffectRuntimeRequestError("inconsistent stored event identity"); - } - known.set(record.event_id, record); - } - const choices: AppendChoice[] = []; - for (const raw of request.events) { - const item = identity(raw), prior = known.get(item.event_id); - if (prior) { - if (prior.fingerprint !== item.fingerprint) { - return result("rejected", {reason_code: "event_id_conflict", event_id: item.event_id}); - } - choices.push({kind: "replay", event_id: item.event_id, append_sequence: prior.append_sequence}); - } else { - if (last === Number.MAX_SAFE_INTEGER) { - return result("rejected", {reason_code: "event_sequence_exhausted"}); - } - const record = {...item, append_sequence: ++last}; - known.set(item.event_id, record); - choices.push({kind: "append", event_id: item.event_id, append_sequence: record.append_sequence}); - } - } - return result("planned", {choices}); -} diff --git a/loopx/control_plane/goals/state_event_replay.ts b/loopx/control_plane/goals/state_event_replay.ts deleted file mode 100644 index d2f61adeb9..0000000000 --- a/loopx/control_plane/goals/state_event_replay.ts +++ /dev/null @@ -1,175 +0,0 @@ -/** Replay admission and lifecycle projection for the retained event source. - * Content stays in the host codec: ordinals address that exact input batch. - * This pure plan grants no append, lease, capture or promotion authority. */ -import type {JsonObject} from "../effect_program.ts"; -import {EffectRuntimeRequestError} from "../effect_runtime_errors.ts"; -import {assertNever, requireJsonObject, requireNonEmptyString, requireStringLiteral, requireStringArray} from "../runtime_decode.ts"; -import {authorityUnicodeCompare} from "../coordination/authority_store_codec.ts"; -import {normalizeTodoPriority} from "../todos/priority.ts"; - -const KINDS = ["todo_added", "todo_claimed", "todo_updated", "todo_blocked", "todo_deferred", - "todo_completed", "refresh_recorded", "run_recorded", "quota_spent", "evidence_attached", - "supervisor_proposed", "supervisor_receipt_recorded"] as const; -type Kind = typeof KINDS[number]; -type Status = "open" | "blocked" | "deferred" | "done"; -interface Event { - ordinal: number; id: string; goal: string; kind: Kind; sequence: number | null; - time: string; todo: string | null; role: "user" | "agent" | null; - priority: string | null; plannerOrder: number | null; contentChanged: boolean; - fields: string[]; binding: string | null; continuation: string | null; - removedPolicy: string | null; exclusions: boolean; goalBound: boolean | null; -} -interface Todo { - todo_id: string; field_sources: Record; status: Status; done: boolean; - role: "user" | "agent"; priority: string; planner_order: number | null; - source_section: string; render_priority: boolean; append_sequence: number | null; - binding: string | null; removedPolicy: string | null; -} -function nullableInteger(value: unknown, name: string, positive = false): number | null { - if (value === null) return null; - if (typeof value !== "number" || !Number.isSafeInteger(value) || (positive && value < 1)) { - throw new EffectRuntimeRequestError(`${name} must be ${positive ? "a positive" : "a"} safe integer or null`); - } - return value; -} -function nullableText(value: unknown, name: string): string | null { - return value === null ? null : requireNonEmptyString(value, name); -} -function decode(raw: unknown, index: number, offset: number): Event { - const ordinal = index + offset; - const r = requireJsonObject(raw, "event replay facts"); - const kind = requireStringLiteral(r.event_type, KINDS, "event_type"); - const todo = r.todo_id === null ? null : requireNonEmptyString(r.todo_id, "todo_id"); - if (["todo_added", "todo_claimed", "todo_updated", "todo_blocked", "todo_deferred", "todo_completed"].includes(kind) && todo === null) throw new EffectRuntimeRequestError(`${kind} requires refs.todo_id`); - if (typeof r.content_changed !== "boolean") throw new EffectRuntimeRequestError("content_changed must be boolean"); - if (typeof r.has_exclusions !== "boolean" || (r.goal_bound !== null && typeof r.goal_bound !== "boolean")) { - throw new EffectRuntimeRequestError("event ownership facts require explicit booleans"); - } - return {ordinal, id: requireNonEmptyString(r.event_id, "event_id"), goal: requireNonEmptyString(r.goal_id, "goal_id"), - kind, todo, sequence: nullableInteger(r.append_sequence, "append_sequence", true), - time: requireNonEmptyString(r.recorded_at, "recorded_at"), - role: r.role === null ? null : requireStringLiteral(r.role, ["user", "agent"] as const, "role"), - priority: r.priority === null ? null : normalizeTodoPriority(r.priority), - plannerOrder: nullableInteger(r.planner_order, "planner_order"), contentChanged: r.content_changed, - fields: requireStringArray(r.fields, "event content fields"), - binding: nullableText(r.capability_binding_ref, "capability_binding_ref"), - continuation: nullableText(r.continuation_policy, "continuation_policy"), - removedPolicy: nullableText(r.removed_continuation_policy, "removed_continuation_policy"), - exclusions: r.has_exclusions, goalBound: r.goal_bound}; -} - -/** Continuation state is ephemeral replay data, never a durable authority token. */ -function decodeTodo(value: unknown): Todo { - const r = requireJsonObject(value, "replay continuation"); - const sources = requireJsonObject(r.field_sources, "content sources"); - const field_sources: Record = {}; - for (const [key, index] of Object.entries(sources)) { - const n = nullableInteger(index, "content source ordinal"); - if (n === null || n < 0) throw new EffectRuntimeRequestError("invalid content source ordinal"); - Object.defineProperty(field_sources, key, {value: n, enumerable: true, writable: true, configurable: true}); - } - const role = requireStringLiteral(r.role, ["user", "agent"] as const, "role"); - const status = requireStringLiteral(r.status, ["open", "done", "blocked", "deferred"] as const, "status"); - if (typeof r.render_priority !== "boolean") throw new EffectRuntimeRequestError("render_priority must be boolean"); - const priority = normalizeTodoPriority(r.priority); - if (priority === null) throw new EffectRuntimeRequestError("continuation priority is required"); - return {todo_id: requireNonEmptyString(r.todo_id, "todo_id"), field_sources, - status, done: status === "done", role, priority, - source_section: role === "user" ? "User Todo / Owner Review Reading Queue" : "Agent Todo", - render_priority: r.render_priority, planner_order: nullableInteger(r.planner_order, "planner_order"), - append_sequence: nullableInteger(r.append_sequence, "append_sequence", true), - binding: nullableText(r.binding, "capability binding"), removedPolicy: nullableText(r.removedPolicy, "removed policy")}; -} - -export function planStateEventReplay(value: unknown): JsonObject { - const r = requireJsonObject(value, "state event replay request"); - if (r.schema_version !== "state_event_replay_request_v0" || !Array.isArray(r.events)) { - throw new EffectRuntimeRequestError("state event replay requires its schema and complete event facts"); - } - const offset = nullableInteger(r.offset ?? 0, "batch offset"); - if (offset === null || offset < 0 || offset > Number.MAX_SAFE_INTEGER - r.events.length) { - throw new EffectRuntimeRequestError("invalid batch offset"); - } - const events = r.events.map((event, index) => decode(event, index, offset)); - const ids = new Set(); - for (const event of events) { - if (ids.has(event.id)) throw new EffectRuntimeRequestError("event replay facts must be deduplicated by the source codec"); - ids.add(event.id); - } - events.sort((a, b) => (a.sequence ?? 0) - (b.sequence ?? 0) || authorityUnicodeCompare(a.time, b.time) || authorityUnicodeCompare(a.id, b.id)); - const goal = r.goal_id === null ? events[0]?.goal ?? "" : requireNonEmptyString(r.goal_id, "goal_id"); - const todos = new Map(); - if (r.initial_todos !== undefined && !Array.isArray(r.initial_todos)) { - throw new EffectRuntimeRequestError("initial_todos must be an array"); - } - for (const raw of (r.initial_todos ?? []) as unknown[]) { - const todo = decodeTodo(raw); - if (todos.has(todo.todo_id) || Object.values(todo.field_sources).some(index => index >= offset)) { - throw new EffectRuntimeRequestError("continuation does not precede this replay batch"); - } - todos.set(todo.todo_id, todo); - } - const timeline: number[] = []; - for (const event of events) { - if (event.goal !== goal) throw new EffectRuntimeRequestError("all events in a projection must share one goal_id"); - const kind = event.kind; - switch (kind) { - case "todo_added": { - const id = event.todo!; // decode requires identity for every Todo event. - if (todos.has(id)) throw new EffectRuntimeRequestError(`todo_id already exists: ${id}; use todo_updated`); - const role = event.role ?? "agent"; - todos.set(id, {todo_id: id, field_sources: Object.fromEntries(event.fields.map(field => [field, event.ordinal])), status: "open", done: false, - role, priority: event.priority ?? "P2", planner_order: event.plannerOrder, - source_section: role === "user" ? "User Todo / Owner Review Reading Queue" : "Agent Todo", - render_priority: event.priority !== null, append_sequence: event.sequence, - binding: event.binding, removedPolicy: event.removedPolicy}); - break; - } - case "todo_claimed": case "todo_updated": case "todo_blocked": case "todo_deferred": case "todo_completed": { - const todo = todos.get(event.todo!); - if (!todo) throw new EffectRuntimeRequestError(`${kind} references unknown todo_id: ${event.todo}`); - for (const field of event.fields) Object.defineProperty(todo.field_sources, field, - {value: event.ordinal, enumerable: true, configurable: true, writable: true}); - if (!event.fields.includes("last_actor_agent_id")) delete todo.field_sources.last_actor_agent_id; - if (kind === "todo_updated") { - if (event.binding !== null) { - if (todo.binding !== null && todo.binding !== event.binding) { - throw new EffectRuntimeRequestError("capability_binding_ref is immutable once set"); - } - todo.binding = event.binding; - } - if (event.removedPolicy !== null) { - delete todo.field_sources.continuation_policy; - todo.removedPolicy = event.removedPolicy; - } else if (event.continuation !== null && todo.removedPolicy !== null) { - if (event.continuation === "independent_handoff" && event.exclusions) { - delete todo.field_sources.removed_continuation_policy; - todo.removedPolicy = null; - } else delete todo.field_sources.continuation_policy; - } - if (event.fields.includes("bound_agent") && event.goalBound === null) delete todo.field_sources.goal_bound; - if (event.goalBound === true) delete todo.field_sources.bound_agent; - if (event.role !== null) todo.role = event.role; - if (event.priority !== null) todo.priority = event.priority; - if (event.priority !== null || event.contentChanged) todo.render_priority = true; - todo.source_section = todo.role === "user" ? "User Todo / Owner Review Reading Queue" : "Agent Todo"; - } else if (kind === "todo_blocked") todo.status = "blocked"; - else if (kind === "todo_deferred") todo.status = "deferred"; - else if (kind === "todo_completed") todo.status = "done"; - todo.done = todo.status === "done"; - break; - } - case "refresh_recorded": case "run_recorded": case "quota_spent": case "evidence_attached": - timeline.push(event.ordinal); break; - case "supervisor_proposed": case "supervisor_receipt_recorded": break; - default: assertNever(kind, "unhandled event replay kind"); - } - } - const items = [...todos.values()].sort((a, b) => Number(a.role !== "user") - Number(b.role !== "user") || - authorityUnicodeCompare(a.priority, b.priority) || (a.planner_order ?? 9999) - (b.planner_order ?? 9999) || - (a.append_sequence ?? 0) - (b.append_sequence ?? 0)); - return {schema_version: "state_event_replay_plan_v0", goal_id: goal, - event_indices: events.map(e => e.ordinal), timeline_indices: timeline, - todos: items.map(item => ({...item, sort_key: [Number(item.role !== "user"), item.priority, - item.planner_order ?? 9999, item.append_sequence ?? 0]}))}; -} diff --git a/loopx/control_plane/runtime/event_store_migration_bridge.py b/loopx/control_plane/runtime/event_store_migration_bridge.py deleted file mode 100644 index 67afd8b6e7..0000000000 --- a/loopx/control_plane/runtime/event_store_migration_bridge.py +++ /dev/null @@ -1,196 +0,0 @@ -from __future__ import annotations - -from typing import Any - - -EVENT_STORE_MIGRATION_BRIDGE_SCHEMA_VERSION = "event_store_migration_bridge_v0" -EVENT_STORE_MIGRATION_CANARY_SCHEMA_VERSION = "event_store_migration_canary_v0" - -MARKDOWN_ACTIVE_STATE_SOURCE = "markdown_active_state" -EVENT_PROJECTION_SOURCE = "event_projection" - - -def _compact_text(value: Any) -> str: - return " ".join(str(value or "").strip().split()) - - -def _bool(value: Any) -> bool: - return value is True - - -def build_event_store_migration_bridge( - *, - goal_id: str, - event_read_path_ready: bool, - dual_read_parity_clean: bool = False, - rollback_plan_recorded: bool = False, - bounded_canary_passed: bool = False, - idempotency_conflicts_clean: bool = False, - public_boundary_clean: bool = False, - active_state_projection_ready: bool = False, - event_projection_head_matches_store: bool = False, - canary_goal_limit: int = 1, - canary_duration_minutes: int = 30, - evidence_refs: list[str] | None = None, -) -> dict[str, Any]: - """Build a fail-closed migration bridge before event projection promotion.""" - - normalized_goal_id = _compact_text(goal_id) - if not normalized_goal_id: - raise ValueError("goal_id is required") - canary_goal_limit = max(1, int(canary_goal_limit or 1)) - canary_duration_minutes = max(1, int(canary_duration_minutes or 1)) - evidence = [_compact_text(ref) for ref in (evidence_refs or []) if _compact_text(ref)] - - checks = { - "event_read_path_ready": _bool(event_read_path_ready), - "active_state_projection_ready": _bool(active_state_projection_ready), - "dual_read_parity_clean": _bool(dual_read_parity_clean), - "event_projection_head_matches_store": _bool(event_projection_head_matches_store), - "rollback_plan_recorded": _bool(rollback_plan_recorded), - "bounded_canary_passed": _bool(bounded_canary_passed), - "idempotency_conflicts_clean": _bool(idempotency_conflicts_clean), - "public_boundary_clean": _bool(public_boundary_clean), - } - required_for_shadow = [ - "event_read_path_ready", - "active_state_projection_ready", - ] - required_for_canary = [ - *required_for_shadow, - "dual_read_parity_clean", - "event_projection_head_matches_store", - "rollback_plan_recorded", - "idempotency_conflicts_clean", - "public_boundary_clean", - ] - required_for_promotion = [ - *required_for_canary, - "bounded_canary_passed", - ] - missing_for_shadow = [key for key in required_for_shadow if not checks[key]] - missing_for_canary = [key for key in required_for_canary if not checks[key]] - missing_for_promotion = [key for key in required_for_promotion if not checks[key]] - - if missing_for_shadow: - stage = "wait_for_event_read_path" - next_action = ( - "finish event read-path prerequisites before dual-read migration work" - ) - elif missing_for_canary: - stage = "dual_read_shadow" - next_action = ( - "compare Markdown read model and event projection until parity, rollback, " - "idempotency, and public-boundary checks are clean" - ) - elif missing_for_promotion: - stage = "bounded_canary" - next_action = ( - "run the bounded canary on a small goal set before promotion" - ) - else: - stage = "promotion_candidate" - next_action = ( - "promote event projection only through an explicit reviewed write-path change" - ) - - return { - "schema_version": EVENT_STORE_MIGRATION_BRIDGE_SCHEMA_VERSION, - "goal_id": normalized_goal_id, - "source_of_truth": MARKDOWN_ACTIVE_STATE_SOURCE, - "candidate_source": EVENT_PROJECTION_SOURCE, - "stage": stage, - "promotion_allowed": False, - "promotion_candidate": not missing_for_promotion, - "next_action": next_action, - "checks": checks, - "missing_for_shadow": missing_for_shadow, - "missing_for_canary": missing_for_canary, - "missing_for_promotion": missing_for_promotion, - "dual_read": { - "enabled": not missing_for_shadow, - "read_order": [ - MARKDOWN_ACTIVE_STATE_SOURCE, - EVENT_PROJECTION_SOURCE, - ], - "failure_policy": "prefer_markdown_and_record_parity_delta", - "required_equality": [ - "todo ids", - "todo status", - "priority and planner order", - "claimed_by", - "user todo binding", - "gate refs", - "projection head sequence", - ], - }, - "rollback": { - "required": True, - "recorded": checks["rollback_plan_recorded"], - "fallback_source": MARKDOWN_ACTIVE_STATE_SOURCE, - "trigger": [ - "parity delta", - "projection head mismatch", - "event append conflict", - "public boundary warning", - "canary regression", - ], - "action": "disable event projection preference and keep Markdown parser as canonical read fallback", - }, - "canary": build_event_store_migration_canary( - goal_id=normalized_goal_id, - ready=not missing_for_canary, - passed=checks["bounded_canary_passed"], - goal_limit=canary_goal_limit, - duration_minutes=canary_duration_minutes, - evidence_refs=evidence, - ), - "evidence_refs": evidence, - } - - -def build_event_store_migration_canary( - *, - goal_id: str, - ready: bool, - passed: bool = False, - goal_limit: int = 1, - duration_minutes: int = 30, - evidence_refs: list[str] | None = None, -) -> dict[str, Any]: - """Describe the bounded rollout canary for event-store read promotion.""" - - normalized_goal_id = _compact_text(goal_id) - if not normalized_goal_id: - raise ValueError("goal_id is required") - goal_limit = max(1, int(goal_limit or 1)) - duration_minutes = max(1, int(duration_minutes or 1)) - evidence = [_compact_text(ref) for ref in (evidence_refs or []) if _compact_text(ref)] - - return { - "schema_version": EVENT_STORE_MIGRATION_CANARY_SCHEMA_VERSION, - "goal_id": normalized_goal_id, - "ready": bool(ready), - "passed": bool(passed), - "scope": { - "max_goals": goal_limit, - "duration_minutes": duration_minutes, - "write_path": "disabled", - "read_preference": MARKDOWN_ACTIVE_STATE_SOURCE, - }, - "observe": [ - "status todo summaries", - "quota selected todo", - "review packet todo refs", - "dashboard/frontstage projection", - "event projection head sequence", - ], - "success_criteria": [ - "no parity delta", - "no idempotency conflict", - "no private-boundary warning", - "rollback remains one-command safe", - ], - "failure_policy": "stop canary and keep Markdown parser canonical", - "evidence_refs": evidence, - } diff --git a/loopx/control_plane/status/active_state_projection.py b/loopx/control_plane/status/active_state_projection.py index 18b1295fa8..c394cd709e 100644 --- a/loopx/control_plane/status/active_state_projection.py +++ b/loopx/control_plane/status/active_state_projection.py @@ -3,22 +3,14 @@ from __future__ import annotations import re -from pathlib import Path from typing import Any -from ..goals.active_state_event_projection import ( - active_state_event_projection_fields as _active_state_event_projection_fields, - state_event_log_candidates as _state_event_log_candidates, -) from ..goals.active_state_sections import ( active_state_section_entries as _active_state_section_entries, active_state_sections as _active_state_sections, ) -from ..goals.path_resolution import resolve_goal_local_path from ..runtime.public_safety import public_safe_compact_text -from ..todos.active_state_todo_parser import parse_active_state_todos from ..todos.todo_summary import ( - MAX_STATUS_TODOS_PER_ROLE, normalize_todo_text, ) from ..work_items.backlog_hygiene import ( @@ -39,33 +31,8 @@ ) -def state_event_log_candidates(goal: dict[str, Any], *, state_path: Path) -> list[Path]: - return _state_event_log_candidates( - goal, - state_path=state_path, - resolve_goal_local_path=resolve_goal_local_path, - event_log_basename=STATE_EVENT_LOG_BASENAME, - ) -def active_state_event_projection_fields( - goal: dict[str, Any], - *, - state_path: Path, - preferred_todo_ids: set[str] | None = None, - rollout_events: list[dict[str, Any]] | None = None, - item_limit: int | None = MAX_STATUS_TODOS_PER_ROLE, -) -> dict[str, Any]: - return _active_state_event_projection_fields( - goal, - state_path=state_path, - resolve_goal_local_path=resolve_goal_local_path, - parse_active_state_todos=parse_active_state_todos, - preferred_todo_ids=preferred_todo_ids, - rollout_events=rollout_events, - item_limit=item_limit, - event_log_basename=STATE_EVENT_LOG_BASENAME, - ) def active_state_sections(state_text: str, headings: tuple[str, ...]) -> dict[str, list[str]]: diff --git a/loopx/control_plane/testing/authority_e2e_ladder.py b/loopx/control_plane/testing/authority_e2e_ladder.py index a2416b3d26..0caf242fdd 100644 --- a/loopx/control_plane/testing/authority_e2e_ladder.py +++ b/loopx/control_plane/testing/authority_e2e_ladder.py @@ -660,7 +660,7 @@ def _row_postgresql_conformance_live(context: RowContext) -> RowOutcome: LadderRow( id="s2c2.event_only_todo_source_holds", stage="2c2", - title="An event-only Todo source holds qualification and candidate reads fail-closed until rollback and rebootstrap", + title="A retired Todo event source refuses reads and primary writes without changing either source", product_path="real_cli", gate="deterministic", posix_only=False, diff --git a/loopx/control_plane/testing/authority_e2e_rows_stage2c2.py b/loopx/control_plane/testing/authority_e2e_rows_stage2c2.py index 50e3cbac80..9400b9f864 100644 --- a/loopx/control_plane/testing/authority_e2e_rows_stage2c2.py +++ b/loopx/control_plane/testing/authority_e2e_rows_stage2c2.py @@ -18,7 +18,6 @@ from __future__ import annotations -import importlib import json import select import subprocess @@ -96,7 +95,7 @@ # byte delta may grow by about one Todo record per transaction. A larger jump # means something beyond the live projection is being re-published. GROWTH_DELTA_ACCELERATION_ENVELOPE_BYTES = 2048 -EVENT_ONLY_HOLD = "event_log_writer_not_bound" +EVENT_ONLY_HOLD = "legacy_todo_event_source_retired" CONTINUITY_HOLD = "source_partition_continuity_unproved" SHADOW_READ_MODULE = Path("loopx") / "control_plane" / "coordination" / "local_authority_shadow.ts" SHADOW_READ_REQUEST_SCHEMA = "loopx_coordination_runtime_shadow_outbox_read_v0" @@ -919,7 +918,7 @@ def row_parity_divergent_detects_foreign_edit(context: RowContext) -> RowOutcome def row_event_only_todo_source_holds(context: RowContext) -> RowOutcome: - """An event-only Todo source holds qualification and candidate reads fail-closed; recovery needs rollback and rebootstrap.""" + """A retired source refuses candidate/primary operations; preserving both sources allows explicit cleanup.""" workspace = capture_workspace(context, "ladder-event") todo_ids: list[str] = [] @@ -929,47 +928,24 @@ def row_event_only_todo_source_holds(context: RowContext) -> RowOutcome: todo_ids.append(str(added["todo_id"])) qualified(qualify(workspace), label="baseline") log = workspace.state_path.with_name("events.jsonl") - # The product's own state-event store writes the event-only source. It is - # loaded lazily so this strictly typed ladder module does not follow the - # untyped state-event module at type-check time. - state_events = importlib.import_module("loopx.event_sourced_state") - state_events.AppendOnlyStateEventStore(log).append( - state_events.make_state_event( - event_id="ladder-event-only-todo", - goal_id=workspace.goal_id, - event_type=state_events.TODO_ADDED, - refs={"todo_id": "todo_event_only"}, - payload={"role": "agent", "title": "An event-only todo without a Markdown writer.", "task_class": "advancement_task"}, - recorded_at="2026-09-06T00:00:00+00:00", - ) - ) + log.write_text(json.dumps({"schema_version": "loopx_state_event_v0", + "event_type": "todo_added", "refs": {"todo_id": "todo_event_only"}}) + "\n", encoding="utf-8") log_bytes = log.read_bytes() surfaces = {"inspect": inspect(workspace), "qualify": qualify(workspace), "read-candidate": read_candidate(workspace, todo_ids[0])} for label, payload in surfaces.items(): - expect(payload.get("ok") is False and payload.get("error") == EVENT_ONLY_HOLD, f"{label} must hold on the unbound event source") + expect(payload.get("ok") is False and str(payload.get("error") or "").startswith(EVENT_ONLY_HOLD), f"{label} must refuse retired source: {payload.get('error')}") status = shadow_status(workspace) expect(status.get("ok") is True and management_status(status) == "active", "status must stay readable while the lineage is held") - during = add_todo(workspace, "Markdown write during the event-only hold.") - expect(during.get("added") is True, "the primary write must still commit") - held = capture_evidence(during, label="todo add (during hold)") - expect(held.get("outcome") == "pending" and held.get("reason_code") == CONTINUITY_HOLD, "the capture must hold on unproven continuity") - expect(log.read_bytes() == log_bytes, "the hold must not touch the event log") - expect(backlog(shadow_status(workspace), "todos").get("committed_pending") == 1, "the held entry must stay pending") - log.unlink() - removed = rejected(qualify(workspace), "qualification", label="qualify after removal") - expect(removed.get("reason_code") == "outbox_pending", "removing the event source must not requalify the held lineage") - stopped = drain(workspace) - expect(stopped.get("outcome") == "stopped" and stopped.get("reason_code") == CONTINUITY_HOLD, "drain must keep holding the entry") - summary = _recover_by_rollback_and_rebootstrap(workspace, label="event-only") - return passed( - hold=EVENT_ONLY_HOLD, - held_surfaces=sorted(surfaces), - primary_write_during_hold=CONTINUITY_HOLD, - event_log_untouched=True, - removal_requalifies=False, - recovered_by="rollback_then_bootstrap", - rebootstrap_baseline_todos=summary.get("todo_count"), - ) + before = workspace.state_path.read_bytes() + during = goal_cli(workspace, "todo", "add", "--role", "agent", + "--text", "Markdown write during the retired event hold.", check=False) + expect(during.get("ok") is False, "retired source must reject primary writes") + expect(workspace.state_path.read_bytes() == before, "refusal must preserve Markdown") + expect(log.read_bytes() == log_bytes, "refusal must preserve the event file") + log.unlink() # The fixture owns this file; product never removes it. + qualified(qualify(workspace), label="retired source removed without any write") + return passed(hold=EVENT_ONLY_HOLD, held_surfaces=sorted(surfaces), + primary_write_refused=True, event_log_untouched=True) @dataclass(frozen=True) diff --git a/loopx/control_plane/todos/active_state_todos.py b/loopx/control_plane/todos/active_state_todos.py index 26c800bd5c..ec8cd120ed 100644 --- a/loopx/control_plane/todos/active_state_todos.py +++ b/loopx/control_plane/todos/active_state_todos.py @@ -11,37 +11,6 @@ from .succession_warning import public_todo_summary -MONITOR_WRITEBACK_CONTRACT_SCHEMA_VERSION = "monitor_writeback_contract_v0" - - -def _attach_monitor_writeback_contract( - fields: dict[str, Any], - *, - supported: bool, - source: str, -) -> None: - if supported: - return - contract = { - "schema_version": MONITOR_WRITEBACK_CONTRACT_SCHEMA_VERSION, - "supported": False, - "source": source, - } - for key in ("user_todos", "agent_todos"): - summary = fields.get(key) - if isinstance(summary, dict): - summary["monitor_writeback"] = dict(contract) - - -def attach_monitor_writeback_contract( - fields: dict[str, Any], - *, - supported: bool, - source: str, -) -> None: - _attach_monitor_writeback_contract(fields, supported=supported, source=source) - - def _redacted_status_todo_fields(fields: dict[str, Any]) -> dict[str, Any]: redacted = dict(fields) for key in ("user_todos", "agent_todos"): @@ -87,16 +56,13 @@ def active_state_todo_fields( load_rollout_events: Callable[..., list[dict[str, Any]]], rollout_event_log_path: Callable[[Path, str], Path], max_todo_index_rollout_events_per_goal: int, - active_state_event_projection_fields: Callable[..., dict[str, Any]], parse_active_state_todos: Callable[..., dict[str, Any]], parse_issue_meta_surface: Callable[[str], dict[str, Any] | None], backlog_hygiene_warning: Callable[..., dict[str, Any] | None], completed_todo_archive_warning: Callable[[dict[str, Any] | None], dict[str, Any] | None], state_projection_gap_warning: Callable[..., dict[str, Any] | None], - attach_monitor_writeback_contract: Callable[..., None] | None = None, redacted_status_todo_fields: Callable[[dict[str, Any]], dict[str, Any]] | None = None, ) -> dict[str, Any]: - monitor_writeback_contract_writer = attach_monitor_writeback_contract or _attach_monitor_writeback_contract todo_field_redactor = redacted_status_todo_fields or _redacted_status_todo_fields goal_id = str(goal.get("id") or "").strip() # Inspect authority before the display file. A missing/stale projection is @@ -106,6 +72,9 @@ def active_state_todo_fields( if runtime_root is not None and goal_id else None ) state_path = resolve_goal_local_path(goal.get("state_file"), goal, fallback_base=Path.cwd()) + if canonical is None and state_path is not None: + from ..goals.legacy_event_source import require_no_legacy_todo_events + require_no_legacy_todo_events(goal, state_path=state_path) if canonical is None and (state_path is None or not state_path.exists()): return {} try: @@ -132,12 +101,6 @@ def active_state_todo_fields( rollout_event_log_path(runtime_root, goal_id), limit=max_todo_index_rollout_events_per_goal, ) - event_fields = {} if canonical is not None else active_state_event_projection_fields( - goal, - state_path=state_path, - preferred_todo_ids=preferred_todo_ids, - rollout_events=events, - ) if canonical is not None: fields = canonical_todo_summary_fields( canonical["todos"], @@ -150,23 +113,6 @@ def active_state_todo_fields( # Canonical observation/successor transactions now support current # lease proof. Scheduling exposes due work; mutation admission still # validates the caller's proof and never falls back to the old writer. - elif event_fields.get("user_todos") or event_fields.get("agent_todos"): - fields = event_fields - markdown_fields = parse_active_state_todos( - state_text, - goal=goal, - state_path=state_path, - preferred_todo_ids=preferred_todo_ids, - rollout_events=events, - ) - standing_decision_authority = markdown_fields.get("standing_decision_authority") - if isinstance(standing_decision_authority, dict): - fields["standing_decision_authority"] = standing_decision_authority - monitor_writeback_contract_writer( - fields, - supported=False, - source="event_projection_read_model", - ) else: fields = parse_active_state_todos( state_text, @@ -175,13 +121,6 @@ def active_state_todo_fields( preferred_todo_ids=preferred_todo_ids, rollout_events=events, ) - monitor_writeback_contract_writer( - fields, - supported=True, - source="markdown_active_state", - ) - if event_fields: - fields.update(event_fields) issue_meta_surface = parse_issue_meta_surface(state_text) if issue_meta_surface: fields["issue_meta_surface"] = issue_meta_surface diff --git a/loopx/control_plane/todos/completion_fence.ts b/loopx/control_plane/todos/completion_fence.ts index 33efb552ff..55d33ac67c 100644 --- a/loopx/control_plane/todos/completion_fence.ts +++ b/loopx/control_plane/todos/completion_fence.ts @@ -21,7 +21,7 @@ const TODO_STATUSES = ["open", "done", "blocked", "deferred"] as const; const TODO_ID_PATTERN = /^todo_[a-z0-9_-]{3,64}$/; export type TodoStatus = (typeof TODO_STATUSES)[number]; -export type TodoCompletionProjectionSource = "materialized" | "event_log"; +export type TodoCompletionProjectionSource = "materialized"; export type TodoCompletionIdentitySource = | "turn_settlement" | "unscoped_completion" @@ -134,7 +134,7 @@ function todoStatus(value: unknown): TodoStatus { } function projectionSource(value: unknown): TodoCompletionProjectionSource { - if (value === "materialized" || value === "event_log") return value; + if (value === "materialized") return value; throw new EffectRuntimeRequestError("projection_source is unsupported"); } @@ -209,8 +209,7 @@ export function evaluateTodoCompletionFence( request.todo.completion_continuation, ); const done = status === "done"; - const terminalBeforeRequest = done || - (request.projection_source === "event_log" && status === "deferred"); + const terminalBeforeRequest = done; const requestedKey = request.requested_completion_turn_key; const storedKey = request.todo.completion_turn_key; const expectedLocalKey = request.goal_id !== null && request.todo_id !== null diff --git a/loopx/control_plane/todos/completion_policy.py b/loopx/control_plane/todos/completion_policy.py index a99b6f8f82..6d28ca17ec 100644 --- a/loopx/control_plane/todos/completion_policy.py +++ b/loopx/control_plane/todos/completion_policy.py @@ -12,7 +12,6 @@ from .contract import ( normalize_todo_claimed_by, normalize_todo_continuation_policy, - normalize_todo_id, ) @@ -60,9 +59,8 @@ def linked_successors_from_state( *, lines: list[str], successor_todo_ids: Iterable[str], - event_fields: Mapping[str, Any] | None = None, ) -> list[LinkedSuccessor]: - """Resolve declared successor rows from Markdown or event projection.""" + """Resolve declared successor rows from Markdown.""" successors: list[LinkedSuccessor] = [] for todo_id in successor_todo_ids: @@ -71,21 +69,6 @@ def linked_successors_from_state( role, _section, _start, _end, block = match successors.append(linked_successor_from_todo({**block, "role": role})) continue - for role in ("user", "agent"): - summary = (event_fields or {}).get(f"{role}_todos") - items = summary.get("items") if isinstance(summary, Mapping) else [] - item = next( - ( - value - for value in items or [] - if isinstance(value, Mapping) - and normalize_todo_id(value.get("todo_id")) == todo_id - ), - None, - ) - if item: - successors.append(linked_successor_from_todo({**item, "role": role})) - break return successors diff --git a/loopx/control_plane/todos/completion_transaction.ts b/loopx/control_plane/todos/completion_transaction.ts index e2cdfd2d4b..473383c475 100644 --- a/loopx/control_plane/todos/completion_transaction.ts +++ b/loopx/control_plane/todos/completion_transaction.ts @@ -46,7 +46,7 @@ export const TODO_COMPLETION_POLICY_FAILURE_SCHEMA = "loopx_todo_completion_policy_failure_v0"; const CALLER_VALIDATION_RECEIPT_SCHEMA = "issue_fix_validation_command_v0"; -const PROJECTION_SOURCES = ["materialized", "event_log"] as const; +const PROJECTION_SOURCES = ["materialized"] as const; const COMPLETION_IDENTITY_SOURCES = [ "turn_settlement", "unscoped_completion", diff --git a/loopx/control_plane/todos/completion_validation.py b/loopx/control_plane/todos/completion_validation.py index 41e9916151..85ac24563a 100644 --- a/loopx/control_plane/todos/completion_validation.py +++ b/loopx/control_plane/todos/completion_validation.py @@ -36,10 +36,6 @@ read_completion_validation_declaration, ) from .contract import TODO_STATUS_DONE, normalize_todo_status -from .event_writeback import ( - event_projection_source_authority, - event_projection_todo_context, -) # Kept safely under the 30s outer CLI/MCP subprocess budget so a timed-out # validation still produces a typed receipt before the outer call is killed. @@ -467,18 +463,6 @@ def resolve_private_completion_validation_declaration( todo_id=todo_id, role=role, ) - if source is None: - event_context = event_projection_todo_context( - registry_path=registry_path, - goal_id=goal_id, - state_path=state_file, - todo_id=todo_id, - role=role, - ) - if event_context is not None: - source = dict( - event_context.get("raw_item") or event_context.get("item") or {} - ) declaration = ( completion_validation_declaration(source) if isinstance(source, dict) @@ -529,29 +513,17 @@ def run_completion_validation_gate_with_source( w.r.t. the state file and safe to call before acquiring the mutation lock, so a multi-second validation command does not block concurrent Todo writes. """ + from ...history import load_registry + from ...registry import registry_goals + from ..goals.legacy_event_source import require_no_legacy_todo_events + goal = next((item for item in registry_goals(load_registry(registry_path)) if item.get("id") == goal_id), {}) + require_no_legacy_todo_events(goal, state_path=state_file) projection_source = "materialized" source_authority: dict[str, Any] | None = None - event_context: dict[str, Any] | None = None todo = _materialized_todo_item(state_file=state_file, todo_id=todo_id, role=role) if todo is None: - event_context = event_projection_todo_context( - registry_path=registry_path, - goal_id=goal_id, - state_path=state_file, - todo_id=todo_id, - role=role, - ) - if event_context is None: - return { - "failure": None, - "source_authority": None, - "source_snapshot": None, - "transaction": None, - } - projection_source = "event_log" - todo = dict(event_context.get("raw_item") or event_context["item"]) - todo["role"] = event_context["role"] - source_authority = event_projection_source_authority(event_context) + return {"todo": None, "validation": None, "failure": None, + "source_authority": None, "source_snapshot": None, "transaction": None} source_snapshot = todo_completion_source_snapshot(todo) completion_policy_source = None if completion_policy_facts is not None: @@ -564,9 +536,6 @@ def run_completion_validation_gate_with_source( goal_id=goal_id, lines=lines, successor_todo_ids=requested_successor_todo_ids or [], - event_fields=( - event_context.get("fields") if event_context is not None else None - ), facts=completion_policy_facts, ) transaction = reduce_todo_completion_transaction( @@ -645,7 +614,6 @@ def completion_policy_source_from_state( goal_id: str, lines: list[str], successor_todo_ids: list[str], - event_fields: Mapping[str, Any] | None, facts: Mapping[str, Any], ) -> dict[str, Any]: """Project lock-comparable facts for the TS completion policy.""" @@ -666,7 +634,6 @@ def completion_policy_source_from_state( linked_successors=linked_successors_from_state( lines=lines, successor_todo_ids=successor_todo_ids, - event_fields=event_fields, ), ), ) @@ -716,29 +683,16 @@ def locked_todo_completion_source( goal_id: str, todo_id: str, role: str | None, -) -> tuple[Any, dict[str, Any] | None, dict[str, Any] | None]: - """Resolve the materialized or event-projected Todo under the write lock.""" +) -> tuple[Any, dict[str, Any] | None]: + """Resolve the materialized Todo under the write lock.""" match = find_todo_block(lines, todo_id=todo_id, role=role) if match: item_role, _section, _start, _end, block = match todo = dict(block) todo["role"] = item_role - return match, todo, None - event_context = event_projection_todo_context( - registry_path=registry_path, - goal_id=goal_id, - state_path=state_file, - todo_id=todo_id, - role=role, - ) - if event_context is None: - return None, None, None - event_context["state_file"] = state_file - event_context["project"] = project - todo = dict(event_context["item"]) - todo["role"] = event_context["role"] - return None, todo, event_context + return match, todo + return None, None def execute_completion_validation_effects( diff --git a/loopx/control_plane/todos/completion_validation_plan.ts b/loopx/control_plane/todos/completion_validation_plan.ts index 767697cd2f..63fab39b40 100644 --- a/loopx/control_plane/todos/completion_validation_plan.ts +++ b/loopx/control_plane/todos/completion_validation_plan.ts @@ -44,7 +44,7 @@ export type TodoCompletionValidationPlanResult = function projectionSource(value: unknown): TodoCompletionProjectionSource { return requireStringLiteral( value, - ["materialized", "event_log"] as const, + ["materialized"] as const, "projection_source", ); } diff --git a/loopx/control_plane/todos/durable_completion.py b/loopx/control_plane/todos/durable_completion.py index b09b695a8c..125dc5e2ba 100644 --- a/loopx/control_plane/todos/durable_completion.py +++ b/loopx/control_plane/todos/durable_completion.py @@ -36,10 +36,9 @@ normalize_todo_completion_recovery, ) from .completion_fence import evaluate_todo_completion_fence -from .event_writeback import event_projection_todo_context -TodoCompletionProjectionSource = Literal["materialized", "event_log"] +TodoCompletionProjectionSource = Literal["materialized"] def read_persisted_todo_record( @@ -132,26 +131,6 @@ def read_persisted_todo_record_with_source( item_todo_id = normalize_todo_id(item.get("todo_id")) if item_todo_id: existing_todo_ids.add(item_todo_id) - if block is None and registry_path is not None and goal_id is not None: - context = event_projection_todo_context( - registry_path=registry_path, - goal_id=goal_id, - state_path=state_file, - todo_id=todo_id, - role=None, - ) - if context is not None: - block = dict(context["item"]) - projection_source = "event_log" - for candidate_role in ("user", "agent"): - summary = context["fields"].get(f"{candidate_role}_todos") - items = summary.get("items") if isinstance(summary, dict) else [] - for item in items if isinstance(items, list) else []: - if not isinstance(item, dict): - continue - item_todo_id = normalize_todo_id(item.get("todo_id")) - if item_todo_id: - existing_todo_ids.add(item_todo_id) if block is None: normalized_todo_id = normalize_todo_id(todo_id) or todo_id raise ValueError( diff --git a/loopx/control_plane/todos/event_writeback.py b/loopx/control_plane/todos/event_writeback.py deleted file mode 100644 index de69d37f2d..0000000000 --- a/loopx/control_plane/todos/event_writeback.py +++ /dev/null @@ -1,660 +0,0 @@ -from __future__ import annotations - -from contextlib import nullcontext -from ..coordination.legacy_writer_fence import legacy_todo_write_transaction, require_legacy_coordination_write_allowed -from ..coordination.shadow_management import require_shadow_primary_write_allowed -from ..coordination.local_authority_shadow_adapter import effective_runtime_root - -import hashlib -from pathlib import Path -from typing import Any, Mapping - -from ...event_sourced_state import ( - AppendOnlyStateEventStore, - StateEventError, - StateEventSourceChangedError, - TODO_ADDED, - TODO_CLAIMED, - TODO_COMPLETED, - build_state_projection, - make_state_event, -) -from ...history import load_registry -from ..goals.active_state_event_projection import ( - active_state_event_projection_fields, - state_event_log_candidates, -) -from ..goals.path_resolution import resolve_goal_local_path -from ..runtime.validation_command import CALLER_VALIDATION_RECEIPT_SCHEMA_VERSION -from .active_state_todo_parser import parse_active_state_todos -from .contract import ( - TODO_STATUS_DONE, - build_todo_id, - merge_todo_id_lists, - normalize_todo_claimed_by, - normalize_todo_id, - normalize_todo_id_list, -) -from .completion_transaction import reduce_todo_completion_transaction -from .successor_derivation import ( - build_successor_intents, - derive_successor_proposals, -) -from .todo_semantics import todo_priority_parts - - -TODO_SECTION_HEADINGS = { - "user": "User Todo / Owner Review Reading Queue", - "agent": "Agent Todo", -} - - -def _registry_goal(registry_path: Path, goal_id: str) -> dict[str, Any] | None: - registry = load_registry(registry_path) - for goal in registry.get("goals") or []: - if isinstance(goal, dict) and str(goal.get("id") or "") == goal_id: - return goal - return None - - -def _raw_event_projection_todo_item( - *, - projection: Mapping[str, Any], - todo_id: str, - roles: list[str], -) -> dict[str, Any] | None: - for item_role in roles: - if item_role not in TODO_SECTION_HEADINGS: - continue - summary = projection.get(f"{item_role}_todos") - items = summary.get("items") if isinstance(summary, dict) else [] - for item in items if isinstance(items, list) else []: - if not isinstance(item, dict): - continue - if normalize_todo_id(item.get("todo_id")) == todo_id: - return dict(item) - return None - - -def _canonical_event_projection_source( - *, - goal: dict[str, Any], - state_path: Path, - projection_authority: Mapping[str, Any], -) -> tuple[Path, dict[str, Any]] | None: - """Resolve the exact log whose fingerprint produced the public projection.""" - authority_keys = ( - "source_event_count", - "source_checksum", - "last_event_id", - "last_append_sequence", - "projection_version", - ) - goal_id = str(goal.get("id") or "").strip() - for event_log_path in state_event_log_candidates( - goal, - state_path=state_path, - resolve_goal_local_path=resolve_goal_local_path, - ): - if not event_log_path.exists(): - continue - try: - events = AppendOnlyStateEventStore(event_log_path).load() - if not events: - continue - projection = build_state_projection(events, goal_id=goal_id or None) - except (OSError, StateEventError): - continue - if all( - projection.get(key) == projection_authority.get(key) - for key in authority_keys - ): - return event_log_path, projection - return None - - -def event_projection_source_authority(context: Mapping[str, Any]) -> dict[str, Any]: - fields = context.get("fields") if isinstance(context.get("fields"), dict) else {} - projection = ( - fields.get("state_event_projection") - if isinstance(fields, dict) - else None - ) - projection = projection if isinstance(projection, dict) else {} - return { - "projection_source": "event_log", - "event_log_path": str(context.get("event_log_path") or ""), - "source_checksum": projection.get("source_checksum"), - "last_event_id": projection.get("last_event_id"), - "last_append_sequence": projection.get("last_append_sequence"), - } - - -def event_projection_source_matches( - context: Mapping[str, Any], - source_authority: Mapping[str, Any] | None, -) -> bool: - if not isinstance(source_authority, Mapping): - return True - if source_authority.get("projection_source") != "event_log": - return True - current = event_projection_source_authority(context) - return all( - current.get(key) == source_authority.get(key) - for key in ( - "event_log_path", - "source_checksum", - "last_event_id", - "last_append_sequence", - ) - ) - - -def _completion_validation_source_drift_failure( - *, - goal_id: str, - todo_id: str, - dry_run: bool, -) -> dict[str, Any]: - return { - "ok": False, - "dry_run": dry_run, - "completed": False, - "goal_id": goal_id, - "todo_id": todo_id, - "changed": False, - "validation": { - "schema_version": CALLER_VALIDATION_RECEIPT_SCHEMA_VERSION, - "command_label": "todo completion validation", - "exit_code": None, - "passed": False, - "status": "source_drift", - "summary": ( - "event-log validation source changed before completion append; " - "retry the completion against the current canonical source" - ), - "stdout_captured": False, - "stderr_captured": False, - "local_path_captured": False, - }, - "validation_blocked_completion": True, - } - - -def event_projection_todo_context( - *, - registry_path: Path, - goal_id: str, - state_path: Path, - todo_id: str, - role: str | None, -) -> dict[str, Any] | None: - goal = _registry_goal(registry_path, goal_id) - if not goal: - return None - fields = active_state_event_projection_fields( - goal, - state_path=state_path, - resolve_goal_local_path=resolve_goal_local_path, - parse_active_state_todos=parse_active_state_todos, - item_limit=None, - ) - if not fields.get("state_event_projection"): - return None - normalized_todo_id = normalize_todo_id(todo_id) - if not normalized_todo_id: - return None - roles = [role] if role else ["user", "agent"] - matched_role: str | None = None - matched_item: dict[str, Any] | None = None - for item_role in roles: - if item_role not in TODO_SECTION_HEADINGS: - continue - summary = fields.get(f"{item_role}_todos") - items = summary.get("items") if isinstance(summary, dict) else [] - for item in items if isinstance(items, list) else []: - if not isinstance(item, dict): - continue - if normalize_todo_id(item.get("todo_id")) == normalized_todo_id: - matched_role = item_role - matched_item = dict(item) - break - if matched_item: - break - if not matched_item or matched_role is None: - return None - projection_authority = fields.get("state_event_projection") - source = _canonical_event_projection_source( - goal=goal, - state_path=state_path, - projection_authority=( - projection_authority - if isinstance(projection_authority, Mapping) - else {} - ), - ) - if source is None: - return None - event_log_path, raw_projection = source - raw_item = _raw_event_projection_todo_item( - projection=raw_projection, - todo_id=normalized_todo_id, - roles=roles, - ) - return { - "goal": goal, - "fields": fields, - "event_log_path": event_log_path, - "registry_path": registry_path, - "state_path": state_path, - "role": matched_role, - "item": matched_item, - "raw_item": raw_item or matched_item, - } - - -def _todo_write_event_id( - *, - goal_id: str, - todo_id: str, - action: str, - updated_at: str, - text: str | None = None, -) -> str: - digest = hashlib.sha1( - "|".join([goal_id, todo_id, action, updated_at, str(text or "")]).encode("utf-8") - ).hexdigest()[:16] - return f"todo-write-{action}-{digest}" - - -def _encode_event_projected_successor( - *, - goal_id: str, - proposal: Mapping[str, Any], - updated_at: str, - fields: dict[str, Any], - actor_agent_id: str | None, -) -> tuple[dict[str, Any], list[dict[str, Any]]]: - """Encode a TS-derived successor without deciding defaults or writing it.""" - role = str(proposal["role"]) - section = TODO_SECTION_HEADINGS[role] - summary = fields.get(f"{role}_todos") - items = summary.get("items") if isinstance(summary, dict) else [] - index = len(items if isinstance(items, list) else []) + 1 - text = str(proposal["text"]) - todo_id = build_todo_id(role=role, source_section=section, index=index, text=text) - priority, title = todo_priority_parts(text) - metadata_fields = ( - "task_class", - "action_kind", - "capability_binding_ref", - "task_repository", - "required_capabilities", - "continuation_policy", - "bound_agent", - "goal_bound", - "blocks_agent", - "excluded_agents", - "unblocks_todo_id", - ) - metadata = { - key: proposal[key] for key in metadata_fields - if key in proposal and proposal[key] not in (None, "", []) - } - payload = { - "role": role, - "priority": priority or "P2", - "title": title, - "planner_order": index, - "updated_at": updated_at, - **metadata, - } - events = [ - make_state_event( - event_id=_todo_write_event_id( - goal_id=goal_id, - todo_id=todo_id, - action="add", - updated_at=updated_at, - text=text, - ), - goal_id=goal_id, - event_type=TODO_ADDED, - refs={"todo_id": todo_id}, - payload=payload, - recorded_at=updated_at, - producer="loopx.todo.complete", - actor_agent_id=actor_agent_id, - ) - ] - claimed_by = proposal.get("claimed_by") - if claimed_by: - events.append( - make_state_event( - event_id=_todo_write_event_id( - goal_id=goal_id, - todo_id=todo_id, - action="claim", - updated_at=updated_at, - text=claimed_by, - ), - goal_id=goal_id, - event_type=TODO_CLAIMED, - refs={"todo_id": todo_id}, - payload={"claimed_by": claimed_by}, - recorded_at=updated_at, - producer="loopx.todo.complete", - actor_agent_id=actor_agent_id, - ) - ) - return { - "added": True, - "already_exists": False, - "metadata_updated": False, - "role": role, - "section": section, - "todo": text, - "todo_id": todo_id, - **{key: proposal.get(key) for key in metadata_fields}, - "required_capabilities": proposal.get("required_capabilities", []), - "excluded_agents": proposal.get("excluded_agents", []), - "claimed_by": claimed_by, - "updated_at": updated_at, - "source": "event_log", - }, events - - -def complete_event_projected_goal_todo( - *, - goal_id: str, - context: dict[str, Any], - evidence: str | None, - note: str | None, - no_followup: bool, - successor_todo_ids: list[str] | None, - claimed_by: str | None, - clear_claim: bool, - next_agent_todo: str | None, - next_user_todo: str | None, - next_user_task_class: str, - next_claimed_by: str | None, - next_task_class: str | None, - next_action_kind: str | None, - next_task_repository: str | None, - next_required_capabilities: list[str] | None, - next_continuation_policy: str | None, - self_merged: bool, - next_excluded_agents: list[str], - registered_agents: list[str], - updated_at: str, - dry_run: bool, - completion_turn_key: str | None = None, - completion_identity_source: str | None = None, - actor_agent_id: str | None = None, - completion_fence: dict[str, Any] | None = None, - completion_state: Mapping[str, Any] | None = None, - completion_validation_source_authority: dict[str, Any] | None = None, - runtime_root: Path | None = None, - primary_lock_held: bool = False, -) -> dict[str, Any]: - registry_path = Path(context["registry_path"]) - state_path = Path(context["state_path"]) - root = runtime_root or effective_runtime_root(registry_path, None) - transaction = ( - nullcontext() - if primary_lock_held - else legacy_todo_write_transaction( - registry_path, - goal_id, - state_path, - actor_agent_id, - "todo_event_complete", - dry_run, - runtime_root=root, - ) - ) - with transaction: - if not dry_run: - require_shadow_primary_write_allowed(root, goal_id) - require_legacy_coordination_write_allowed( - runtime_root=root, goal_id=goal_id - ) - item = dict(context["item"]) - role = str(context["role"]) - todo_id = normalize_todo_id(item.get("todo_id")) - if not todo_id: - raise ValueError("event-projected todo has no stable todo_id") - if not event_projection_source_matches( - context, - completion_validation_source_authority, - ): - return _completion_validation_source_drift_failure( - goal_id=goal_id, - todo_id=todo_id, - dry_run=dry_run, - ) - if clear_claim and item.get("claimed_by"): - item.pop("claimed_by", None) - effective_claimed_by = claimed_by or normalize_todo_claimed_by( - item.get("claimed_by") - ) - store = AppendOnlyStateEventStore(Path(context["event_log_path"])) - source_checksum = context["fields"]["state_event_projection"]["source_checksum"] - if completion_fence is None or completion_state is None: - transaction = reduce_todo_completion_transaction( - todo=item, - projection_source="event_log", - completion_turn_key=completion_turn_key, - no_followup=no_followup, - goal_id=goal_id, - todo_id=todo_id, - completion_identity_source=completion_identity_source, - requested_has_successor=bool( - normalize_todo_id_list(successor_todo_ids) - or normalize_todo_id_list(item.get("successor_todo_ids")) - or next_agent_todo - or next_user_todo - ), - dry_run=dry_run, - ) - if transaction["decision"] in {"execute_validation", "reject"}: - raise RuntimeError( - "event-projected Todo completion validation must run through " - "the completion gate" - ) - completion_fence = dict(transaction["fence"]) - candidate_state = transaction.get("completion_state") - completion_state = ( - dict(candidate_state) if isinstance(candidate_state, Mapping) else None - ) - already_done = bool(completion_fence["terminal_before_request"]) - terminal_upgrade = completion_fence["reason"] in { - "same_turn_terminal_upgrade", - "lifecycle_reentry_terminal_upgrade", - } - untyped_completion_repair = ( - completion_fence["reason"] == "untyped_completion_repair" - ) - unscoped_identity_repair = ( - completion_fence["reason"] == "unscoped_completion_identity_repair" - ) - if completion_fence["outcome"] == "replay": - if not dry_run: - try: - store.append_many([], expected_checksum=source_checksum) - except StateEventSourceChangedError: - return _completion_validation_source_drift_failure( - goal_id=goal_id, todo_id=todo_id, dry_run=dry_run - ) - return { - "ok": True, - "dry_run": dry_run, - "completed": True, - "idempotent_replay": True, - "changed": False, - "goal_id": goal_id, - "role": role, - "section": TODO_SECTION_HEADINGS[role], - "todo": item.get("text") or item.get("title"), - "todo_id": todo_id, - "status": TODO_STATUS_DONE, - "completion_continuation": completion_fence.get( - "completion_continuation" - ), - "completion_recovery": item.get("completion_recovery"), - "status_changed": False, - "next_todos": [], - "state_file": str(context.get("state_file") or ""), - "project": str(context.get("project") or "") or None, - "updated_at": item.get("updated_at"), - "source": "event_log", - } - successor_intents = build_successor_intents( - next_agent_todo=next_agent_todo, - next_user_todo=next_user_todo, - next_user_task_class=next_user_task_class, - next_claimed_by=next_claimed_by, - next_task_class=next_task_class, - next_action_kind=next_action_kind, - next_task_repository=next_task_repository, - next_required_capabilities=next_required_capabilities, - next_continuation_policy=next_continuation_policy, - next_excluded_agents=next_excluded_agents, - ) - successor_proposals = derive_successor_proposals( - command="complete", - predecessor=item, - registered_agents=registered_agents, - actor_agent_id=actor_agent_id, - completion_policy={ - "effective_claimed_by": effective_claimed_by, - "effective_next_claimed_by": next_claimed_by, - "effective_next_excluded_agents": next_excluded_agents, - }, - successor_intents=successor_intents, - ) - encoded_successors = [ - _encode_event_projected_successor( - goal_id=goal_id, - proposal=proposal, - updated_at=updated_at, - fields=context["fields"], - actor_agent_id=actor_agent_id, - ) - for proposal in successor_proposals - ] - next_results = [result for result, _ in encoded_successors] - pending_events = [event for _, events in encoded_successors for event in events] - - normalized_successor_todo_ids = merge_todo_id_lists( - successor_todo_ids, - [item.get("todo_id") for item in next_results], - normalize_todo_id_list(item.get("successor_todo_ids")), - ) - if not isinstance(completion_state, Mapping): - raise RuntimeError( - "event-projected Todo completion requires the TypeScript " - "transaction state" - ) - completion_continuation = completion_state.get("continuation") - completion_recovery = completion_state.get("recovery") - if completion_continuation not in { - "active_goal", - "successor", - "no_followup", - } or completion_recovery not in { - None, - "same_turn_terminal_closeout", - "lifecycle_reentry_terminal_closeout", - }: - raise RuntimeError( - "TypeScript Todo completion transaction state shape mismatch" - ) - completion_payload: dict[str, Any] = {"updated_at": updated_at} - completion_payload["completion_continuation"] = completion_continuation - if completion_recovery: - completion_payload["completion_recovery"] = completion_recovery - if not already_done: - completion_payload["completed_at"] = updated_at - if evidence: - completion_payload["evidence"] = evidence - if completion_turn_key: - completion_payload["completion_turn_key"] = completion_turn_key - if note: - completion_payload["note"] = note - if no_followup: - completion_payload["no_followup"] = "true" - if normalized_successor_todo_ids: - completion_payload["successor_todo_ids"] = normalized_successor_todo_ids - completion_event = make_state_event( - event_id=_todo_write_event_id( - goal_id=goal_id, - todo_id=todo_id, - action="complete", - updated_at=updated_at, - text=evidence or note, - ), - goal_id=goal_id, - event_type=TODO_COMPLETED, - refs={"todo_id": todo_id}, - payload=completion_payload, - recorded_at=updated_at, - producer="loopx.todo.complete", - actor_agent_id=actor_agent_id, - ) - if ( - not already_done - or terminal_upgrade - or untyped_completion_repair - or unscoped_identity_repair - ): - pending_events.append(completion_event) - if not dry_run: - try: - store.append_many(pending_events, expected_checksum=source_checksum) - except StateEventSourceChangedError: - return _completion_validation_source_drift_failure( - goal_id=goal_id, todo_id=todo_id, dry_run=dry_run - ) - - result = { - "ok": True, - "dry_run": dry_run, - "completed": True, - "goal_id": goal_id, - "role": role, - "section": TODO_SECTION_HEADINGS[role], - "todo": item.get("text") or item.get("title"), - "todo_id": todo_id, - "status": TODO_STATUS_DONE, - "status_changed": not already_done, - "text_changed": False, - "metadata_updated": ( - (not already_done) - or terminal_upgrade - or untyped_completion_repair - or unscoped_identity_repair - ), - "changed": ( - (not already_done) - or terminal_upgrade - or untyped_completion_repair - or unscoped_identity_repair - or bool(next_results) - ), - "claimed_by": normalize_todo_claimed_by(effective_claimed_by), - "task_class": item.get("task_class"), - "action_kind": item.get("action_kind"), - "capability_binding_ref": item.get("capability_binding_ref"), - "continuation_policy": item.get("continuation_policy"), - "successor_todo_ids": normalized_successor_todo_ids, - "completion_continuation": completion_continuation, - "completion_recovery": completion_recovery, - "next_todos": next_results, - "state_file": str(context.get("state_file") or ""), - "project": str(context.get("project") or "") or None, - "updated_at": updated_at, - "source": "event_log", - } - result["self_merged"] = self_merged - return result diff --git a/loopx/control_plane/todos/goal_todo_projection.py b/loopx/control_plane/todos/goal_todo_projection.py index 0b0b48ef7a..8cf94de6fa 100644 --- a/loopx/control_plane/todos/goal_todo_projection.py +++ b/loopx/control_plane/todos/goal_todo_projection.py @@ -3,7 +3,8 @@ The text is a parameter rather than a file read so a writer that still holds the state-file lock can project the exact bytes it is about to commit; ``loopx.todos.list_goal_todos`` passes the on-disk text. Everything here is a -deterministic function of the text, the goal record, and the event projection. +projection of those bytes and rollout metadata. Retired event sources are +rejected before any Markdown substitution. """ from __future__ import annotations @@ -11,13 +12,12 @@ from pathlib import Path from typing import Any -from ..status.active_state_projection import active_state_event_projection_fields +from ..goals.legacy_event_source import require_no_legacy_todo_events from .active_state_editing import TODO_SECTION_HEADINGS from .active_state_todo_parser import parse_active_state_todos from .list_projection import compact_explicit_limit_todo_summary from .succession_warning import public_todo_summary from .contract import ( - build_todo_id, normalize_todo_claimed_by, normalize_todo_id, normalize_todo_status, @@ -68,112 +68,12 @@ def summary_items(fields: dict[str, Any], role: str) -> list[dict[str, Any]]: return [] return [item for item in summary.get("items") or [] if isinstance(item, dict)] -def merge_todo_projection_fields( - *, - markdown_fields: dict[str, Any], - event_fields: dict[str, Any], -) -> tuple[dict[str, Any], dict[str, Any]]: - merged: dict[str, Any] = {} - merged_items: dict[str, list[dict[str, Any]]] = {"user": [], "agent": []} - source_sections: dict[str, str] = {} - overlay: dict[str, Any] = { - "schema_version": "todo_list_projection_overlay_v0", - "base": "markdown_active_state", - "overlay": "event_projection", - "markdown_only_todo_ids": [], - "event_only_todo_ids": [], - "overlaid_todo_ids": [], - } - - # A todo_id is goal-wide identity. Merge both sources before splitting by - # role so an event-projected role change replaces the stale Markdown item. - by_id: dict[str, dict[str, Any]] = {} - order: list[str] = [] - markdown_ids: set[str] = set() - markdown_ids_by_role: dict[str, set[str]] = {"user": set(), "agent": set()} - event_ids: set[str] = set() - event_order: list[str] = [] - for role in ("user", "agent"): - markdown_items = summary_items(markdown_fields, role) - for item in markdown_items: - todo_id = normalize_todo_id(item.get("todo_id")) or build_todo_id( - role=role, - source_section=item.get("source_section"), - index=item.get("index"), - text=item.get("text"), - ) - if todo_id not in by_id: - order.append(todo_id) - markdown_ids.add(todo_id) - markdown_ids_by_role[role].add(todo_id) - by_id[todo_id] = dict(item) - - for role in ("user", "agent"): - event_items = summary_items(event_fields, role) - for item in event_items: - todo_id = normalize_todo_id(item.get("todo_id")) or build_todo_id( - role=role, - source_section=item.get("source_section"), - index=item.get("index"), - text=item.get("text"), - ) - if todo_id not in by_id: - order.append(todo_id) - if todo_id not in event_ids: - event_order.append(todo_id) - event_ids.add(todo_id) - by_id[todo_id] = dict(item) - - markdown_only_todo_ids: list[str] = [] - seen_markdown_only_ids: set[str] = set() - for role in ("user", "agent"): - for todo_id in sorted(markdown_ids_by_role[role] - event_ids): - if todo_id not in seen_markdown_only_ids: - markdown_only_todo_ids.append(todo_id) - seen_markdown_only_ids.add(todo_id) - overlay["markdown_only_todo_ids"] = markdown_only_todo_ids - overlay["event_only_todo_ids"] = [ - todo_id for todo_id in event_order if todo_id not in markdown_ids - ] - overlay["overlaid_todo_ids"] = [ - todo_id for todo_id in event_order if todo_id in markdown_ids - ] - - for todo_id in order: - item = by_id[todo_id] - final_role = "user" if item.get("role") == "user" else "agent" - merged_items[final_role].append(item) - - for role in ("user", "agent"): - source_section = str( - (markdown_fields.get(f"{role}_todos") or {}).get("source_section") - or (event_fields.get(f"{role}_todos") or {}).get("source_section") - or TODO_SECTION_HEADINGS[role] - ) - source_sections[role] = source_section - - resume_source_items = [*merged_items["user"], *merged_items["agent"]] - for role in ("user", "agent"): - if not merged_items[role]: - continue - summary = compact_todo_group( - merged_items[role], - source_section=source_sections[role], - role=role, - resume_source_items=resume_source_items, - item_limit=None, - ) - if summary: - merged[f"{role}_todos"] = summary - return merged, overlay class GoalTodoSummaries: """Role summaries and todo items projected from one active-state text.""" __slots__ = ( "source", - "projection_fields", - "projection_overlay", "summaries", "todos", "unfiltered_count", @@ -184,16 +84,12 @@ def __init__( self, *, source: str, - projection_fields: dict[str, Any], - projection_overlay: dict[str, Any] | None, summaries: dict[str, dict[str, Any]], todos: list[dict[str, Any]], unfiltered_count: int, uncapped_todo_count: int, ) -> None: self.source = source - self.projection_fields = projection_fields - self.projection_overlay = projection_overlay self.summaries = summaries self.todos = todos self.unfiltered_count = unfiltered_count @@ -218,15 +114,7 @@ def goal_todo_summaries( commit; ``list_goal_todos`` passes the on-disk text. """ - projection_fields = active_state_event_projection_fields( - goal or {}, - state_path=state_path, - item_limit=None, - rollout_events=rollout_events, - ) - projection_has_todos = bool( - projection_fields.get("user_todos") or projection_fields.get("agent_todos") - ) + require_no_legacy_todo_events(goal or {}, state_path=state_path) markdown_fields = parse_active_state_todos( state_text, goal=goal, @@ -234,28 +122,9 @@ def goal_todo_summaries( item_limit=None, rollout_events=rollout_events, ) - markdown_has_todos = bool( - markdown_fields.get("user_todos") or markdown_fields.get("agent_todos") - ) - projection_overlay: dict[str, Any] | None = None - if projection_has_todos and markdown_has_todos: - fields, projection_overlay = merge_todo_projection_fields( - markdown_fields=markdown_fields, - event_fields=projection_fields, - ) - source = "event_projection_with_markdown_overlay" - elif projection_has_todos: - fields = projection_fields - source = "event_projection" - else: - fields = markdown_fields - source = "markdown_active_state" - return todo_summaries_from_fields( - fields=fields, - source=source, - projection_fields=projection_fields, - projection_overlay=projection_overlay, + fields=markdown_fields, + source="markdown_active_state", rollout_events=rollout_events, roles=roles, status=status, @@ -269,8 +138,6 @@ def todo_summaries_from_fields( *, fields: dict[str, Any], source: str, - projection_fields: dict[str, Any] | None, - projection_overlay: dict[str, Any] | None, rollout_events: list[dict[str, Any]], roles: list[str], status: str | None, @@ -308,8 +175,6 @@ def todo_summaries_from_fields( uncapped_todo_count += int(summary.get("total_count") or 0) return GoalTodoSummaries( source=source, - projection_fields=projection_fields or {}, - projection_overlay=projection_overlay, summaries=summaries, todos=todos, unfiltered_count=unfiltered_count, @@ -321,8 +186,6 @@ def exact_archived_todo_summaries( *, archived_items: list[dict[str, Any]], source: str, - projection_fields: dict[str, Any] | None, - projection_overlay: dict[str, Any] | None, rollout_events: list[dict[str, Any]], roles: list[str], status: str | None, @@ -360,8 +223,6 @@ def exact_archived_todo_summaries( return todo_summaries_from_fields( fields={f"{item_role}_todos": summary}, source=source, - projection_fields=projection_fields, - projection_overlay=projection_overlay, rollout_events=rollout_events, roles=roles, status=status, @@ -402,7 +263,6 @@ def project_goal_todo_items( "exact_archived_todo_summaries", "filtered_todo_summary", "goal_todo_summaries", - "merge_todo_projection_fields", "project_goal_todo_items", "summary_items", "todo_summaries_from_fields", diff --git a/loopx/control_plane/todos/handoff_mode.py b/loopx/control_plane/todos/handoff_mode.py index 873e94f064..b7cf1cb1e9 100644 --- a/loopx/control_plane/todos/handoff_mode.py +++ b/loopx/control_plane/todos/handoff_mode.py @@ -22,9 +22,9 @@ transaction. Stale or missing Markdown and local lease files are not fallback sources. The legacy mode below remains a frontmatter compatibility contract. -Unpromoted transitions read the complete Todo event overlay under the append -store locks and local leases under the lease mutex. The same typed quiescence -rule governs both paths; malformed event sources cannot become empty evidence. +Unpromoted transitions read complete Markdown Todos and local leases under +their writer mutexes. The same typed quiescence rule governs both paths; +nonempty retired Todo event sources are refused, never treated as empty. """ from __future__ import annotations diff --git a/loopx/control_plane/todos/handoff_mode_source.py b/loopx/control_plane/todos/handoff_mode_source.py index 23c04e0f01..3ab28a85b7 100644 --- a/loopx/control_plane/todos/handoff_mode_source.py +++ b/loopx/control_plane/todos/handoff_mode_source.py @@ -1,7 +1,7 @@ """Locked compatibility inputs for the typed handoff transition planner. -The active-state writer mutex is held by the caller. Event locks use the -append store's own lock and remain held until the mode write is durable. +The caller holds the active-state mutex; local leases share their existing +mutex. Retired Todo event sources are rejected before quiescence is projected. """ from __future__ import annotations @@ -11,45 +11,30 @@ from typing import Any -def _event_paths(registry_path: Path, goal_id: str, state_path: Path) -> tuple[dict[str, Any], list[Path]]: - from ...history import load_registry - from ...registry import find_registry_goal - from ..status.active_state_projection import state_event_log_candidates - - goal = find_registry_goal(load_registry(registry_path), goal_id) or {"id": goal_id} - paths = state_event_log_candidates(goal, state_path=state_path) - return goal, sorted({path.expanduser().resolve() for path in paths}, key=str) - - @contextmanager def handoff_mode_source( *, registry_path: Path, goal_id: str, state_path: Path, state_text: str, runtime_root: Path, ) -> Iterator[dict[str, Any]]: - from ...event_sourced_state import AppendOnlyStateEventStore, StateEventError - from ...file_lock import exclusive_file_lock, exclusive_cross_runtime_file_lock + from ..goals.legacy_event_source import require_no_legacy_todo_events, RetiredTodoEventSourceError + from ...file_lock import exclusive_cross_runtime_file_lock from ..runtime.time import now_local_iso from ..work_items.task_lease import read_lease, task_lease_dir, task_lease_lock_path from .goal_todo_projection import project_goal_todo_items from .handoff_mode import HandoffModeError - goal, paths = _event_paths(registry_path, goal_id, state_path) + from ...history import load_registry + from ...registry import find_registry_goal + goal = find_registry_goal(load_registry(registry_path), goal_id) or {"id": goal_id} with ExitStack() as stack: - # Same ordering for every mode writer; state -> event logs -> leases. - # Lock absent candidates too, so the first append cannot race the scan. - for path in paths: - stack.enter_context(exclusive_file_lock(path, operation="handoff_mode_set")) stack.enter_context(exclusive_cross_runtime_file_lock( task_lease_lock_path(runtime_root=runtime_root, goal_id=goal_id), operation="handoff_mode_set")) try: - # Display projection can warn and fall back on malformed events. - # A safety decision must instead prove every candidate readable. - for path in paths: - AppendOnlyStateEventStore(path).load() + require_no_legacy_todo_events(goal, state_path=state_path) todos = project_goal_todo_items(goal, state_text=state_text, state_path=state_path, rollout_events=[]) - except (OSError, StateEventError) as error: - raise HandoffModeError("cannot establish handoff quiescence from the event sources", + except (OSError, RetiredTodoEventSourceError) as error: + raise HandoffModeError("cannot establish handoff quiescence from the legacy source", code="handoff_mode_source_unavailable") from error fields = ("todo_id", "done", "status", "claimed_by", "archive_state") leases = [] diff --git a/loopx/control_plane/todos/list_projection.py b/loopx/control_plane/todos/list_projection.py index 18eef4f1fd..3db00ff405 100644 --- a/loopx/control_plane/todos/list_projection.py +++ b/loopx/control_plane/todos/list_projection.py @@ -347,20 +347,6 @@ def compact_thin_todo_list_payload(payload: dict[str, Any]) -> dict[str, Any]: compact["todo"] = matched compact["relations"] = todo_item_relations(matched) if matched else {} - overlay = compact.get("projection_overlay") - if isinstance(overlay, dict): - compact["projection_overlay"] = compact_todo_projection_overlay( - overlay, - full_detail_cold_path="todo list without --thin or active state", - ) - state_event_projection = compact.get("state_event_projection") - if isinstance(state_event_projection, dict): - compact["state_event_projection"] = { - key: state_event_projection[key] - for key in ("schema_version", "source_event_count", "last_event_id") - if key in state_event_projection - } - compact["thin"] = True compact["todo_list_field_projection"] = thin_todo_list_field_projection_contract( matched_todo_count=matched_todo_count, @@ -501,31 +487,6 @@ def compact_explicit_limit_todo_summary( return compact -AGENT_LANE_OVERLAY_FULL_DETAIL_COLD_PATH = ( - "todo list without --agent-id or active state" -) -EXPLICIT_LIMIT_OVERLAY_FULL_DETAIL_COLD_PATH = ( - "todo list without --limit or active state" -) - - -def compact_todo_projection_overlay( - value: Any, - *, - full_detail_cold_path: str = AGENT_LANE_OVERLAY_FULL_DETAIL_COLD_PATH, -) -> Any: - if not isinstance(value, dict): - return value - compact = { - key: child for key, child in value.items() if not isinstance(child, list) - } - for key, child in value.items(): - if isinstance(child, list): - compact[f"{key.removesuffix('_todo_ids')}_count"] = len(child) - compact["full_detail_cold_path"] = full_detail_cold_path - return compact - - def todo_list_projection_contract( *, matched_todo_count: int, diff --git a/loopx/control_plane/todos/markdown.py b/loopx/control_plane/todos/markdown.py index 108cb68c96..84d9f8b9a5 100644 --- a/loopx/control_plane/todos/markdown.py +++ b/loopx/control_plane/todos/markdown.py @@ -119,18 +119,6 @@ def render_todo_markdown(payload: dict[str, Any]) -> str: lines.append( f"- returned_todo_count: `{payload.get('returned_todo_count')}`" ) - state_event_projection = payload.get("state_event_projection") - if isinstance(state_event_projection, dict): - lines.extend( - [ - f"- event_log: `{state_event_projection.get('event_log')}`", - ( - "- source_event_count: `" - f"{state_event_projection.get('source_event_count')}`" - ), - f"- last_event_id: `{state_event_projection.get('last_event_id')}`", - ] - ) for key, heading in ( ("user_todos", "User Todo"), ("agent_todos", "Agent Todo"), diff --git a/loopx/control_plane/work_items/task_lease_acquire_adapter.py b/loopx/control_plane/work_items/task_lease_acquire_adapter.py index 162fd1e0ff..e009d15825 100644 --- a/loopx/control_plane/work_items/task_lease_acquire_adapter.py +++ b/loopx/control_plane/work_items/task_lease_acquire_adapter.py @@ -20,10 +20,9 @@ TASK_LEASE_LIFECYCLE_REQUEST_SCHEMA, ) from ..coordination.runtime_shadow import resolve_coordination_runtime_shadow_config -from ..goals.active_state_event_projection import ( +from ..goals.legacy_event_source import ( state_event_log_candidates as _state_event_log_candidates, ) -from ..goals.path_resolution import resolve_goal_local_path from ..todos.contract import normalize_todo_id from ..todos.handoff_mode import HANDOFF_MODE_LEGACY from .local_lease_record import TASK_LEASE_SCHEMA_VERSION, TaskLeaseError @@ -91,7 +90,6 @@ def _task_lease_authority_source_paths( _state_event_log_candidates( goal, state_path=state_file, - resolve_goal_local_path=resolve_goal_local_path, ) ): sources.append((f"state_event_{index}", path)) diff --git a/loopx/event_sourced_state.py b/loopx/event_sourced_state.py deleted file mode 100644 index 82f97dffc0..0000000000 --- a/loopx/event_sourced_state.py +++ /dev/null @@ -1,1134 +0,0 @@ -from __future__ import annotations - -import hashlib -import json -import re -from dataclasses import dataclass -from pathlib import Path -from typing import Any, Iterable - -from .control_plane.runtime.time import now_utc_iso as runtime_now_utc_iso -from .file_lock import exclusive_file_lock -from .control_plane.todos.contract import ( - TODO_MONITOR_METADATA_FIELDS, - TODO_STATUS_DONE, - TODO_STATUS_BLOCKED, - TODO_STATUS_DEFERRED, - TODO_STATUS_OPEN, - TODO_TASK_PATTERN, - build_todo_id, - format_todo_metadata_line, - normalize_explicit_todo_task_class, - normalize_required_capabilities, - normalize_required_write_scopes, - normalize_removed_todo_continuation_policy, - normalize_todo_action_kind, - normalize_todo_blocks_agent, - normalize_todo_bound_agent, - normalize_todo_capability_binding_ref, - normalize_todo_claimed_by, - normalize_todo_continuation_policy, - normalize_todo_excluded_agents, - normalize_todo_global_gate, - normalize_todo_goal_bound, - normalize_todo_id, - normalize_todo_id_list, - normalize_todo_status, - normalize_todo_task_domain, - normalize_todo_task_repository, - parse_todo_metadata_line, - todo_done_for_status, - todo_marker_for_status, - todo_status_from_marker, -) - - -STATE_EVENT_SCHEMA_VERSION = "loopx_state_event_v0" -STATE_PROJECTION_SCHEMA_VERSION = "event_sourced_state_projection_v0" -STATE_PROJECTION_VERSION = "event_sourced_state_contract_v0" - -PUBLIC_PRIVACY = "public_safe" -LOCAL_PRIVATE_PRIVACY = "local_private" -PRIVATE_POINTER_PRIVACY = "private_pointer" -PRIVACY_VALUES = {PUBLIC_PRIVACY, LOCAL_PRIVATE_PRIVACY, PRIVATE_POINTER_PRIVACY} - -TODO_ADDED = "todo_added" -TODO_CLAIMED = "todo_claimed" -TODO_UPDATED = "todo_updated" -TODO_BLOCKED = "todo_blocked" -TODO_DEFERRED = "todo_deferred" -TODO_COMPLETED = "todo_completed" -REFRESH_RECORDED = "refresh_recorded" -RUN_RECORDED = "run_recorded" -QUOTA_SPENT = "quota_spent" -EVIDENCE_ATTACHED = "evidence_attached" -SUPERVISOR_PROPOSED = "supervisor_proposed" -SUPERVISOR_RECEIPT_RECORDED = "supervisor_receipt_recorded" - -MARKDOWN_BACKFILL_PRODUCER = "loopx.backfill" -MARKDOWN_HEADING_PATTERN = re.compile(r"^##\s+(.+?)\s*$") -TODO_PRIORITY_PREFIX_PATTERN = re.compile(r"^\[(P[0-4])\]\s+(.+)$", re.IGNORECASE) -PUBLIC_BACKFILL_REDACTION = "[redacted-private-state]" -PUBLIC_BACKFILL_UNSAFE_PATTERNS = ( - re.compile(r"(?i)(?:^|[\s`'\"])(?:/Users/|/private/|/var/folders/)"), - re.compile(r"(?i)\b(?:secret|password|credential|token)\b"), - re.compile(r"https?://"), -) - -SUPPORTED_EVENT_TYPES = { - TODO_ADDED, - TODO_CLAIMED, - TODO_UPDATED, - TODO_BLOCKED, - TODO_DEFERRED, - TODO_COMPLETED, - REFRESH_RECORDED, - RUN_RECORDED, - QUOTA_SPENT, - EVIDENCE_ATTACHED, - SUPERVISOR_PROPOSED, - SUPERVISOR_RECEIPT_RECORDED, -} - -TODO_EVENT_TYPES = { - TODO_ADDED, - TODO_CLAIMED, - TODO_UPDATED, - TODO_BLOCKED, - TODO_DEFERRED, - TODO_COMPLETED, -} - - -class StateEventError(ValueError): - """Raised when a state event cannot be accepted or replayed.""" - - -class StateEventConflictError(StateEventError): - """Raised when a duplicate event id carries different event content.""" - - -class StateEventSourceChangedError(StateEventError): - """The locked event stream differs from the basis used to plan the write.""" - - -class StateEventCommitUnknownError(StateEventError): - """Publication may have landed; read back before repeating business work.""" - - -def now_utc_iso() -> str: - return runtime_now_utc_iso() - - -def compact_text(value: Any) -> str: - return " ".join(str(value or "").strip().split()) - - -def _copy_todo_added_validation_fields( - source: dict[str, Any], target: dict[str, Any] -) -> None: - """Copy caller-approved completion validation fields from a TODO_ADDED payload.""" - for key in ( - "validation_command", - "validation_label", - "validation_timeout_seconds", - ): - if key in source and source.get(key) is not None: - target[key] = source[key] - if "validation_command_argv" in source: - target["validation_command_argv"] = source.get("validation_command_argv") - - -def _redact_public_backfill_text(value: Any, *, privacy: str) -> str: - text = compact_text(value) - if privacy != PUBLIC_PRIVACY: - return text - if any(pattern.search(text) for pattern in PUBLIC_BACKFILL_UNSAFE_PATTERNS): - return PUBLIC_BACKFILL_REDACTION - return text - - -def _redact_public_backfill_source_ref(value: Any, *, privacy: str) -> str: - text = compact_text(value) or "ACTIVE_GOAL_STATE.md" - if privacy != PUBLIC_PRIVACY: - return text - if any(pattern.search(text) for pattern in PUBLIC_BACKFILL_UNSAFE_PATTERNS): - return "ACTIVE_GOAL_STATE.md" - return text - - -def _role_for_markdown_heading(heading: str) -> str | None: - normalized = compact_text(heading).lower() - if normalized.startswith("user todo") or "owner review" in normalized: - return "user" - if normalized.startswith("agent todo"): - return "agent" - return None - - -def _todo_priority_and_title(text: Any, *, privacy: str) -> tuple[str, str]: - compact = compact_text(text) - match = TODO_PRIORITY_PREFIX_PATTERN.match(compact) - if match: - return match.group(1).upper(), _redact_public_backfill_text(match.group(2), privacy=privacy) - return "P2", _redact_public_backfill_text(compact, privacy=privacy) - - -def _backfill_event_id(*, goal_id: str, todo_id: str, suffix: str) -> str: - digest = hashlib.sha1(f"{goal_id}|{todo_id}|{suffix}".encode("utf-8")).hexdigest()[:16] - return f"backfill-{suffix}-{digest}" - - -def _backfill_source_refs( - *, - source_ref: str, - source_section: str, - source_line: int | None, - privacy: str, -) -> dict[str, Any]: - refs: dict[str, Any] = { - "source_ref": _redact_public_backfill_source_ref(source_ref, privacy=privacy), - "source_section": compact_text(source_section), - } - if source_line is not None: - refs["source_line"] = source_line - return refs - - -def _markdown_todo_records(state_text: str) -> list[dict[str, Any]]: - role: str | None = None - source_section: str | None = None - current: dict[str, Any] | None = None - records: list[dict[str, Any]] = [] - role_indexes = {"user": 0, "agent": 0} - - for line_number, line in enumerate(state_text.splitlines(), start=1): - heading_match = MARKDOWN_HEADING_PATTERN.match(line) - if heading_match: - source_section = compact_text(heading_match.group(1)) - role = _role_for_markdown_heading(source_section) - current = None - continue - if role is None or source_section is None: - continue - todo_match = TODO_TASK_PATTERN.match(line) - if todo_match: - marker, text = todo_match.groups() - role_indexes[role] += 1 - current = { - "role": role, - "source_section": source_section, - "source_line": line_number, - "planner_order": role_indexes[role], - "status": todo_status_from_marker(marker), - "text": compact_text(text), - } - records.append(current) - continue - if current is None or not line.startswith((" ", "\t")): - continue - metadata = parse_todo_metadata_line(line) - if metadata: - current.update(metadata) - continue - continuation = compact_text(line) - if continuation: - current["text"] = compact_text(f"{current.get('text', '')} {continuation}") - for record in records: - role = str(record.get("role") or "agent") - source_section = str(record.get("source_section") or "") - title_text = compact_text(record.get("text")) - if not normalize_todo_id(record.get("todo_id")): - record["todo_id"] = build_todo_id( - role=role, - source_section=source_section, - index=record.get("planner_order"), - text=title_text, - ) - record["status"] = normalize_todo_status(record.get("status")) or TODO_STATUS_OPEN - return records - - -def backfill_todo_events_from_markdown( - state_text: str, - *, - goal_id: str, - source_ref: str = "ACTIVE_GOAL_STATE.md", - recorded_at: str | None = None, - producer: str = MARKDOWN_BACKFILL_PRODUCER, - privacy: str = LOCAL_PRIVATE_PRIVACY, -) -> list[dict[str, Any]]: - """Convert Markdown workbench todos into idempotent append-only events. - - The helper does not mutate the Markdown source. When writing a public-safe - stream, compact todo titles/evidence/reasons that look like private state - are redacted; local-private streams preserve the workbench text. - """ - normalized_goal_id = compact_text(goal_id) - if not normalized_goal_id: - raise StateEventError("goal_id is required") - if privacy not in PRIVACY_VALUES: - raise StateEventError(f"privacy must be one of: {', '.join(sorted(PRIVACY_VALUES))}") - - events: list[dict[str, Any]] = [] - for record in _markdown_todo_records(state_text): - role = str(record.get("role") or "agent") - todo_id = normalize_todo_id(record.get("todo_id")) or build_todo_id( - role=role, - source_section=record.get("source_section"), - index=record.get("planner_order"), - text=record.get("text"), - ) - priority, title = _todo_priority_and_title(record.get("text"), privacy=privacy) - refs = { - "todo_id": todo_id, - **_backfill_source_refs( - source_ref=source_ref, - source_section=str(record.get("source_section") or ""), - source_line=record.get("source_line"), - privacy=privacy, - ), - } - payload: dict[str, Any] = { - "role": role, - "priority": priority, - "title": title, - "planner_order": record.get("planner_order"), - } - task_class = normalize_explicit_todo_task_class(record.get("task_class")) - action_kind = normalize_todo_action_kind(record.get("action_kind")) - task_domain = normalize_todo_task_domain(record.get("task_domain")) - capability_binding_ref = normalize_todo_capability_binding_ref( - record.get("capability_binding_ref") - ) - task_repository = normalize_todo_task_repository( - record.get("task_repository") - ) - continuation_policy = normalize_todo_continuation_policy( - record.get("continuation_policy") - ) - removed_continuation_policy = normalize_removed_todo_continuation_policy( - record.get("removed_continuation_policy") - ) - required_write_scopes = normalize_required_write_scopes( - record.get("required_write_scopes") - ) - required_capabilities = normalize_required_capabilities( - record.get("required_capabilities") - ) - blocks_agent = normalize_todo_blocks_agent(record.get("blocks_agent")) - bound_agent = normalize_todo_bound_agent(record.get("bound_agent")) - goal_bound = normalize_todo_goal_bound(record.get("goal_bound")) - global_gate = normalize_todo_global_gate(record.get("global_gate")) - excluded_agents = normalize_todo_excluded_agents(record.get("excluded_agents")) - if task_class: - payload["task_class"] = task_class - if action_kind: - payload["action_kind"] = action_kind - if task_domain: - payload["task_domain"] = task_domain - if capability_binding_ref: - payload["capability_binding_ref"] = capability_binding_ref - if task_repository: - payload["task_repository"] = task_repository - if continuation_policy: - payload["continuation_policy"] = continuation_policy - if removed_continuation_policy: - payload["removed_continuation_policy"] = removed_continuation_policy - if required_write_scopes: - payload["required_write_scopes"] = required_write_scopes - if required_capabilities: - payload["required_capabilities"] = required_capabilities - if blocks_agent: - payload["blocks_agent"] = blocks_agent - if bound_agent: - payload["bound_agent"] = bound_agent - if goal_bound is not None: - payload["goal_bound"] = goal_bound - if global_gate is not None: - payload["global_gate"] = global_gate - if excluded_agents: - payload["excluded_agents"] = excluded_agents - _copy_todo_added_validation_fields(record, payload) - if privacy == PUBLIC_PRIVACY: - for key in ( - "validation_command", - "validation_command_argv", - "validation_label", - ): - if key in payload: - payload[key] = _redact_public_backfill_text( - payload[key], privacy=privacy - ) - events.append( - make_state_event( - event_id=_backfill_event_id(goal_id=normalized_goal_id, todo_id=todo_id, suffix="add"), - goal_id=normalized_goal_id, - event_type=TODO_ADDED, - refs=refs, - payload=payload, - recorded_at=recorded_at, - producer=producer, - privacy=privacy, - ) - ) - - claimed_by = normalize_todo_claimed_by(record.get("claimed_by")) - if claimed_by: - events.append( - make_state_event( - event_id=_backfill_event_id(goal_id=normalized_goal_id, todo_id=todo_id, suffix="claim"), - goal_id=normalized_goal_id, - event_type=TODO_CLAIMED, - refs=refs, - payload={"claimed_by": claimed_by}, - recorded_at=recorded_at, - producer=producer, - privacy=privacy, - ) - ) - - status = normalize_todo_status(record.get("status")) or TODO_STATUS_OPEN - if status == TODO_STATUS_DONE: - completion_payload: dict[str, Any] = {} - for key in ("evidence", "reason"): - if record.get(key): - completion_payload[key] = _redact_public_backfill_text(record[key], privacy=privacy) - events.append( - make_state_event( - event_id=_backfill_event_id(goal_id=normalized_goal_id, todo_id=todo_id, suffix="complete"), - goal_id=normalized_goal_id, - event_type=TODO_COMPLETED, - refs=refs, - payload=completion_payload, - recorded_at=recorded_at, - producer=producer, - privacy=privacy, - ) - ) - elif status == TODO_STATUS_DEFERRED: - deferred_payload = {} - for key in ("reason", "resume_when"): - if record.get(key): - deferred_payload[key] = _redact_public_backfill_text(record[key], privacy=privacy) - events.append( - make_state_event( - event_id=_backfill_event_id(goal_id=normalized_goal_id, todo_id=todo_id, suffix="defer"), - goal_id=normalized_goal_id, - event_type=TODO_DEFERRED, - refs=refs, - payload=deferred_payload, - recorded_at=recorded_at, - producer=producer, - privacy=privacy, - ) - ) - elif status == TODO_STATUS_BLOCKED: - blocked_payload = {} - if record.get("reason"): - blocked_payload["reason"] = _redact_public_backfill_text(record["reason"], privacy=privacy) - events.append( - make_state_event( - event_id=_backfill_event_id(goal_id=normalized_goal_id, todo_id=todo_id, suffix="block"), - goal_id=normalized_goal_id, - event_type=TODO_BLOCKED, - refs=refs, - payload=blocked_payload, - recorded_at=recorded_at, - producer=producer, - privacy=privacy, - ) - ) - return events - - -def _sorted_dict(value: dict[str, Any]) -> dict[str, Any]: - return {key: value[key] for key in sorted(value)} - - -def event_fingerprint(event: dict[str, Any]) -> str: - comparable = {key: value for key, value in event.items() if key != "append_sequence"} - return json.dumps(comparable, sort_keys=True, separators=(",", ":"), ensure_ascii=False) - - -def event_stream_checksum(events: Iterable[dict[str, Any]]) -> str: - body = "\n".join( - json.dumps(event, sort_keys=True, separators=(",", ":"), ensure_ascii=False) - for event in events - ) - return "sha256:" + hashlib.sha256(body.encode("utf-8")).hexdigest() - - -def _require_dict(value: Any, *, field_name: str) -> dict[str, Any]: - if value is None: - return {} - if not isinstance(value, dict): - raise StateEventError(f"{field_name} must be an object") - return dict(value) - - -def normalize_state_event(event: dict[str, Any], *, append_sequence: int | None = None) -> dict[str, Any]: - if not isinstance(event, dict): - raise StateEventError("event must be an object") - - event_id = compact_text(event.get("event_id")) - goal_id = compact_text(event.get("goal_id")) - event_type = compact_text(event.get("event_type")) - if not event_id: - raise StateEventError("event_id is required") - if not goal_id: - raise StateEventError("goal_id is required") - if event_type not in SUPPORTED_EVENT_TYPES: - raise StateEventError(f"unsupported event_type: {event_type}") - - refs = _require_dict(event.get("refs"), field_name="refs") - if refs.get("mutates_prior_event_id"): - raise StateEventError("events must not mutate prior events") - payload = _require_dict(event.get("payload"), field_name="payload") - - if event_type in TODO_EVENT_TYPES: - todo_id = normalize_todo_id(refs.get("todo_id")) - if not todo_id: - raise StateEventError(f"{event_type} requires refs.todo_id") - refs["todo_id"] = todo_id - if payload.get("capability_binding_ref") is not None: - capability_binding_ref = normalize_todo_capability_binding_ref( - payload.get("capability_binding_ref") - ) - if not capability_binding_ref: - raise StateEventError( - "capability_binding_ref must be a public-safe namespaced token" - ) - payload["capability_binding_ref"] = capability_binding_ref - if payload.get("task_domain") is not None: - task_domain = normalize_todo_task_domain(payload.get("task_domain")) - if not task_domain: - raise StateEventError( - "task_domain must be a public-safe lowercase token" - ) - payload["task_domain"] = task_domain - - privacy = compact_text(event.get("privacy") or PUBLIC_PRIVACY) - if privacy not in PRIVACY_VALUES: - raise StateEventError(f"privacy must be one of: {', '.join(sorted(PRIVACY_VALUES))}") - - sequence = append_sequence if append_sequence is not None else event.get("append_sequence") - if sequence is not None: - if isinstance(sequence, bool) or not isinstance(sequence, int): - raise StateEventError("append_sequence must be an integer") - if sequence < 1: - raise StateEventError("append_sequence must be positive") - - normalized = { - "schema_version": compact_text(event.get("schema_version") or STATE_EVENT_SCHEMA_VERSION), - "event_id": event_id, - "goal_id": goal_id, - "event_type": event_type, - "recorded_at": compact_text(event.get("recorded_at") or now_utc_iso()), - "producer": compact_text(event.get("producer") or "loopx.event_sourced_state"), - "privacy": privacy, - "projection_version": compact_text(event.get("projection_version") or STATE_PROJECTION_VERSION), - "refs": _sorted_dict(refs), - "payload": _sorted_dict(payload), - } - actor_agent_id = normalize_todo_claimed_by(event.get("actor_agent_id")) - if actor_agent_id: - normalized["actor_agent_id"] = actor_agent_id - if normalized["schema_version"] != STATE_EVENT_SCHEMA_VERSION: - raise StateEventError(f"schema_version must be {STATE_EVENT_SCHEMA_VERSION}") - if sequence is not None: - normalized["append_sequence"] = sequence - return normalized - - -def make_state_event( - *, - event_id: str, - goal_id: str, - event_type: str, - refs: dict[str, Any] | None = None, - payload: dict[str, Any] | None = None, - recorded_at: str | None = None, - producer: str | None = None, - privacy: str = PUBLIC_PRIVACY, - projection_version: str = STATE_PROJECTION_VERSION, - actor_agent_id: str | None = None, -) -> dict[str, Any]: - return normalize_state_event( - { - "schema_version": STATE_EVENT_SCHEMA_VERSION, - "event_id": event_id, - "goal_id": goal_id, - "event_type": event_type, - "recorded_at": recorded_at or now_utc_iso(), - "producer": producer or "loopx.event_sourced_state", - "privacy": privacy, - "projection_version": projection_version, - "refs": refs or {}, - "payload": payload or {}, - "actor_agent_id": actor_agent_id, - } - ) - - -@dataclass -class AppendOnlyStateEventStore: - path: Path - - def load(self) -> list[dict[str, Any]]: - events: list[dict[str, Any]] = [] - if self.path.exists(): - for line_number, line in enumerate(self.path.read_text(encoding="utf-8").splitlines(), start=1): - if not line.strip(): - continue - try: - raw = json.loads(line) - except json.JSONDecodeError as exc: - raise StateEventError(f"invalid JSONL at line {line_number}: {exc}") from exc - events.append(normalize_state_event(raw)) - return _dedupe_events(events) - - def append(self, event: dict[str, Any]) -> dict[str, Any]: - return self.append_many((event,))[0] - - def append_many( - self, - events: Iterable[dict[str, Any]], - *, - expected_checksum: str | None = None, - ) -> list[dict[str, Any]]: - """Publish an eager batch atomically; lazy iterables retain per-item visibility. - - A compare-and-append requires an eager batch. Materialize caller-owned - iterators outside this method when one atomic publication is intended. - """ - if type(events) not in (list, tuple): - if expected_checksum is not None: - raise StateEventError("a source-bound append requires a list or tuple") - return [self.append(event) for event in events] - if not events and expected_checksum is None: - return [] - - from .control_plane.effect_runtime import effect_runtime_result - from .control_plane.todos.active_state_editing import ( - atomic_write_state_text, - verify_state_text_durable, - ) - - # Validate all caller data before entering the lock or publishing any - # bytes. Sequence allocation is deliberately deferred to the TS owner. - normalized = [ - normalize_state_event(event, append_sequence=1) for event in events - ] - requested_ids = {item["event_id"] for item in normalized} - - def identity(item: dict[str, Any]) -> dict[str, Any]: - return { - "event_id": item["event_id"], - "fingerprint": hashlib.sha256( - event_fingerprint(item).encode("utf-8") - ).hexdigest(), - } - - with exclusive_file_lock(self.path): - stored = self.load() - existing = {item["event_id"]: item for item in stored} - plan = effect_runtime_result( - "goal.state_event.plan_append", - { - "schema_version": "loopx_state_event_append_plan_v0", - "source_checksum": event_stream_checksum( - sorted(stored, key=event_sort_key) - ), - "expected_checksum": expected_checksum, - "last_sequence": max( - (int(item["append_sequence"]) for item in stored), default=0 - ), - "existing": [ - {**identity(item), "append_sequence": item["append_sequence"]} - for item in stored - if item["event_id"] in requested_ids - ], - "events": [identity(item) for item in normalized], - }, - ) - if plan.get("schema_version") != "loopx_state_event_append_result_v0": - raise StateEventError("invalid event append plan result schema") - if plan.get("status") != "planned": - reason = plan.get("reason_code") - if reason == "event_source_changed": - raise StateEventSourceChangedError( - "event source changed before append" - ) - if reason == "event_id_conflict": - raise StateEventConflictError( - f"conflicting event_id: {plan['event_id']}" - ) - raise StateEventError(str(reason or "invalid event append plan")) - choices = plan.get("choices") - if not isinstance(choices, list) or len(choices) != len(normalized): - raise StateEventError("invalid event append plan choices") - appended: list[dict[str, Any]] = [] - additions: list[dict[str, Any]] = [] - for event, choice in zip(normalized, choices, strict=True): - if ( - not isinstance(choice, dict) - or choice.get("event_id") != event["event_id"] - or choice.get("kind") not in {"append", "replay"} - or isinstance(choice.get("append_sequence"), bool) - or not isinstance(choice.get("append_sequence"), int) - or not 1 <= choice["append_sequence"] <= 2**53 - 1 - ): - raise StateEventError("invalid event append plan choice") - if choice["kind"] == "append": - event["append_sequence"] = choice["append_sequence"] - existing[event["event_id"]] = event - additions.append(event) - appended.append(existing[event["event_id"]]) - # Preserve all historical bytes, including harmless blank lines. - # Replacing the whole file changes no prior event or sequence. - prior_text = ( - self.path.read_bytes().decode("utf-8") if self.path.exists() else "" - ) - if additions: - separator = "\n" if prior_text and not prior_text.endswith("\n") else "" - suffix = "".join( - json.dumps(item, sort_keys=True, ensure_ascii=False) + "\n" - for item in additions - ) - try: - atomic_write_state_text(self.path, prior_text + separator + suffix) - except OSError as error: - raise StateEventCommitUnknownError( - "event append outcome uncertain; read back the event stream before retrying the original operation" - ) from error - else: - # A prior replace may have succeeded before directory fsync - # failed. Exact replay must establish durability, not just see it. - if self.path.exists(): - try: - verify_state_text_durable(self.path, prior_text) - except OSError as error: - raise StateEventCommitUnknownError( - "event replay durability uncertain; read back the event stream before retrying" - ) from error - return appended - - -def _dedupe_events(events: Iterable[dict[str, Any]]) -> list[dict[str, Any]]: - by_id: dict[str, dict[str, Any]] = {} - ordered: list[dict[str, Any]] = [] - for event in events: - prior = by_id.get(event["event_id"]) - if prior is not None: - if event_fingerprint(prior) != event_fingerprint(event): - raise StateEventConflictError( - f"conflicting event_id: {event['event_id']}" - ) - continue - by_id[event["event_id"]] = event - ordered.append(event) - return ordered - - -def event_sort_key(event: dict[str, Any]) -> tuple[int, str, str]: - return ( - int(event.get("append_sequence") or 0), - str(event.get("recorded_at") or ""), - str(event.get("event_id") or ""), - ) - - -def _decode_added_todo_content(event: dict[str, Any]) -> dict[str, Any]: - payload = event.get("payload") or {} - refs = event.get("refs") or {} - text = compact_text(payload.get("text") or payload.get("title")) - todo_id = refs["todo_id"] # The legacy decoder already requires this identity. - task_class = normalize_explicit_todo_task_class(payload.get("task_class")) - action_kind = normalize_todo_action_kind(payload.get("action_kind")) - task_domain = normalize_todo_task_domain(payload.get("task_domain")) - capability_binding_ref = normalize_todo_capability_binding_ref( - payload.get("capability_binding_ref") - ) - task_repository = normalize_todo_task_repository(payload.get("task_repository")) - continuation_policy = normalize_todo_continuation_policy( - payload.get("continuation_policy") - ) - removed_continuation_policy = normalize_removed_todo_continuation_policy( - payload.get("removed_continuation_policy") - or payload.get("continuation_policy") - ) - required_write_scopes = normalize_required_write_scopes( - payload.get("required_write_scopes") - ) - required_capabilities = normalize_required_capabilities( - payload.get("required_capabilities") - ) - blocks_agent = normalize_todo_blocks_agent(payload.get("blocks_agent")) - bound_agent = normalize_todo_bound_agent(payload.get("bound_agent")) - goal_bound = normalize_todo_goal_bound(payload.get("goal_bound")) - global_gate = normalize_todo_global_gate(payload.get("global_gate")) - excluded_agents = normalize_todo_excluded_agents(payload.get("excluded_agents")) - claimed_by = normalize_todo_claimed_by(payload.get("claimed_by")) - actor_agent_id = normalize_todo_claimed_by(event.get("actor_agent_id")) - todo: dict[str, Any] = { - "schema_version": "todo_item_v0", "todo_id": todo_id, - "title": text, "text": text, "planner_order": payload.get("planner_order"), - "append_sequence": event.get("append_sequence"), - "last_event_id": event.get("event_id"), - "updated_at": compact_text(payload.get("updated_at")), - } - if task_class: - todo["task_class"] = task_class - if action_kind: - todo["action_kind"] = action_kind - if task_domain: - todo["task_domain"] = task_domain - if capability_binding_ref: - todo["capability_binding_ref"] = capability_binding_ref - if task_repository: - todo["task_repository"] = task_repository - if removed_continuation_policy: - todo["removed_continuation_policy"] = removed_continuation_policy - elif continuation_policy: - todo["continuation_policy"] = continuation_policy - if required_write_scopes: - todo["required_write_scopes"] = required_write_scopes - if required_capabilities: - todo["required_capabilities"] = required_capabilities - if blocks_agent: - todo["blocks_agent"] = blocks_agent - if bound_agent: - todo["bound_agent"] = bound_agent - if goal_bound is not None: - todo["goal_bound"] = goal_bound - if global_gate is not None: - todo["global_gate"] = global_gate - if excluded_agents: - todo["excluded_agents"] = excluded_agents - unblocks_todo_id = normalize_todo_id(payload.get("unblocks_todo_id")) - if unblocks_todo_id: - todo["unblocks_todo_id"] = unblocks_todo_id - for key in TODO_MONITOR_METADATA_FIELDS: - if payload.get(key): - todo[key] = compact_text(payload[key]) - if claimed_by: - todo["claimed_by"] = claimed_by - if actor_agent_id: - todo["created_by"] = actor_agent_id - _copy_todo_added_validation_fields(payload, todo) - return todo - - -def _decode_todo_event_content(event: dict[str, Any]) -> dict[str, Any]: - """Normalize historical payload values without reading prior Todo state.""" - todo: dict[str, Any] = {} - payload = event.get("payload") or {} - event_type = event.get("event_type") - actor_agent_id = normalize_todo_claimed_by(event.get("actor_agent_id")) - if actor_agent_id: - todo["last_actor_agent_id"] = actor_agent_id - if event_type == TODO_CLAIMED: - claimed_by = normalize_todo_claimed_by(payload.get("claimed_by")) - if claimed_by: - todo["claimed_by"] = claimed_by - elif event_type == TODO_UPDATED: - for key in ( - "title", - "task_class", - "action_kind", - ): - if payload.get(key): - todo[key] = compact_text(payload[key]) - task_domain = normalize_todo_task_domain(payload.get("task_domain")) - if task_domain: - todo["task_domain"] = task_domain - capability_binding_ref = normalize_todo_capability_binding_ref( - payload.get("capability_binding_ref") - ) - if capability_binding_ref: - todo["capability_binding_ref"] = capability_binding_ref - continuation_policy = normalize_todo_continuation_policy( - payload.get("continuation_policy") - ) - removed_continuation_policy = normalize_removed_todo_continuation_policy( - payload.get("removed_continuation_policy") - or payload.get("continuation_policy") - ) - update_excluded_agents = normalize_todo_excluded_agents( - payload.get("excluded_agents") - ) - if removed_continuation_policy: - todo["removed_continuation_policy"] = removed_continuation_policy - elif continuation_policy: - todo["continuation_policy"] = continuation_policy - required_write_scopes = normalize_required_write_scopes( - payload.get("required_write_scopes") - ) - if required_write_scopes: - todo["required_write_scopes"] = required_write_scopes - task_repository = normalize_todo_task_repository( - payload.get("task_repository") - ) - if task_repository: - todo["task_repository"] = task_repository - required_capabilities = normalize_required_capabilities( - payload.get("required_capabilities") - ) - if required_capabilities: - todo["required_capabilities"] = required_capabilities - blocks_agent = normalize_todo_blocks_agent(payload.get("blocks_agent")) - if blocks_agent: - todo["blocks_agent"] = blocks_agent - bound_agent = normalize_todo_bound_agent(payload.get("bound_agent")) - if bound_agent: - todo["bound_agent"] = bound_agent - goal_bound = normalize_todo_goal_bound(payload.get("goal_bound")) - if goal_bound is not None: - todo["goal_bound"] = goal_bound - global_gate = normalize_todo_global_gate(payload.get("global_gate")) - if global_gate is not None: - todo["global_gate"] = global_gate - if update_excluded_agents: - todo["excluded_agents"] = update_excluded_agents - for key in TODO_MONITOR_METADATA_FIELDS: - if payload.get(key): - todo[key] = compact_text(payload[key]) - if payload.get("text") or payload.get("title"): - title = compact_text(payload.get("text") or payload.get("title")) - todo["title"] = title - elif event_type == TODO_BLOCKED: - if payload.get("reason"): - todo["reason"] = compact_text(payload["reason"]) - elif event_type == TODO_DEFERRED: - if payload.get("reason"): - todo["reason"] = compact_text(payload["reason"]) - if payload.get("resume_when"): - todo["resume_when"] = compact_text(payload["resume_when"]) - elif event_type == TODO_COMPLETED: - for key in ( - "evidence", - "reason", - "note", - "completed_at", - "updated_at", - "no_followup", - "completion_turn_key", - "completion_continuation", - "completion_recovery", - ): - if payload.get(key) is not None: - todo[key] = compact_text(payload[key]) - successor_todo_ids = normalize_todo_id_list(payload.get("successor_todo_ids")) - if successor_todo_ids: - todo["successor_todo_ids"] = successor_todo_ids - todo["last_event_id"] = event.get("event_id") - todo["last_append_sequence"] = event.get("append_sequence") - return todo - - -def build_state_projection( - events: Iterable[dict[str, Any]], - *, - goal_id: str | None = None, - generated_at: str | None = None, -) -> dict[str, Any]: - # Python owns legacy event normalization and exact historical checksum bytes. - # TS receives bounded semantic facts, never evidence, validation commands or - # arbitrary payloads; returned ordinals address this exact normalized batch. - from .control_plane.effect_runtime import EffectRuntimeRejected, effect_runtime_result - - normalized = sorted(_dedupe_events(normalize_state_event(event) for event in events), key=event_sort_key) - facts = [] - contents = [] - for event in normalized: - payload = event["payload"] - kind = event["event_type"] - edits = kind in (TODO_ADDED, TODO_UPDATED) - order = payload.get("planner_order") if kind == TODO_ADDED else None - # The typed fold rejects non-integer orders, so the adapter must not - # coerce them first: truncating 1.5 to 1 would sort a Todo by one value - # and report another. Every producer of this payload writes an integer. - if order is not None and (isinstance(order, bool) or not isinstance(order, int)): - raise StateEventError("planner_order must be an integer") - sequence = event.get("append_sequence") - for value in (order, sequence): - if value is not None and abs(value) > 2**53 - 1: - raise StateEventError("event replay integers must be safe integers") - content = (_decode_added_todo_content(event) if kind == TODO_ADDED else - _decode_todo_event_content(event) if kind in TODO_EVENT_TYPES else {}) - contents.append(content) - facts.append({ - "event_id": event["event_id"], "goal_id": event["goal_id"], - "event_type": kind, "append_sequence": sequence, - "recorded_at": event["recorded_at"], - "todo_id": event["refs"].get("todo_id") if kind in TODO_EVENT_TYPES else None, - "role": compact_text(payload.get("role")) or None if edits else None, - "priority": compact_text(payload.get("priority")) or None if edits else None, - "planner_order": order, "fields": list(content), - "capability_binding_ref": content.get("capability_binding_ref"), - "continuation_policy": content.get("continuation_policy"), - "removed_continuation_policy": content.get("removed_continuation_policy"), - "has_exclusions": bool(content.get("excluded_agents")), - "goal_bound": content.get("goal_bound"), - "content_changed": bool(payload.get("text") or payload.get("title")) if edits else False, - }) - # Only touched Todo continuation rows cross each bounded call. The host - # retains content and prior results; TS alone decides all state transitions. - rows: dict[str, dict[str, Any]] = {} - timeline_indices: list[int] = [] - inferred_goal = goal_id or (normalized[0]["goal_id"] if normalized else "") - for offset in range(0, len(facts), 256): - batch = facts[offset:offset + 256] - touched = {fact["todo_id"] for fact in batch if fact["todo_id"] is not None} - try: - plan = effect_runtime_result("goal.state_event.plan_replay", { - "schema_version": "state_event_replay_request_v0", "events": batch, - "goal_id": inferred_goal, "offset": offset, - "initial_todos": [rows[key] for key in touched if key in rows], - }) - except EffectRuntimeRejected as exc: - raise StateEventError(str(exc)) from exc - if not isinstance(plan, dict) or plan.get("schema_version") != "state_event_replay_plan_v0": - raise RuntimeError("invalid typed state event replay plan") - for row in plan["todos"]: - rows[row["todo_id"]] = row - timeline_indices.extend(plan["timeline_indices"]) - ordered = normalized # Exact historical checksum order remains in the codec. - todo_items = [] - for row in sorted(rows.values(), key=lambda row: row["sort_key"]): - todo = {key: contents[index][key] for key, index in row["field_sources"].items()} - for key in ("role", "priority", "status", "done", "source_section"): - todo[key] = row[key] - if row["render_priority"]: - todo["text"] = f"[{row['priority']}] {todo['title']}" - todo_items.append(todo) - timeline = [] - for index in timeline_indices: - event = normalized[index] - entry = {key: event.get(key) for key in ( - "event_id", "event_type", "append_sequence", "recorded_at", "refs", - )} - entry["summary"] = compact_text(event["payload"].get("summary")) - if event.get("actor_agent_id"): - entry["actor_agent_id"] = event["actor_agent_id"] - timeline.append(entry) - last_event = ordered[-1] if ordered else {} - return { - "schema_version": STATE_PROJECTION_SCHEMA_VERSION, - "goal_id": inferred_goal, - "generated_at": generated_at or now_utc_iso(), - "source_event_count": len(ordered), - "source_checksum": event_stream_checksum(ordered), - "last_event_id": last_event.get("event_id"), - "last_append_sequence": last_event.get("append_sequence"), - "projection_version": STATE_PROJECTION_VERSION, - "user_todos": _todo_summary([item for item in todo_items if item["role"] == "user"], role="user"), - "agent_todos": _todo_summary([item for item in todo_items if item["role"] != "user"], role="agent"), - "timeline": timeline, - } - - -def _todo_summary(items: list[dict[str, Any]], *, role: str) -> dict[str, Any]: - open_items = [item for item in items if not todo_done_for_status(item.get("status"))] - done_items = [item for item in items if todo_done_for_status(item.get("status"))] - return { - "schema_version": "todo_summary_v0", - "role": role, - "total_count": len(items), - "open_count": len(open_items), - "done_count": len(done_items), - "items": items, - "first_open_items": open_items[:5], - } - - -def render_todo_markdown(item: dict[str, Any]) -> list[str]: - status = normalize_todo_status(item.get("status")) or TODO_STATUS_OPEN - marker = todo_marker_for_status(status) - text = compact_text(item.get("text") or item.get("title")) - if not text.startswith("[") and item.get("priority"): - text = f"[{compact_text(item.get('priority'))}] {text}" - lines = [f"- [{marker}] {text}"] - monitor_metadata = { - key: item.get(key) - for key in TODO_MONITOR_METADATA_FIELDS - } - metadata = format_todo_metadata_line( - todo_id=item.get("todo_id"), - status=status, - task_class=item.get("task_class"), - action_kind=item.get("action_kind"), - task_domain=item.get("task_domain"), - capability_binding_ref=item.get("capability_binding_ref"), - task_repository=item.get("task_repository"), - continuation_policy=item.get("continuation_policy"), - removed_continuation_policy=item.get("removed_continuation_policy"), - required_write_scopes=item.get("required_write_scopes"), - required_capabilities=item.get("required_capabilities"), - claimed_by=item.get("claimed_by"), - bound_agent=item.get("bound_agent"), - goal_bound=item.get("goal_bound"), - blocks_agent=item.get("blocks_agent"), - excluded_agents=item.get("excluded_agents"), - global_gate=item.get("global_gate"), - unblocks_todo_id=item.get("unblocks_todo_id"), - resume_when=item.get("resume_when"), - successor_todo_ids=item.get("successor_todo_ids"), - completion_continuation=item.get("completion_continuation"), - completion_recovery=item.get("completion_recovery"), - no_followup=True if item.get("no_followup") == "true" else None, - **monitor_metadata, - note=item.get("note"), - evidence=item.get("evidence"), - validation_command=item.get("validation_command"), - validation_command_argv=( - json.dumps( - item.get("validation_command_argv"), - separators=(",", ":"), - ensure_ascii=False, - ) - if isinstance(item.get("validation_command_argv"), list) - else item.get("validation_command_argv") - ), - validation_label=item.get("validation_label"), - validation_timeout_seconds=( - str(item.get("validation_timeout_seconds")) - if item.get("validation_timeout_seconds") is not None - else None - ), - completion_turn_key=item.get("completion_turn_key"), - reason=item.get("reason"), - completed_at=item.get("completed_at"), - updated_at=item.get("updated_at"), - ) - if metadata: - lines.append(metadata) - return lines - - -def render_active_state_sections(projection: dict[str, Any]) -> str: - lines: list[str] = [] - for heading, summary_key in ( - ("User Todo / Owner Review Reading Queue", "user_todos"), - ("Agent Todo", "agent_todos"), - ): - items = ((projection.get(summary_key) or {}).get("items") or []) - if not items: - continue - if lines: - lines.append("") - lines.extend([f"## {heading}", ""]) - for item in items: - lines.extend(render_todo_markdown(item)) - - timeline = projection.get("timeline") or [] - if timeline: - if lines: - lines.append("") - lines.extend(["## Progress Ledger", ""]) - for event in timeline: - event_type = compact_text(event.get("event_type")) - summary = compact_text(event.get("summary")) or "event recorded" - lines.append(f"- {event_type}: {summary}") - return "\n".join(lines).rstrip() + "\n" diff --git a/loopx/semantics/project_registry_io_manifest_v1.json b/loopx/semantics/project_registry_io_manifest_v1.json index 52363fcdef..d2253f8376 100644 --- a/loopx/semantics/project_registry_io_manifest_v1.json +++ b/loopx/semantics/project_registry_io_manifest_v1.json @@ -181,6 +181,14 @@ "api": "load_registry", "classification": "codec_api" }, + { + "site": "loopx/capabilities/manager_context/__init__.py::.configure_delivery_target::codec_read:load_registry#1", + "line": 317, + "column": 20, + "kind": "codec_read", + "api": "load_registry", + "classification": "codec_api" + }, { "site": "loopx/capabilities/manager_context/__init__.py::.configure_evidence_scope::codec_read:load_registry#1", "line": 275, @@ -599,7 +607,7 @@ }, { "site": "loopx/cli_commands/manager_inbox.py::.handle_manager_inbox::codec_read:load_registry#1", - "line": 78, + "line": 93, "column": 20, "kind": "codec_read", "api": "load_registry", @@ -791,7 +799,7 @@ }, { "site": "loopx/cli_commands/todo_continuation.py::.handle_todo_continuation::codec_read:load_registry#1", - "line": 126, + "line": 136, "column": 20, "kind": "codec_read", "api": "load_registry", @@ -847,7 +855,7 @@ }, { "site": "loopx/control_plane/coordination/legacy_writer_fence.py::.legacy_todo_write_transaction::codec_read:load_registry#1", - "line": 297, + "line": 302, "column": 9, "kind": "codec_read", "api": "load_registry", @@ -1143,7 +1151,7 @@ }, { "site": "loopx/control_plane/goals/shared_goal_alignment.py::._project_shared_goal_alignment::codec_read:load_registry#1", - "line": 228, + "line": 121, "column": 28, "kind": "codec_read", "api": "load_registry", @@ -1351,23 +1359,23 @@ }, { "site": "loopx/control_plane/todos/completion_validation.py::._resolve_goal_repo_workspace::codec_read:load_registry#1", - "line": 72, + "line": 68, "column": 31, "kind": "codec_read", "api": "load_registry", "classification": "codec_api" }, { - "site": "loopx/control_plane/todos/event_writeback.py::._registry_goal::codec_read:load_registry#1", - "line": 53, - "column": 16, + "site": "loopx/control_plane/todos/completion_validation.py::.run_completion_validation_gate_with_source::codec_read:load_registry#1", + "line": 519, + "column": 50, "kind": "codec_read", "api": "load_registry", "classification": "codec_api" }, { - "site": "loopx/control_plane/todos/handoff_mode_source.py::._event_paths::codec_read:load_registry#1", - "line": 19, + "site": "loopx/control_plane/todos/handoff_mode_source.py::.handoff_mode_source::codec_read:load_registry#1", + "line": 28, "column": 31, "kind": "codec_read", "api": "load_registry", @@ -1399,7 +1407,7 @@ }, { "site": "loopx/control_plane/work_items/task_lease_acquire_adapter.py::._canonical_lease_authority_facts::codec_read:load_registry#1", - "line": 314, + "line": 312, "column": 31, "kind": "codec_read", "api": "load_registry", @@ -1407,7 +1415,7 @@ }, { "site": "loopx/control_plane/work_items/task_lease_acquire_adapter.py::._task_lease_authority_snapshot_attempt::codec_read:load_registry#1", - "line": 229, + "line": 227, "column": 16, "kind": "codec_read", "api": "load_registry", @@ -1415,7 +1423,7 @@ }, { "site": "loopx/control_plane/work_items/task_lease_acquire_adapter.py::._task_lease_authority_snapshot_attempt::codec_read:load_registry#2", - "line": 252, + "line": 250, "column": 22, "kind": "codec_read", "api": "load_registry", @@ -1423,7 +1431,7 @@ }, { "site": "loopx/control_plane/work_items/task_lease_acquire_adapter.py::.execute_native_task_lease_acquire::codec_read:load_registry#1", - "line": 397, + "line": 395, "column": 20, "kind": "codec_read", "api": "load_registry", @@ -1431,7 +1439,7 @@ }, { "site": "loopx/control_plane/work_items/task_lease_acquire_adapter.py::.execute_native_task_lease_lifecycle::codec_read:load_registry#1", - "line": 709, + "line": 707, "column": 24, "kind": "codec_read", "api": "load_registry", @@ -1863,7 +1871,7 @@ }, { "site": "loopx/todos.py::.list_goal_todos::codec_read:load_registry#1", - "line": 198, + "line": 192, "column": 16, "kind": "codec_read", "api": "load_registry", diff --git a/loopx/status.py b/loopx/status.py index 26ee24cf16..8ada8c66e6 100644 --- a/loopx/status.py +++ b/loopx/status.py @@ -90,7 +90,6 @@ parse_state_frontmatter, ) from .control_plane.todos.active_state_todos import ( - MONITOR_WRITEBACK_CONTRACT_SCHEMA_VERSION as _MONITOR_WRITEBACK_CONTRACT_SCHEMA_VERSION, active_state_todo_fields as _active_state_todo_fields_read_model, ) from .control_plane.todos.active_state_todo_parser import ( @@ -271,7 +270,6 @@ STATUS_CONTRACT_SCHEMA_VERSION = 2 MINIMUM_DASHBOARD_STATUS_CONTRACT_SCHEMA_VERSION = 2 STATUS_CONTRACT_SIGNAL_LIMIT = 3 -MONITOR_WRITEBACK_CONTRACT_SCHEMA_VERSION = _MONITOR_WRITEBACK_CONTRACT_SCHEMA_VERSION EVENT_LEDGER_DECISION_CLASSIFICATIONS = USER_OR_CONTROLLER_CLASSIFICATIONS | { "operator_gate_approved", } @@ -360,33 +358,8 @@ -def state_event_log_candidates(goal: dict[str, Any], *, state_path: Path) -> list[Path]: - from .control_plane.status.active_state_projection import ( - state_event_log_candidates as _state_event_log_candidates, - ) - - return _state_event_log_candidates(goal, state_path=state_path) -def active_state_event_projection_fields( - goal: dict[str, Any], - *, - state_path: Path, - preferred_todo_ids: set[str] | None = None, - rollout_events: list[dict[str, Any]] | None = None, - item_limit: int | None = MAX_STATUS_TODOS_PER_ROLE, -) -> dict[str, Any]: - from .control_plane.status.active_state_projection import ( - active_state_event_projection_fields as _active_state_event_projection_fields, - ) - - return _active_state_event_projection_fields( - goal, - state_path=state_path, - preferred_todo_ids=preferred_todo_ids, - rollout_events=rollout_events, - item_limit=item_limit, - ) def active_state_sections(state_text: str, headings: tuple[str, ...]) -> dict[str, list[str]]: @@ -723,7 +696,6 @@ def active_state_todo_fields( load_rollout_events=load_rollout_events, rollout_event_log_path=rollout_event_log_path, max_todo_index_rollout_events_per_goal=MAX_TODO_INDEX_ROLLOUT_EVENTS_PER_GOAL, - active_state_event_projection_fields=active_state_event_projection_fields, parse_active_state_todos=parse_active_state_todos, parse_issue_meta_surface=parse_issue_meta_surface, backlog_hygiene_warning=backlog_hygiene_warning, diff --git a/loopx/todos.py b/loopx/todos.py index fb2165c5a7..43da08364a 100644 --- a/loopx/todos.py +++ b/loopx/todos.py @@ -69,9 +69,6 @@ user_todo_completion_metadata_updates, ) from .control_plane.todos import completion_validation as completion_validation_module -from .control_plane.todos.event_writeback import ( - complete_event_projected_goal_todo, -) from .control_plane.todos.line_update import ( apply_todo_update_to_lines, link_generated_successor_todo_ids, @@ -80,11 +77,8 @@ ) from .control_plane.todos.next_action_runtime import apply_added_todo_next_action, settle_completed_todo_next_action from .control_plane.todos.list_projection import ( - AGENT_LANE_OVERLAY_FULL_DETAIL_COLD_PATH, - EXPLICIT_LIMIT_OVERLAY_FULL_DETAIL_COLD_PATH, compact_agent_lane_todo_summary, compact_thin_todo_list_payload, - compact_todo_projection_overlay, todo_item_relations, todo_list_projection_contract, ) @@ -229,8 +223,6 @@ def list_goal_todos( goal_acceptance_work_guards=canonical_read.get("goal_acceptance_work_guards"), ), source="file_authority", - projection_fields={}, - projection_overlay=None, rollout_events=rollout_events, roles=roles, status=status, @@ -269,8 +261,6 @@ def list_goal_todos( archived_projection = exact_archived_todo_summaries( archived_items=archived_items, source=projected.source, - projection_fields=projected.projection_fields, - projection_overlay=projected.projection_overlay, rollout_events=rollout_events, roles=roles, status=status, @@ -281,8 +271,6 @@ def list_goal_todos( if archived_projection is not None: projected = archived_projection source = projected.source - projection_fields = projected.projection_fields - projection_overlay = projected.projection_overlay summaries = projected.summaries todos = projected.todos unfiltered_count = projected.unfiltered_count @@ -372,28 +360,8 @@ def list_goal_todos( if not todos: payload["not_found"] = True payload.update(summaries) - if source == "event_projection" and projection_fields.get("state_event_projection"): - payload["state_event_projection"] = projection_fields["state_event_projection"] - if source == "event_projection_with_markdown_overlay": - if projection_fields.get("state_event_projection"): - payload["state_event_projection"] = projection_fields["state_event_projection"] - payload["projection_overlay"] = ( - compact_todo_projection_overlay( - projection_overlay, - full_detail_cold_path=( - EXPLICIT_LIMIT_OVERLAY_FULL_DETAIL_COLD_PATH - if limit is not None - else AGENT_LANE_OVERLAY_FULL_DETAIL_COLD_PATH - ), - ) - if agent_lane_hot_path or limit is not None - else projection_overlay - ) - if projection_fields.get("state_event_projection_warning"): - payload["state_event_projection_warning"] = projection_fields["state_event_projection_warning"] return compact_thin_todo_list_payload(payload) if thin else payload - def add_todo_to_lines( lines: list[str], *, @@ -1648,7 +1616,7 @@ def complete_goal_todo( ) lines = original.splitlines() updated_at = now_local() - completion_match, completion_todo, event_context = ( + completion_match, completion_todo = ( completion_validation_module.locked_todo_completion_source( lines=lines, state_file=resolved_state_file, @@ -1675,7 +1643,6 @@ def complete_goal_todo( goal_id=goal_id, lines=lines, successor_todo_ids=normalized_successor_todo_ids, - event_fields=event_context.get("fields") if event_context else None, facts=completion_policy_facts, ) ) @@ -1742,7 +1709,6 @@ def complete_goal_todo( runtime_root=shadow_runtime_root, ) ) - completion_fence = completion_transaction["fence"] completion_state = completion_transaction.get("completion_state") completion_policy = completion_policy_from_transaction(completion_transaction) effective_claimed_by = completion_policy.effective_claimed_by @@ -1752,58 +1718,6 @@ def complete_goal_todo( completion_policy.effective_next_excluded_agents ) effective_self_merged = completion_policy.self_merged - if not completion_match: - if event_context: - event_result = complete_event_projected_goal_todo( - goal_id=goal_id, - context=event_context, - runtime_root=shadow_runtime_root, - primary_lock_held=True, - evidence=evidence, - completion_turn_key=completion_turn_key, - completion_identity_source=completion_identity_source, - note=note, - no_followup=no_followup, - successor_todo_ids=normalized_successor_todo_ids, - claimed_by=effective_claimed_by, - clear_claim=clear_claim, - next_agent_todo=next_agent_todo, - next_user_todo=next_user_todo, - next_user_task_class=effective_next_user_task_class, - next_claimed_by=effective_next_claimed_by, - next_task_class=next_task_class, - next_action_kind=next_action_kind, - next_task_repository=next_task_repository, - next_required_capabilities=next_required_capabilities, - next_continuation_policy=next_continuation_policy, - self_merged=effective_self_merged, - next_excluded_agents=effective_next_excluded_agents, - registered_agents=registered_agents, - updated_at=updated_at, - dry_run=dry_run, - actor_agent_id=mutation_authority.get("actor_agent_id"), - completion_fence=completion_fence, - completion_state=completion_state, - completion_validation_source_authority=validation_gate.get("source_authority"), - ) - event_result["linked_successor_id"] = completion_policy.linked_successor_id - event_result["mutation_authority"] = mutation_authority - event_result["task_lease_fence"] = task_lease_fence - event_result.update(completion_handoff) - release_verified_task_lease_fence( - task_lease_fence, - committed=bool(event_result.get("changed")) and not dry_run, - ) - # This branch can append multiple state-log events inside the - # event writer. Capturing after that call would be observation, - # not a transaction-bound prepare/commit pair. Keep the gap - # explicit until the event writer owns the outbox boundary. - shadow_capture.skip("event_log_writer_not_bound") - return settle_todo_runtime_shadow_capture( - event_result, registry_path=registry_path, - runtime_root=shadow_runtime_root, goal_id=goal_id, - capture=shadow_capture, - ) if not isinstance(completion_state, dict): raise RuntimeError( "TypeScript Todo completion transaction did not authorize a commit" diff --git a/pyproject.toml b/pyproject.toml index 9dc6ba5869..6de9e6e9a0 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -57,6 +57,7 @@ include = ["loopx*"] "loopx.control_plane.coordination" = ["*.ts", "*.json"] "loopx.control_plane.goals" = ["*.ts"] "loopx.control_plane.quota" = ["*.ts"] +"loopx.control_plane.runtime" = ["*.ts"] "loopx.control_plane.scheduler" = ["*.ts"] "loopx.control_plane.todos" = ["*.ts"] "loopx.control_plane.turn_driver" = ["*.ts"] @@ -149,7 +150,6 @@ files = [ "loopx/control_plane/coordination/local_authority_shadow_projection.py", "loopx/control_plane/quota/effect_program.py", "loopx/control_plane/quota/states.py", - "loopx/control_plane/runtime/event_store_migration_bridge.py", "loopx/control_plane/runtime/time.py", "loopx/control_plane/runtime/trajectory_hygiene.py", "loopx/control_plane/scheduler/time.py", diff --git a/tests/control_plane/test_canonical_goal_governance.py b/tests/control_plane/test_canonical_goal_governance.py index 45a5549279..2d0a3fc71e 100644 --- a/tests/control_plane/test_canonical_goal_governance.py +++ b/tests/control_plane/test_canonical_goal_governance.py @@ -6,7 +6,7 @@ import pytest from canonical_authority_fixture import initialize_canonical_authority -from test_goal_amendment_proposal import _write_fixture, _proposal, _admit, _default_events, GOAL_ID +from test_goal_amendment_proposal import _write_fixture, _proposal, _admit, GOAL_ID from loopx.control_plane.coordination.runtime_shadow import build_todo_runtime_shadow_projection from loopx.control_plane.coordination.local_authority import LocalCoordinationAuthorityUnavailable from loopx.control_plane.goals.shared_goal_alignment import project_shared_goal_alignment @@ -21,7 +21,7 @@ def _record(todo_id="todo_current", **fields): def _canonical(tmp_path, records=None, *, events=None, leases=None, native=False): - paths = _write_fixture(tmp_path, events=events) + paths = _write_fixture(tmp_path) projection = build_todo_runtime_shadow_projection(goal_id=GOAL_ID, todos=[_record()] if records is None else records, leases=leases or [], handoff_mode="soft_claim") if native: @@ -38,6 +38,8 @@ def _canonical(tmp_path, records=None, *, events=None, leases=None, native=False "contract_fields": list(TODO_DOMAIN_RECORD_FIELDS), "todo_count": len(projection["todos"]), "records_sha256": sha256(canonical_bytes(projection["todos"])).hexdigest()} initialize_canonical_authority(paths["runtime"], GOAL_ID, projection, state_path=paths["state_file"]) + if events: + paths["state_file"].with_name("events.jsonl").write_text("retired unrelated source\n") return paths @@ -96,7 +98,7 @@ def test_empty_canonical_is_authoritative_and_missing_display_is_not_repaired(tm assert not paths["state_file"].exists() -@pytest.mark.parametrize("events", [None, _default_events()]) +@pytest.mark.parametrize("events", [False, True]) def test_canonical_revision_changes_proposal_basis_without_changing_event_sequence(tmp_path, events): paths = _canonical(tmp_path, events=events) proposal = _proposal(paths, {"affected_todo_ids": ["todo_current"]}) diff --git a/tests/control_plane/test_canonical_status_todos.py b/tests/control_plane/test_canonical_status_todos.py index 97d2f5414f..db5d5e419a 100644 --- a/tests/control_plane/test_canonical_status_todos.py +++ b/tests/control_plane/test_canonical_status_todos.py @@ -78,7 +78,7 @@ def test_status_uses_provider_and_allows_native_monitor_writeback_without_displa original = state.read_bytes() if display == "missing": state.unlink() - for name in ("active_state_event_projection_fields", "parse_active_state_todos"): + for name in ("parse_active_state_todos",): monkeypatch.setattr( f"loopx.status.{name}", lambda *_args, **_kwargs: pytest.fail("promoted read must not parse legacy Todos"), diff --git a/tests/control_plane/test_cli_output_budget.py b/tests/control_plane/test_cli_output_budget.py index fef2bcbc6a..962d59ab4a 100644 --- a/tests/control_plane/test_cli_output_budget.py +++ b/tests/control_plane/test_cli_output_budget.py @@ -24,11 +24,6 @@ measure_cli_output, public_manifest, ) -from loopx.event_sourced_state import ( - AppendOnlyStateEventStore, - TODO_ADDED, - make_state_event, -) from loopx.heartbeat_prompt import build_heartbeat_prompt from loopx.help_surface import COMMAND_GROUPS from loopx.rollout_event_log import rollout_event_log_path @@ -309,48 +304,6 @@ def _write_todo_list_limit_stress_fixture(state_file: Path) -> None: state_file.write_text("\n".join(lines) + "\n", encoding="utf-8") -def _write_todo_list_limit_overlay_stress_fixture(state_file: Path) -> None: - lines = [ - "---", - "status: active", - "updated_at: 2026-01-01T00:00:00+00:00", - "---", - "", - "# Todo List Limit Overlay Stress Fixture", - "", - "## Agent Todo", - "", - ] - for index in range(3_000): - lines.extend( - [ - f"- [ ] Observe overlay lineage {index:04d}.", - ( - " " - ), - ] - ) - state_file.write_text("\n".join(lines) + "\n", encoding="utf-8") - store = AppendOnlyStateEventStore(state_file.with_name("events.jsonl")) - store.append( - make_state_event( - event_id="evt-limit-overlay", - goal_id=GOAL_ID, - event_type=TODO_ADDED, - refs={"todo_id": "todo_overlay_0000"}, - payload={ - "role": "agent", - "title": "Overlay the first advancement step.", - "task_class": "continuous_monitor", - "claimed_by": AGENT_IDS[0], - }, - recorded_at="2026-01-01T00:00:00Z", - producer="todo-list-limit-overlay-budget", - ) - ) def _invoke_cli(args: list[str]) -> tuple[int, str]: @@ -1736,44 +1689,6 @@ def test_todo_list_explicit_limit_bounds_monitor_and_blocker_lanes( } -def test_todo_list_explicit_limit_bounds_projection_overlay_ids( - tmp_path: Path, -) -> None: - with _stable_budget_fixture_root( - tmp_path / "todo-list-limit-overlay", - ) as stable_root: - project, runtime, registry_path, state_file = _write_fixture( - stable_root, - SCENARIOS[1], - ) - _write_todo_list_limit_overlay_stress_fixture(state_file) - command = _mode_variant_commands( - project=project, - runtime=runtime, - registry_path=registry_path, - state_file=state_file, - output_format="json", - )["todo_list_limited"] - exit_code, text = _invoke_cli(command) - - assert exit_code == 0, text - spec = CLI_OUTPUT_MODE_VARIANT_BY_ID["todo_list_limited"] - measurement = measure_cli_output(text, output_format="json") - assert_cli_output_mode_variant( - spec, - output_format="json", - text=text, - measurement=measurement, - ) - payload = json.loads(text) - overlay = payload["projection_overlay"] - assert overlay["markdown_only_count"] == 2_999 - assert overlay["event_only_count"] == 0 - assert overlay["overlaid_count"] == 1 - assert overlay["full_detail_cold_path"] == ( - "todo list without --limit or active state" - ) - assert [key for key in overlay if key.endswith("_todo_ids")] == [] def test_turn_envelope_cli_preserves_codex_app_scheduler_binding( diff --git a/tests/control_plane/test_durable_completion_projection.py b/tests/control_plane/test_durable_completion_projection.py index 2a4ed81d1a..659f807d60 100644 --- a/tests/control_plane/test_durable_completion_projection.py +++ b/tests/control_plane/test_durable_completion_projection.py @@ -1,6 +1,5 @@ from __future__ import annotations -import json from pathlib import Path import pytest @@ -12,12 +11,6 @@ read_persisted_todo_record, read_persisted_todo_record_with_source, ) -from loopx.event_sourced_state import ( - TODO_ADDED, - TODO_COMPLETED, - AppendOnlyStateEventStore, - make_state_event, -) TODO_DONE = { "todo_id": "todo_fixture0001", @@ -90,7 +83,6 @@ def test_projects_terminal_intent_without_premature_completion() -> None: [ ("blocked", "materialized"), ("deferred", "materialized"), - ("deferred", "event_log"), ], ) def test_terminal_readback_does_not_treat_ambiguous_state_as_absent( @@ -325,95 +317,3 @@ def test_read_persisted_todo_record_fails_closed_when_todo_is_missing( state_file, todo_id="todo_fixture0001", ) - - -def test_read_persisted_todo_record_falls_back_to_event_projection( - tmp_path: Path, -) -> None: - repo = tmp_path / "repo" - repo.mkdir() - state_file = repo / "ACTIVE_GOAL_STATE.md" - state_file.write_text( - "\n".join( - [ - "---", - "status: active", - "updated_at: 2026-01-01T00:00:00+00:00", - "---", - "", - "# Fixture", - "", - "## Agent Todo", - "", - ] - ) - + "\n", - encoding="utf-8", - ) - # The event-sourced Todo lives only in the event log, mirroring the - # completion payload shape written by complete_event_projected_goal_todo. - goal_id = "fixture-goal" - events = [ - make_state_event( - event_id="add-fixture-1", - goal_id=goal_id, - event_type=TODO_ADDED, - refs={"todo_id": "todo_fixture0001"}, - payload={ - "role": "agent", - "priority": "P0", - "title": "Advance one public fixture.", - "text": "Advance one public fixture.", - "planner_order": 1, - "task_class": "advancement_task", - }, - ), - make_state_event( - event_id="complete-fixture-1", - goal_id=goal_id, - event_type=TODO_COMPLETED, - refs={"todo_id": "todo_fixture0001"}, - payload={ - "no_followup": "true", - "completion_continuation": "no_followup", - }, - ), - ] - AppendOnlyStateEventStore(repo / "events.jsonl").append_many(events) - registry = tmp_path / "registry.global.json" - registry.write_text( - json.dumps( - { - "common_runtime_root": str(tmp_path / "runtime"), - "goals": [ - { - "id": goal_id, - "domain": "fixture", - "status": "active", - "repo": str(repo), - "state_file": state_file.name, - "adapter": {"kind": "fixture_v0"}, - } - ], - } - ), - encoding="utf-8", - ) - - todo, existing_todo_ids, projection_source = read_persisted_todo_record_with_source( - state_file, - todo_id="todo_fixture0001", - registry_path=registry, - goal_id=goal_id, - ) - assert projection_source == "event_log" - assert existing_todo_ids == {"todo_fixture0001"} - outcome = project_durable_completion_outcome( - todo=todo, - expected_todo_id="todo_fixture0001", - existing_todo_ids=existing_todo_ids, - ) - assert outcome == { - "todo_id": "todo_fixture0001", - "continuation": "no_followup", - } diff --git a/tests/control_plane/test_event_replay_integrity.py b/tests/control_plane/test_event_replay_integrity.py deleted file mode 100644 index f113f4a296..0000000000 --- a/tests/control_plane/test_event_replay_integrity.py +++ /dev/null @@ -1,147 +0,0 @@ -"""Event replay must preserve Todo identity and coherent scheduling/display.""" -import pytest - -from loopx.event_sourced_state import StateEventError, build_state_projection, make_state_event - - -def event(kind, number, *, todo_id="todo_alpha", **payload): - return {**make_state_event(event_id=f"event-{number}", goal_id="example-goal", - event_type=kind, refs={"todo_id": todo_id}, payload=payload, - recorded_at="2026-09-24T00:00:00Z"), "append_sequence": number} - - -def test_second_creation_cannot_erase_a_completed_commitment(): - events = [event("todo_added", 1, title="Retain commitment", role="agent"), - event("todo_completed", 2, evidence="Verified"), - event("todo_added", 3, title="Replacement", role="agent")] - with pytest.raises(StateEventError, match="already exists"): - build_state_projection(events) - - -def test_priority_only_update_updates_rendered_text(): - result = build_state_projection([event("todo_added", 1, title="Keep title", priority="P2"), - event("todo_updated", 2, priority="P0")]) - todo = result["agent_todos"]["items"][0] - assert (todo["priority"], todo["title"], todo["text"]) == ("P0", "Keep title", "[P0] Keep title") - - -def test_role_update_moves_both_summary_and_source_section(): - result = build_state_projection([event("todo_added", 1, title="Owner decision", role="agent"), - event("todo_updated", 2, role="user")]) - assert result["agent_todos"]["total_count"] == 0 - assert result["user_todos"]["items"][0]["source_section"] == "User Todo / Owner Review Reading Queue" - - -def test_zero_planner_order_is_a_real_order_not_a_missing_value(): - result = build_state_projection([event("todo_added", 1, todo_id="todo_first", title="First", planner_order=0), - event("todo_added", 2, todo_id="todo_second", title="Second", planner_order=1)]) - assert [todo["todo_id"] for todo in result["agent_todos"]["items"]] == ["todo_first", "todo_second"] - - -def test_large_content_remains_complete_outside_the_typed_facts(monkeypatch): - import json - from loopx.control_plane import effect_runtime - - invoke = effect_runtime.effect_runtime_result - request_sizes = [] - - def measured(method, params, **kwargs): - if method == "goal.state_event.plan_replay": - request_sizes.append(len(json.dumps(params).encode())) - assert "Retain this complete evidence" not in json.dumps(params) - return invoke(method, params, **kwargs) - - monkeypatch.setattr(effect_runtime, "effect_runtime_result", measured) - evidence = "Retain this complete evidence. " * 100_000 - source = [event("todo_added", 1, title="Large result"), - event("todo_completed", 2, evidence=evidence)] - result = build_state_projection(source) - assert result["agent_todos"]["items"][0]["evidence"] == evidence.strip() - assert len(request_sizes) == 1 and request_sizes[0] < 4096 - - -def test_identical_event_replay_keeps_checksum_and_no_duplicate_todo(): - from loopx.event_sourced_state import StateEventConflictError, event_stream_checksum - - source = event("todo_added", 1, title="Original", role="agent") - result = build_state_projection([source, source]) - assert result["source_event_count"] == result["agent_todos"]["total_count"] == 1 - assert result["source_checksum"] == event_stream_checksum([source]) - with pytest.raises(StateEventConflictError): - build_state_projection([source, {**source, "payload": {"title": "Changed"}}]) - - -def test_mixed_history_preserves_content_attribution_and_dependency_edges(): - source = [ - event("todo_added", 1, title="Source", priority="P1", claimed_by="author", - task_class="advancement_task", capability_binding_ref="review:source", - validation_command_argv=["python", "-c", "print('ok')"]), - event("todo_deferred", 2, reason="Wait for review", resume_when="todo_done:todo_review"), - event("todo_added", 3, todo_id="todo_review", title="Review", role="agent", - task_class="advancement_task", claimed_by="reviewer"), - event("todo_completed", 4, todo_id="todo_review", evidence="Review passed"), - event("todo_updated", 5, title="Source revised", priority="P0"), - event("todo_completed", 6, evidence="Delivered", successor_todo_ids=["todo_followup"]), - event("todo_added", 7, todo_id="todo_followup", title="Follow-up", role="user", - goal_bound=True, task_class="user_action"), - ] - result = build_state_projection(source) - parent = next(row for row in result["agent_todos"]["items"] if row["todo_id"] == "todo_alpha") - assert parent["text"] == "[P0] Source revised" - assert parent["claimed_by"] == "author" - assert parent["capability_binding_ref"] == "review:source" - assert parent["successor_todo_ids"] == ["todo_followup"] - assert parent["validation_command_argv"] == ["python", "-c", "print('ok')"] - assert result["agent_todos"]["done_count"] == 2 - assert result["user_todos"]["open_count"] == 1 - - -def test_long_history_folds_across_bounded_calls_without_losing_old_fields(monkeypatch): - import json - from loopx.control_plane import effect_runtime - invoke = effect_runtime.effect_runtime_result - sizes = [] - def measured(method, params, **kwargs): - if method == "goal.state_event.plan_replay": - sizes.append(len(json.dumps(params).encode())) - assert len(params["events"]) <= 256 - return invoke(method, params, **kwargs) - monkeypatch.setattr(effect_runtime, "effect_runtime_result", measured) - source = [event("todo_added", 1, title="Original", claimed_by="author", capability_binding_ref="test:bound")] - source.extend(event("todo_updated", i, title=f"Revision {i}") for i in range(2, 4100)) - source.append(event("todo_completed", 4100, evidence="Long history verified")) - result = build_state_projection(source) - todo = result["agent_todos"]["items"][0] - assert todo["title"] == "Revision 4099" - assert todo["status"] == "done" and todo["claimed_by"] == "author" - assert todo["capability_binding_ref"] == "test:bound" - assert len(sizes) == 17 and max(sizes) < 256_000 - # Duplicate identity protection must survive a batch boundary as well. - with pytest.raises(StateEventError, match="already exists"): - build_state_projection([*source, event("todo_added", 4101, title="Cannot reset history")]) - - -def test_fractional_planner_order_is_rejected_before_typing(): - # The adapter used to truncate 1.5 to 1, so the fold sorted by one value - # while the projection still reported the original fraction. - with pytest.raises(StateEventError, match="planner_order must be an integer"): - build_state_projection([ - event("todo_added", 1, title="Fractional", planner_order=1.5), - event("todo_added", 2, title="Integer", planner_order=1, todo_id="todo_beta"), - ]) - - -@pytest.mark.parametrize("order", [True, "1", [1], {"value": 1}]) -def test_non_integer_planner_order_forms_are_rejected(order): - with pytest.raises(StateEventError, match="planner_order must be an integer"): - build_state_projection([event("todo_added", 1, title="Bad order", planner_order=order)]) - - -def test_integer_and_absent_planner_order_still_project(): - result = build_state_projection([ - event("todo_added", 1, title="First", planner_order=3), - event("todo_added", 2, title="Second", todo_id="todo_beta"), - ]) - items = result["agent_todos"]["items"] - assert [item["todo_id"] for item in items] == ["todo_alpha", "todo_beta"] - assert items[0]["planner_order"] == 3 diff --git a/tests/control_plane/test_event_store_migration_bridge.py b/tests/control_plane/test_event_store_migration_bridge.py deleted file mode 100644 index 6bdbefc462..0000000000 --- a/tests/control_plane/test_event_store_migration_bridge.py +++ /dev/null @@ -1,139 +0,0 @@ -from __future__ import annotations - -import pytest - -from loopx.control_plane.runtime.event_store_migration_bridge import ( - EVENT_PROJECTION_SOURCE, - MARKDOWN_ACTIVE_STATE_SOURCE, - build_event_store_migration_bridge, -) - - -GOAL_ID = "event-store-migration-bridge-fixture" - - -def test_bridge_fails_closed_before_event_read_path() -> None: - bridge = build_event_store_migration_bridge( - goal_id=GOAL_ID, - event_read_path_ready=False, - active_state_projection_ready=False, - ) - - assert bridge["schema_version"] == "event_store_migration_bridge_v0" - assert bridge["source_of_truth"] == MARKDOWN_ACTIVE_STATE_SOURCE - assert bridge["candidate_source"] == EVENT_PROJECTION_SOURCE - assert bridge["stage"] == "wait_for_event_read_path" - assert bridge["promotion_allowed"] is False - assert bridge["promotion_candidate"] is False - assert bridge["dual_read"]["enabled"] is False - assert bridge["canary"]["ready"] is False - assert bridge["rollback"]["fallback_source"] == MARKDOWN_ACTIVE_STATE_SOURCE - assert bridge["missing_for_shadow"] == [ - "event_read_path_ready", - "active_state_projection_ready", - ] - assert "bounded_canary_passed" in bridge["missing_for_promotion"] - - -def test_shadow_mode_requires_parity_and_rollback() -> None: - bridge = build_event_store_migration_bridge( - goal_id=GOAL_ID, - event_read_path_ready=True, - active_state_projection_ready=True, - dual_read_parity_clean=False, - rollback_plan_recorded=False, - idempotency_conflicts_clean=False, - public_boundary_clean=False, - ) - - assert bridge["stage"] == "dual_read_shadow" - assert bridge["dual_read"]["enabled"] is True - assert bridge["dual_read"]["failure_policy"] == ( - "prefer_markdown_and_record_parity_delta" - ) - assert bridge["promotion_allowed"] is False - assert bridge["canary"]["ready"] is False - assert bridge["missing_for_canary"] == [ - "dual_read_parity_clean", - "event_projection_head_matches_store", - "rollback_plan_recorded", - "idempotency_conflicts_clean", - "public_boundary_clean", - ] - - -def test_canary_ready_does_not_promote_automatically() -> None: - bridge = build_event_store_migration_bridge( - goal_id=GOAL_ID, - event_read_path_ready=True, - active_state_projection_ready=True, - dual_read_parity_clean=True, - event_projection_head_matches_store=True, - rollback_plan_recorded=True, - idempotency_conflicts_clean=True, - public_boundary_clean=True, - bounded_canary_passed=False, - canary_goal_limit=2, - canary_duration_minutes=45, - evidence_refs=["event-projection-parity-smoke"], - ) - - assert bridge["stage"] == "bounded_canary" - assert bridge["promotion_candidate"] is False - assert bridge["promotion_allowed"] is False - assert bridge["missing_for_canary"] == [] - assert bridge["missing_for_promotion"] == ["bounded_canary_passed"] - assert bridge["canary"]["ready"] is True - assert bridge["canary"]["scope"] == { - "max_goals": 2, - "duration_minutes": 45, - "write_path": "disabled", - "read_preference": MARKDOWN_ACTIVE_STATE_SOURCE, - } - assert bridge["evidence_refs"] == ["event-projection-parity-smoke"] - - -def test_promotion_candidate_still_requires_reviewed_write_path_change() -> None: - bridge = build_event_store_migration_bridge( - goal_id=GOAL_ID, - event_read_path_ready=True, - active_state_projection_ready=True, - dual_read_parity_clean=True, - event_projection_head_matches_store=True, - rollback_plan_recorded=True, - idempotency_conflicts_clean=True, - public_boundary_clean=True, - bounded_canary_passed=True, - ) - - assert bridge["stage"] == "promotion_candidate" - assert bridge["promotion_candidate"] is True - assert bridge["promotion_allowed"] is False - assert bridge["missing_for_promotion"] == [] - assert "explicit reviewed write-path change" in bridge["next_action"] - assert bridge["rollback"]["recorded"] is True - assert bridge["canary"]["passed"] is True - - -@pytest.mark.parametrize("goal_id", ["", " "]) -def test_bridge_requires_a_goal_id(goal_id: str) -> None: - with pytest.raises(ValueError, match="goal_id is required"): - build_event_store_migration_bridge( - goal_id=goal_id, - event_read_path_ready=False, - ) - - -def test_bridge_normalizes_bounded_canary_inputs() -> None: - bridge = build_event_store_migration_bridge( - goal_id=" fixture goal ", - event_read_path_ready=False, - canary_goal_limit=0, - canary_duration_minutes=0, - evidence_refs=[" parity packet ", "", " "], - ) - - assert bridge["goal_id"] == "fixture goal" - assert bridge["canary"]["scope"]["max_goals"] == 1 - assert bridge["canary"]["scope"]["duration_minutes"] == 1 - assert bridge["evidence_refs"] == ["parity packet"] diff --git a/tests/control_plane/test_event_todo_transaction.py b/tests/control_plane/test_event_todo_transaction.py deleted file mode 100644 index 8296d05798..0000000000 --- a/tests/control_plane/test_event_todo_transaction.py +++ /dev/null @@ -1,189 +0,0 @@ -"""The public completion entrypoint must never publish half a continuation.""" - -from __future__ import annotations - -import json -from pathlib import Path - -import pytest - -from loopx.control_plane.todos import active_state_editing as io -from loopx.control_plane.todos import event_writeback -from loopx.event_sourced_state import ( - TODO_ADDED, - TODO_COMPLETED, - TODO_UPDATED, - AppendOnlyStateEventStore, - StateEventCommitUnknownError, - build_state_projection, - make_state_event, -) -from loopx.todos import complete_goal_todo - - -@pytest.fixture -def event_goal(tmp_path: Path): - state = tmp_path / "ACTIVE_GOAL_STATE.md" - state.write_text( - "---\ngoal_id: event-transaction\n---\n\n## Agent Todo\n", encoding="utf-8" - ) - registry = tmp_path / "registry.json" - registry.write_text( - json.dumps( - { - "common_runtime_root": str(tmp_path / "runtime"), - "goals": [ - { - "id": "event-transaction", - "status": "active", - "repo": str(tmp_path), - "state_file": state.name, - "domain": "harness_self_improvement", - "adapter": {"kind": "harness_self_improvement"}, - "coordination": { - "agent_model": "peer_v1", - "registered_agents": ["worker"], - }, - } - ], - } - ), - encoding="utf-8", - ) - store = AppendOnlyStateEventStore(tmp_path / "events.jsonl") - store.append( - make_state_event( - event_id="parent", - goal_id="event-transaction", - event_type=TODO_ADDED, - refs={"todo_id": "todo_parent"}, - payload={ - "role": "agent", - "title": "Validate the implementation before independent review.", - "task_class": "advancement_task", - "claimed_by": "worker", - }, - recorded_at="2026-09-24T00:00:00Z", - ) - ) - request = dict( - registry_path=registry, - goal_id="event-transaction", - todo_id="todo_parent", - claimed_by="worker", - evidence="Implementation validation passed.", - completion_turn_key="completion-transaction", - next_agent_todo="Review the implementation independently.", - next_task_class="advancement_task", - next_claimed_by="worker", - ) - return store, request, state - - -def test_late_completion_encoding_failure_cannot_orphan_successor( - event_goal, monkeypatch -): - store, request, state = event_goal - before, markdown = store.path.read_bytes(), state.read_bytes() - encode = event_writeback.make_state_event - - def fail_completion(**kwargs): - if kwargs["event_type"] == TODO_COMPLETED: - raise ValueError("injected completion encoding failure") - return encode(**kwargs) - - monkeypatch.setattr(event_writeback, "make_state_event", fail_completion) - with pytest.raises(ValueError, match="encoding failure"): - complete_goal_todo(**request) - assert store.path.read_bytes() == before - assert state.read_bytes() == markdown - - -def test_concurrent_source_change_rejects_entire_completion(event_goal, monkeypatch): - store, request, _ = event_goal - derive = event_writeback.derive_successor_proposals - - def concurrent_update(**kwargs): - proposals = derive(**kwargs) - store.append( - make_state_event( - event_id="concurrent-update", - goal_id="event-transaction", - event_type=TODO_UPDATED, - refs={"todo_id": "todo_parent"}, - payload={"title": "A revised validation requirement."}, - recorded_at="2026-09-24T00:01:00Z", - ) - ) - return proposals - - monkeypatch.setattr( - event_writeback, "derive_successor_proposals", concurrent_update - ) - result = complete_goal_todo(**request) - assert result["ok"] is False - assert result["completed"] is False - assert [row["event_id"] for row in store.load()] == ["parent", "concurrent-update"] - - -def test_lost_sync_ack_retries_public_completion_without_duplicate_work( - event_goal, monkeypatch -): - store, request, _ = event_goal - sync = io.fsync_state_directory - failures = [] - - def fail_event_log_once(path): - if path == store.path and not failures: - failures.append(path) - raise OSError("injected event-log directory sync failure") - return sync(path) - - monkeypatch.setattr(io, "fsync_state_directory", fail_event_log_once) - with pytest.raises(StateEventCommitUnknownError): - complete_goal_todo(**request) - landed = store.path.read_bytes() - projection = build_state_projection(store.load()) - todos = projection["agent_todos"]["items"] - parent = next(row for row in todos if row["todo_id"] == "todo_parent") - assert parent["status"] == "done" - assert len(parent["successor_todo_ids"]) == 1 - assert parent["successor_todo_ids"][0] in {row["todo_id"] for row in todos} - replay = complete_goal_todo(**request) - assert replay["idempotent_replay"] is True - assert replay["changed"] is False - assert store.path.read_bytes() == landed - - -def test_complete_successors_are_visible_together_and_dry_run_is_read_only(event_goal): - store, request, state = event_goal - request.update( - next_user_todo="Approve the proposed delivery.", - next_user_task_class="user_gate", - ) - before, markdown = store.path.read_bytes(), state.read_bytes() - preview = complete_goal_todo(**request, dry_run=True) - assert preview["ok"] is True - assert store.path.read_bytes() == before - result = complete_goal_todo(**request) - assert result["ok"] is True - assert len(result["next_todos"]) == 2 - for successor in result["next_todos"]: - assert successor["required_capabilities"] == [] - assert successor["excluded_agents"] == [] - events = store.load() - assert events[-1]["event_type"] == TODO_COMPLETED - projection = build_state_projection(events) - parent = next( - row - for row in projection["agent_todos"]["items"] - if row["todo_id"] == "todo_parent" - ) - all_ids = { - row["todo_id"] - for role in ("agent_todos", "user_todos") - for row in projection[role]["items"] - } - assert len(parent["successor_todo_ids"]) == 2 - assert set(parent["successor_todo_ids"]) <= all_ids - assert state.read_bytes() == markdown diff --git a/tests/control_plane/test_goal_amendment_proposal.py b/tests/control_plane/test_goal_amendment_proposal.py index 4e088c4c14..3395e41927 100644 --- a/tests/control_plane/test_goal_amendment_proposal.py +++ b/tests/control_plane/test_goal_amendment_proposal.py @@ -44,11 +44,6 @@ from loopx.control_plane.todos.active_state_todo_parser import ( parse_active_state_todos, ) -from loopx.event_sourced_state import ( - TODO_ADDED, - AppendOnlyStateEventStore, - make_state_event, -) from loopx.history import load_index GOAL_ID = "goal-stage2" @@ -291,7 +286,6 @@ def _goal_state_text(specs: list[dict[str, str]]) -> str: def _write_fixture( root: Path, *, - events: list[dict[str, str]] | None = None, with_other_goal: bool = False, runs: Any = _DEFAULT_RUNS, other_goal_runs: Any = _DEFAULT_RUNS, @@ -359,21 +353,7 @@ def _goal_state(goal_id: str, specs: list[dict[str, str]]) -> tuple[Path, str]: encoding="utf-8", ) - if events: - store = AppendOnlyStateEventStore(state_file.with_name(EVENT_LOG_NAME)) - for event in events: - store.append( - make_state_event( - event_id=event["event_id"], - goal_id=GOAL_ID, - event_type=TODO_ADDED, - actor_agent_id=event["actor_agent_id"], - refs={"todo_id": event["todo_id"]}, - payload={"text": f"Fixture event for {event['todo_id']}."}, - ) - ) - else: - runtime.mkdir(parents=True, exist_ok=True) + runtime.mkdir(parents=True, exist_ok=True) # The quota run-history ledger is the causal authority: by default the # fixture carries a real open obligation (typed stall runs), and each @@ -426,24 +406,6 @@ def _assert_fixture_todos_parsed( ) -def _default_events() -> list[dict[str, str]]: - return [ - { - "event_id": "evt_stage2_001", - "actor_agent_id": "agent-a", - "todo_id": "todo_stage2_a", - }, - { - "event_id": "evt_stage2_002", - "actor_agent_id": "agent-b", - "todo_id": "todo_stage2_b", - }, - { - "event_id": "evt_stage2_003", - "actor_agent_id": "agent-a", - "todo_id": "todo_stage2_c", - }, - ] def _derived_source_basis(paths: dict[str, Path]) -> dict[str, object]: @@ -543,7 +505,7 @@ def _journal_rows(paths: dict[str, Path]) -> list[dict[str, object]]: def test_admits_and_retains_a_well_formed_proposal(tmp_path: Path) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) record = _admit(paths, _proposal(paths)) @@ -553,7 +515,7 @@ def test_admits_and_retains_a_well_formed_proposal(tmp_path: Path) -> None: assert record["proposer_agent_id"] == "agent-a" assert record["amendment_class"] == "shared_acceptance" assert record["admission"] == "admitted" - assert record["admission_facts"] == [] + assert record["admission_facts"] == ["base_source_basis_unverifiable"] assert record["canonical_effect"] == "none" assert record["journal_append_sequence"] == 1 assert record["recorded_at"] @@ -568,7 +530,7 @@ def test_admission_binds_the_authority_derived_obligation_id( # produced — never a free-standing well-shaped string. The derivation # helper used here is the same read-only status projection entry point # the adapter calls at submit time. - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) record = _admit(paths, _proposal(paths)) @@ -587,7 +549,6 @@ def test_unscoped_goal_obligation_folds_into_a_deterministic_peer_lane( # lane admits, the other fails closed. paths = _write_fixture( tmp_path, - events=_default_events(), runs=_stall_runs(agent_id=None), ) unscoped = _derived_obligation(paths, agent_id=None) @@ -614,7 +575,7 @@ def test_unscoped_goal_obligation_folds_into_a_deterministic_peer_lane( def test_nonexistent_replan_obligation_fails_closed(tmp_path: Path) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) with pytest.raises(ValueError, match="does not match an open replan obligation"): _admit( @@ -635,7 +596,7 @@ def test_settlement_ack_run_closes_the_derived_obligation( # run into the same ledger. Derivation stops at the ack, the open # inventory empties, and a proposal naming the previously open id # fails closed with nothing retained. - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) proposal = _proposal(paths) obligation_id = _derived_obligation(paths)["obligation_id"] @@ -661,7 +622,6 @@ def test_later_utc_ack_closes_obligation_across_offsets( stalled_runs[1]["generated_at"] = "2026-09-01T08:01:00+08:00" paths = _write_fixture( tmp_path, - events=_default_events(), runs=stalled_runs, ) proposal = _proposal(paths) @@ -685,7 +645,6 @@ def test_cross_goal_replan_obligation_fails_closed(tmp_path: Path) -> None: # valid. paths = _write_fixture( tmp_path, - events=_default_events(), with_other_goal=True, other_goal_runs=_stall_runs( goal_id=OTHER_GOAL_ID, hypothesis="hypothesis-stage2-peer" @@ -712,7 +671,6 @@ def test_mismatched_agent_lane_fails_closed(tmp_path: Path) -> None: # is agent-scoped to agent-b and agent-a's proposal fails closed. paths = _write_fixture( tmp_path, - events=_default_events(), runs=_stall_runs(agent_id="agent-b"), ) @@ -729,7 +687,7 @@ def test_mismatched_agent_lane_fails_closed(tmp_path: Path) -> None: def test_missing_run_history_fails_closed(tmp_path: Path) -> None: # No run ledger at all: the obligation inventory is empty and # admission must not trust a causal chain on string shape alone. - paths = _write_fixture(tmp_path, events=_default_events(), runs=[]) + paths = _write_fixture(tmp_path, runs=[]) with pytest.raises(ValueError, match="does not match an open replan obligation"): _admit( @@ -762,7 +720,7 @@ def test_incomplete_run_rows_derive_no_obligation(tmp_path: Path) -> None: "progress_observation": {"result_class": "blocked"}, }, ] - paths = _write_fixture(tmp_path, events=_default_events(), runs=forged_rows) + paths = _write_fixture(tmp_path, runs=forged_rows) assert ( autonomous_replan_obligation_from_runs( @@ -796,7 +754,7 @@ def test_untyped_run_row_interrupting_the_streak_derives_no_obligation( }, _stall_runs()[1], ] - paths = _write_fixture(tmp_path, events=_default_events(), runs=interrupted) + paths = _write_fixture(tmp_path, runs=interrupted) assert ( autonomous_replan_obligation_from_runs( @@ -840,7 +798,7 @@ def test_module_exposes_no_obligation_writer_api() -> None: def test_legacy_receipt_journal_is_inert(tmp_path: Path) -> None: # The retired receipts.jsonl path is read by nothing: appending a # self-minted "open" receipt row there does not change admission. - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) legacy = ( paths["runtime"] / "goals" / GOAL_ID / "replan-obligations" / "receipts.jsonl" ) @@ -867,7 +825,7 @@ def test_legacy_receipt_journal_is_inert(tmp_path: Path) -> None: def test_peer_claimed_affected_todo_still_admits(tmp_path: Path) -> None: # Shared amendments legitimately affect peer-claimed work: admission # checks goal membership and openness, not proposer ownership. - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) record = _admit( paths, @@ -879,7 +837,7 @@ def test_peer_claimed_affected_todo_still_admits(tmp_path: Path) -> None: def test_nonexistent_affected_todo_fails_closed(tmp_path: Path) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) with pytest.raises(ValueError, match="not open on goal"): _admit( @@ -891,7 +849,7 @@ def test_nonexistent_affected_todo_fails_closed(tmp_path: Path) -> None: def test_done_affected_todo_fails_closed(tmp_path: Path) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) with pytest.raises(ValueError, match="not open on goal"): _admit( @@ -903,7 +861,7 @@ def test_done_affected_todo_fails_closed(tmp_path: Path) -> None: def test_cross_goal_affected_todo_fails_closed(tmp_path: Path) -> None: - paths = _write_fixture(tmp_path, events=_default_events(), with_other_goal=True) + paths = _write_fixture(tmp_path, with_other_goal=True) with pytest.raises(ValueError, match="not open on goal"): _admit( @@ -917,7 +875,7 @@ def test_cross_goal_affected_todo_fails_closed(tmp_path: Path) -> None: def test_proposal_digest_matches_the_python_canonical_recipe( tmp_path: Path, ) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) proposal = _proposal(paths) record = _admit(paths, proposal) @@ -929,44 +887,10 @@ def test_proposal_digest_matches_the_python_canonical_recipe( assert record["proposal_digest"] == expected -def test_stale_base_is_retained_with_needs_rebase(tmp_path: Path) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) - - record = _admit(paths, _proposal(paths, {"base_state_event_basis_sequence": 1})) - - assert record["admission_facts"] == [ - "base_state_event_basis_sequence_behind_derived_head" - ] - assert record["canonical_effect"] == "none" - assert len(_journal_rows(paths)) == 1, "a stale proposal must still be retained" - - -def test_future_base_fails_closed_without_retention(tmp_path: Path) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) - - with pytest.raises(ValueError, match="ahead of the derived state event basis head"): - _admit(paths, _proposal(paths, {"base_state_event_basis_sequence": 99})) - - assert _journal_rows(paths) == [] -def test_equal_sequence_with_mismatched_digest_needs_rebase( - tmp_path: Path, -) -> None: - # A proposal claiming the current sequence but binding to a different - # source basis identity must never be admitted fresh: the digest - # participates in admission, not only the sequence (review P1b). - paths = _write_fixture(tmp_path, events=_default_events()) - record = _admit( - paths, - _proposal(paths, {"base_source_basis_digest": MISMATCHED_DIGEST}), - ) - assert record["admission"] == "needs_rebase" - assert record["admission_facts"] == ["base_source_basis_digest_mismatch"] - assert record["canonical_effect"] == "none" - assert [row["admission"] for row in _journal_rows(paths)] == ["needs_rebase"] def test_replan_obligation_ids_follow_the_todo_contract( @@ -976,7 +900,7 @@ def test_replan_obligation_ids_follow_the_todo_contract( # normalize_todo_replan_obligation_id's "replan-<16 lowercase hex>" # (real values such as the derived fixture id); the colon namespace # must be rejected end to end through the managed TS runtime. - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) record = _admit(paths, _proposal(paths)) # default uses the derived id @@ -998,7 +922,7 @@ def test_replan_obligation_ids_follow_the_todo_contract( def test_unknown_amendment_class_is_rejected_without_retention( tmp_path: Path, ) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) with pytest.raises(ValueError, match="amendment class is unsupported"): _admit(paths, _proposal(paths, {"amendment_class": "emergency_powers"})) @@ -1007,7 +931,7 @@ def test_unknown_amendment_class_is_rejected_without_retention( def test_evidence_refs_over_budget_are_rejected(tmp_path: Path) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) with pytest.raises(ValueError, match="exceeds 8 pointers"): _admit( @@ -1026,14 +950,14 @@ def test_evidence_refs_over_budget_are_rejected(tmp_path: Path) -> None: def test_unregistered_proposer_fails_closed(tmp_path: Path) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) with pytest.raises(ValueError, match="not registered"): _admit(paths, _proposal(paths, {"proposer_agent_id": "agent-z"})) def test_unknown_goal_fails_closed(tmp_path: Path) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) with pytest.raises(ValueError, match="not registered"): _admit(paths, _proposal(paths, {"goal_id": "goal-unknown"})) @@ -1042,7 +966,7 @@ def test_unknown_goal_fails_closed(tmp_path: Path) -> None: def test_journal_is_append_only_and_replay_is_idempotent( tmp_path: Path, ) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) journal = _proposal_journal(paths) _admit(paths, _proposal(paths)) @@ -1065,7 +989,7 @@ def test_journal_is_append_only_and_replay_is_idempotent( def test_conflicting_proposal_id_replay_fails_closed(tmp_path: Path) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) _admit(paths, _proposal(paths)) with pytest.raises(ValueError, match="conflicting proposal_id"): @@ -1085,7 +1009,7 @@ def test_markdown_basis_zero_from_real_projection_admits(tmp_path: Path) -> None # proposal must be able to consume that real basis verbatim — a decoder # demanding a positive integer here forces proposers to fabricate # history. (_proposal binds the live derived basis, so sequence is 0.) - paths = _write_fixture(tmp_path, events=None) + paths = _write_fixture(tmp_path) proposal = _proposal(paths) assert proposal["base_state_event_basis_sequence"] == 0 @@ -1105,7 +1029,7 @@ def test_markdown_basis_with_fabricated_positive_sequence_fails_closed( # decoder happily admitted the proposal as unverifiable. 0 is the only # markdown basis the Stage 1 producer can emit, so any other value is # not a producible base and must fail closed instead of being retained. - paths = _write_fixture(tmp_path, events=None) + paths = _write_fixture(tmp_path) with pytest.raises(ValueError, match="markdown_active_state"): _admit(paths, _proposal(paths, {"base_state_event_basis_sequence": 5})) @@ -1117,84 +1041,28 @@ def test_event_log_basis_rejects_zero_sequence(tmp_path: Path) -> None: # Event-log bases stay strictly positive: an append sequence of 0 cannot # exist under revision_basis=state_event_log, so it is a schema-level # rejection, not a "behind the head" needs_rebase retention. - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) with pytest.raises(ValueError, match="state_event_log"): - _admit(paths, _proposal(paths, {"base_state_event_basis_sequence": 0})) + _admit(paths, _proposal(paths, {"base_revision_basis": "state_event_log", "base_state_event_basis_sequence": 0})) assert _journal_rows(paths) == [] -def test_superseded_markdown_basis_is_retained_as_needs_rebase( - tmp_path: Path, -) -> None: - # Review round 8 counterexample: the proposal binds the REAL markdown - # basis (sequence 0 from the live Stage 1 projection of an event-less - # Goal) and admits. The Goal then gains its first state event through - # the real AppendOnlyStateEventStore producer; replaying the same base - # under a new proposal id must not be rejected as a fabricated history — - # it enters the explicit needs_rebase reconciliation outcome, is - # retained, and reads back from the journal. - paths = _write_fixture(tmp_path, events=None) - - markdown_proposal = _proposal(paths) - assert markdown_proposal["base_revision_basis"] == "markdown_active_state" - first = _admit(paths, markdown_proposal) - assert first["admission"] == "admitted" - assert first["admission_facts"] == ["base_source_basis_unverifiable"] - - store = AppendOnlyStateEventStore(paths["state_file"].with_name(EVENT_LOG_NAME)) - store.append( - make_state_event( - event_id="evt_stage2_first", - goal_id=GOAL_ID, - event_type=TODO_ADDED, - actor_agent_id="agent-a", - refs={"todo_id": "todo_stage2_a"}, - payload={"text": "First fixture event for todo_stage2_a."}, - ) - ) - evolved = _derived_source_basis(paths) - assert evolved["revision_basis"] == "state_event_log" - assert evolved["state_event_basis_sequence"] >= 1 - - replay = _admit(paths, {**markdown_proposal, "proposal_id": "gap_stage2_002"}) - - assert replay["admission"] == "needs_rebase" - assert replay["admission_facts"] == ["base_revision_basis_superseded"] - assert replay["canonical_effect"] == "none" - assert replay["base_revision_basis"] == "markdown_active_state" - assert replay["base_state_event_basis_sequence"] == 0 - rows = _journal_rows(paths) - assert [row["admission"] for row in rows] == ["admitted", "needs_rebase"] - - # Not every zero under the evolved basis is a superseded markdown base: - # a proposal claiming state_event_log with sequence 0 invents an append - # that can never have existed and still fails closed, nothing retained. - with pytest.raises(ValueError, match="state_event_log"): - _admit( - paths, - { - **markdown_proposal, - "proposal_id": "gap_stage2_003", - "base_revision_basis": "state_event_log", - }, - ) - assert len(_journal_rows(paths)) == 2 def test_admission_has_zero_canonical_effect(tmp_path: Path) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) event_log = paths["state_file"].with_name(EVENT_LOG_NAME) before_state = paths["state_file"].read_bytes() - before_events = event_log.read_bytes() + assert not event_log.exists() before_registry = paths["registry"].read_bytes() _admit(paths, _proposal(paths)) _admit(paths, _proposal(paths, {"proposal_id": "gap_stage2_002"})) assert paths["state_file"].read_bytes() == before_state - assert event_log.read_bytes() == before_events + assert not event_log.exists() assert paths["registry"].read_bytes() == before_registry assert _journal_rows(paths)[0]["canonical_effect"] == "none" @@ -1240,7 +1108,7 @@ def test_registered_effect_method_rejects_an_illegal_request() -> None: def test_concurrent_admissions_serialize_journal_appends( tmp_path: Path, ) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) repo_root = Path(__file__).resolve().parents[2] child_code = """ import json @@ -1307,7 +1175,7 @@ def test_concurrent_admissions_serialize_journal_appends( def test_corrupt_journal_line_fails_closed_and_retains_nothing( tmp_path: Path, ) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) _admit(paths, _proposal(paths)) journal = _proposal_journal(paths) with journal.open("a", encoding="utf-8") as stream: @@ -1329,7 +1197,7 @@ def test_corrupt_journal_line_fails_closed_and_retains_nothing( def test_retention_does_not_advance_the_derived_head(tmp_path: Path) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) before_tree = _canonical_tree_snapshot(paths) first = _admit(paths, _proposal(paths)) @@ -1339,29 +1207,11 @@ def test_retention_does_not_advance_the_derived_head(tmp_path: Path) -> None: ) assert first["admission"] == "admitted" - assert first["admission_facts"] == [] + assert first["admission_facts"] == ["base_source_basis_unverifiable"] # The first journal append must not move the canonical head the second # proposal reports against: retention lives outside the revision # carrier, so the same base stays fresh (not needs_rebase) and every # non-journal runtime byte is unchanged. assert second["admission"] == "admitted" - assert second["admission_facts"] == [] - assert _canonical_tree_snapshot(paths) == before_tree - - -def test_needs_rebase_admission_has_zero_canonical_effect( - tmp_path: Path, -) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) - before_tree = _canonical_tree_snapshot(paths) - - record = _admit(paths, _proposal(paths, {"base_state_event_basis_sequence": 1})) - - assert record["admission"] == "needs_rebase" - assert record["admission_facts"] == [ - "base_state_event_basis_sequence_behind_derived_head" - ] + assert second["admission_facts"] == ["base_source_basis_unverifiable"] assert _canonical_tree_snapshot(paths) == before_tree - rows = _journal_rows(paths) - assert [row["admission"] for row in rows] == ["needs_rebase"] - assert rows[0]["canonical_effect"] == "none" diff --git a/tests/control_plane/test_goal_amendment_proposal_cli.py b/tests/control_plane/test_goal_amendment_proposal_cli.py index 381bd3db10..211db3afc1 100644 --- a/tests/control_plane/test_goal_amendment_proposal_cli.py +++ b/tests/control_plane/test_goal_amendment_proposal_cli.py @@ -20,19 +20,12 @@ import pytest from loopx.cli import main as cli_main -from loopx.event_sourced_state import ( - TODO_ADDED, - AppendOnlyStateEventStore, - make_state_event, -) from tests.control_plane.test_goal_amendment_proposal import ( AGENTS, - EVENT_LOG_NAME, GOAL_ID, OTHER_GOAL_ID, _ack_run, _append_runs, - _default_events, _derived_obligation, _proposal, _stall_runs, @@ -91,7 +84,6 @@ def _write_dual_registry_fixture(root: Path) -> dict[str, Path]: """ from tests.control_plane.test_goal_amendment_proposal import ( - _default_events, _default_todo_specs, _goal_state_text, ) @@ -103,19 +95,6 @@ def _write_dual_registry_fixture(root: Path) -> dict[str, Path]: state_file = project / state_relative state_file.parent.mkdir(parents=True) state_file.write_text(_goal_state_text(_default_todo_specs()), encoding="utf-8") - store = AppendOnlyStateEventStore(state_file.with_name(EVENT_LOG_NAME)) - for event in _default_events(): - store.append( - make_state_event( - event_id=event["event_id"], - goal_id=GOAL_ID, - event_type=TODO_ADDED, - actor_agent_id=event["actor_agent_id"], - refs={"todo_id": event["todo_id"]}, - payload={"text": f"Fixture event for {event['todo_id']}."}, - ) - ) - def _registry(common_runtime_root: Path) -> str: return ( json.dumps( @@ -166,7 +145,7 @@ def test_cli_submits_proposal_and_lists_journal_json( tmp_path: Path, capsys: pytest.CaptureFixture[str], ) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) proposal = _proposal(paths) proposal_json = _write_submit_inputs(tmp_path, proposal) @@ -285,7 +264,7 @@ def test_cli_single_registry_submit_and_list_positive_control( # Positive control: with one registry there is no routing ambiguity — # submit and --list agree, including when --project names the registry's # own project directory (whose local registry IS the selected registry). - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) proposal = _proposal(paths) proposal_json = _write_submit_inputs(tmp_path, proposal) @@ -321,7 +300,7 @@ def test_cli_submits_proposal_markdown( tmp_path: Path, capsys: pytest.CaptureFixture[str], ) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) proposal = _proposal(paths) proposal_json = _write_submit_inputs(tmp_path, proposal) @@ -361,7 +340,7 @@ def test_cli_alias_amendment_proposal_matches( tmp_path: Path, capsys: pytest.CaptureFixture[str], ) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) proposal = _proposal(paths) proposal_json = _write_submit_inputs(tmp_path, proposal) @@ -383,7 +362,7 @@ def test_cli_resubmission_is_idempotent( tmp_path: Path, capsys: pytest.CaptureFixture[str], ) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) proposal = _proposal(paths) proposal_json = _write_submit_inputs(tmp_path, proposal) @@ -427,7 +406,7 @@ def test_cli_nonexistent_obligation_fails_closed( tmp_path: Path, capsys: pytest.CaptureFixture[str], ) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) proposal = _proposal(paths, {"replan_obligation_id": "replan-deadbeefdeadbeef"}) proposal_json = _write_submit_inputs(tmp_path, proposal) @@ -453,7 +432,7 @@ def test_cli_unregistered_proposer_fails_closed( tmp_path: Path, capsys: pytest.CaptureFixture[str], ) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) proposal = _proposal(paths, {"proposer_agent_id": "agent-z"}) proposal_json = _write_submit_inputs(tmp_path, proposal) @@ -475,7 +454,7 @@ def test_cli_list_without_goal_id_fails_closed( tmp_path: Path, capsys: pytest.CaptureFixture[str], ) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) exit_code, payload, _ = _run_amendment_cli( capsys, @@ -495,7 +474,7 @@ def test_cli_malformed_proposal_json_fails_closed( tmp_path: Path, capsys: pytest.CaptureFixture[str], ) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) proposal_json = tmp_path / "broken.json" proposal_json.write_text('{"schema_version": tru', encoding="utf-8") @@ -523,7 +502,7 @@ def test_cli_submit_without_run_history_fails_closed( # No run ledger at all: the obligation inventory is empty and the # causal chain cannot be verified from string shape alone. There is no # CLI flag that could supply an authority payload instead. - paths = _write_fixture(tmp_path, events=_default_events(), runs=[]) + paths = _write_fixture(tmp_path, runs=[]) proposal = _proposal(paths, {"replan_obligation_id": "replan-0123456789abcdef"}) proposal_json = _write_submit_inputs(tmp_path, proposal) @@ -545,7 +524,7 @@ def test_cli_list_path_traversal_sibling_fails_closed( tmp_path: Path, capsys: pytest.CaptureFixture[str], ) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) # Place a sibling journal under runtime/goals/victim/amendment-proposals/journal.jsonl sibling_journal = ( paths["runtime"] / "goals" / "victim" / "amendment-proposals" / "journal.jsonl" @@ -579,7 +558,7 @@ def test_cli_list_path_traversal_dotdot_fails_closed( tmp_path: Path, capsys: pytest.CaptureFixture[str], ) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) exit_code, payload, _ = _run_amendment_cli( capsys, @@ -601,7 +580,7 @@ def test_cli_list_absolute_path_fails_closed( tmp_path: Path, capsys: pytest.CaptureFixture[str], ) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) exit_code, payload, _ = _run_amendment_cli( capsys, @@ -623,7 +602,7 @@ def test_cli_list_path_separator_fails_closed( tmp_path: Path, capsys: pytest.CaptureFixture[str], ) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) exit_code, payload, _ = _run_amendment_cli( capsys, @@ -645,7 +624,7 @@ def test_cli_list_unknown_goal_fails_closed( tmp_path: Path, capsys: pytest.CaptureFixture[str], ) -> None: - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) exit_code, payload, _ = _run_amendment_cli( capsys, @@ -687,7 +666,7 @@ def test_cli_submit_forged_run_rows_fails_closed( "progress_observation": {"result_class": "blocked"}, }, ] - paths = _write_fixture(tmp_path, events=_default_events(), runs=forged_rows) + paths = _write_fixture(tmp_path, runs=forged_rows) proposal = _proposal(paths, {"replan_obligation_id": "replan-0123456789abcdef"}) proposal_json = _write_submit_inputs(tmp_path, proposal) @@ -716,7 +695,7 @@ def test_cli_legacy_receipt_journal_is_inert( # The retired receipts.jsonl path is read by nothing: appending a # self-minted "open" receipt row there neither admits a proposal nor # changes the derived causal id. - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) legacy = ( paths["runtime"] / "goals" / GOAL_ID / "replan-obligations" / "receipts.jsonl" ) @@ -761,7 +740,7 @@ def test_cli_submit_after_settlement_ack_run_fails_closed( # The real close path: refresh-state appends an autonomous_replan_ack # run into the quota ledger, derivation stops there, and a proposal # naming the previously open obligation fails closed. - paths = _write_fixture(tmp_path, events=_default_events()) + paths = _write_fixture(tmp_path) proposal = _proposal(paths) obligation_id = _derived_obligation(paths)["obligation_id"] @@ -791,7 +770,6 @@ def test_cli_submit_wrong_goal_obligation_fails_closed( ) -> None: paths = _write_fixture( tmp_path, - events=_default_events(), with_other_goal=True, other_goal_runs=_stall_runs( goal_id=OTHER_GOAL_ID, hypothesis="hypothesis-stage2-peer" diff --git a/tests/control_plane/test_goal_amendment_proposal_lifecycle.py b/tests/control_plane/test_goal_amendment_proposal_lifecycle.py index 25a3b1c1b3..018fc78df3 100644 --- a/tests/control_plane/test_goal_amendment_proposal_lifecycle.py +++ b/tests/control_plane/test_goal_amendment_proposal_lifecycle.py @@ -408,187 +408,3 @@ def test_production_quota_obligation_survives_the_full_amendment_lifecycle( ) assert final_rc == 0, final assert [row["proposal_id"] for row in final["rows"]] == ["gap_lifecycle_001"] - - -def test_markdown_proposal_reconciles_after_the_goal_gains_an_event_log( - tmp_path: Path, -) -> None: - """Producer → basis transition → submit/list regression (review r8). - - A proposal that legitimately bound the real markdown basis (sequence 0) - stays explicitly reconcilable after the Goal's first state event moves - the derived basis to a typed event log: resubmitting the same base - under a new proposal id returns the needs_rebase reconciliation outcome - through the production CLI and reads back from the journal — never the - "fabricated positive integer" rejection — while a forged - state_event_log claim of sequence 0 keeps failing closed. - """ - from loopx.event_sourced_state import ( - TODO_ADDED, - AppendOnlyStateEventStore, - make_state_event, - ) - - project, runtime, registry_path = _write_fixture(tmp_path) - for _ in range(2): - _record_stall_observation(registry_path, runtime) - guard_rc, guard = _run_cli( - registry_path, - runtime, - "quota", - "should-run", - "--codex-app", - "--goal-id", - GOAL_ID, - "--agent-id", - AGENT_ID, - "--turn-instance-id", - TURN_ID, - "--scan-path", - str(project), - ) - assert guard_rc == 0, guard - obligation_id = guard["replan_action_packet"]["obligation_id"] - - align_rc, alignment = _run_cli( - registry_path, - runtime, - "shared-goal-alignment", - "--goal-id", - GOAL_ID, - "--agent-id", - AGENT_ID, - "--project", - str(project), - ) - assert align_rc == 0, alignment - basis = alignment["source_basis"] - assert basis["revision_basis"] == "markdown_active_state" - assert basis["state_event_basis_sequence"] == 0 - - proposal = { - "schema_version": "goal_amendment_proposal_v0", - "proposal_id": "gap_lifecycle_101", - "goal_id": GOAL_ID, - "proposer_agent_id": AGENT_ID, - "amendment_class": "lane_route", - "base_revision_basis": basis["revision_basis"], - "base_state_event_basis_sequence": basis["state_event_basis_sequence"], - "base_source_basis_digest": basis["source_basis_digest"], - "retained": ["the settled objective stays unchanged"], - "changed": ["the lane reroutes around the blocked surface"], - "stopped": [], - "evidence_refs": ["evidence:lifecycle-stall"], - "affected_todo_ids": [TODO_ID], - "replan_obligation_id": obligation_id, - } - proposal_json = tmp_path / "proposal-markdown.json" - proposal_json.write_text(json.dumps(proposal), encoding="utf-8") - submit_rc, submit = _run_cli( - registry_path, - runtime, - "goal-amendment-proposal", - "--proposal-json", - str(proposal_json), - "--project", - str(project), - ) - assert submit_rc == 0, submit - assert submit["admission"] == "admitted" - assert submit["admission_facts"] == ["base_source_basis_unverifiable"] - - # The real producer moves the Goal to a typed event log: its first - # state event lands through the actual AppendOnlyStateEventStore. - state_file = project / f".codex/goals/{GOAL_ID}/ACTIVE_GOAL_STATE.md" - AppendOnlyStateEventStore(state_file.with_name("events.jsonl")).append( - make_state_event( - event_id="evt_lifecycle_first", - goal_id=GOAL_ID, - event_type=TODO_ADDED, - actor_agent_id=AGENT_ID, - refs={"todo_id": TODO_ID}, - payload={"text": "First lifecycle event."}, - ) - ) - evolved_rc, evolved = _run_cli( - registry_path, - runtime, - "shared-goal-alignment", - "--goal-id", - GOAL_ID, - "--agent-id", - AGENT_ID, - "--project", - str(project), - ) - assert evolved_rc == 0, evolved - assert evolved["source_basis"]["revision_basis"] == "state_event_log" - assert evolved["source_basis"]["state_event_basis_sequence"] >= 1 - - # Replay the real markdown base under a new proposal id: an explicit, - # read-back reconciliation outcome — not a rejected "fabricated" base. - replay = {**proposal, "proposal_id": "gap_lifecycle_102"} - replay_json = tmp_path / "proposal-replay.json" - replay_json.write_text(json.dumps(replay), encoding="utf-8") - replay_rc, replay_payload = _run_cli( - registry_path, - runtime, - "goal-amendment-proposal", - "--proposal-json", - str(replay_json), - "--project", - str(project), - ) - assert replay_rc == 0, replay_payload - assert replay_payload["admission"] == "needs_rebase" - assert replay_payload["admission_facts"] == ["base_revision_basis_superseded"] - assert replay_payload["canonical_effect"] == "none" - assert replay_payload["base_revision_basis"] == "markdown_active_state" - - list_rc, listed = _run_cli( - registry_path, - runtime, - "goal-amendment-proposal", - "--list", - "--goal-id", - GOAL_ID, - ) - assert list_rc == 0, listed - assert [row["proposal_id"] for row in listed["rows"]] == [ - "gap_lifecycle_101", - "gap_lifecycle_102", - ] - assert listed["rows"][1]["admission"] == "needs_rebase" - - # The illegal claim stays illegal: sequence 0 can never have existed - # under a state_event_log base, so it is rejected with nothing retained. - forged = { - **proposal, - "proposal_id": "gap_lifecycle_103", - "base_revision_basis": "state_event_log", - } - forged_json = tmp_path / "proposal-forged.json" - forged_json.write_text(json.dumps(forged), encoding="utf-8") - forged_rc, forged_payload = _run_cli( - registry_path, - runtime, - "goal-amendment-proposal", - "--proposal-json", - str(forged_json), - "--project", - str(project), - ) - assert forged_rc == 1, forged_payload - assert "state_event_log" in forged_payload["error"] - final_rc, final = _run_cli( - registry_path, - runtime, - "goal-amendment-proposal", - "--list", - "--goal-id", - GOAL_ID, - ) - assert [row["proposal_id"] for row in final["rows"]] == [ - "gap_lifecycle_101", - "gap_lifecycle_102", - ] diff --git a/tests/control_plane/test_goal_handoff_mode.py b/tests/control_plane/test_goal_handoff_mode.py index 986ec21059..fd2a148c53 100644 --- a/tests/control_plane/test_goal_handoff_mode.py +++ b/tests/control_plane/test_goal_handoff_mode.py @@ -48,16 +48,10 @@ transfer_task_lease, write_lease, ) -from loopx.event_sourced_state import ( - TODO_ADDED, - AppendOnlyStateEventStore, - make_state_event, -) from loopx.status import parse_active_state_todos from loopx.todos import ( add_goal_todo, complete_goal_todo, - list_goal_todos, supersede_goal_todo, update_goal_todo, ) @@ -258,28 +252,6 @@ def test_goal_handoff_mode_without_frontmatter_is_legacy() -> None: assert goal_handoff_mode("## Agent Todo\n") == HANDOFF_MODE_LEGACY -def test_event_only_claim_blocks_mode_switch_without_mutating_either_source(tmp_path: Path) -> None: - """Unmaterialized claims are ownership facts, including during migration.""" - registry, state = _write_workspace(tmp_path) - event_log = state.with_name("events.jsonl") - todo_id = "todo_event_only_claimed" - AppendOnlyStateEventStore(event_log).append(make_state_event( - event_id="evt-event-only-claimed", goal_id=GOAL_ID, event_type=TODO_ADDED, - refs={"todo_id": todo_id}, payload={"role": "agent", "title": "Complete the event-only task.", - "task_class": "advancement_task", "claimed_by": AGENT_A}, - recorded_at="2026-08-01T00:01:00+00:00", producer="handoff-mode-regression")) - assert todo_id not in state.read_text(encoding="utf-8") - projected = list_goal_todos(registry_path=registry, goal_id=GOAL_ID, todo_id=todo_id) - assert projected["todo"]["claimed_by"] == AGENT_A - before = state.read_bytes(), event_log.read_bytes() - for dry_run in (True, False): - with pytest.raises(HandoffModeError) as error: - set_goal_handoff_mode(registry_path=registry, goal_id=GOAL_ID, - mode=HANDOFF_MODE_HARD_LEASE, dry_run=dry_run) - assert error.value.code == "handoff_mode_not_quiescent" - assert error.value.payload["claimed_todos"] == [ - {"todo_id": todo_id, "claimed_by": AGENT_A, "status": "open"}] - assert (state.read_bytes(), event_log.read_bytes()) == before # --------------------------------------------------------------------------- diff --git a/tests/control_plane/test_handoff_mode_source.py b/tests/control_plane/test_handoff_mode_source.py index 2dfb0e8f6e..e925ca563c 100644 --- a/tests/control_plane/test_handoff_mode_source.py +++ b/tests/control_plane/test_handoff_mode_source.py @@ -6,9 +6,8 @@ import pytest -from loopx.control_plane.todos.handoff_mode import HandoffModeError, set_goal_handoff_mode, show_goal_handoff_mode -from loopx.event_sourced_state import AppendOnlyStateEventStore, TODO_ADDED, make_state_event -from loopx.file_lock import LockAcquireTimeoutError, exclusive_file_lock +from loopx.control_plane.todos.handoff_mode import HandoffModeError, set_goal_handoff_mode +from loopx.control_plane.goals.legacy_event_source import RetiredTodoEventSourceError def workspace(root: Path) -> tuple[Path, Path, Path]: @@ -21,63 +20,14 @@ def workspace(root: Path) -> tuple[Path, Path, Path]: return registry, state, runtime -def append(path: Path, *, claimed: bool, todo_id: str = "todo_event") -> None: - AppendOnlyStateEventStore(path).append(make_state_event(event_id=f"add-{todo_id}", goal_id="mode-source", - event_type=TODO_ADDED, refs={"todo_id": todo_id}, payload={"role": "agent", "title": "Durable event task", - "task_class": "advancement_task", **({"claimed_by": "worker"} if claimed else {})}, - recorded_at="2026-09-20T00:00:00Z", producer="mode-source-fixture")) def switch(registry: Path, **kwargs): return set_goal_handoff_mode(registry_path=registry, goal_id="mode-source", mode="hard_lease", **kwargs) -@pytest.mark.parametrize("registered", [False, True]) -def test_unmaterialized_claim_at_end_of_large_event_source_blocks(tmp_path: Path, registered: bool) -> None: - registry, state, _ = workspace(tmp_path) - path = tmp_path / ("declared-events.jsonl" if registered else "events.jsonl") - if registered: - value = json.loads(registry.read_text()) - value["goals"][0]["state_event_log"] = str(path) - registry.write_text(json.dumps(value)) - events = [make_state_event(event_id=f"add-{i}", goal_id="mode-source", event_type=TODO_ADDED, - refs={"todo_id": f"todo_event_{i}"}, payload={"role": "user" if i % 2 else "agent", - "title": f"Task {i}", "task_class": "advancement_task", **({"claimed_by": "worker"} if i == 520 else {})}, - recorded_at="2026-09-20T00:00:00Z", producer="mode-source-fixture") for i in range(521)] - AppendOnlyStateEventStore(path).append_many(events) - before = state.read_bytes(), path.read_bytes() - for dry_run in (True, False): - with pytest.raises(HandoffModeError) as caught: - switch(registry, dry_run=dry_run) - assert caught.value.code == "handoff_mode_not_quiescent" - assert [row["todo_id"] for row in caught.value.payload["claimed_todos"]] == ["todo_event_520"] - assert (state.read_bytes(), path.read_bytes()) == before - - -def test_unclaimed_overlay_stays_unmaterialized_and_locked_until_durable_write(tmp_path: Path, monkeypatch) -> None: - registry, state, _ = workspace(tmp_path) - path = state.with_name("events.jsonl") - append(path, claimed=False) - from loopx.control_plane.coordination import runtime_shadow_writer_adapter as adapter - original_write = adapter.write_captured_todo_state - seen = [] - - def locked_write(*args, **kwargs): - with pytest.raises(LockAcquireTimeoutError): - with exclusive_file_lock(path, timeout_seconds=0): - pytest.fail("event append lock was released before mode write") - seen.append(True) - return original_write(*args, **kwargs) - - monkeypatch.setattr(adapter, "write_captured_todo_state", locked_write) - before = path.read_bytes() - result = switch(registry) - assert result["changed"] is True and seen == [True] - assert path.read_bytes() == before - assert "todo_event" not in state.read_text() - assert show_goal_handoff_mode(registry_path=registry, goal_id="mode-source")["handoff_mode"] == "hard_lease" - with exclusive_file_lock(path, timeout_seconds=0): - pass + + @pytest.mark.parametrize("contents", ["not json\n", '{"schema_version":"unknown"}\n']) @@ -86,31 +36,20 @@ def test_unreadable_event_source_is_not_quiescence(tmp_path: Path, contents: str path = state.with_name("events.jsonl") path.write_text(contents) before = state.read_bytes() - with pytest.raises(HandoffModeError) as caught: + with pytest.raises(RetiredTodoEventSourceError): switch(registry) - assert caught.value.code == "handoff_mode_source_unavailable" assert state.read_bytes() == before -def test_invalid_unused_event_candidate_does_not_fall_back_silently(tmp_path: Path) -> None: - registry, state, _ = workspace(tmp_path) - declared = tmp_path / "declared.jsonl" - append(declared, claimed=False) - value = json.loads(registry.read_text()) - value["goals"][0]["state_event_log"] = str(declared) - registry.write_text(json.dumps(value)) - state.with_name("events.jsonl").write_text("corrupt fallback\n") - with pytest.raises(HandoffModeError) as caught: - switch(registry) - assert caught.value.code == "handoff_mode_source_unavailable" -def test_noop_does_not_need_to_repair_or_scan_source(tmp_path: Path) -> None: +def test_retired_source_refuses_even_a_mode_noop(tmp_path: Path) -> None: registry, state, _ = workspace(tmp_path) state.write_text(state.read_text().replace("legacy", "hard_lease")) state.with_name("events.jsonl").write_text("corrupt unrelated event\n") before = state.read_bytes() - assert switch(registry)["changed"] is False + with pytest.raises(RetiredTodoEventSourceError): + switch(registry) assert state.read_bytes() == before diff --git a/tests/control_plane/test_local_authority_shadow_outbox.py b/tests/control_plane/test_local_authority_shadow_outbox.py index 19d6cebeec..fcd02e0357 100644 --- a/tests/control_plane/test_local_authority_shadow_outbox.py +++ b/tests/control_plane/test_local_authority_shadow_outbox.py @@ -262,20 +262,6 @@ def test_disabled_capture_creates_nothing(tmp_path: Path) -> None: assert not (runtime_root / "authority-shadow").exists() -def test_event_only_capture_holds_without_inventing_projection_or_retiring_entries(tmp_path: Path) -> None: - registry, state, runtime_root = _fixture(tmp_path) - _record_change(registry, state, runtime_root, "Baseline coordination fact.") - original = state.read_text() - before = _files(_todo_dir(runtime_root)) - for event_id, proposed in (("evt-noop", original), ("evt-change", original + "\n## Operator Notes\nEvent evidence.\n")): - capture = _capture(registry, state, runtime_root, original_text=original, - write_class="todo_complete_event_projection") - capture.prepare(proposed, event_id=event_id) - capture.committed() - assert capture.outcome.entry_id is None - assert capture.outcome.skipped_reason == "event_log_writer_not_bound" - assert _files(_todo_dir(runtime_root)) == before - assert [entry.seq for entry in outbox.list_entries(_todo_dir(runtime_root))] == [1] def test_cursor_allocation_hint_does_not_authorize_a_gap_or_candidate_write(tmp_path: Path) -> None: diff --git a/tests/control_plane/test_long_history_todo_reads.py b/tests/control_plane/test_long_history_todo_reads.py index bf60640fe9..06bbd0d4d9 100644 --- a/tests/control_plane/test_long_history_todo_reads.py +++ b/tests/control_plane/test_long_history_todo_reads.py @@ -33,8 +33,8 @@ def test_public_list_preserves_completed_dependency_after_archive(tmp_path: Path fields = parse_active_state_todos(after.decode(), item_limit=None) assert fields["agent_todos"]["items"][0]["resume_ready"] is True for limit in [None, 1]: - projected = todo_summaries_from_fields(fields=fields, source="markdown_active_state", projection_fields={}, - projection_overlay=None, rollout_events=[], roles=["agent"], status="open", todo_id="todo_successor", + projected = todo_summaries_from_fields(fields=fields, source="markdown_active_state", + rollout_events=[], roles=["agent"], status="open", todo_id="todo_successor", agent_id="agent-a", limit=limit) assert projected.todos[0]["resume_ready"] is True if limit is None: diff --git a/tests/control_plane/test_quota_authority_settlement_journey.py b/tests/control_plane/test_quota_authority_settlement_journey.py index 04a9e1799a..42c8d1d374 100644 --- a/tests/control_plane/test_quota_authority_settlement_journey.py +++ b/tests/control_plane/test_quota_authority_settlement_journey.py @@ -12,7 +12,6 @@ from loopx.control_plane.coordination.runtime_shadow import build_todo_runtime_shadow_projection from loopx.control_plane.todos.active_state_todo_parser import parse_active_state_todos from loopx.control_plane.quota.settlement import read_heartbeat_settlement -from loopx.event_sourced_state import AppendOnlyStateEventStore, TODO_ADDED, make_state_event from loopx.rollout_event_log import rollout_event_log_path from loopx.presentation.renderers.quota_event_markdown import render_quota_slot_preview_markdown from loopx.state_refresh import render_state_refresh_markdown @@ -116,21 +115,6 @@ def test_failed_canonical_read_cannot_fall_back_to_markdown(tmp_path): assert cli._heartbeat_receipt_count(runtime, cli.TURN_ID) == 0 -def test_legacy_event_overlay_keeps_current_role_over_stale_markdown(tmp_path): - project, runtime, registry, path, _ = _source(tmp_path, provider="legacy") - AppendOnlyStateEventStore(path.with_name("events.jsonl")).append(make_state_event( - event_id="event-current-role", goal_id=cli.GOAL_ID, event_type=TODO_ADDED, - refs={"todo_id": cli.TODO_ID}, - payload={"role": "user", "title": "Review the proposed result", "task_class": "user_action"}, - recorded_at="2026-08-04T00:00:00Z", producer="regression-fixture", - )) - listed = list_goal_todos(registry_path=registry, goal_id=cli.GOAL_ID, - runtime_root_arg=str(runtime), todo_id=cli.TODO_ID) - assert listed["todo"]["role"] == "user" - code, guard = _guard(project, runtime, registry) - assert code == 1 - assert guard["decision"] == "skip" - assert guard["heartbeat_receipt"]["status"] == "not_committed" def _refresh(project: Path, runtime: Path, registry: Path): diff --git a/tests/control_plane/test_retired_todo_event_source.py b/tests/control_plane/test_retired_todo_event_source.py new file mode 100644 index 0000000000..d6d6b9b04f --- /dev/null +++ b/tests/control_plane/test_retired_todo_event_source.py @@ -0,0 +1,56 @@ +"""Retirement never loses an event-owned Todo or executes its validation.""" +import json +import pytest + +from loopx.control_plane.goals.legacy_event_source import ( + RetiredTodoEventSourceError, require_no_legacy_todo_events, +) +from loopx.control_plane.testing.canary_harness import run_json_cli_result +from loopx.control_plane.todos import completion_validation +from loopx.todos import complete_goal_todo + +@pytest.mark.parametrize("alias", [None, "state_event_log", "state_events_file", "event_log"]) +@pytest.mark.parametrize("contents", ["{broken\n", '{"event_type":"todo_added"}\n']) +def test_all_old_source_selectors_refuse_reads_and_writes_without_data_loss(tmp_path, alias, contents): + state = tmp_path / "ACTIVE_GOAL_STATE.md" + state.write_text("---\nstatus: active\n---\n\n## Agent Todo\n\n- [ ] [P1] Materialized work\n" + " \n") + old = tmp_path / ("aliased.jsonl" if alias else "events.jsonl") + old.write_text(contents) + goal = {"id": "retired-fixture", "repo": str(tmp_path), "state_file": state.name, + "coordination": {"agent_model": "peer_v1", "registered_agents": ["agent-a"]}} + if alias: + goal[alias] = old.name + registry = tmp_path / "registry.json" + registry.write_text(json.dumps({"common_runtime_root": str(tmp_path / "runtime"), "goals": [goal]})) + before = state.read_bytes(), old.read_bytes() + for args in [("todo", "list"), + ("shared-goal-alignment", "--agent-id", "agent-a", "--project", str(tmp_path)), ("todo", "add", "--text", "New work", "--role", "agent"), + ("todo", "complete", "--todo-id", "todo_existing", "--evidence", "validated")]: + code, result = run_json_cli_result(*args, "--goal-id", goal["id"], registry_path=registry) + assert code != 0 or result.get("ok") is False, result + assert "legacy_todo_event_source_retired" in json.dumps(result), result + assert (state.read_bytes(), old.read_bytes()) == before + + +def test_completion_rejects_before_any_validation_effect(tmp_path, monkeypatch): + state = tmp_path / "ACTIVE_GOAL_STATE.md" + state.write_text("## Agent Todo\n- [ ] [P1] Work\n" + " \n") + state.with_name("events.jsonl").write_text("retired\n") + registry = tmp_path / "registry.json" + registry.write_text(json.dumps({"goals": [{"id": "retired-fixture", "repo": str(tmp_path), "state_file": state.name}]})) + def forbidden(**kwargs): + raise AssertionError("a retired source executed a completion effect") + monkeypatch.setattr(completion_validation, "run_declared_completion_validation_effect", forbidden) + with pytest.raises(RetiredTodoEventSourceError): + complete_goal_todo(registry_path=registry, goal_id="retired-fixture", todo_id="todo_existing", + evidence="validated", no_followup=True) + + +def test_empty_file_is_not_an_event_authority_and_is_preserved(tmp_path): + state = tmp_path / "ACTIVE_GOAL_STATE.md" + old = state.with_name("events.jsonl") + old.touch() + require_no_legacy_todo_events({}, state_path=state) + assert old.exists() and old.read_bytes() == b"" diff --git a/tests/control_plane/test_shadow_drain_adversarial.py b/tests/control_plane/test_shadow_drain_adversarial.py index 976b9ab387..e3aacd4519 100644 --- a/tests/control_plane/test_shadow_drain_adversarial.py +++ b/tests/control_plane/test_shadow_drain_adversarial.py @@ -172,31 +172,6 @@ def test_registry_runtime_override_cannot_bypass_an_active_source_binding(tmp_pa assert w.state.read_bytes() == before -@pytest.mark.parametrize("overlay", [False, True], ids=["event_only", "event_overlay"]) -def test_public_qualification_and_candidate_reads_hold_unbound_event_todos( - tmp_path: Path, overlay: bool, -) -> None: - from loopx.event_sourced_state import AppendOnlyStateEventStore, TODO_ADDED, make_state_event - - w = workspace(tmp_path) - ids = [w.add(f"Markdown evidence {index}")["todo_id"] for index in range(3)] - assert w.cli("coordination-shadow", "qualify")["qualification"]["qualified"] is True - event_id = ids[0] if overlay else "todo_unbound_event" - log = w.state.with_name("events.jsonl") - store = AppendOnlyStateEventStore(log) - store.append(make_state_event( - event_id="evt-unbound-todo", goal_id=w.goal, event_type=TODO_ADDED, - refs={"todo_id": event_id}, payload={"role": "agent", "title": "Event source remains independently writable", "task_class": "advancement_task"}, - recorded_at="2026-09-06T00:00:00+00:00", - )) - assert len(store.load()) == 1 - evidence = log.read_bytes() - for command in (("qualify",), ("read-candidate", "--todo-id", ids[0])): - result = w.cli("coordination-shadow", *command, success=False) - assert result["ok"] is False, result - assert result["error"] == "event_log_writer_not_bound", result - assert result["decision_read_from_shadow"] is False - assert log.read_bytes() == evidence @pytest.mark.parametrize("missing", ["identity", "candidate"]) diff --git a/tests/control_plane/test_shadow_observable_native_e2e.py b/tests/control_plane/test_shadow_observable_native_e2e.py index dfa2e930e3..fc927e75a9 100644 --- a/tests/control_plane/test_shadow_observable_native_e2e.py +++ b/tests/control_plane/test_shadow_observable_native_e2e.py @@ -156,27 +156,14 @@ def test_http_reward_preserves_preview_gate_and_persisted_arguments(caller: Call assert w.read(todo) == record -def test_event_writer_retains_primary_semantics_and_cannot_qualify(caller: Caller) -> None: + + +def test_retired_event_source_refuses_without_primary_write(caller: Caller) -> None: w = caller - seed = """ -import json,sys -from pathlib import Path -from loopx.event_sourced_state import AppendOnlyStateEventStore, TODO_ADDED, make_state_event -store=AppendOnlyStateEventStore(Path(sys.argv[1])) -store.append(make_state_event(event_id='evt-caller-fixture', goal_id='observable',event_type=TODO_ADDED, - refs={'todo_id':'todo_event_fixture'},payload={'role':'agent','title':'Event-owned task', - 'task_class':'advancement_task','claimed_by':'agent-a'},recorded_at='2026-09-01T00:00:00Z')) -print(json.dumps({'events':len(store.load())})) -""" - log = w.path / 'events.jsonl' - assert w.invoke([sys.executable, '-c', seed, str(log)], ['event-source', 'seed'])['events'] == 1 - before = w.state.read_bytes() - completed = w.call('todo', 'complete', '--todo-id', 'todo_event_fixture', '--agent-id', 'agent-a', - '--evidence', 'validation://event-caller', '--no-follow-up') - assert completed['ok'] is True, completed - assert w.state.read_bytes() == before - assert w.read('todo_event_fixture')['done'] is True - assert len(log.read_text().splitlines()) > 1 - if w.mode == 'enabled': - result = w.call('coordination-shadow', 'qualify') - assert result['ok'] is False and result.get('error') == 'event_log_writer_not_bound', result + log = w.path / "events.jsonl" + log.write_text('{"event_type":"todo_added","refs":{"todo_id":"todo_event_fixture"}}\n') + before = w.state.read_bytes(), log.read_bytes() + result = w.call('todo', 'complete', '--todo-id', 'todo_event_fixture', '--agent-id', 'agent-a', + '--evidence', 'validation://retired-source', '--no-follow-up') + assert result['ok'] is False, result + assert (w.state.read_bytes(), log.read_bytes()) == before diff --git a/tests/control_plane/test_shared_goal_alignment.py b/tests/control_plane/test_shared_goal_alignment.py index 8ed9f97d35..d4b1bad644 100644 --- a/tests/control_plane/test_shared_goal_alignment.py +++ b/tests/control_plane/test_shared_goal_alignment.py @@ -25,11 +25,6 @@ from loopx.control_plane.todos.active_state_todo_parser import ( parse_active_state_todos, ) -from loopx.event_sourced_state import ( - AppendOnlyStateEventStore, - TODO_ADDED, - make_state_event, -) GOAL_ID = "goal-stage1" AGENTS = ("agent-a", "agent-b") @@ -114,7 +109,6 @@ def _write_fixture( root: Path, *, todo_specs: list[dict[str, str]], - events: list[dict[str, str]] | None = None, leases: dict[str, dict[str, object]] | None = None, agents: tuple[str, ...] = AGENTS, goal_id: str = GOAL_ID, @@ -159,19 +153,7 @@ def _write_fixture( encoding="utf-8", ) - if events: - store = AppendOnlyStateEventStore(state_file.with_name(EVENT_LOG_NAME)) - for event in events: - store.append( - make_state_event( - event_id=event["event_id"], - goal_id=goal_id, - event_type=TODO_ADDED, - actor_agent_id=event["actor_agent_id"], - refs={"todo_id": event["todo_id"]}, - payload={"text": f"Fixture event for {event['todo_id']}."}, - ) - ) + runtime.mkdir(parents=True, exist_ok=True) if leases: for todo_id, lease in leases.items(): @@ -252,24 +234,6 @@ def _default_todo_specs() -> list[dict[str, str]]: ] -def _default_events() -> list[dict[str, str]]: - return [ - { - "event_id": "evt_stage1_001", - "actor_agent_id": "agent-b", - "todo_id": "todo_monitor", - }, - { - "event_id": "evt_stage1_002", - "actor_agent_id": "agent-a", - "todo_id": "todo_lane_a", - }, - { - "event_id": "evt_stage1_003", - "actor_agent_id": "agent-a", - "todo_id": "todo_unclaimed", - }, - ] def test_projects_basis_binding_and_unclaimed_work( @@ -278,7 +242,6 @@ def test_projects_basis_binding_and_unclaimed_work( paths = _write_fixture( tmp_path, todo_specs=_default_todo_specs(), - events=_default_events(), ) projection = project_shared_goal_alignment( @@ -292,14 +255,14 @@ def test_projects_basis_binding_and_unclaimed_work( assert projection["agent_id"] == "agent-a" assert projection["read_only"] is True basis = projection["source_basis"] - assert basis["revision_basis"] == "state_event_log" - assert basis["state_event_basis_sequence"] == 3 + assert basis["revision_basis"] == "markdown_active_state" + assert basis["state_event_basis_sequence"] == 0 assert basis["source_basis_digest"].startswith("sha256:") assert basis["state_updated_at"] == "2026-09-01T00:00:00+00:00" frontier = projection["frontier_basis"] - assert frontier["based_on_state_event_sequence"] == 3 - assert frontier["basis_source"] == "state_event_log" - assert frontier["last_agent_event_id"] == "evt_stage1_003" + assert frontier["based_on_state_event_sequence"] is None + assert frontier["basis_source"] == "unbound" + assert frontier["last_agent_event_id"] is None assert projection["frontier_counts"] == { "current_agent_claimed_advancement_count": 1, "unclaimed_advancement_count": 1, @@ -313,7 +276,7 @@ def test_projects_basis_binding_and_unclaimed_work( for item in projection["unclaimed_eligible_work"] ) assert projection["drift_facts"] == [] - assert projection["conflict_facts"] == [] + assert projection["conflict_facts"] == ["frontier_basis_unverifiable"] def test_a_goal_id_without_the_goal_prefix_projects( @@ -326,7 +289,6 @@ def test_a_goal_id_without_the_goal_prefix_projects( tmp_path, goal_id="loopx-meta", todo_specs=_default_todo_specs(), - events=_default_events(), ) projection = project_shared_goal_alignment( @@ -336,74 +298,13 @@ def test_a_goal_id_without_the_goal_prefix_projects( ) assert projection["goal_id"] == "loopx-meta" - assert projection["source_basis"]["state_event_basis_sequence"] == 3 + assert projection["source_basis"]["state_event_basis_sequence"] == 0 assert projection["drift_facts"] == [] - assert projection["conflict_facts"] == [] - - -def test_peer_events_do_not_advance_another_agents_basis( - tmp_path: Path, -) -> None: - paths = _write_fixture( - tmp_path, - todo_specs=_default_todo_specs(), - events=_default_events(), - ) + assert projection["conflict_facts"] == ["frontier_basis_unverifiable"] - projection = project_shared_goal_alignment( - goal_id=GOAL_ID, - agent_id="agent-b", - project=paths["project"], - ) - # agent-a authored events 2 and 3; agent-b's basis stays at its own - # latest attributed event (1) and never inherits the peer sequences. - assert projection["source_basis"]["state_event_basis_sequence"] == 3 - assert projection["frontier_basis"]["based_on_state_event_sequence"] == 1 - assert projection["frontier_basis"]["last_agent_event_id"] == ( - "evt_stage1_001" - ) - assert projection["drift_facts"] == ["frontier_basis_behind"] -def test_appending_one_event_rotates_the_projection_into_frontier_behind( - tmp_path: Path, -) -> None: - paths = _write_fixture( - tmp_path, - todo_specs=_default_todo_specs(), - events=_default_events(), - ) - event_log = paths["state_file"].with_name(EVENT_LOG_NAME) - - before = project_shared_goal_alignment( - goal_id=GOAL_ID, - agent_id="agent-a", - project=paths["project"], - ) - assert before["source_basis"]["state_event_basis_sequence"] == 3 - assert before["frontier_basis"]["based_on_state_event_sequence"] == 3 - assert before["drift_facts"] == [] - - AppendOnlyStateEventStore(event_log).append( - make_state_event( - event_id="evt_stage1_004", - goal_id=GOAL_ID, - event_type=TODO_ADDED, - actor_agent_id="agent-b", - refs={"todo_id": "todo_new_peer_work"}, - payload={"text": "Fixture event that moves the state event basis head."}, - ) - ) - - after = project_shared_goal_alignment( - goal_id=GOAL_ID, - agent_id="agent-a", - project=paths["project"], - ) - assert after["source_basis"]["state_event_basis_sequence"] == 4 - assert after["frontier_basis"]["based_on_state_event_sequence"] == 3 - assert after["drift_facts"] == ["frontier_basis_behind"] def test_without_an_event_log_the_basis_is_unverifiable_not_behind( @@ -412,7 +313,6 @@ def test_without_an_event_log_the_basis_is_unverifiable_not_behind( paths = _write_fixture( tmp_path, todo_specs=_default_todo_specs(), - events=None, ) projection = project_shared_goal_alignment( @@ -440,7 +340,6 @@ def test_next_action_prose_never_changes_the_source_basis_digest( paths = _write_fixture( tmp_path, todo_specs=_default_todo_specs(), - events=_default_events(), ) first = project_shared_goal_alignment( @@ -473,7 +372,6 @@ def test_blocked_and_monitor_todos_stay_out_of_unclaimed_work( paths = _write_fixture( tmp_path, todo_specs=_default_todo_specs(), - events=_default_events(), ) projection = project_shared_goal_alignment( @@ -496,7 +394,6 @@ def test_lease_owner_mismatch_projects_a_conflict_fact( paths = _write_fixture( tmp_path, todo_specs=_default_todo_specs(), - events=_default_events(), leases={ "todo_lane_a": { "schema_version": "task_lease_v0", @@ -517,7 +414,7 @@ def test_lease_owner_mismatch_projects_a_conflict_fact( project=paths["project"], ) - assert projection["conflict_facts"] == ["lease_owner_mismatch"] + assert projection["conflict_facts"] == ["frontier_basis_unverifiable", "lease_owner_mismatch"] assert projection["drift_facts"] == [] @@ -527,7 +424,6 @@ def test_matching_lease_owner_is_not_a_conflict( paths = _write_fixture( tmp_path, todo_specs=_default_todo_specs(), - events=_default_events(), leases={ "todo_lane_a": { "schema_version": "task_lease_v0", @@ -557,7 +453,6 @@ def test_corrupt_lease_epoch_fails_closed( paths = _write_fixture( tmp_path, todo_specs=_default_todo_specs(), - events=_default_events(), leases={ "todo_lane_a": { "schema_version": "task_lease_v0", @@ -584,7 +479,6 @@ def test_open_lane_replan_obligation_projects_a_conflict_fact( paths = _write_fixture( tmp_path, todo_specs=_default_todo_specs(), - events=_default_events(), ) projection = project_shared_goal_alignment( @@ -623,7 +517,6 @@ def test_peer_claimed_bound_todo_projects_a_conflict_fact( paths = _write_fixture( tmp_path, todo_specs=specs, - events=_default_events(), ) projection = project_shared_goal_alignment( @@ -639,7 +532,6 @@ def test_unregistered_agent_fails_closed(tmp_path: Path) -> None: paths = _write_fixture( tmp_path, todo_specs=_default_todo_specs(), - events=_default_events(), ) with pytest.raises(ValueError, match="not registered"): @@ -654,7 +546,6 @@ def test_unknown_goal_fails_closed(tmp_path: Path) -> None: paths = _write_fixture( tmp_path, todo_specs=_default_todo_specs(), - events=_default_events(), ) with pytest.raises(ValueError, match="not registered"): @@ -665,64 +556,8 @@ def test_unknown_goal_fails_closed(tmp_path: Path) -> None: ) -def test_registered_agent_without_any_events_projects_an_unbound_basis( - tmp_path: Path, -) -> None: - paths = _write_fixture( - tmp_path, - todo_specs=_default_todo_specs(), - events=_default_events(), - agents=("agent-a", "agent-b", "agent-c"), - ) - - projection = project_shared_goal_alignment( - goal_id=GOAL_ID, - agent_id="agent-c", - project=paths["project"], - ) - - # The state event basis head stays verifiable (the log exists at - # sequence 3), but an Agent with zero attributed events must not - # fabricate a frontier: the basis is unbound and reported as - # unverifiable instead of behind. - assert projection["source_basis"]["revision_basis"] == "state_event_log" - assert projection["source_basis"]["state_event_basis_sequence"] == 3 - assert projection["frontier_basis"] == { - "based_on_state_event_sequence": None, - "basis_source": "unbound", - "last_agent_event_id": None, - } - assert projection["drift_facts"] == [] - assert projection["conflict_facts"] == ["frontier_basis_unverifiable"] - - -def test_a_corrupt_event_log_falls_back_to_the_markdown_basis( - tmp_path: Path, -) -> None: - paths = _write_fixture( - tmp_path, - todo_specs=_default_todo_specs(), - events=None, - ) - event_log = paths["state_file"].with_name(EVENT_LOG_NAME) - event_log.write_text("{ this line is not jsonl\n", encoding="utf-8") - projection = project_shared_goal_alignment( - goal_id=GOAL_ID, - agent_id="agent-a", - project=paths["project"], - ) - # A present-but-corrupt log must not fabricate a basis sequence: the - # adapter falls back to the markdown active state and reports the - # frontier as unverifiable instead of trusting the head. - assert projection["source_basis"]["revision_basis"] == ( - "markdown_active_state" - ) - assert projection["source_basis"]["state_event_basis_sequence"] == 0 - assert projection["frontier_basis"]["basis_source"] == "unbound" - assert projection["drift_facts"] == [] - assert projection["conflict_facts"] == ["frontier_basis_unverifiable"] def test_non_numeric_lease_epoch_fails_closed( @@ -731,7 +566,6 @@ def test_non_numeric_lease_epoch_fails_closed( paths = _write_fixture( tmp_path, todo_specs=_default_todo_specs(), - events=_default_events(), leases={ "todo_lane_a": { "schema_version": "task_lease_v0", @@ -758,7 +592,6 @@ def test_released_lease_record_does_not_project_ownership_conflict( paths = _write_fixture( tmp_path, todo_specs=_default_todo_specs(), - events=_default_events(), leases={ "todo_lane_a": { "schema_version": "task_lease_v0", @@ -780,7 +613,7 @@ def test_released_lease_record_does_not_project_ownership_conflict( ) assert "lease_owner_mismatch" not in projection["conflict_facts"] - assert projection["conflict_facts"] == [] + assert projection["conflict_facts"] == ["frontier_basis_unverifiable"] def test_expired_lease_record_does_not_project_ownership_conflict( @@ -789,7 +622,6 @@ def test_expired_lease_record_does_not_project_ownership_conflict( paths = _write_fixture( tmp_path, todo_specs=_default_todo_specs(), - events=_default_events(), leases={ "todo_lane_a": { "schema_version": "task_lease_v0", @@ -811,7 +643,7 @@ def test_expired_lease_record_does_not_project_ownership_conflict( ) assert "lease_owner_mismatch" not in projection["conflict_facts"] - assert projection["conflict_facts"] == [] + assert projection["conflict_facts"] == ["frontier_basis_unverifiable"] def test_active_lease_owner_mismatch_projects_conflict_fact( @@ -820,7 +652,6 @@ def test_active_lease_owner_mismatch_projects_conflict_fact( paths = _write_fixture( tmp_path, todo_specs=_default_todo_specs(), - events=_default_events(), leases={ "todo_lane_a": { "schema_version": "task_lease_v0", @@ -841,7 +672,7 @@ def test_active_lease_owner_mismatch_projects_conflict_fact( project=paths["project"], ) - assert projection["conflict_facts"] == ["lease_owner_mismatch"] + assert projection["conflict_facts"] == ["frontier_basis_unverifiable", "lease_owner_mismatch"] @pytest.mark.parametrize( @@ -873,7 +704,6 @@ def test_active_lease_without_a_valid_owner_fails_closed( paths = _write_fixture( tmp_path, todo_specs=_default_todo_specs(), - events=_default_events(), leases={"todo_lane_a": lease}, ) @@ -891,7 +721,6 @@ def test_projection_is_deterministic_across_repeated_calls( paths = _write_fixture( tmp_path, todo_specs=_default_todo_specs(), - events=_default_events(), ) first = project_shared_goal_alignment( @@ -919,7 +748,6 @@ def test_adapter_sends_typed_facts_only(monkeypatch, tmp_path: Path) -> None: paths = _write_fixture( tmp_path, todo_specs=_default_todo_specs(), - events=_default_events(), ) captured: dict[str, object] = {} @@ -942,8 +770,8 @@ def call(method: str, params: dict[str, object]) -> dict[str, object]: assert "prose" not in json.dumps(request) assert request["goal_id"] == GOAL_ID assert request["agent_id"] == "agent-a" - assert request["source_basis"]["state_event_basis_sequence"] == 3 - assert request["frontier_basis"]["based_on_state_event_sequence"] == 3 + assert request["source_basis"]["state_event_basis_sequence"] == 0 + assert request["frontier_basis"]["based_on_state_event_sequence"] is None assert "claims" not in request # Selection now belongs to TS, not the adapter. own = next(item for item in request["work_items"] if item["todo_id"] == "todo_lane_a") assert own["claimed_by"] == "agent-a" diff --git a/tests/control_plane/test_shared_goal_alignment_cli.py b/tests/control_plane/test_shared_goal_alignment_cli.py index 61b00c6b29..76260d061b 100644 --- a/tests/control_plane/test_shared_goal_alignment_cli.py +++ b/tests/control_plane/test_shared_goal_alignment_cli.py @@ -17,7 +17,6 @@ from tests.control_plane.test_shared_goal_alignment import ( GOAL_ID, - _default_events, _default_todo_specs, _write_fixture, ) @@ -43,7 +42,6 @@ def test_cli_projects_shared_goal_alignment_json( paths = _write_fixture( tmp_path, todo_specs=_default_todo_specs(), - events=_default_events(), ) exit_code, payload, _ = _run_alignment_cli( @@ -66,8 +64,8 @@ def test_cli_projects_shared_goal_alignment_json( assert payload["goal_id"] == GOAL_ID assert payload["agent_id"] == "agent-a" assert payload["read_only"] is True - assert payload["source_basis"]["state_event_basis_sequence"] == 3 - assert payload["frontier_basis"]["based_on_state_event_sequence"] == 3 + assert payload["source_basis"]["state_event_basis_sequence"] == 0 + assert payload["frontier_basis"]["based_on_state_event_sequence"] is None assert payload["frontier_counts"]["current_agent_claimed_advancement_count"] == 1 assert payload["unclaimed_eligible_work"][0]["todo_id"] == "todo_unclaimed" @@ -79,7 +77,6 @@ def test_cli_projects_shared_goal_alignment_markdown( paths = _write_fixture( tmp_path, todo_specs=_default_todo_specs(), - events=_default_events(), ) exit_code, _, stdout = _run_alignment_cli( @@ -111,7 +108,6 @@ def test_cli_alias_goal_alignment( paths = _write_fixture( tmp_path, todo_specs=_default_todo_specs(), - events=_default_events(), ) exit_code, payload, _ = _run_alignment_cli( @@ -140,7 +136,6 @@ def test_cli_unregistered_agent_fails_closed( paths = _write_fixture( tmp_path, todo_specs=_default_todo_specs(), - events=_default_events(), ) exit_code, payload, _ = _run_alignment_cli( @@ -169,7 +164,6 @@ def test_cli_missing_goal_fails_closed( paths = _write_fixture( tmp_path, todo_specs=_default_todo_specs(), - events=_default_events(), ) exit_code, payload, _ = _run_alignment_cli( @@ -198,7 +192,6 @@ def test_cli_missing_state_file_fails_closed( paths = _write_fixture( tmp_path, todo_specs=_default_todo_specs(), - events=_default_events(), ) paths["state_file"].unlink() diff --git a/tests/control_plane/test_supervisor_log.py b/tests/control_plane/test_supervisor_log.py new file mode 100644 index 0000000000..ccf4c24072 --- /dev/null +++ b/tests/control_plane/test_supervisor_log.py @@ -0,0 +1,73 @@ +"""Real file publication, concurrency and preview guarantees of the experimental log.""" +from concurrent.futures import ProcessPoolExecutor +from pathlib import Path +import json + +import pytest + +from loopx.control_plane.agents import supervisor_event_log as log +from loopx.control_plane.agents.supervisor_events import record_supervisor_proposal, record_supervisor_receipt + +GOAL = "supervisor-log-fixture" +SUPERVISOR = {"enabled": True, "agent_id": "agent-a", "supervised_agents": ["agent-b"]} +DECISION = {"decision_id": "inject-1", "kind": "inject", "target_agent_id": "agent-b", + "message": "Check the validation evidence.", "reason_codes": ["evidence-gap"], "evidence_refs": ["effect:1"]} + +def _receipt(path: Path, receipt_id: str): + try: + result = record_supervisor_receipt(log_path=path, goal_id=GOAL, execute=True, + host_capabilities=["session_message_injection"], receipt={"receipt_id": receipt_id, "decision_id": "inject-1", + "adapter_id": "fixture-host", "outcome": "executed", "authority_ref": "owner:1", + "rollback_boundary": {"mode": "compensating_action", "ref": "policy:1", "automatic": False}, + "evidence_refs": ["effect:1"], "reason_codes": ["adapter-success"]}) + return result["appended"] + except ValueError as exc: + return str(exc) + +def _proposal(path: Path, *, execute=True, goal_id=GOAL): + return record_supervisor_proposal(log_path=path, goal_id=goal_id, + supervisor=SUPERVISOR, decision=DECISION, execute=execute) + +def test_competing_executed_receipts_are_serialized(tmp_path): + path = tmp_path / "supervisor-events.jsonl" + _proposal(path) + with ProcessPoolExecutor(max_workers=2) as pool: + results = list(pool.map(_receipt, [path, path], ["receipt-a", "receipt-b"])) + assert results.count(True) == 1, results + assert sum(isinstance(row, str) and "already has an executed receipt" in row for row in results) == 1 + assert len(log.SupervisorEventStore(path).load()) == 2 + +def test_preview_does_not_publish_or_sync_and_checks_full_identity(tmp_path, monkeypatch): + path = tmp_path / "supervisor-events.jsonl" + _proposal(path) + before = path.read_bytes() + def forbidden(*args, **kwargs): + raise AssertionError("preview attempted durable IO") + monkeypatch.setattr(log, "atomic_write_state_text", forbidden) + monkeypatch.setattr(log, "verify_state_text_durable", forbidden) + assert _proposal(path, execute=False)["appended"] is False + with pytest.raises(log.SupervisorEventConflictError, match="another Goal"): + _proposal(path, execute=False, goal_id="other-goal") + assert path.read_bytes() == before + +def test_commit_unknown_retry_recovers_same_record(tmp_path, monkeypatch): + path = tmp_path / "supervisor-events.jsonl" + write = log.atomic_write_state_text + def uncertain(*args, **kwargs): + write(*args, **kwargs) + raise OSError("simulated acknowledgement loss") + with monkeypatch.context() as patch: + patch.setattr(log, "atomic_write_state_text", uncertain) + with pytest.raises(log.SupervisorEventCommitUnknownError): + _proposal(path) + assert _proposal(path)["appended"] is False + assert len(log.SupervisorEventStore(path).load()) == 1 + +@pytest.mark.parametrize("row", ["{broken", json.dumps({"schema_version": "loopx_state_event_v0"})]) +def test_invalid_or_retired_schema_is_not_empty_history(tmp_path, row): + path = tmp_path / "supervisor-events.jsonl" + path.write_text(row + "\n") + before = path.read_bytes() + with pytest.raises(log.SupervisorEventError): + _proposal(path) + assert path.read_bytes() == before diff --git a/tests/control_plane/test_todo_completion_validation.py b/tests/control_plane/test_todo_completion_validation.py index b2f11328fe..727c2a058f 100644 --- a/tests/control_plane/test_todo_completion_validation.py +++ b/tests/control_plane/test_todo_completion_validation.py @@ -25,16 +25,6 @@ persist_completion_validation_declaration, read_completion_validation_declaration, ) -from loopx.event_sourced_state import ( - TODO_ADDED, - TODO_COMPLETED, - TODO_DEFERRED, - TODO_UPDATED, - AppendOnlyStateEventStore, - build_state_projection, - make_state_event, - render_todo_markdown, -) from loopx.status import parse_active_state_todos from loopx.todos import add_goal_todo, complete_goal_todo, update_goal_todo @@ -1151,63 +1141,10 @@ def test_agent_todo_update_done_keeps_guard_error_without_running_gate( assert _agent_todo(state, str(todo["todo_id"]))["status"] == "open" -def _add_event_only_todo( - state: Path, - *, - todo_id: str = "todo_event_validation", - validation_command: Any = None, - validation_command_argv: Any = None, - validation_label: Any = None, - validation_timeout_seconds: Any = None, - event_log: Path | None = None, -) -> str: - store = AppendOnlyStateEventStore(event_log or state.with_name("events.jsonl")) - payload: dict[str, Any] = { - "role": "agent", - "title": "Deliver one event-projected change.", - "task_class": "advancement_task", - "claimed_by": AGENT, - } - if validation_command is not None: - payload["validation_command"] = validation_command - if validation_command_argv is not None: - payload["validation_command_argv"] = validation_command_argv - if validation_label is not None: - payload["validation_label"] = validation_label - if validation_timeout_seconds is not None: - payload["validation_timeout_seconds"] = validation_timeout_seconds - store.append( - make_state_event( - event_id=f"evt-{todo_id}-add", - goal_id=GOAL_ID, - event_type=TODO_ADDED, - refs={"todo_id": todo_id}, - payload=payload, - recorded_at="2026-08-22T00:00:00+00:00", - ) - ) - return todo_id -def _defer_event_todo(state: Path, *, todo_id: str) -> None: - AppendOnlyStateEventStore(state.with_name("events.jsonl")).append( - make_state_event( - event_id=f"evt-{todo_id}-defer", - goal_id=GOAL_ID, - event_type=TODO_DEFERRED, - refs={"todo_id": todo_id}, - payload={ - "reason": "waiting for owner signal", - "resume_when": "manual", - }, - recorded_at="2026-08-22T00:01:00+00:00", - ) - ) -def _event_todo_completed_count(event_log: Path) -> int: - events = AppendOnlyStateEventStore(event_log).load() - return sum(event["event_type"] == TODO_COMPLETED for event in events) def _set_registry_event_log(registry: Path, event_log: Path) -> None: @@ -1216,42 +1153,6 @@ def _set_registry_event_log(registry: Path, event_log: Path) -> None: registry.write_text(json.dumps(data), encoding="utf-8") -def test_event_projection_preserves_private_validation_and_public_marker() -> None: - todo_id = "todo_event_projected_validation" - events = [ - make_state_event( - event_id=f"evt-{todo_id}-add", - goal_id=GOAL_ID, - event_type=TODO_ADDED, - refs={"todo_id": todo_id}, - payload={ - "role": "agent", - "title": "Deliver one event-projected change.", - "task_class": "advancement_task", - "claimed_by": AGENT, - "validation_command_argv": [sys.executable, "-c", "pass"], - "validation_label": "event argv smoke", - "validation_timeout_seconds": 5, - }, - recorded_at="2026-08-22T00:00:00+00:00", - ) - ] - - projection = build_state_projection(events, goal_id=GOAL_ID) - item = projection["agent_todos"]["items"][0] - - assert item["validation_command_argv"] == [sys.executable, "-c", "pass"] - assert item["validation_label"] == "event argv smoke" - assert item["validation_timeout_seconds"] == 5 - rendered = "\n".join(render_todo_markdown(item)) - assert "validation_command_argv=" in rendered - public = project_completion_validation_authority(item) - assert public["completion_validation_required"] is True - assert len(public["completion_validation_sha256"]) == 64 - assert "validation_command" not in public - assert "validation_command_argv" not in public - assert "validation_label" not in public - assert "validation_timeout_seconds" not in public def test_private_validation_store_is_owner_only_and_detects_tampering( @@ -1349,249 +1250,3 @@ def test_canonical_validation_digest_rejects_different_private_declaration( role="agent", persist_if_resolved=False, ) - - -def test_event_projected_failing_validation_blocks_completion( - tmp_path: Path, - monkeypatch: pytest.MonkeyPatch, -) -> None: - registry, state = _write_fixture(tmp_path) - todo_id = _add_event_only_todo( - state, - validation_command=_FAIL_COMMAND, - validation_label="event-projected smoke", - ) - calls = _spy_validation_runner(monkeypatch) - - result = complete_goal_todo( - registry_path=registry, - goal_id=GOAL_ID, - todo_id=todo_id, - agent_id=AGENT, - evidence="claim of completion", - no_followup=True, - ) - - assert calls["count"] == 1 - assert result["ok"] is False - assert result["validation_blocked_completion"] is True - assert result["validation"]["passed"] is False - assert _event_todo_completed_count(state.with_name("events.jsonl")) == 0 - - -def test_event_projected_deferred_validation_replays_without_running_command( - tmp_path: Path, - monkeypatch: pytest.MonkeyPatch, -) -> None: - registry, state = _write_fixture(tmp_path) - todo_id = _add_event_only_todo( - state, - todo_id="todo_event_deferred_validation", - validation_command=_FAIL_COMMAND, - ) - _defer_event_todo(state, todo_id=todo_id) - calls = _spy_validation_runner(monkeypatch) - - result = complete_goal_todo( - registry_path=registry, - goal_id=GOAL_ID, - todo_id=todo_id, - agent_id=AGENT, - evidence="terminal replay request", - no_followup=True, - ) - - assert calls["count"] == 0 - assert result["ok"] is True - assert result["idempotent_replay"] is True - assert result["changed"] is False - assert "validation_blocked_completion" not in result - assert _event_todo_completed_count(state.with_name("events.jsonl")) == 0 - - -def test_noncanonical_sidecar_cannot_inject_validation_command( - tmp_path: Path, - monkeypatch: pytest.MonkeyPatch, -) -> None: - registry, state = _write_fixture(tmp_path) - todo_id = "todo_event_two_logs" - canonical = state.with_name("canonical-events.jsonl") - _add_event_only_todo(state, todo_id=todo_id, event_log=canonical) - _add_event_only_todo( - state, - todo_id=todo_id, - validation_command=_FAIL_COMMAND, - ) - _set_registry_event_log(registry, canonical) - calls = _spy_validation_runner(monkeypatch) - - result = complete_goal_todo( - registry_path=registry, - goal_id=GOAL_ID, - todo_id=todo_id, - agent_id=AGENT, - evidence="canonical source has no declaration", - no_followup=True, - ) - - assert calls["count"] == 0 - assert result["ok"] is True - assert result["changed"] is True - assert result["source"] == "event_log" - assert _event_todo_completed_count(canonical) == 1 - assert _event_todo_completed_count(state.with_name("events.jsonl")) == 0 - - -def test_empty_earlier_candidate_does_not_replace_canonical_validation_source( - tmp_path: Path, - monkeypatch: pytest.MonkeyPatch, -) -> None: - registry, state = _write_fixture(tmp_path) - todo_id = _add_event_only_todo( - state, - todo_id="todo_event_after_empty_candidate", - validation_command=_FAIL_COMMAND, - ) - empty_candidate = state.with_name("empty-events.jsonl") - empty_candidate.touch() - _set_registry_event_log(registry, empty_candidate) - calls = _spy_validation_runner(monkeypatch) - - result = complete_goal_todo( - registry_path=registry, - goal_id=GOAL_ID, - todo_id=todo_id, - agent_id=AGENT, - evidence="canonical fallback validation must run", - no_followup=True, - ) - - assert calls["count"] == 1 - assert result["ok"] is False - assert result["validation_blocked_completion"] is True - assert _event_todo_completed_count(state.with_name("events.jsonl")) == 0 - assert empty_candidate.read_text(encoding="utf-8") == "" - - -def test_corrupted_earlier_candidate_does_not_block_canonical_completion_source( - tmp_path: Path, - monkeypatch: pytest.MonkeyPatch, -) -> None: - registry, state = _write_fixture(tmp_path) - todo_id = _add_event_only_todo( - state, - todo_id="todo_event_after_corrupted_candidate", - validation_command=_PASS_COMMAND, - ) - canonical = state.with_name("events.jsonl") - corrupted_candidate = state.with_name("corrupted-events.jsonl") - corrupted_candidate.write_text("{not-json\n", encoding="utf-8") - _set_registry_event_log(registry, corrupted_candidate) - calls = _spy_validation_runner(monkeypatch) - - result = complete_goal_todo( - registry_path=registry, - goal_id=GOAL_ID, - todo_id=todo_id, - agent_id=AGENT, - evidence="canonical source remains usable", - no_followup=True, - ) - - assert calls["count"] == 1 - assert result["ok"] is True - assert result["changed"] is True - assert result["source"] == "event_log" - assert _event_todo_completed_count(canonical) == 1 - assert corrupted_candidate.read_text(encoding="utf-8") == "{not-json\n" - - -@pytest.mark.parametrize("timeout_value", [0, 30, "not-int", {"seconds": 5}]) -def test_event_projected_invalid_timeout_rejects_without_running_command( - tmp_path: Path, - monkeypatch: pytest.MonkeyPatch, - timeout_value: Any, -) -> None: - registry, state = _write_fixture(tmp_path) - todo_id = _add_event_only_todo( - state, - todo_id="todo_event_invalid_timeout", - validation_command=_PASS_COMMAND, - validation_timeout_seconds=timeout_value, - ) - calls = _spy_validation_runner(monkeypatch) - - result = complete_goal_todo( - registry_path=registry, - goal_id=GOAL_ID, - todo_id=todo_id, - agent_id=AGENT, - evidence="invalid persisted timeout must fail closed", - no_followup=True, - ) - - assert calls["count"] == 0 - assert result["ok"] is False - assert result["validation_blocked_completion"] is True - assert result["validation"]["status"] == "declaration_invalid" - assert "validation_timeout_seconds" in result["validation"]["summary"] - assert _event_todo_completed_count(state.with_name("events.jsonl")) == 0 - - -def test_event_projected_source_drift_after_validation_blocks_append( - tmp_path: Path, - monkeypatch: pytest.MonkeyPatch, -) -> None: - registry, state = _write_fixture(tmp_path) - todo_id = _add_event_only_todo( - state, - todo_id="todo_event_source_drift", - validation_command=_PASS_COMMAND, - ) - event_log = state.with_name("events.jsonl") - calls = {"count": 0} - - def drifting_runner(*_args: Any, **_kwargs: Any) -> dict[str, Any]: - calls["count"] += 1 - AppendOnlyStateEventStore(event_log).append( - make_state_event( - event_id=f"evt-{todo_id}-drift", - goal_id=GOAL_ID, - event_type=TODO_UPDATED, - refs={"todo_id": todo_id}, - payload={"title": "Changed after validation."}, - recorded_at="2026-08-22T00:02:00+00:00", - ) - ) - return { - "schema_version": completion_validation_module.CALLER_VALIDATION_RECEIPT_SCHEMA_VERSION, - "command_label": "todo completion validation", - "exit_code": 0, - "passed": True, - "status": "passed", - "summary": "validation passed before source drift", - "stdout_captured": False, - "stderr_captured": False, - "local_path_captured": False, - } - - monkeypatch.setattr( - completion_validation_module, - "run_caller_validation", - drifting_runner, - ) - - result = complete_goal_todo( - registry_path=registry, - goal_id=GOAL_ID, - todo_id=todo_id, - agent_id=AGENT, - evidence="validated stale source", - no_followup=True, - ) - - assert calls["count"] == 1 - assert result["ok"] is False - assert result["validation_blocked_completion"] is True - assert result["validation"]["status"] == "source_drift" - assert _event_todo_completed_count(event_log) == 0 diff --git a/tests/control_plane/test_todo_list_agent_lane_projection.py b/tests/control_plane/test_todo_list_agent_lane_projection.py index dcb78d6244..365b9363c5 100644 --- a/tests/control_plane/test_todo_list_agent_lane_projection.py +++ b/tests/control_plane/test_todo_list_agent_lane_projection.py @@ -1,18 +1,10 @@ from __future__ import annotations import json -import os -import subprocess -import sys from pathlib import Path from loopx.control_plane.todos.contract import encode_metadata_value from loopx.control_plane.todos.markdown import render_todo_markdown -from loopx.event_sourced_state import ( - AppendOnlyStateEventStore, - TODO_ADDED, - make_state_event, -) from loopx.todos import list_goal_todos GOAL_ID = "todo-list-agent-lane-goal" @@ -302,69 +294,3 @@ def test_explicit_done_filter_remains_a_full_detail_cold_path( assert len(payload["agent_todos"]["items"]) == 220 assert "returned_todo_count" not in payload assert "todo_list_projection" not in payload - - -def test_event_projection_role_change_keeps_one_todo_identity(tmp_path: Path) -> None: - registry_path, state_file = _write_fixture(tmp_path) - store = AppendOnlyStateEventStore(state_file.with_name("events.jsonl")) - store.append( - make_state_event( - event_id="evt-role-change", - goal_id=GOAL_ID, - event_type=TODO_ADDED, - refs={"todo_id": "todo_agent_unclaimed"}, - payload={ - "role": "user", - "title": "Review the formerly unclaimed advancement.", - "task_class": "user_action", - "bound_agent": AGENT_ID, - }, - recorded_at="2026-08-04T00:00:00Z", - producer="todo-list-role-change-regression", - ) - ) - - result = subprocess.run( - [ - sys.executable, - "-m", - "loopx.cli", - "todo", - "list", - "--format", - "json", - "--goal-id", - GOAL_ID, - ], - cwd=REPO_ROOT, - env={**os.environ, "LOOPX_REGISTRY": str(registry_path)}, - check=False, - capture_output=True, - text=True, - ) - assert result.returncode == 0, result.stderr - payload = json.loads(result.stdout) - - todo_ids = [item["todo_id"] for item in payload["todos"]] - assert len(todo_ids) == len(set(todo_ids)) - assert payload["todo_count"] == len(todo_ids) - matching = [ - item - for item in payload["todos"] - if item["todo_id"] == "todo_agent_unclaimed" - ] - assert len(matching) == 1 - assert matching[0]["role"] == "user" - assert all( - item["todo_id"] != "todo_agent_unclaimed" - for item in payload["agent_todos"]["items"] - ) - assert payload["projection_overlay"]["overlaid_todo_ids"] == [ - "todo_agent_unclaimed" - ] - assert "todo_agent_unclaimed" not in payload["projection_overlay"][ - "event_only_todo_ids" - ] - assert "todo_agent_unclaimed" not in payload["projection_overlay"][ - "markdown_only_todo_ids" - ] diff --git a/tests/control_plane/test_todo_list_explicit_limit.py b/tests/control_plane/test_todo_list_explicit_limit.py index 5cdf1dd5e0..6052b2cf92 100644 --- a/tests/control_plane/test_todo_list_explicit_limit.py +++ b/tests/control_plane/test_todo_list_explicit_limit.py @@ -6,11 +6,6 @@ import sys from pathlib import Path -from loopx.event_sourced_state import ( - AppendOnlyStateEventStore, - TODO_ADDED, - make_state_event, -) from loopx.todos import list_goal_todos GOAL_ID = "todo-list-explicit-limit-goal" @@ -318,68 +313,3 @@ def test_cli_limit_path_and_typed_rejections(tmp_path: Path) -> None: non_integer = _run_cli(registry_path, "--limit", "many") assert non_integer.returncode == 2 assert "invalid int value" in non_integer.stderr - - -def test_explicit_limit_bounds_projection_overlay_lineage( - tmp_path: Path, -) -> None: - registry_path = _write_fixture( - tmp_path, - item_count_per_role=200, - agent_id=AGENT_ID, - ) - state_file = ( - tmp_path / "project" / ".local/goals" / GOAL_ID / "ACTIVE_GOAL_STATE.md" - ) - store = AppendOnlyStateEventStore(state_file.with_name("events.jsonl")) - store.append( - make_state_event( - event_id="evt-explicit-limit-overlay", - goal_id=GOAL_ID, - event_type=TODO_ADDED, - refs={"todo_id": "todo_agent_open_000"}, - payload={ - "role": "agent", - "title": "Overlay the first advancement step.", - "task_class": "continuous_monitor", - "claimed_by": AGENT_ID, - }, - recorded_at="2026-01-01T00:00:00Z", - producer="todo-list-explicit-limit-overlay", - ) - ) - - result = subprocess.run( - [ - sys.executable, - "-m", - "loopx.cli", - "todo", - "list", - "--format", - "json", - "--goal-id", - GOAL_ID, - "--limit", - "3", - ], - cwd=REPO_ROOT, - env={**os.environ, "LOOPX_REGISTRY": str(registry_path)}, - check=False, - capture_output=True, - text=True, - ) - assert result.returncode == 0, result.stderr - payload = json.loads(result.stdout) - - assert payload["source"] == "event_projection_with_markdown_overlay" - assert payload["explicit_limit"] == 3 - assert payload["returned_todo_count"] == 6 - overlay = payload["projection_overlay"] - assert overlay["overlaid_count"] == 1 - assert overlay["event_only_count"] == 0 - assert overlay["markdown_only_count"] == 399 - assert overlay["full_detail_cold_path"] == ( - "todo list without --limit or active state" - ) - assert [key for key in overlay if key.endswith("_todo_ids")] == [] diff --git a/tests/control_plane/test_todo_mutation_authority.py b/tests/control_plane/test_todo_mutation_authority.py index da91d47c08..e65263948b 100644 --- a/tests/control_plane/test_todo_mutation_authority.py +++ b/tests/control_plane/test_todo_mutation_authority.py @@ -9,9 +9,6 @@ from loopx.control_plane.scheduler.monitor_poll_writeback import ( write_monitor_poll_todo_state, ) -from loopx.control_plane.todos.event_writeback import ( - complete_event_projected_goal_todo, -) import loopx.control_plane.todos.mutation_authority as mutation_authority_module import loopx.control_plane.work_items.task_lease as task_lease_module from loopx.control_plane.work_items.task_lease import ( @@ -19,18 +16,6 @@ acquire_task_lease, release_task_lease, ) -from loopx.event_sourced_state import ( - TODO_ADDED, - TODO_COMPLETED, - TODO_DEFERRED, - TODO_UPDATED, - AppendOnlyStateEventStore, - StateEventError, - backfill_todo_events_from_markdown, - build_state_projection, - make_state_event, - render_active_state_sections, -) from loopx.status import parse_active_state_todos from loopx.todos import ( add_goal_todo, @@ -1096,65 +1081,6 @@ def test_terminal_upgrade_rejects_untyped_legacy_completion_until_repaired( assert upgraded["completion_recovery"] == "same_turn_terminal_closeout" -def test_event_projected_completion_reports_task_lease_fence( - tmp_path: Path, -) -> None: - registry, state = _write_fixture(tmp_path, multi_agent=False) - event_log = state.with_name("events.jsonl") - store = AppendOnlyStateEventStore(event_log) - todo_id = "todo_event_lease" - store.append( - make_state_event( - event_id="evt-event-lease-parent", - goal_id=GOAL_ID, - event_type=TODO_ADDED, - refs={"todo_id": todo_id}, - payload={ - "role": "agent", - "title": "Complete the event-projected leased task.", - "task_class": "advancement_task", - "claimed_by": AUTHOR_AGENT, - }, - recorded_at="2026-07-18T00:00:00+00:00", - ) - ) - lease_key = "event-projection-instance" - acquire_task_lease( - registry_path=registry, - runtime_root=tmp_path / "runtime", - goal_id=GOAL_ID, - todo_id=todo_id, - owner=AUTHOR_AGENT, - idempotency_key=lease_key, - ttl_seconds=600, - ) - - result = complete_goal_todo( - registry_path=registry, - goal_id=GOAL_ID, - todo_id=todo_id, - claimed_by=AUTHOR_AGENT, - task_lease_idempotency_key=lease_key, - task_lease_expected_version=1, - evidence="event-projected lease owner validated the result", - no_followup=True, - ) - - assert result["source"] == "event_log" - assert result["task_lease_fence"] == { - "schema_version": "task_lease_v0", - "required": True, - "active": True, - "owner": AUTHOR_AGENT, - "version": 1, - "lease_epoch": 1, - "execution_instance_verified": True, - "released": True, - } - assert ( - json.loads(_lease_path(tmp_path, todo_id).read_text(encoding="utf-8"))["status"] - == "released" - ) def test_unfenced_completion_leaves_no_lease_artifacts(tmp_path: Path) -> None: @@ -1489,220 +1415,10 @@ def test_exception_after_verified_fence_leaves_lease_intact( assert state.read_text(encoding="utf-8") == state_before -def test_event_projected_dry_run_completion_does_not_release_lease( - tmp_path: Path, -) -> None: - registry, state = _write_fixture(tmp_path, multi_agent=False) - event_log = state.with_name("events.jsonl") - store = AppendOnlyStateEventStore(event_log) - todo_id = "todo_event_dry_run_lease" - store.append( - make_state_event( - event_id="evt-event-dry-run-lease", - goal_id=GOAL_ID, - event_type=TODO_ADDED, - refs={"todo_id": todo_id}, - payload={ - "role": "agent", - "title": "Dry-run the event-projected leased task.", - "task_class": "advancement_task", - "claimed_by": AUTHOR_AGENT, - }, - recorded_at="2026-07-18T00:00:00+00:00", - ) - ) - lease_key = "event-dry-run-instance" - acquire_task_lease( - registry_path=registry, - runtime_root=tmp_path / "runtime", - goal_id=GOAL_ID, - todo_id=todo_id, - owner=AUTHOR_AGENT, - idempotency_key=lease_key, - ttl_seconds=600, - ) - log_before = event_log.read_text(encoding="utf-8") - - result = complete_goal_todo( - registry_path=registry, - goal_id=GOAL_ID, - todo_id=todo_id, - claimed_by=AUTHOR_AGENT, - task_lease_idempotency_key=lease_key, - task_lease_expected_version=1, - evidence="event-projected dry run must not touch the lease", - no_followup=True, - dry_run=True, - ) - - assert result["source"] == "event_log" - assert result["dry_run"] is True - assert result["task_lease_fence"]["execution_instance_verified"] is True - assert "released" not in result["task_lease_fence"] - lease_path = _lease_path(tmp_path, todo_id) - assert json.loads(lease_path.read_text(encoding="utf-8"))["status"] == "active" - assert event_log.read_text(encoding="utf-8") == log_before - - -def test_event_projected_terminal_replay_does_not_release_lease( - tmp_path: Path, -) -> None: - """Terminal event-projected replay never releases a leftover lease. - - A verified fence cannot coexist with the already-done writeback branch: - the owner constraint self-disarms any fence over a non-open todo. The - reachable shape is a lease acquired while the todo was open, the todo - then deferred, and a completion without the key: the fence reports - inactive, the writeback replays with changed=False, and the stale lease - file is left exactly as it was. - """ - - registry, state = _write_fixture(tmp_path, multi_agent=False) - event_log = state.with_name("events.jsonl") - store = AppendOnlyStateEventStore(event_log) - todo_id = "todo_event_deferred_lease" - store.append( - make_state_event( - event_id="evt-event-deferred-parent", - goal_id=GOAL_ID, - event_type=TODO_ADDED, - refs={"todo_id": todo_id}, - payload={ - "role": "agent", - "title": "Terminal event-projected leased task.", - "task_class": "advancement_task", - "claimed_by": AUTHOR_AGENT, - }, - recorded_at="2026-07-18T00:00:00+00:00", - ) - ) - lease_key = "event-deferred-instance" - acquire_task_lease( - registry_path=registry, - runtime_root=tmp_path / "runtime", - goal_id=GOAL_ID, - todo_id=todo_id, - owner=AUTHOR_AGENT, - idempotency_key=lease_key, - ttl_seconds=600, - ) - store.append( - make_state_event( - event_id="evt-event-deferred-terminal", - goal_id=GOAL_ID, - event_type=TODO_DEFERRED, - refs={"todo_id": todo_id}, - payload={"reason": "deferred after the lease was acquired"}, - recorded_at="2026-07-18T00:01:00+00:00", - ) - ) - lease_path = _lease_path(tmp_path, todo_id) - lease_before = lease_path.read_text(encoding="utf-8") - - result = complete_goal_todo( - registry_path=registry, - goal_id=GOAL_ID, - todo_id=todo_id, - claimed_by=AUTHOR_AGENT, - evidence="terminal event-projected todo replays without a write", - no_followup=True, - ) - - assert result["changed"] is False - assert result["idempotent_replay"] is True - assert result["task_lease_fence"] == { - "schema_version": "task_lease_v0", - "required": False, - "active": False, - } - assert lease_path.read_text(encoding="utf-8") == lease_before - assert json.loads(lease_before)["status"] == "active" - - -def test_event_projected_completion_appends_same_turn_terminal_upgrade( - tmp_path: Path, -) -> None: - registry, state = _write_fixture(tmp_path, multi_agent=False) - event_log = state.with_name("events.jsonl") - store = AppendOnlyStateEventStore(event_log) - todo_id = "todo_event_terminal_upgrade" - store.append( - make_state_event( - event_id="evt-event-terminal-parent", - goal_id=GOAL_ID, - event_type=TODO_ADDED, - refs={"todo_id": todo_id}, - payload={ - "role": "agent", - "title": "Append a same-turn terminal upgrade.", - "task_class": "advancement_task", - "claimed_by": AUTHOR_AGENT, - }, - recorded_at="2026-07-18T00:00:00+00:00", - ) - ) - store.append( - make_state_event( - event_id="evt-event-ordinary-completion", - goal_id=GOAL_ID, - event_type=TODO_COMPLETED, - refs={"todo_id": todo_id}, - payload={ - "completed_at": "2026-07-18T00:01:00+00:00", - "updated_at": "2026-07-18T00:01:00+00:00", - "completion_turn_key": "turn-a", - "completion_continuation": "active_goal", - }, - recorded_at="2026-07-18T00:01:00+00:00", - ) - ) - with pytest.raises(ValueError, match="different completion_turn_key"): - complete_goal_todo( - registry_path=registry, - goal_id=GOAL_ID, - todo_id=todo_id, - claimed_by=AUTHOR_AGENT, - evidence="a different turn cannot append terminal closeout", - completion_turn_key="turn-b", - no_followup=True, - ) - assert sum(event["event_type"] == TODO_COMPLETED for event in store.load()) == 1 - upgraded = complete_goal_todo( - registry_path=registry, - goal_id=GOAL_ID, - todo_id=todo_id, - claimed_by=AUTHOR_AGENT, - completion_turn_key="turn-a", - evidence="same-turn terminal closeout validated", - no_followup=True, - ) - assert upgraded["source"] == "event_log" - assert upgraded["changed"] is True - upgraded_events = AppendOnlyStateEventStore(event_log).load() - projected = build_state_projection(upgraded_events) - projected_todo = projected["agent_todos"]["items"][0] - assert projected_todo["no_followup"] == "true" - assert projected_todo["completion_continuation"] == "no_followup" - assert projected_todo["completion_recovery"] == ("same_turn_terminal_closeout") - assert projected_todo["completed_at"] == "2026-07-18T00:01:00+00:00" - assert sum(event["event_type"] == TODO_COMPLETED for event in upgraded_events) == 2 - replayed = complete_goal_todo( - registry_path=registry, - goal_id=GOAL_ID, - todo_id=todo_id, - claimed_by=AUTHOR_AGENT, - completion_turn_key="turn-a", - evidence="same-turn terminal closeout replayed", - no_followup=True, - ) - assert replayed["idempotent_replay"] is True - assert replayed["changed"] is False - replayed_events = AppendOnlyStateEventStore(event_log).load() - assert sum(event["event_type"] == TODO_COMPLETED for event in replayed_events) == 2 def test_capability_binding_cannot_be_rebound_by_duplicate_add(tmp_path: Path) -> None: @@ -1730,242 +1446,14 @@ def test_capability_binding_cannot_be_rebound_by_duplicate_add(tmp_path: Path) - assert state.read_text(encoding="utf-8") == before -def test_capability_binding_follows_event_projected_successor(tmp_path: Path) -> None: - registry, state = _write_fixture(tmp_path) - event_log = tmp_path / "todo-events.jsonl" - store = AppendOnlyStateEventStore(event_log) - store.append( - make_state_event( - event_id="evt-binding-parent", - goal_id=GOAL_ID, - event_type=TODO_ADDED, - refs={"todo_id": "todo_binding_parent"}, - payload={ - "role": "agent", - "title": "Advance the admitted issue-fix route.", - "task_class": "advancement_task", - "action_kind": "issue_fix_branch_validation", - "capability_binding_ref": "issue-fix:feasibility-a1b2c3d4", - "claimed_by": AUTHOR_AGENT, - }, - recorded_at="2026-07-18T00:00:00+00:00", - ) - ) - projection = build_state_projection(store.load()) - parent = projection["agent_todos"]["items"][0] - - result = complete_event_projected_goal_todo( - goal_id=GOAL_ID, - context={ - "registry_path": registry, - "state_path": state, - "item": parent, - "role": "agent", - "event_log_path": event_log, - "fields": { - "agent_todos": projection["agent_todos"], - "state_event_projection": {"source_checksum": projection["source_checksum"]}, - }, - }, - evidence="bounded validation passed", - note=None, - no_followup=False, - successor_todo_ids=[], - claimed_by=AUTHOR_AGENT, - clear_claim=False, - next_agent_todo="Open the validated issue-fix review packet.", - next_user_todo=None, - next_user_task_class="user_gate", - next_claimed_by=AUTHOR_AGENT, - next_task_class="advancement_task", - next_action_kind="issue_fix_reviewer_request", - next_task_repository=None, - next_required_capabilities=None, - next_continuation_policy="same_agent_non_delivery", - self_merged=False, - next_excluded_agents=[], - registered_agents=[AUTHOR_AGENT], - updated_at="2026-07-18T00:01:00+00:00", - dry_run=False, - ) - - successor_id = result["next_todos"][0]["todo_id"] - replayed = build_state_projection(AppendOnlyStateEventStore(event_log).load()) - replayed_agent_todos = { - item["todo_id"]: item for item in replayed["agent_todos"]["items"] - } - assert successor_id in replayed_agent_todos, (result, replayed) - successor = next( - item - for item in replayed["agent_todos"]["items"] - if item["todo_id"] == successor_id - ) - assert successor["capability_binding_ref"] == ("issue-fix:feasibility-a1b2c3d4") - - event_count = len(AppendOnlyStateEventStore(event_log).load()) - completed_parent = next( - item - for item in replayed["agent_todos"]["items"] - if item["todo_id"] == parent["todo_id"] - ) - duplicate = complete_event_projected_goal_todo( - goal_id=GOAL_ID, - context={ - "registry_path": registry, - "state_path": state, - "item": completed_parent, - "role": "agent", - "event_log_path": event_log, - "fields": { - "agent_todos": replayed["agent_todos"], - "state_event_projection": {"source_checksum": replayed["source_checksum"]}, - }, - }, - evidence="late stale completion", - note=None, - no_followup=False, - successor_todo_ids=[], - claimed_by=AUTHOR_AGENT, - clear_claim=False, - next_agent_todo="Create a duplicate event successor.", - next_user_todo=None, - next_user_task_class="user_gate", - next_claimed_by=AUTHOR_AGENT, - next_task_class="advancement_task", - next_action_kind="issue_fix_reviewer_request", - next_task_repository=None, - next_required_capabilities=None, - next_continuation_policy="same_agent_non_delivery", - self_merged=False, - next_excluded_agents=[], - registered_agents=[AUTHOR_AGENT], - updated_at="2026-07-18T00:02:00+00:00", - dry_run=False, - ) - assert duplicate["idempotent_replay"] is True - assert duplicate["changed"] is False - assert len(AppendOnlyStateEventStore(event_log).load()) == event_count -def test_task_domain_survives_markdown_event_projection() -> None: - events = backfill_todo_events_from_markdown( - "\n".join( - [ - "## Agent Todo", - "", - "- [ ] [P0] Validate one adaptive lane.", - ( - " " - ), - ] - ), - goal_id=GOAL_ID, - ) - projection = build_state_projection(events) - assert projection["agent_todos"]["items"][0]["task_domain"] == "validation" -def test_updated_at_survives_todo_add_projection() -> None: - added = make_state_event( - event_id="evt-updated-at-add", - goal_id=GOAL_ID, - event_type=TODO_ADDED, - refs={"todo_id": "todo_updated_at001"}, - payload={ - "role": "agent", - "title": "Validate one adaptive lane.", - "updated_at": "2026-07-18T00:00:00+00:00", - }, - recorded_at="2026-07-18T00:00:00+00:00", - ) - projection = build_state_projection([added]) - assert ( - projection["agent_todos"]["items"][0]["updated_at"] - == "2026-07-18T00:00:00+00:00" - ) - - -def test_resume_when_survives_markdown_render_round_trip() -> None: - added = make_state_event( - event_id="evt-resume-when-add", - goal_id=GOAL_ID, - event_type=TODO_ADDED, - refs={"todo_id": "todo_resume_when1"}, - payload={ - "role": "agent", - "title": "Wait for the issuer fix.", - "task_class": "blocker", - }, - recorded_at="2026-07-18T00:00:00+00:00", - ) - deferred = make_state_event( - event_id="evt-resume-when-defer", - goal_id=GOAL_ID, - event_type=TODO_DEFERRED, - refs={"todo_id": "todo_resume_when1"}, - payload={ - "reason": "blocked on the issuer fix", - "resume_when": "todo_done:todo_issuerfix", - }, - recorded_at="2026-07-18T00:01:00+00:00", - ) - - projection = build_state_projection([added, deferred]) - - assert ( - projection["agent_todos"]["items"][0]["resume_when"] - == "todo_done:todo_issuerfix" - ) - rendered = render_active_state_sections(projection) - assert "resume_when=todo_done:todo_issuerfix" in rendered - reparsed = parse_active_state_todos(rendered) - assert ( - reparsed["agent_todos"]["items"][0]["resume_when"] - == "todo_done:todo_issuerfix" - ) - - -def test_task_domain_event_update_is_normalized_and_invalid_values_fail() -> None: - added = make_state_event( - event_id="evt-domain-add", - goal_id=GOAL_ID, - event_type=TODO_ADDED, - refs={"todo_id": "todo_domain_update"}, - payload={ - "role": "agent", - "title": "Validate one adaptive lane.", - "task_class": "advancement_task", - "task_domain": "validation", - }, - recorded_at="2026-07-18T00:00:00+00:00", - ) - updated = make_state_event( - event_id="evt-domain-update", - goal_id=GOAL_ID, - event_type=TODO_UPDATED, - refs={"todo_id": "todo_domain_update"}, - payload={"task_domain": "docs.review"}, - recorded_at="2026-07-18T00:01:00+00:00", - ) - - projection = build_state_projection([added, updated]) - - assert projection["agent_todos"]["items"][0]["task_domain"] == "docs.review" - with pytest.raises(StateEventError, match="task_domain"): - make_state_event( - event_id="evt-domain-invalid", - goal_id=GOAL_ID, - event_type=TODO_UPDATED, - refs={"todo_id": "todo_domain_update"}, - payload={"task_domain": "../private"}, - recorded_at="2026-07-18T00:02:00+00:00", - ) def test_duplicate_todo_add_rejects_invalid_task_domain_atomically( diff --git a/tests/control_plane_ts/state_event_append.test.ts b/tests/control_plane_ts/state_event_append.test.ts deleted file mode 100644 index 6cd909c1b4..0000000000 --- a/tests/control_plane_ts/state_event_append.test.ts +++ /dev/null @@ -1,59 +0,0 @@ -import assert from "node:assert/strict"; -import test from "node:test"; -import {planStateEventAppend} from "../../loopx/control_plane/goals/state_event_append.ts"; - -const old = {event_id: "old", fingerprint: "a".repeat(64), append_sequence: 7}; -const fresh = {event_id: "new", fingerprint: "b".repeat(64)}; -const request = {schema_version: "loopx_state_event_append_plan_v0", source_checksum: "basis", - expected_checksum: "basis", last_sequence: 7, existing: [old], events: [fresh, old, fresh]}; - -test("allocate only once for a duplicate, preserving historical replay order", () => { - assert.deepEqual(planStateEventAppend(request).choices, [ - {kind: "append", event_id: "new", append_sequence: 8}, - {kind: "replay", event_id: "old", append_sequence: 7}, - {kind: "replay", event_id: "new", append_sequence: 8}, - ]); - assert.equal(request.last_sequence, 7); -}); - -test("late conflicts never return a partial append plan", () => { - for (const event_id of ["old", "new"]) { - const result = planStateEventAppend({...request, - events: [fresh, {event_id, fingerprint: "c".repeat(64)}]}); - assert.equal(result.status, "rejected"); - assert.equal(result.reason_code, "event_id_conflict"); - assert.equal(result.choices, undefined); - } -}); - -test("source mismatch wins even for an empty durability confirmation", () => { - const result = planStateEventAppend({...request, source_checksum: "changed", events: []}); - assert.equal(result.reason_code, "event_source_changed"); - assert.equal(result.choices, undefined); -}); - -test("sequence exhaustion cannot round two new events onto one identity", () => { - assert.equal(planStateEventAppend({...request, last_sequence: Number.MAX_SAFE_INTEGER}).reason_code, - "event_sequence_exhausted"); - assert.throws(() => planStateEventAppend({...request, last_sequence: Number.MAX_SAFE_INTEGER + 1})); - assert.equal(planStateEventAppend({...request, last_sequence: Number.MAX_SAFE_INTEGER, events: [old]}).status, - "planned"); -}); - -test("malformed or contradictory compact source witnesses reject", () => { - for (const changed of [{source_checksum: null}, {expected_checksum: undefined}, {last_sequence: true}, {existing: [old, old]}, - {existing: [{...old, append_sequence: 8}]}, {events: [{...fresh, fingerprint: "not-a-hash"}]}]) { - assert.throws(() => planStateEventAppend({...request, ...changed})); - } -}); - -test("valid event ids cannot collide with JavaScript object prototype names", () => { - const events = ["__proto__", "constructor", "toString"].map(event_id => ({...fresh, event_id})); - assert.deepEqual(planStateEventAppend({...request, existing: [], last_sequence: 0, - events: [...events, events[0]]}).choices, [ - {kind: "append", event_id: "__proto__", append_sequence: 1}, - {kind: "append", event_id: "constructor", append_sequence: 2}, - {kind: "append", event_id: "toString", append_sequence: 3}, - {kind: "replay", event_id: "__proto__", append_sequence: 1}, - ]); -}); diff --git a/tests/control_plane_ts/state_event_replay.test.ts b/tests/control_plane_ts/state_event_replay.test.ts deleted file mode 100644 index d9f1649af0..0000000000 --- a/tests/control_plane_ts/state_event_replay.test.ts +++ /dev/null @@ -1,116 +0,0 @@ -import assert from "node:assert/strict"; -import {test} from "node:test"; -import {planStateEventReplay} from "../../loopx/control_plane/goals/state_event_replay.ts"; -import type {JsonObject} from "../../loopx/control_plane/effect_program.ts"; - -function event(kind: string, n: number, fields: JsonObject = {}): JsonObject { - return {event_id: `event-${n}`, goal_id: "sample", event_type: kind, append_sequence: n, - recorded_at: "2026-09-24T00:00:00Z", todo_id: "todo_alpha", role: null, priority: null, - planner_order: null, content_changed: false, fields: ["title"], capability_binding_ref: null, - continuation_policy: null, removed_continuation_policy: null, has_exclusions: false, - goal_bound: null, ...fields}; -} -function plan(events: JsonObject[], goal: string | null = null): JsonObject { - return planStateEventReplay({schema_version: "state_event_replay_request_v0", goal_id: goal, events}); -} -function rows(result: JsonObject): JsonObject[] {return result.todos as JsonObject[];} - -test("replay orders facts without changing caller arrays; ordinals still address original content", () => { - const input = [event("todo_completed", 3), event("todo_added", 1), event("todo_claimed", 2)]; - const before = structuredClone(input); - const result = plan(input); - assert.deepEqual(input, before); - assert.deepEqual(result.event_indices, [1, 2, 0]); - assert.deepEqual(rows(result)[0].field_sources, {title: 0}); - assert.equal(rows(result)[0].status, "done"); - assert.equal(rows(result)[0].done, true); -}); - -test("different event IDs cannot overwrite an existing Todo even after completion", () => { - assert.throws(() => plan([event("todo_added", 1), event("todo_completed", 2), event("todo_added", 3)]), /already exists/); -}); - -test("mixed Goal, orphan, duplicate identity and unsupported event kind reject", () => { - assert.throws(() => plan([event("todo_added", 1)], "another"), /share one goal/); - assert.throws(() => plan([event("todo_completed", 1)]), /unknown todo_id/); - assert.throws(() => plan([event("todo_added", 1), event("todo_added", 1)]), /deduplicated/); - assert.throws(() => plan([event("todo_reopened", 1)]), /unsupported/); - assert.throws(() => plan([event("todo_added", 1, {todo_id: null})]), /requires refs.todo_id/); -}); - -test("summary order preserves zero, user lanes and the actual source section", () => { - const result = plan([event("todo_added", 1, {planner_order: 0}), - event("todo_added", 2, {todo_id: "todo_beta", planner_order: 1}), - event("todo_updated", 3, {todo_id: "todo_beta", role: "user", priority: "P0"})]); - const [user, agent] = rows(result); - assert.equal(user.todo_id, "todo_beta"); - assert.equal(user.source_section, "User Todo / Owner Review Reading Queue"); - assert.equal(user.render_priority, true); - assert.equal(agent.planner_order, 0); - assert.equal(agent.render_priority, false); -}); - -test("legacy lexical tie ordering follows Unicode scalar order, not UTF-16 order", () => { - const input = [event("refresh_recorded", 1, {append_sequence: null, event_id: "\u{10000}"}), - event("run_recorded", 2, {append_sequence: null, event_id: "\ue000"})]; - assert.deepEqual(plan(input).event_indices, [1, 0]); -}); - -test("malformed facts and unsafe integers cannot silently change replay ordering", () => { - for (const field of ["append_sequence", "planner_order"]) { - for (const value of [true, 1.5, Number.MAX_SAFE_INTEGER + 1]) { - assert.throws(() => plan([event("todo_added", 1, {[field]: value})]), /safe integer/); - } - } - assert.throws(() => plan([event("todo_added", 1, {role: "superuser"})]), /unsupported/); - assert.throws(() => plan([event("todo_updated", 1, {priority: "P99"})]), /priority/); -}); - -test("binding cannot change; owner addressing is exclusive and actor attribution can clear", () => { - const added = event("todo_added", 1, {capability_binding_ref: "domain:alpha", - fields: ["capability_binding_ref", "goal_bound", "last_actor_agent_id"]}); - assert.throws(() => plan([added, event("todo_updated", 2, {capability_binding_ref: "domain:beta"})]), /immutable/); - const result = rows(plan([added, event("todo_updated", 2, {fields: ["bound_agent"]})]))[0]; - assert.deepEqual(result.field_sources, {capability_binding_ref: 0, bound_agent: 1}); - const rebound = rows(plan([added, event("todo_updated", 2, {fields: ["bound_agent"]}), - event("todo_updated", 3, {fields: ["goal_bound"], goal_bound: true})]))[0]; - assert.deepEqual(rebound.field_sources, {capability_binding_ref: 0, goal_bound: 2}); - for (const flag of [false, true]) { - const both = rows(plan([added, event("todo_updated", 2, {fields: ["bound_agent", "goal_bound"], goal_bound: flag})]))[0]; - assert.deepEqual(both.field_sources, flag - ? {capability_binding_ref: 0, goal_bound: 1} - : {capability_binding_ref: 0, bound_agent: 1, goal_bound: 1}); - } -}); - -test("removed continuation remains blocked until an explicit independent handoff repair", () => { - const added = event("todo_added", 1, {removed_continuation_policy: "author_handoff", - fields: ["removed_continuation_policy"]}); - const denied = event("todo_updated", 2, {continuation_policy: "independent_handoff", fields: ["continuation_policy"]}); - assert.deepEqual(rows(plan([added, denied]))[0].field_sources, {removed_continuation_policy: 0}); - const repaired = {...denied, has_exclusions: true, fields: ["continuation_policy", "excluded_agents"]}; - assert.deepEqual(rows(plan([added, repaired]))[0].field_sources, {continuation_policy: 1, excluded_agents: 1}); -}); - -test("all historical event kinds have an explicit projection disposition", () => { - const input = [event("todo_added", 1), event("todo_claimed", 2), event("todo_updated", 3), - event("todo_blocked", 4), event("todo_deferred", 5), event("todo_completed", 6), - event("refresh_recorded", 7), event("run_recorded", 8), event("quota_spent", 9), - event("evidence_attached", 10), event("supervisor_proposed", 11), event("supervisor_receipt_recorded", 12)]; - const result = plan(input); - assert.deepEqual(result.timeline_indices, [6, 7, 8, 9]); - assert.equal(rows(result)[0].status, "done"); - assert.equal((result.event_indices as number[]).length, 12); -}); - -test("continuation is equivalent to one fold and cannot point into the new batch", () => { - const first = [event("todo_added", 1, {fields: ["title", "claimed_by"]})]; - const next = [event("todo_updated", 2, {priority: "P0", fields: ["title"]}), event("todo_completed", 3, {fields: ["evidence"]})]; - const seed = rows(plan(first)); - const resumed = planStateEventReplay({schema_version: "state_event_replay_request_v0", goal_id: "sample", - offset: 1, initial_todos: seed, events: next}); - assert.deepEqual(rows(resumed), rows(plan([...first, ...next]))); - assert.throws(() => planStateEventReplay({schema_version: "state_event_replay_request_v0", goal_id: "sample", - offset: 0, initial_todos: seed, events: next}), /does not precede/); - assert.deepEqual(seed, rows(plan(first)), "continuation input must remain unchanged"); -}); diff --git a/tests/control_plane_ts/supervisor_event_append.test.ts b/tests/control_plane_ts/supervisor_event_append.test.ts new file mode 100644 index 0000000000..9a5bce7c53 --- /dev/null +++ b/tests/control_plane_ts/supervisor_event_append.test.ts @@ -0,0 +1,27 @@ +import assert from "node:assert/strict"; +import test from "node:test"; +import {planSupervisorEventAppend as plan} from "../../loopx/control_plane/agents/supervisor_event_append.ts"; +const event = {event_id: "receipt-1", fingerprint: "a".repeat(64)}; +const request = {schema_version: "loopx_supervisor_event_append_plan_v0", last_sequence: 7, + existing: null, event}; +test("one new receipt advances the sequence; replay preserves the original slot", () => { + assert.deepEqual(plan(request), {status: "planned", kind: "append", append_sequence: 8}); + assert.deepEqual(plan({...request, existing: {...event, append_sequence: 3}}), + {status: "planned", kind: "replay", append_sequence: 3}); +}); +test("changed content for an existing id is rejected", () => { + assert.equal(plan({...request, existing: {...event, fingerprint: "b".repeat(64), + append_sequence: 3}}).reason_code, "event_id_conflict"); +}); +test("sequence exhaustion rejects new writes but permits replay", () => { + const full = {...request, last_sequence: Number.MAX_SAFE_INTEGER}; + assert.equal(plan(full).reason_code, "event_sequence_exhausted"); + assert.equal(plan({...full, existing: {...event, append_sequence: 3}}).kind, "replay"); + for (const last_sequence of [true, -1, 1.5, Number.MAX_SAFE_INTEGER + 1]) { + assert.throws(() => plan({...request, last_sequence})); + } +}); +test("stored identities must refer to the requested event and a real sequence", () => { + for (const existing of [{...event, append_sequence: 8}, {...event, append_sequence: 0}, + {...event, event_id: "other", append_sequence: 3}]) assert.throws(() => plan({...request, existing})); +}); diff --git a/tests/control_plane_ts/todo_completion_fence.test.ts b/tests/control_plane_ts/todo_completion_fence.test.ts index d051129d78..fc29bc53ca 100644 --- a/tests/control_plane_ts/todo_completion_fence.test.ts +++ b/tests/control_plane_ts/todo_completion_fence.test.ts @@ -27,13 +27,13 @@ const fixture = JSON.parse( }>; }; -test("pinned Python completion-fence characterization remains exact", () => { +test("surviving materialized completion-fence characterization remains exact", () => { assert.equal( fixture.schema_version, "loopx_todo_completion_fence_characterization_v0", ); assert.equal(fixture.source_baseline, "58adc1783"); - assert.equal(fixture.cases.length >= 8, true); + assert.equal(fixture.cases.length >= 7, true); for (const item of fixture.cases) { if (item.expected_error) { assert.throws( diff --git a/tests/control_plane_ts/todo_completion_validation_plan.test.ts b/tests/control_plane_ts/todo_completion_validation_plan.test.ts index b24f510de1..6bfdccc1a4 100644 --- a/tests/control_plane_ts/todo_completion_validation_plan.test.ts +++ b/tests/control_plane_ts/todo_completion_validation_plan.test.ts @@ -17,7 +17,7 @@ const baseTodo = { function request(todo: Record) { return { schema_version: TODO_COMPLETION_VALIDATION_PLAN_REQUEST_SCHEMA, - projection_source: "event_log", + projection_source: "materialized", todo: { ...baseTodo, ...todo }, requested_no_followup: false, requested_completion_turn_key: null, @@ -25,16 +25,10 @@ function request(todo: Record) { }; } -test("event-log deferred todos replay before validation execution", () => { - const result = evaluateTodoCompletionValidationPlan( - request({ - status: "deferred", - validation_command: "false", - }), - ); - - assert.equal(result.effect, "skip"); - assert.equal(result.reason, "terminal_replay"); +test("retired event source cannot admit a validation effect", () => { + assert.throws(() => evaluateTodoCompletionValidationPlan({ + ...request({status: "deferred", validation_command: "false"}), projection_source: "event_log", + }), /projection_source is unsupported/); }); test("valid command and argv declarations produce run effects", () => { diff --git a/tests/fixtures/control_plane/todo_completion_fence_characterization_v0.json b/tests/fixtures/control_plane/todo_completion_fence_characterization_v0.json index b2eb469568..bc0065d544 100644 --- a/tests/fixtures/control_plane/todo_completion_fence_characterization_v0.json +++ b/tests/fixtures/control_plane/todo_completion_fence_characterization_v0.json @@ -50,30 +50,6 @@ "completion_continuation": null } }, - { - "name": "event-projected deferred Todo replays", - "request": { - "schema_version": "loopx_todo_completion_fence_request_v0", - "projection_source": "event_log", - "todo": { - "status": "deferred", - "no_followup": null, - "completion_continuation": null, - "completion_turn_key": null, - "successor_todo_ids": [] - }, - "requested_no_followup": true, - "requested_completion_turn_key": null - }, - "expected_result": { - "schema_version": "loopx_todo_completion_fence_result_v0", - "outcome": "replay", - "reason": "already_terminal", - "status": "deferred", - "terminal_before_request": true, - "completion_continuation": null - } - }, { "name": "typed done Todo replays", "request": { @@ -122,30 +98,6 @@ "completion_continuation": null } }, - { - "name": "same-turn terminal upgrade continues", - "request": { - "schema_version": "loopx_todo_completion_fence_request_v0", - "projection_source": "event_log", - "todo": { - "status": "done", - "no_followup": "false", - "completion_continuation": "active_goal", - "completion_turn_key": "turn-a", - "successor_todo_ids": [] - }, - "requested_no_followup": true, - "requested_completion_turn_key": "turn-a" - }, - "expected_result": { - "schema_version": "loopx_todo_completion_fence_result_v0", - "outcome": "continue", - "reason": "same_turn_terminal_upgrade", - "status": "done", - "terminal_before_request": true, - "completion_continuation": "active_goal" - } - }, { "name": "cross-turn replay is rejected", "request": { @@ -173,7 +125,9 @@ "no_followup": false, "completion_continuation": "successor", "completion_turn_key": "turn-a", - "successor_todo_ids": ["todo_successor"] + "successor_todo_ids": [ + "todo_successor" + ] }, "requested_no_followup": true, "requested_completion_turn_key": "turn-a" diff --git a/tests/test_event_sourced_state_store.py b/tests/test_event_sourced_state_store.py deleted file mode 100644 index e09bd8c903..0000000000 --- a/tests/test_event_sourced_state_store.py +++ /dev/null @@ -1,431 +0,0 @@ -import json -from contextlib import contextmanager -from pathlib import Path - -import pytest - -import loopx.event_sourced_state as event_sourced_state -from loopx.event_sourced_state import ( - TODO_ADDED, - AppendOnlyStateEventStore, - StateEventError, - make_state_event, -) - - -def test_load_observes_events_appended_by_another_store(tmp_path: Path) -> None: - event_log = tmp_path / "events.jsonl" - reader = AppendOnlyStateEventStore(event_log) - writer = AppendOnlyStateEventStore(event_log) - assert reader.load() == [] - - appended = writer.append( - make_state_event( - event_id="evt-concurrent-writer", - goal_id="goal-a", - event_type=TODO_ADDED, - refs={"todo_id": "todo_concurrent_writer"}, - payload={"role": "agent", "title": "Observe the durable event."}, - recorded_at="2026-09-06T00:00:00Z", - ) - ) - - assert appended["append_sequence"] == 1 - assert reader.load() == [appended] - - -def test_append_many_loads_once_and_preserves_idempotent_sequence( - tmp_path: Path, - monkeypatch: pytest.MonkeyPatch, -) -> None: - store = AppendOnlyStateEventStore(tmp_path / "events.jsonl") - events = [ - make_state_event( - event_id=f"event-{index}", - goal_id="goal-a", - event_type=TODO_ADDED, - refs={"todo_id": f"todo_event_{index}"}, - payload={"role": "agent", "title": f"Event {index}"}, - recorded_at="2026-09-08T00:00:00Z", - ) - for index in range(3) - ] - first = store.append(events[0]) - load = store.load - load_count = 0 - - def counted_load() -> list[dict[str, object]]: - nonlocal load_count - load_count += 1 - return load() - - monkeypatch.setattr(store, "load", counted_load) - appended = store.append_many([events[1], events[1], events[0], events[2]]) - - assert load_count == 1 - assert [event["append_sequence"] for event in appended] == [2, 2, 1, 3] - assert [event["event_id"] for event in load()] == [ - "event-0", - "event-1", - "event-2", - ] - assert appended[2] == first - - -def test_append_many_preserves_lazy_iterable_visibility_and_reentrancy( - tmp_path: Path, - monkeypatch: pytest.MonkeyPatch, -) -> None: - store = AppendOnlyStateEventStore(tmp_path / "events.jsonl") - events = [ - make_state_event( - event_id=f"lazy-event-{index}", - goal_id="goal-a", - event_type=TODO_ADDED, - refs={"todo_id": f"todo_lazy_event_{index}"}, - payload={"role": "agent", "title": f"Lazy event {index}"}, - recorded_at="2026-09-09T00:00:00Z", - ) - for index in range(3) - ] - lock_held = False - - @contextmanager - def non_reentrant_lock(_path: Path): - nonlocal lock_held - assert not lock_held - lock_held = True - try: - yield - finally: - lock_held = False - - monkeypatch.setattr( - event_sourced_state, - "exclusive_file_lock", - non_reentrant_lock, - ) - observed_prefix: list[str] = [] - - def lazy_events(): - yield events[0] - observed_prefix.extend(event["event_id"] for event in store.load()) - store.append(events[1]) - yield events[2] - - appended = store.append_many(lazy_events()) - - assert observed_prefix == ["lazy-event-0"] - assert [event["event_id"] for event in appended] == [ - "lazy-event-0", - "lazy-event-2", - ] - assert [event["event_id"] for event in store.load()] == [ - "lazy-event-0", - "lazy-event-1", - "lazy-event-2", - ] - - -def test_append_many_does_not_iterate_list_subclasses_under_lock( - tmp_path: Path, - monkeypatch: pytest.MonkeyPatch, -) -> None: - store = AppendOnlyStateEventStore(tmp_path / "events.jsonl") - events = [ - make_state_event( - event_id=f"subclass-event-{index}", - goal_id="goal-a", - event_type=TODO_ADDED, - refs={"todo_id": f"todo_subclass_event_{index}"}, - payload={"role": "agent", "title": f"Subclass event {index}"}, - recorded_at="2026-09-10T00:00:00Z", - ) - for index in range(2) - ] - lock_held = False - - @contextmanager - def non_reentrant_lock(_path: Path): - nonlocal lock_held - assert not lock_held - lock_held = True - try: - yield - finally: - lock_held = False - - monkeypatch.setattr(event_sourced_state, "exclusive_file_lock", non_reentrant_lock) - - class ReentrantList(list): - def __iter__(self): - store.append(events[1]) - return super().__iter__() - - appended = store.append_many(ReentrantList([events[0]])) - - assert [event["event_id"] for event in appended] == ["subclass-event-0"] - assert [event["event_id"] for event in store.load()] == [ - "subclass-event-1", - "subclass-event-0", - ] - - -def test_eager_batch_has_no_visible_prefix_while_validating_later_events( - tmp_path: Path, -) -> None: - event_log = tmp_path / "events.jsonl" - store = AppendOnlyStateEventStore(event_log) - first = make_state_event( - event_id="flush-event-0", - goal_id="goal-a", - event_type=TODO_ADDED, - refs={"todo_id": "todo_flush_event_0"}, - payload={"role": "agent", "title": "First event"}, - recorded_at="2026-09-10T00:00:00Z", - ) - observed_prefix: list[str] = [] - - class ObservingEvent(dict): - def get(self, key, default=None): - if not observed_prefix: - observed_prefix.append(event_log.read_text(encoding="utf-8") if event_log.exists() else "") - return super().get(key, default) - - second = ObservingEvent( - make_state_event( - event_id="flush-event-1", - goal_id="goal-a", - event_type=TODO_ADDED, - refs={"todo_id": "todo_flush_event_1"}, - payload={"role": "agent", "title": "Second event"}, - recorded_at="2026-09-10T00:00:00Z", - ) - ) - - store.append_many([first, second]) - - assert observed_prefix == [""] - assert len(store.load()) == 2 - - -def _event(identity: str, title: str = "A complete event") -> dict: - return make_state_event( - event_id=identity, - goal_id="goal-a", - event_type=TODO_ADDED, - refs={"todo_id": f"todo_{identity}"}, - payload={"role": "agent", "title": title}, - recorded_at="2026-09-24T00:00:00Z", - ) - - -@pytest.mark.parametrize("failure", ["invalid", "stored_conflict", "batch_conflict"]) -def test_late_batch_failure_preserves_every_prior_byte( - tmp_path: Path, failure: str -) -> None: - store = AppendOnlyStateEventStore(tmp_path / "events.jsonl") - store.append(_event("existing")) - before = store.path.read_bytes() - tail = ( - {"event_id": "invalid"} - if failure == "invalid" - else _event( - "existing" if failure == "stored_conflict" else "new", "Conflicting payload" - ) - ) - with pytest.raises(StateEventError): - store.append_many([_event("new"), tail]) - assert store.path.read_bytes() == before - - -def test_stale_basis_cannot_publish_successor_prefix(tmp_path: Path) -> None: - store = AppendOnlyStateEventStore(tmp_path / "events.jsonl") - store.append(_event("original")) - checksum = event_sourced_state.event_stream_checksum(store.load()) - store.append(_event("concurrent")) - before = store.path.read_bytes() - with pytest.raises(event_sourced_state.StateEventSourceChangedError): - store.append_many([_event("successor")], expected_checksum=checksum) - assert store.path.read_bytes() == before - - -def test_lost_publication_ack_replays_without_new_events_and_syncs_again( - tmp_path: Path, - monkeypatch: pytest.MonkeyPatch, -) -> None: - from loopx.control_plane.todos import active_state_editing as io - - store = AppendOnlyStateEventStore(tmp_path / "events.jsonl") - sync = io.fsync_state_directory - attempts = [] - - def fail_once(path): - attempts.append(path) - if len(attempts) == 1: - raise OSError("injected directory sync failure after replacement") - return sync(path) - - monkeypatch.setattr(io, "fsync_state_directory", fail_once) - batch = [_event("first"), _event("second")] - with pytest.raises(event_sourced_state.StateEventCommitUnknownError): - store.append_many(batch) - landed = store.path.read_bytes() - assert [item["event_id"] for item in store.load()] == ["first", "second"] - assert [item["append_sequence"] for item in store.append_many(batch)] == [1, 2] - assert store.path.read_bytes() == landed - assert len(attempts) == 2 - - -def test_replay_durability_failure_remains_an_uncertain_commit( - tmp_path: Path, monkeypatch: pytest.MonkeyPatch, -) -> None: - from loopx.control_plane.todos import active_state_editing as io - - store = AppendOnlyStateEventStore(tmp_path / "events.jsonl") - event = _event("first") - store.append(event) - before = store.path.read_bytes() - - def fail_sync(path: Path) -> None: - raise OSError("injected replay sync failure") - - monkeypatch.setattr(io, "fsync_state_directory", fail_sync) - with pytest.raises(event_sourced_state.StateEventCommitUnknownError): - store.append(event) - assert store.path.read_bytes() == before - - -@pytest.mark.parametrize("sequence", [True, False, 1.5, "2"]) -def test_load_rejects_non_integer_append_sequence( - tmp_path: Path, sequence: object -) -> None: - event_log = tmp_path / "events.jsonl" - event = make_state_event( - event_id="evt-bool-sequence", - goal_id="goal-a", - event_type=TODO_ADDED, - refs={"todo_id": "todo_bool_sequence"}, - payload={"role": "agent", "title": "Reject corrupt sequence."}, - recorded_at="2026-09-07T00:00:00Z", - ) - event["append_sequence"] = sequence - event_log.write_text(json.dumps(event) + "\n", encoding="utf-8") - - with pytest.raises(StateEventError, match="append_sequence must be an integer"): - AppendOnlyStateEventStore(event_log).load() - - -@pytest.mark.parametrize("ending", [b"\r\n\r\n", b""]) -def test_atomic_append_preserves_historical_bytes( - tmp_path: Path, ending: bytes -) -> None: - store = AppendOnlyStateEventStore(tmp_path / "events.jsonl") - first = store.append(_event("first")) - historical = json.dumps(first, ensure_ascii=False).encode() + ending - store.path.write_bytes(historical) - store.append(_event("second")) - assert store.path.read_bytes().startswith(historical) - assert len(store.load()) == 2 - store.append(_event("first")) - assert len(store.load()) == 2 - - -def test_failure_before_replace_leaves_old_stream_intact( - tmp_path: Path, monkeypatch -) -> None: - from loopx.control_plane.todos import active_state_editing as io - - store = AppendOnlyStateEventStore(tmp_path / "events.jsonl") - store.append(_event("first")) - before = store.path.read_bytes() - - def fail_replace(*args): - raise OSError("injected pre-publication failure") - - monkeypatch.setattr(io.os, "replace", fail_replace) - with pytest.raises(event_sourced_state.StateEventCommitUnknownError): - store.append_many([_event("second"), _event("third")]) - assert store.path.read_bytes() == before - assert not list(tmp_path.glob(".*.tmp")) - - -@pytest.mark.parametrize( - "choice", - [ - {"kind": "unknown", "event_id": "second", "append_sequence": 2}, - {"kind": "append", "event_id": "wrong", "append_sequence": 2}, - {"kind": "append", "event_id": "second", "append_sequence": True}, - ], -) -def test_invalid_native_append_reply_does_not_publish( - tmp_path: Path, - monkeypatch: pytest.MonkeyPatch, - choice: dict, -) -> None: - from loopx.control_plane import effect_runtime - - store = AppendOnlyStateEventStore(tmp_path / "events.jsonl") - store.append(_event("first")) - before = store.path.read_bytes() - monkeypatch.setattr( - effect_runtime, - "effect_runtime_result", - lambda *args: { - "schema_version": "loopx_state_event_append_result_v0", - "status": "planned", - "choices": [choice], - }, - ) - with pytest.raises(StateEventError, match="invalid event append plan choice"): - store.append(_event("second")) - assert store.path.read_bytes() == before - - -def test_concurrent_processes_publish_contiguous_batches(tmp_path: Path) -> None: - import subprocess - import sys - - event_log = tmp_path / "events.jsonl" - start = tmp_path / "start" - script = """ -import sys, time -from pathlib import Path -from loopx.event_sourced_state import AppendOnlyStateEventStore, make_state_event, TODO_ADDED -path, barrier, worker = sys.argv[1:] -while not Path(barrier).exists(): - time.sleep(0.01) -AppendOnlyStateEventStore(Path(path)).append_many([ - make_state_event(event_id=f"{worker}-{i}", goal_id="goal-a", event_type=TODO_ADDED, - refs={"todo_id": f"todo_{worker}_{i}"}, payload={"role": "agent", "title": "Concurrent batch"}, - recorded_at="2026-09-24T00:00:00Z") for i in range(3) -]) -""" - workers = [ - subprocess.Popen( - [sys.executable, "-c", script, str(event_log), str(start), str(i)], - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - text=True, - ) - for i in range(3) - ] - try: - start.touch() - for process in workers: - _, error = process.communicate(timeout=40) - assert process.returncode == 0, error - finally: - for process in workers: - if process.poll() is None: - process.kill() - process.wait() - events = AppendOnlyStateEventStore(event_log).load() - assert [row["append_sequence"] for row in events] == list(range(1, 10)) - for worker in range(3): - positions = [ - i - for i, row in enumerate(events) - if row["event_id"].startswith(f"{worker}-") - ] - assert positions == list(range(positions[0], positions[0] + 3))