Skip to content

test(runtime): pin the projection writer's refuse-before-write and readback guards - #5044

Merged
huangruiteng merged 2 commits into
loopx-project:mainfrom
NIU-123370:codex/test-projection-writer-guards
Sep 25, 2026
Merged

huangruiteng merged 2 commits into
loopx-project:mainfrom
NIU-123370:codex/test-projection-writer-guards

Conversation

@NIU-123370

@NIU-123370 NIU-123370 commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Goal And Delivered Outcome

  • Amendment on review: the first revision's readback case used an index_record
    without the projection marker, so the writer refused for a fixture defect
    rather than for the injected fault. The fixture now carries the same marker the
    writer scans for, and an unpatched control case asserts the normal success at
    this boundary before the blinded-read case asserts the refusal.
  • Goal/source and gap: write_compact_runtime_projection
    (loopx/control_plane/runtime/runtime_projection_writer.py:12) is the single
    shared writer behind both shipped compact runtime projections
    (loopx/control_plane/runtime/shared_runtime_material_projection.py:190 and
    loopx/control_plane/runtime/shared_runtime_refresh_projection.py:169, which
    pass their own dry_run straight through). Its happy path, its
    already_current replay and its readback success are all exercised by the
    projection smokes — but its three refusal paths were named nowhere in
    tests/, examples/ or regression/: the must include object marker
    rejection, the is missing identity fields rejection, and the
    runtime projection append did not pass index readback failure. A writer that
    is shared by two callers is exactly where a silently-weakened guard spreads.
  • Observable before → after, with the validation row that proves it: before, a
    caller could pass a record whose marker field is absent, a string, or missing
    one identity component, and a future edit that dropped the pre-write
    validation would be caught by no test at all; after, six tests pin that the
    writer raises before creating any path under the runtime root, that the
    rejection still happens on a dry run, that a dry run reports would_project
    without writing, and that a blind read side turns the append into an OSError
    instead of a verified projection. Proven by the unit and regression_parity
    rows below.
  • Issue/task and intended base: self-contained test gap; no issue is completed
    by this diff, so no Closes. Base main.

Scope And Continuation

  • Completed scope and remaining work: only the refusals the smokes never reach.
    Deliberately not re-asserted here: the happy-path bytes, the markdown renderer
    output and the already_current replay, which the owning smokes already pin
    (mutation E below shows the smoke still catches the replay, so moving those
    assertions here would duplicate an existing owner). Not covered and not
    claimed: concurrent appenders under exclusive_run_index_lock — this diff adds
    no cross-process contention case.
  • Slice boundary / successor: N/A; complete within this scope.

Validation

Public-safe summaries only.

  • Tested revision: 7ec5a3216
  • Run state: finished
  • Input classes: synthetic
Check kind Result Public-safe evidence / limitation
unit passed python -m pytest -q tests/control_plane/test_runtime_projection_writer_guards.py → 7 passed in 0.27s.
unit passed Same selection python -m pytest -q tests/control_plane -k "projection" --ignore=tests/control_plane/test_reviewed_promotion_cli.py: base 19358f114 in a separate detached worktree → 422 passed / 1 failed; this head → 429 passed / 1 failed, the same-named failure in both runs. The +7 is exactly this PR's tests; no previously-green test turned red.
regression_parity passed Six mutations against the writer. Drop the isinstance(marker, dict) guard → 3 failed; all(identity) → any(identity) → 1 failed; move the dry_run early return above validation → 1 failed; drop the if not readback_verified: raise OSError → 1 failed; force readback_verified = True or any(...) so the guard can never fire → 1 failed. Sixth mutation, not caught by this file: disabling the already_current replay leaves all 7 green — by design, because that behavior has an existing owner; python3 examples/control_plane/shared-runtime-material-projection-smoke.py fails (AssertionError, exit 1) under that mutation and passes on an unmodified tree. The refusal mutations leave the caller smoke green, which is why the writer-level file is needed at all.
real_entrypoint passed The added test_unpatched_append_projects_and_reads_back is the control the first revision lacked: with the same fixture and no monkeypatch the writer returns projected / readback_verified=True and appends exactly one index row, so the readback refusal below is about the blinded read side and not about the fixture.
real_entrypoint passed Both owning callers' journeys still pass unchanged: shared-runtime-material-projection-smoke.py and refresh-state-shared-runtime-projection-smoke.py → ok.
static passed python -m ruff check on the new file → all checks passed; ruff format applied to the new file only (no pre-existing file reformatted).
static not_applicable python -m mypy was not run: [tool.mypy] files=[…] enumerates 24 modules, none under tests/, and the module under test is not among them; this diff changes no product file, so no listed target can change verdict.
  • Coverage and gaps: the new file exercises the writer directly rather than
    through a caller, so it covers the guard ordering (validate → reserve →
    append → read back) and every early return, which is the part a future edit to
    a shared writer tends to break. Gaps named plainly: (1) one failure in the
    neighborhood selection,
    tests/control_plane/test_shared_goal_alignment.py::test_appending_one_event_rotates_the_projection_into_frontier_behind,
    is pre-existing on main — it fails identically with this PR's file removed
    and fails when run alone, so it is not caused by and not fixed by this
    diff; (2) the same selection cannot collect
    tests/control_plane/test_reviewed_promotion_cli.py on main
    (ImportError: cannot import name '_workspace' from 'tests.control_plane.test_local_authority_shadow_cli_e2e', that file's line
    12), which is why it is excluded above — unrelated to this PR and reported
    separately rather than silently widened into this scope; (3) the readback test
    pins that the writer refuses to claim verification, and asserts honestly that
    the appended row is still on disk afterwards — it is a claim guard, not a
    rollback. Interpreter note: checks ran with repository-relative commands
    against a locally installed Python 3.12 environment (an editable install from
    another worktree on this machine), not uv sync --extra test; loopx.__file__
    was confirmed to resolve to this worktree before the runs.

Frontend / Visual Evidence

  • UI impact: none
  • Before: N/A
  • After: N/A
  • States and viewports shown: N/A
  • Source data: synthetic
  • Attention review: N/A

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Refactoring (no functional changes)
  • Documentation update
  • Test update

LoopX Area

  • Control plane (goals, todos, quota, scheduler, registry, runtime)
  • Benchmark boundary (adapters, runners, verifiers, scoring, evidence)
  • Capability or extension (providers, adapters, skills)
  • Public docs or presentation surface (README, protocols, dashboard)
  • Build, packaging, installer, or CI
  • Host or runtime integration

Technical Direction

  • Direction / acceptance reference, when applicable: Core control-plane
    hardening — durable-write guard coverage. No roadmap S/G/R id claimed.

Shared-authority RFC fixture impact

  • Production-scale fixture schema: N/A — no fixture, envelope or generator
    changed.
  • Semantic dimensions changed, or reviewed no-impact rationale: N/A; the added
    tests assert refusals on a disposable runtime root.
  • Provider conformance arms run: N/A — the writer under test is the shared
    Python append path, not a provider arm; no provider behavior changes.
  • Read-only legacy/file/PostgreSQL three-arm rehearsal: N/A — no promotion,
    runtime-routing or compatibility-projection behavior is touched.

Boundary Checklist

  • Neither the diff nor this PR body/comments/attachments disclose private state, credentials, raw traces or verifier output, internal links, or local machine paths (including .loopx/, .codex/goals/, and live ACTIVE_GOAL_STATE.md).
  • I did not duplicate maintainer-owned benchmark work unless a maintainer split out a public issue for it.
  • I kept the change scoped to the linked issue/task.
  • I completed the visual evidence section for UI changes, or marked UI impact none.
  • Every commit includes a DCO Signed-off-by trailer (git commit -s).

…adback guards

Signed-off-by: NIU-123370 <191000457+NIU-123370@users.noreply.github.com>
@NIU-123370
NIU-123370 force-pushed the codex/test-projection-writer-guards branch from d356a01 to cd74cad Compare September 25, 2026 08:24

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact head: cd74cad. This review concerns the test evidence on this head, not remote CI or merge authorization.

动机

共享 runtime projection writer 的拒绝写入与追加后读回是实际产品边界;用持久测试保护它有价值。但本 PR 声称新增的“读回不匹配”用例可以证明该边界,必须先确认测试在正常输入下会成功。

改动思路

新增测试通过 _call 构造 writer 输入,再把 load_index 改成空结果,期望 writer 拒绝声称投影成功。真实路径应先用合法的 record/index_record 投影成功,再仅因读回失配而失败;否则异常可能来自夹具本身,无法验证所宣称的因果关系。

具体改动

tests/control_plane/test_runtime_projection_writer_guards.py 增加 6 个测试(+122 行):四个缺失/非法身份的写前拒绝、一个 dry-run、一个追加后读回失败。写前拒绝测试和真实调用 smoke 有意义。可是 _call 的 index_record 在第 43 行没有 shared_runtime_projection marker,writer 追加它后做索引读回时必然找不到与 record 相同的 marker。

关键代码讲解

  • _record 在第 26–35 行创建 marker;_call 在第 38–50 行却只把 goal_id/kind 放入 index_record,破坏 writer 的正常成功前提。
  • test_readback_mismatch_fails_closed_after_append 在第 106–122 行只断言 OSError 和索引已追加。去掉第 112 行的 monkeypatch,原测试仍因缺 marker 得到同一个 OSError,因此无法证明“load_index 看不见刚写入的行”导致拒绝。

对主干的风险

这是测试本身的阻断,不是远端 CI 状态:在 exact head 运行该文件为 6 passed;两条真实 caller smoke 也通过。但我独立调用无 monkeypatch 的 _call(tmp_path),得到 OSError: runtime projection append did not pass index readback;给 index_record 加上与 record 一致的 marker 才返回 projected/readback_verified=True。这意味着新增读回负例会在实际读回守卫失效时仍可能绿灯。请修正夹具,新增或复用未 monkeypatch 的正向断言,再运行该文件和真实 caller smoke;不要只保留期待异常的用例。

我的整体评价

保护真实共享 writer 的方向正确,写前拒绝测试也提供增量价值;但关键读回用例目前是假阳性,长期回归保护尚未成立。变更不触及运行时或用户入口,所以用户体验无直接变化;也无兼容性迁移。相关的小重构就是让一处夹具复用相同 marker,不需额外测试框架。修复并证实“正常成功、仅故障失败”后可复审。

English verdict: REQUEST_CHANGES - fix the invalid index fixture and prove the unpatched success path before relying on the readback-failure test.

…control case

Signed-off-by: NIU-123370 <191000457+NIU-123370@users.noreply.github.com>
@NIU-123370

Copy link
Copy Markdown
Contributor Author

Amendment pushed on head 7ec5a3216 (previous cd74cad9b). Your probe reproduced exactly, and you were right about the cause: with the submitted fixture, an unpatched _call(tmp_path) raised OSError: runtime projection append did not pass index readback, so that case could not tell the injected fault from a fixture defect.

What changed:

  1. _index_record() now carries the same marker the writer scans for on readback, with a comment saying why — that was the whole defect.
  2. Added test_unpatched_append_projects_and_reads_back as the control the review asked for: same fixture, no monkeypatch → projected, readback_verified=True, one index row, both artifacts exist.
  3. test_readback_mismatch_fails_closed_after_append keeps the blinded load_index and now fails only because of it; the case above is what makes that claim mean something.

Numbers on 7ec5a3216:

  • your probe, re-run against the corrected fixture: status=projected readback_verified=True.
  • focused: 7 passed in 0.27s.
  • selection python -m pytest -q tests/control_plane -k "projection" --ignore=tests/control_plane/test_reviewed_promotion_cli.py: base 19358f114 measured in a separate detached worktree → 422 passed / 1 failed; this head → 429 passed / 1 failed, the same-named failure in both runs (test_shared_goal_alignment.py::test_appending_one_event_rotates_the_projection_into_frontier_behind, pre-existing on base). The +7 is exactly this PR's tests.
  • both real caller smokes: ok.
  • mutations against the writer: drop the isinstance(marker, dict) guard → 3 failed; all(identity) → any(identity) → 1 failed; move the dry_run early return above validation → 1 failed; drop the if not readback_verified: raise → 1 failed; force readback_verified = True or any(...) so the guard can never fire → 1 failed; disable the already_current replay → 7 passed here, and examples/control_plane/shared-runtime-material-projection-smoke.py fails (AssertionError, exit 1) under that same mutation, so it stays owned by the smoke rather than duplicated here.
  • ruff check and ruff format --check clean on the new file; python -m mypy still not applicable ([tool.mypy] files=[…] covers 24 modules, none under tests/).

The --ignore in the selection is the collection error reported in #5045; it is unrelated to and not touched by this PR.

@NIU-123370

Copy link
Copy Markdown
Contributor Author

Attribution note on the red rows here, so they are not charged to this PR: on this head 7ec5a3216 the failing check-runs are stage2c-correctness-e2e and stage2c (installed 0), while stage2c (mutants 0), stage2c (e2e 1) and stage2c (e2e 2) are not among the failures. Current origin/main (85759ba44) shows the same split — stage2c-correctness-e2e: failure, stage2c (installed 0): failure, the other three stage2c rows success — measured in the same window with commits/<sha>/check-runs.

This diff adds one file under tests/control_plane/ and changes no product code, no workflow and no fixture, so it cannot reach the stage2c qualification path. The test-shard (1..4) and kernel-static-checks rows were still in_progress when I measured; I will report here if any of them comes back red rather than assume this split holds.

Local numbers on this head, for the record: focused 7 passed in 0.27s; selection tests/control_plane -k "projection" --ignore=tests/control_plane/test_reviewed_promotion_cli.py → base 19358f114 measured in a separate detached worktree 422 passed / 1 failed, this head 429 passed / 1 failed with the same-named pre-existing failure; both real caller smokes ok.

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact head: 7ec5a32. Re-review after my REQUEST_CHANGES review on cd74cad.

动机

共享 runtime projection writer 同时服务 material 与 refresh 两条真实调用路径。旧有端到端 smoke 已覆盖成功写入和 replay,却没有把“无效 marker 必须写前拒绝”和“追加后读不到不得声称成功”固定为直接回归测试。上一版测试的索引夹具本身缺 marker,导致读回负例即使移除故障注入也会失败;本次更新需要先排除这个假阳性,才谈得上长期保护。

改动思路

本 PR 没改产品 writer,而是在可丢弃的临时 runtime root 上调用真实 write_compact_runtime_projection。修订把与 record 相同的 marker 放进 index_record,然后用一个未 monkeypatch 的成功用例作控制,再让相同夹具只在 load_index 被置盲时得到 OSError。写前拒绝、dry run 和真实调用方分别由该文件及现有 material/refresh smoke 守护,测试责任没有扩成第二套生产实现。

具体改动

整份 PR 只新增一个测试文件(+149 行,零产品代码变更);相对我上次审过的 head,新增 36 行并改写 9 行。_index_record 补足共享 marker,_index_rows 统一读取真实追加记录,test_unpatched_append_projects_and_reads_back 明确检查正常写入的状态、读回布尔值、两个产物路径和一条索引行。旧负例继续验证盲读后仍抛错且追加行已落盘,准确描述这个 guard 是“拒绝成功声明”而不是事务回滚。

关键代码讲解

  • tests/control_plane/test_runtime_projection_writer_guards.py:40 的 _index_record 让索引行包含 writer 在读回时查找的 shared_runtime_projection marker,消除了上一版夹具独立于注入故障就会失败的前提错误。
  • :124 的 test_unpatched_append_projects_and_reads_back 是正向对照:不修改读侧时返回 projected/readback_verified=True,磁盘上同时有 JSON、Markdown 和一条 index 行。
  • :134 的 test_readback_mismatch_fails_closed_after_append 仅把 load_index 改为空结果,然后断言同一写入过程抛出读回 OSError,且真实 index 已追加一行;它和正向对照一起定位了失败原因。

对主干的风险

这是一项只改测试的回归保护,不会改变现有 CLI、runtime 状态或用户入口。我在当前 worktree 的 uv run --extra test 环境核对了 loopx.__file__ 指向该 head,运行新增文件得到 7 passed;material 与 refresh 两条真实调用方 smoke 均通过,ruff check 与 git diff --check 也通过。对照现有 smoke,成功写入与 already_current 已被真实调用方覆盖,而无效 marker、缺失身份和盲读拒绝此前没有直接断言;这个测试文件的 149 行是集中覆盖共享 guard,并非重复制作一个运行时实现。未在本轮查询或等待远端 CI,亦未把作者提及的其他测试失败作为本 PR 的阻断证据。跨进程并发写入仍不在此测试范围,也不是本 PR 声称要解决的问题。

我的整体评价

上次的唯一实质阻断已在同一现行 head 上修复:现在“正常成功、注入盲读才失败”的因果关系成立,且两条真实调用链没有回退。长期效果是未来弱化共享 writer 拒绝条件时更容易被发现;用户体验保持原样,因为没有产品行为变更。相关的未来重构可以考虑让测试索引 marker 直接由同一个 record 参数生成,但目前固定夹具与现有覆写用例一致,不值得为此扩展抽象。基于上述本地证据,我批准这个精确 head 的测试更新;这不是对未观察的远端 CI、并发行为或合并就绪状态的保证。

English verdict: APPROVE - the corrected index fixture and unpatched success control make the readback-failure test causally valid; all focused local tests and both shipped caller smokes passed.

@NIU-123370

Copy link
Copy Markdown
Contributor Author

谢谢复审,P1 接受——7ec5a3216 已经把它要的因果链立起来了(索引行带上同一个 marker + 一条未打桩的正向对照)。下面这条是我在补做变异集时发现的一个残留缺口,我没有往这个已批准的 head 上推任何东西,交给你处置。

残留问题:读回只证明“有 marker”,没证明“是这次的身份”。 runtime_projection_writer.py:81-86 的判定是 isinstance(item.get(marker_field), dict) and tuple(...) == identity。把 == identity 换成恒真(or True),或者干脆只留 isinstance(..., dict),当前这个文件 7 passed 全绿:正向对照照样成功(marker 本来就对),盲读负例照样抛错(rows 为空与身份无关)。也就是说“索引里写了别人的 marker 仍然被报告为 projected”这条路径目前没有回归保护——而这恰好是共享 writer 被两个调用方复用时最可能的误用形状。

我补的用例(在 7ec5a3216 之上 +15 行,只动这一个文件),让"存在"与"身份相等"分开可证伪:

def test_readback_requires_the_same_identity(tmp_path: Path) -> None:
    # Proof that "my append landed" has to be identity equality, not the presence
    # of some well-formed marker: a caller that indexes a different projection
    # must not be reported as projected.
    other = _record(source_projection_sha256_16="ffffffffffffffff")

    with pytest.raises(OSError, match="did not pass index readback"):
        _call(
            tmp_path,
            index_record={**_index_record(), MARKER_FIELD: other[MARKER_FIELD]},
        )

    assert len(_index_rows(tmp_path)) == 1

实测(exact 7ec5a3216 + 上面这段):

  • 本文件 8 passed;ruff check / ruff format --diff / git diff --check 干净。
  • 两条变异各只被这条新用例抓住:== identity → or True;把整个合取条件缩成只剩 isinstance(..., dict)。都是 1 failed, 7 passed。
  • 上一版已确认的变异仍然成立:去掉 raise、让 if not readback_verified 永不触发 → 只有 test_readback_mismatch_fails_closed_after_append 红;把夹具的 marker 摘掉 → 只有正向对照红;not all(identity) → not any(identity) → 只有空身份那条红。
  • 一条按设计没被抓住、如实写在这里:把 :53 的 already_current 身份匹配换成恒真,本文件全绿——replay 分支由 material/refresh 两条 smoke 负责,这个文件刻意不扩职责去覆盖它。

邻域与归因:pytest tests/control_plane -k projection(排除 test_reviewed_promotion_cli.py,见下)在 7ec5a3216 + 这段上 = 430 passed / 1 failed / 4736 deselected(64.56s)。那 1 条是 test_shared_goal_alignment.py::test_appending_one_event_rotates_the_projection_into_frontier_behind,三个 revision 各跑一次定性的:PR base 19358f114(不含本 PR 任何提交)failed、本 head failed、当前 main 85759ba44 passed ⇒ 属于分支落后主干带来的既有失败,不是这个 PR 造成的;要一个跑起来干净的 head 可以在合并前 rebase 一次。顺带一条与 #5045 同源的事实:test_reviewed_promotion_cli.py 在 09fe7399d 之前的基线上会 ImportError 并把整个选择集中断(0 collected),所以我这个基线上必须先排除它。

本地环境披露:跑在检出对应 revision 的干净 worktree 上,复用本机已装好的解释器(Python 3.12.14),并把 Node 22.23.2 放到 PATH 最前以避开 effect_runtime 的 ≥22.18 版本门;没有用 uv sync --extra test。未查询也未依赖远端 CI。

后续默认走最小打扰:不改你已批准的 head。如果你更希望它进这一条,我把这 15 行推成一个新 head 供你重审(父提交就是 7ec5a3216,纯增量、不动夹具与已有用例);否则我等这条合并后,单独开一条只含这个用例的精简 PR。

@huangruiteng
huangruiteng merged commit 3eac235 into loopx-project:main Sep 25, 2026
16 of 23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants