Repository navigation
test(workspace): preserve acceptance observation provenance - #6626
huangruiteng merged 2 commits into
Conversation
Signed-off-by: song <liusongstep@gmail.com>
Signed-off-by: song <liusongstep@gmail.com>
loopx-agent
left a comment
There was a problem hiding this comment.
Reviewer: model_agent · gpt-6.1-sol · OpenAI · runtime_reported · reasoning_effort=max · observation_id=640951a733d2d95ab1643c2a4bf225780a91ce69fa867e6ff3974948cd88ea71
动机
运行打包和共享浏览器验收的维护者,以及查看 Goal 当前交付状态的用户。保留快照只表示上次观察,不能证明当前输出仍相同。旧场景把保留快照当成当前通过,在模拟读取失败后直接检查手机摘要,还在已启用合同的手机页面等待不存在的备用按钮;现在分别验证快照来源、恢复原有读取,并用键盘操作实际的条件折叠区。
三条原始场景分别在旧断言或旧定位器失败;93b1e59e0b70625aa480fa674eda7e388e1b3a94 上三个完整打包场景及隔离 SQLite、HTTP、CLI 验收旅程均通过,原阈值和超时保留。本次不认证完整 CI、真实模型、已安装 App 或 Goal 完成,也不授予合并权限。
改动思路
修复三个已有场景与已发布验收合同的偏差:快照按来源判断;读取失败通过原刷新恢复;启用合同的手机页面操作原生 details。删除断言、让旧快照冒充当前,或修改产品去迎合备用定位器,都会破坏既有边界。
三者分别属于静态来源、读取恢复和原生折叠交互,留在原场景即可。完整新 head 已重新判断,没有沿用前一提交的批准。没有新增通用 helper、产品状态、文案或用户决策步骤。
具体改动
核验 head 93b1e59,共同祖先 d614ed0。完整差异为三个已有文件,+25/-7,两个提交均有 DCO。
关键代码讲解
- goalMastheadScenario.run:保留快照必须显示 3 项未核验、0 项失败;不得声称当前通过,也不得出现“检查交付”动作,条款数量保留。
- goalShowcaseScenario.run:保留读取失败及其单行限制,新增旧快照断言;解除模拟故障后点击既有刷新,等“2/4 项条件当前通过”并移除错误,再执行未变的手机布局检查。
- goalWorkspaceM2Scenario.run:启用合同的路径没有备用“展开目标与条件”按钮。改为聚焦原生 summary 并按 Enter,断言条款隐藏→可见→隐藏,同时读回 details.open;目标保持可见。原 rail 焦点返回、历史/任务切换、远程来源拒绝、零效果及 200 项预算检查保留。
- 未变的 useDeliveryReview、GoalAcceptanceContractSection 和原生 details 仍拥有当前读回与展示规则。没有在夹具外复制权威。
规范为 docs/reference/goal-acceptance-observations.md,固定版本 d614ed0(原文)。无独立编号的条款按原文短语定位:
| criterion_id | 本 head 的落实 |
|---|---|
| Goal Overview counts only currently passing criteria | implemented:Overview只统计当前范围内通过的条款,保留回执不能当当前结果。 |
| Raw canonical/status snapshots | implemented:没有宿主文件观察的原始/status快照仅为recorded,不能认证当前字节。 |
| Readback never runs checks | implemented:读取不会执行检查,变化后的文件不能沿用成功回执。 |
| phones fold them without hiding the required action | implemented:手机折叠详情时,必需动作、影响与失败反馈仍可见。 |
对主干的风险
最强反例是“永远显示未核验也能通过”、隐去故障,或手机只点一下却未验证折叠。新 masthead 负例之外,Showcase 必须恢复 2/4 当前通过;真实验收 oracle 必须读回 1/1 当前且文件变化清除旧绿;M2 必须以键盘双向切换真实可见性及 open 状态。这些反例已有独立检查。
三个基线控制仍记为失败:masthead 的旧当前通过断言、Showcase 的故障后手机断言、M2 的旧备用按钮 30000ms 定位超时。前两项基线在早一 head 上运行,两次 head 间只增加 M2 夹具差异,产品字节未变;M2 基线对照在当前产品上运行。恢复精确 head 后三个完整场景重新通过,分别为 21.308s、4.291s、5.687s。200 项初次渲染约 58ms、视图切换约 61–119ms,原 2s/1s 限制保留;这不是通用性能或持续运行结论。
既有 oracle 在真实隔离 SQLite、HTTP、CLI 上通过(23.333s):一次采用、当前版本读回、输出变化清绿且不隐式执行、失败修复重试、拒绝导入 409 保留合同、停用,以及 390px、键盘和减少动态效果。模型/工具传输为脚本,没有使用活动 Goal。已检查整屏和手机产物;部分截图处于滚动位置,折叠与摘要正确性由原有 DOM/布局断言证明。
当前源打包、三文件语法、diff、语义 advisory 和完整语义检查通过;产品、文案、阈值和超时未变。按 wait_for_ci=false 契约未读取或等待本 PR 的 CI。完整 CI、已安装 App、真实模型和持续资格仍未证明。回滚只还原三处夹具,会带回已复现的错误断言。
我的整体评价
APPROVE,未发现阻断项,结论只绑定上述 head。此命名夹具修复为 goal_achieved;父级完整 CI 和主干采纳尚未完成。long_horizon 与 user_experience 均为 preserved:不增加运行循环、表单、用户决策或权限;原当前/保留状态、显式恢复和手机交互继续由原所有者控制。
已检查既有覆盖及作者近 24 小时完整有界批次。本次修复三个实际失败的已有场景,没有重复新 suite。未来整理检查认为不必抽取参数化 helper。既有未核验词汇和原生折叠被复用,没有新共享协议、默认行为或权限;批准不授予合并权限。
English verdict: APPROVE - 6626@93b1e59e0b70625aa480fa674eda7e388e1b3a94: repair three demonstrated fixture failures in retained provenance, owning-read recovery and native mobile disclosure. All three baseline failures reproduced; all three complete exact-head packaged scenarios, the real disposable SQLite/HTTP/CLI oracle and static/semantic checks pass with unchanged product code, limits and timeouts. Full CI/main adoption, installed App/live-model qualification and merge authority remain separate.
Goal acceptance checks now distinguish current local readback from retained observations. Shared browser scenarios still assumed that a stored snapshot was current, or checked the short phone summary before recovering from their simulated delivery-read failure.
This test-only repair keeps the retained snapshot visibly unverified with no check action. The showcase first proves that failure removes its current-pass claim, then explicitly refreshes and reads back the recovered contract before running the unchanged phone single-line and overflow checks. Product code, UI copy, thresholds and timeouts are unchanged.
Validation on main
d614ed099: the original packaged failures reproduced; the repaired scenarios pass. The existing packaged acceptance-oracle journey also passes with real disposable SQLite/HTTP/CLI, including changed-output invalidation, failure/recovery, rejection, disable, keyboard and 390px checks. Model transport is scripted; live model, installed App and full hosted CI are not qualified. Semantic advisory/full vocabulary and syntax checks pass.Placement and related refactor: retain the two existing scenario owners. Their fixture provenance and failure recovery differ, so no new generic helper or product authority is needed. This restores shared qualification; it does not close the wider workspace acceptance.
The natural first-head CI then reached
goal-workspace-m2and exposed its retained mobile locator for the removed fallback direction button. That same timeout reproduces locally. The scenario now uses the enabled contract's existing native disclosure: the objective remains visible; Enter expands and folds criteria; the owner rail, focus return, task selection/history, remote-source refusal, zero effect requests and the frozen 200-item/render/lens budgets remain checked. The full packaged M2 scenario passes. No UI, timeout or budget is changed.Current scope: three existing browser scenario files. The two initial scenarios and real disposable acceptance-oracle journey retain their completed evidence; the newly exposed M2 path was rerun after its focused repair. Hosted CI and independent review must qualify the new exact head; local passes do not erase the previous hosted failures.