ci: exempt client-only changes and shard heavy qualification

[huangruiteng]

Summary

• Replace the experimental exact-test/vision shadow framework with small, whole-PR job exemptions. Pure Dashboard client source/assets retain common checks and require packaged build/freshness/browser qualification, while skipping unrelated backend Python, Windows and Stage2c jobs. Existing Markdown-only handling remains.
• Backend, prompts, budget rules, dependencies, mixed/unknown changes and main remain full. Exemptions require policy already present on the target branch; ci:full only adds checks. Classification and every required/exempt job result are checked by the stable merge gate.
• Distribute the complete Python collection over 4 runners × 2 workers, replacing 2 × 2. Require all four coverage artifacts before enforcing the unchanged union coverage floor.
• Distribute Stage2c E2E over 2 runners × 2 workers, replacing one runner × 4. A small deterministic module partitioner preserves complete modules, shared fixtures and collection order. Mutant and installed-package qualification remain intact.
• Retire the selected-test runner and shadow comparison machinery; no runtime, authority-provider, automation or paid-model changes.

Rationale / scope

The nearest owner is scripts/ci and the existing Python workflow, not a new capability or impact-analysis framework. We exempt entire jobs only for an explicit client-only ownership boundary instead of maintaining lists of individual backend tests. Label or PR prose cannot grant exemptions. Unknown cases run full. Existing public check names stay stable.

This intentionally supersedes the earlier shadow-only proposal on this PR. Its historical results remain evidence, not another permanent CI obligation. No self-merge authorization is assumed.

Validation

• Run state: finished
• Input classes: synthetic, public_fixture
• Local: 37 tests passed (7 policy/gate + 30 workflow/Sonar). Repeated after rebasing onto current main.
• Real entrypoints: Git revision/diff classification; actual workflow pytest-split/xdist/coverage shell on four synthetic shards, proving disjoint complete test union and rejection of any missing coverage artifact; actual two-runner module-shard commands with ordered shared-fixture tests.
• Current repository collection: all 223 Stage2c cases partition into 112 / 111, complete disjoint union with no module crossing shards. Initial local collection lacked PyYAML; rerun passed after installing that dependency into an isolated temporary directory.
• Negative controls: unknown/mixed paths, missing base, symlink/type changes, unusual filenames, policy changes, force-full, missing/failed/cancelled/skipped jobs and contradictory flags.
• Static: Ruff, YAML parsing/job references and diff whitespace passed. Local tests do not prove a speedup.
• Real backend: not applicable; no persistence/transaction changes.
• Public/private scan: no credentials, private state, local paths or raw production logs included.

Performance evidence and expectations

Previous same-PR run: slowest full Python shard 16m07s; Stage2c E2E job 8m24s (223 tests, test step 7m42s); full merge-gate path roughly 17m40s. The old shadow added a separate 4m55s test job and comparison step.

These are historical baselines, not a matched speedup measurement. Four Python runners increase setup cost and concurrency; Stage2c keeps the same total worker count but spreads CPU pressure. Whole-module partitioning uses test counts, not timings, and a large module can still dominate. The new hosted run will measure critical path and runner-minutes. The policy PR runs full qualification plus the new Dashboard job before enabling any exemption.

Current hosted run, exact head be0fe2575: classification, native Windows, Node compatibility and all Stage2c lanes passed. E2E has 112 + 111 passing cases, with job durations 5m30s / 5m38s (test steps about five minutes each), versus the earlier single job's 8m24s. Total Stage2c latency benefits less because queueing and the 6m52s mutation lane now dominate. The new required packaged Dashboard build/freshness/browser job passed in 2m29s.

All four Python shards passed: 7m16s / 9m28s / 7m38s / 7m44s. The longest shard is about 41% shorter than the prior 16m07s; aggregate Python runner time increases about 8% (32m06s versus 29m49s). JUnit readback contains 7,849 distinct cases, 36 conditionally skipped, no failures or duplicate identities. Independently combining those four hosted coverage artifacts reports 74%, above the unchanged 19.6% floor. This is an observational comparison across nearby revisions, not a controlled benchmark or a successful end-to-end gate latency claim.

Current validation hold: the unchanged common-check development Dashboard smoke failed its existing criterion 6 (reload/session reconnection within ten seconds) in both hosted attempts, although the separate packaged variant passed. The aggregate correctly rejected that failure. Rerunning only failed jobs preserved the already-green Python/Stage2c results. An additional local Node 22 development-browser run with coverage passed after installing the declared dependencies; the first local probe lacked the coverage converter and is not counted as a passing run. No UI assertion, timeout or gate was weakened. The hosted-only restoration failure remains unresolved and blocks merge; do not label the whole workflow green. A useful follow-up is capturing the existing browser acceptance report on Linux to distinguish target-selection/readiness from an actual session-restoration defect. Some runners queued for roughly two minutes and the final gate also queued; queueing is measured separately from execution time.

Boundaries

• No tests removed from full Python/Stage2c qualification, no coverage floor reduction.
• No fabricated full coverage when Python is exempt; Sonar consumes only actual full-suite output.
• Temporary local Git time-window bypass authorized for this PR only.
• Review-ready delivery after hosted qualification; not installed or merged by this update.

[huangruiteng]

Review/update for b951351691f99c6db37322a338bf08d4385959d5:

• The candidate runner has passed in GitHub; full-shard comparison remains pending. Existing full core obligations and full coverage remain authoritative; the shadow does not skip any existing runtime qualification.
• Reviewed typed/authority/default behavior: closed profile names, immutable Git identities, exact inventories and negative gate tests. No text/filename heuristic grants an exemption; no runtime permission or settlement change. This CI-policy change must run full plus shadow.
• The separate frontstage build fails on packaged asset freshness. This is also reproducible on the exact baseline main revision 49cee5b30fef9f5bacddb548dc37809d7aa71be3: baseline run, PR run. Both report the same generated JS/CSS asset names and changed chat index. The PR diff contains no dashboard source, packaged assets or frontstage workflow change. This is recorded as a known baseline failure, not reclassified as a pass or hidden by the selector.
• Local validation: 28 policy/audit tests via both unittest and pytest; 418 actual selected Python tests; five real CLI/performance smokes; YAML/job-reference parsing, Ruff and diff checks. No live Goal state or private artifacts included.

Decision: keep draft until hosted shadow comparison is verified. Do not self-merge this CI policy change. Selective-only activation is intentionally a subsequent evidence-reviewed step; this PR alone does not reduce the full-suite critical path.

[huangruiteng]

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

动机

这个 PR 要解决的是一个真实但需要非常谨慎推进的问题：在不削弱主干门禁的前提下，先收集 Vision checkpoint 精确测试选择是否可信的同轮证据。当前 head 3d013299f380571c1d186d9619037571aae45006 选择 shadow-only，而不是直接启用 selective-only，这个阶段划分是正确的。

改动思路

整体链路是：用 immutable base/head/merge-base/checkout 生成 exact-path impact plan，同时继续跑完整 Python shards；随后执行选中的 Vision 测试与真实 smoke，再用同轮 JUnit identity/outcome 对照完整 shards。任何 unknown/mixed path、非规范路径、删除或 type change、过期 plan、缺失/重复 case、结果差异、selected 非通过或 unselected failure 都 fail closed。selection_is_merge_authority=false 与 merge-gate 的完整 job 依赖共同保证 shadow 结果不会悄悄变成跳过完整测试的权限。

具体改动

• impact_plan.py 把 docs/full/vision 分类、精确路径清单、revision 绑定和可复现校验收拢到一个 owner；CI policy 自身变化只触发 shadow rehearsal，仍然选择 full。
• impact_shadow.py 将执行 receipt 绑定 plan digest 与 checkout，并基于 testcase identity/outcome 对比两个 full shards；输出只保留 public-safe 计数，不携带失败文本或本机路径。
• review_gate.py 与 python-tests.yml 把 shadow job 纳入一致性检查，但不改变 full core jobs 的权威地位。
• 文档与测试明确把“当前 shadow 义务”和“未来是否启用 selective-only”分开。

关键代码讲解

我重点核对了 candidate() 的 fail-closed 分支、plan() 的 exact revision/checkout 绑定、validate_shadow_plan() 的重算与 selection_is_merge_authority=false 校验，以及 compare() 对 missing/omitted/different/skipped/unselected failure 的处理。这里最关键的不是“当前 selected 集合能通过”，而是它即使不完整也不能改变今天的合并结论；当前实现满足这一点。

验证方面，本地运行了 28 个 scripts/ci unittest 与 25 个 workflow pytest，共 53 个测试通过；Ruff 与 git diff --check 通过。远端 exact head 的 full Python shards、impact-tests、impact-shadow、Node、stage2c、pytest 和 merge-gate 均成功。Frontstage Pages 的失败来自未改动的 packaged-asset 基线；SonarCloud 对 scripts/ci 的覆盖与 path/subprocess hotspot 仍应在合并前人工 triage，但我没有找到 shell 注入或不受信任路径直接执行的具体漏洞：subprocess 使用 argv 且 workflow 输入来自受控的 Git/固定参数。

对主干的风险

当前没有阻塞性语义问题。主要残余风险是 exact Vision inventory 尚不能证明覆盖未来所有相关 mutation；但 full suite 仍为唯一 merge authority，且 audit 会捕获 unselected full failure，所以该风险被限定在“证据是否足够支持未来激活”，不会影响本 PR 当前的主干安全。分支目前 behind main，合并就绪前仍需按常规 rebase/readback，并处理或明确处置 Sonar 红项。

我的整体评价

这是一个边界清楚、可回滚、没有提前索取选择性门禁权限的 CI shadow 阶段。仓库现有 full gate、JUnit artifacts 与 review gate 都被复用，没有引入第二套合并权威；代码量与问题风险相称。未来若要启用 selective-only，必须另行提交代表性历史/故障注入证据与 trusted-base 策略，不能把当前绿色同轮比较直接当成充分条件。

English verdict: APPROVE for exact head 3d01329. The shadow plan is fail-closed, full CI remains authoritative, and 53 local policy/workflow tests plus the exact-head hosted full/shadow comparison passed. Future selective-only activation still needs separate representative mutation evidence and a reviewed authority change.

[huangruiteng]

Refined this PR to the simpler job-exemption design: client-only Dashboard changes keep common checks and require real packaged build/browser qualification, but need not rerun backend Python/Windows/Stage2c. Mixed/unknown, prompt, budget, dependency and policy changes remain full. Full Python now uses four complete shards; Stage2c E2E uses two whole-module shards with the same four total workers. The previous selected-test/shadow framework is retired. Local qualification: 37 tests passed after rebase, including real shard/coverage commands and ordered shared-fixture module tests; Ruff, YAML dependency validation and whitespace checks passed. Hosted qualification of exact head be0fe25 is running at https://github.com/huangruiteng/loopx/actions/runs/34607363184. No measured speedup claim yet; runner queue/setup costs and complete coverage still need readback. This policy PR intentionally runs full qualification plus the new presentation job. No runtime/provider or live-state changes; no self-merge performed.

[huangruiteng]

Hosted readback for be0fe25: all four Python shards passed (7,849 distinct JUnit cases, 36 conditional skips), all 223 Stage2c E2E cases passed, mutants/installed/Windows/Node and the new packaged Dashboard job passed. Longest Python shard: 16m07s → 9m28s (~41% shorter), summed Python runner time ~8% higher. Longest Stage2c E2E job: 8m24s → 5m38s (~33% shorter); queueing and the unchanged mutation lane limit overall Stage2c gains. These compare nearby runs, not a controlled benchmark. Four downloaded coverage files combine to ~74%, above the unchanged floor.

Merge hold remains: the unchanged development Dashboard common-check smoke failed criterion 6 (reload/session restoration) in both hosted attempts. Local Node 22 development browser + coverage passes all 24 criteria, and hosted packaged mode passes. I did not weaken the timeout/assertion, waive that check, or call the workflow green. Its original failure correctly propagates through pytest/merge-gate. Capture the existing Linux acceptance-results report next to distinguish target selection/readiness from a real restoration defect. Full CI latency improvement is not claimed while this hold remains. The branch is pushed and the PR body now reflects this final evidence and limitation; no merge performed.

[huangruiteng]

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

重新审阅当前 exact head be0fe25754a41144084b923d223963406aa124cc。此前对 3d013299... 的 shadow-only 结论不适用于这次已重写的 whole-PR exemption 方案；本结论只绑定当前 head。

动机

目标是缩短真实 required CI 的关键路径，同时不让“选中少量测试通过”冒充完整资格。当前方案把问题收窄为两个可审计动作：只对边界清晰的纯 Dashboard client PR 豁免无关的 backend jobs；全量 Python/Stage2c 仍覆盖完整集合，只把它们分布到更多 runner。

改动思路

impact_plan.py 对整份 PR diff 做 fail-closed 分类，不维护单测级启发式清单。presentation exemption 只有在目标分支已经包含完全相同的 policy blobs 时才生效，因此本 PR 无法给自己授权。review_gate.py 再逐项核对分类输出与 required/skipped job 状态；任何 missing、cancelled、unexpected skip 或矛盾 flag 都失败。ci:full、main 和 manual dispatch 只会增加到 full qualification。

具体改动

• 仅 allowlisted Dashboard source/assets（可混合 Markdown）进入 presentation；prompt、依赖、测试、构建配置、backend、mixed/unknown、rename/type/symlink 与 policy 改动全部 full。
• presentation 仍运行 common checks，并新增真实 packaged Dashboard build、生成物 freshness 和 browser smoke；不伪造 Python coverage/Sonar 产物。
• 全量 Python 从 2×2 扩为 4×2，四份 coverage 缺一即失败，合并后仍执行原 19.6% floor。
• Stage2c E2E 改为两个 runner、每个两个 worker；按完整 module 做确定性分片，mutants 与 installed-package lanes 保留。
• 旧的 selected-test / shadow comparison 机制从最终 diff 移除，避免形成第二套 merge authority。

关键代码讲解

关键防线是 plan() 的顺序：先从 merge-base 到 exact head 读取 NUL-delimited、禁 rename 的 Git 变更，任何非 A/M/D、非规范路径或非 regular object 回退 full；即使候选是 presentation，trusted_policy() 也要求 base/head 三个 policy blob 完全一致。随后 job_flags() 只允许 docs/presentation/full 三个封闭 profile。merge-gate 并不相信 job 是否“看起来完成”，而是从 changes.outputs 重建期望 profile，再要求每个 job 精确为 success 或 intentional skipped。Stage2c 的 module plugin 只做 collection partition，不重排 module 内 case；xdist 的 loadfile 再保证一个 module 落在单 worker。

对主干的风险

我重新执行了当前方案而不是沿用旧 head 的 approval：33 个 policy/gate/workflow tests 通过，其中包含真实临时 Git repo 的 trusted-base/self-authorization 反例、异常文件名与 symlink/type/missing-base、四个真实 pytest-split/xdist/coverage checkout 的互斥完整并集、缺任一 coverage 的拒绝，以及两路 module shard 的共享 fixture/顺序验证；Ruff、exact-base classifier readback 和 diff check 通过。Hosted exact head 的四个 Python shards、223 个 Stage2c cases、mutants、installed、Windows、Node、packaged Dashboard、build 与 DCO 也都通过，说明新 job graph 和真实 runner 组合可运行。

剩余 merge hold 不能被本 review 消掉：未改动的 common development Dashboard criterion 6 在 hosted Linux 两次 reload/session reconnect 超时，使 checks、聚合 pytest 和 merge-gate 正确保持红色；PR 也仍落后 main。另一个代价是 Python aggregate runner-minutes 约增 8%，虽然最长 shard 从约 16m 降到 9m28s。合并前仍须 rebase、让 required checks 绿色，并确认这种成本换关键路径的取舍可接受；不得绕过失败后把 CI 叙述成全绿。

我的整体评价

当前重写后的范围比旧 shadow 方案更简单：它不创造“选择哪些测试”的新权威，只在稳定 ownership 边界上豁免整类无关 job，并保留可观察的 impact-plan artifact。typed state、domain neutrality、default behavior 与 guidance/obligation 边界均清楚；这是 CI policy，不授予 runtime、Goal、provider、scheduler 或合并权限。future-facing pass 已通过 shared classifier/merge-gate 和删除 shadow machinery 完成，无需再加新的选择框架。

English verdict: APPROVE exact head be0fe25 after rebase and successful required checks; the current hosted common-check failure remains a merge hold.

[huangruiteng]

Follow-up: #4241 and #4250 are now merged. The detailed #4250 main-run error identifies the remaining failure precisely: getByText('正在整理…') matches both the pending-message span and the prefixed accessibility live region, so Playwright throws a strict-mode violation before it can wait. It is not evidence of a ten-second runtime timeout.

Fix proposed in #4251: exact pending-message matching, with the live region, original timeout and durable completion replay assertions unchanged. Four real-browser controls reproduce the old ambiguity and confirm unfinished replies still block. Full development browser with coverage and packaged browser each pass all 24 criteria; 37 CI/workflow regressions also pass. Hosted Linux checks are queued, so no all-green or merge claim yet. Product UI/backend and the CI exemption policy are unchanged.