feat(tools): 测试发现失效守护,防 --no-tests=pass 静默放行(R-00266) - #10
Merged
Conversation
现状实测(守护落地前):给 lumio-core-root-abi 注入 autotests = false, 30 个测试凭空消失,`just nextest` 仍 exit 0 并报「80 tests run: 80 passed」。 --no-tests=pass 解决的是「合法状态下可能没有测试」,代价是**发现失效与真的 没有测试长得一样**——这个洞由本卡补上,不动 nextest 的 --no-tests 语义。 判据是对 tools/test-discovery.manifest 的**集合相等**断言,不是子集: 登记但发现 0 个 → 失败;发现到测试却没登记 → 失败。只用子集会让清单悄悄腐烂, 而那正是本守护要防的那类「静默」。清单只登记包名不登记数量:数量门槛会让每次 加测试都要改数,人只会把数改小,守护随之失效。 判据不锚在被守护对象自己身上:期望集合来自受版本控制、人工评审的清单, workspace 权威包集合来自 cargo metadata,发现结果来自 cargo nextest list ——三个独立来源。采集失败与「发现到 0 个」分开处置(前者 exit 2,后者 exit 1)。 采集(.sh)与判据(.py)分开,判据是输入的纯函数:反例探针可直接拿构造输入 驱动,不必在发货路径上开测试后门。 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
现状实测:这个洞是真的
守护落地前,给
lumio-core-root-abi注入autotests = false(模拟特性/清单配置错误导致集成测试不再被发现,不动任何测试本体):30 个测试凭空消失,门禁全绿。另单独对照
--no-tests语义本身:对无测试的包,--no-tests=pass→ exit 0,--no-tests=fail→ exit 4。判据
对
tools/test-discovery.manifest做集合相等断言,不是子集:只用子集会让清单悄悄腐烂,而那正是本守护要防的那类「静默」。
清单只登记包名,不登记数量:数量门槛会让每次加测试都要改数,人只会把数改小,守护随之失效;
>0是能长期守住、且恰好覆盖「发现整体失效」这一失败模式的判据。判据不锚在被守护对象自己身上——三个独立来源:期望集合来自受版本控制、人工评审的清单;workspace 权威包集合来自
cargo metadata;发现结果来自cargo nextest list。拿当下发现结果生成期望,恒等式证明不了任何东西。采集失败 ≠ 发现到 0 个:
cargo nextest list执行失败 exit 2,判据失败 exit 1,两者处置不同。结构
采集(
.sh)与判据(.py)分开,判据是输入的纯函数——反例探针可直接拿构造输入驱动,不必在发货路径上开测试后门。对照组
autotests = falsejust nextestexit 1(守护是其前置依赖)--no-tests=pass的原始静默面)边界
不改
nextest的--no-tests语义(两者管的是不同的事),不动任何测试本体,清单只含当前真实有测试的 5 个包。验证
just nextest110/110;just的 fmt-check / clippy / deny / about / tool-lock / runtime-deps、check-contracts、check-generated、spec-lint + 17 项自测:全部 exit 0。🤖 Generated with Claude Code