Skip to content

feat(research): 增加针对研报全部 Claims 的端到端验证聚合器 - #155

Open
Theater-ahyeon wants to merge 1 commit into
helsome:mainfrom
Theater-ahyeon:feat/deep-research-e2e-slice
Open

Theater-ahyeon wants to merge 1 commit into
helsome:mainfrom
Theater-ahyeon:feat/deep-research-e2e-slice

Conversation

@Theater-ahyeon

Copy link
Copy Markdown

改了什么

Refs #102(增量切片:研报端到端 Claim 验证聚合器)

按 Issue #102 的验收标准,Deep Research 的最终报告必须能够对其承载的所有事实与因果论断进行机器可读的核验与分类(supported / contradicted / insufficient_evidence):

  1. packages/shared/src/research/claim-verifier.ts
    • 新增 ReportClaimVerificationSummary 接口,规范化单份研报所有 Claim 的核验统计(reportIdverifiedAttotalClaimssupportedcontradictedinsufficientEvidenceresults);
    • 新增 verifyReportClaims(report, verifier, signal):自动扫描研报所有 sections 的 evidenceEvidenceRef),提取原子 claim 并去重,调用底层 ClaimVerifier 对每个独立结论进行逐一核验,输出结构化统计摘要。
  2. packages/shared/src/research/claim-verifier.test.ts
    • 补充端到端聚合验证测试用例,覆盖跨章节重复 claim 自动去重、SEC 主来源证据绑定、三态统计汇总断言。
  3. packages/shared/src/research/index.ts
    • 导出 verifyReportClaims 及对应接口类型。

为什么要改

Issue #102 明确要求:

  • The final report contains stable claim IDs for factual/causal conclusions and evidence references that resolve to specific source spans/structured facts, not only source homepages.
  • Each material factual/causal claim is machine-readably classified as supported / contradicted / insufficient evidence (or equivalent).
  • Unsupported/contradicted claims are not silently rendered as verified conclusions.

此前 claim-verifier.ts 仅实现了单条 ClaimVerificationInput 的核验核心,缺少针对整份 ResearchReport 各 section 证据的批量解析、去重与端到端判定聚合函数。本 PR 完成了这一串联增量,让生产研报可以一键生成全篇核验摘要。

兼容性

  • 无可见 UI 变化:纯共享领域工具函数与类型增量;
  • 完全向后兼容:不改动现有 ResearchReport 的基础结构与持久化格式。

验证

环境:Bun 1.4.2 / Windows 11 (10.0.26200)

bun test packages/shared/src/research/claim-verifier.test.ts
→ 10 passed / 0 failed (27 expect calls)

bun test packages/shared/src/research
→ 108 passed / 0 failed (373 expect calls across 14 files)

bun run typecheck
→ core / i18n / shared / ui / electron 全部 exit 0

@helsome helsome left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个聚合器切片有价值,但当前实现正好在 #102 最严格的 identity 边界上有两个具体 blocker:

  1. claimId = claim-${idx} 取决于遍历/插入顺序;在报告前面新增一个无关 claim 就会让后续 claim 全部换 ID,不满足 #102 的 stable claim IDs。
  2. evidence id 直接用 runId,缺失时用 evidence-${existing.length + 1}。前者是运行相关 id、后者是局部序号,都不是可解析到具体 source span / structured fact 的稳定 evidence identity;而 #104/#105 正在提供 canonical Source/Evidence/Claim bundle,继续单独生成这一套会形成冲突契约。

请最小化修正为复用实际统一证据 bundle 中的稳定 claim/evidence IDs(或在该依赖尚未进入 main 前把本 PR 明确 stack 在对应契约之上),并补一个 focused 回归:插入/重排无关 section/claim 后原有 claimId 不变,且返回的 evidenceIds 能 resolve 到同一 evidence item/source。当前测试使用 mock judge + 手工 report,应标为 FIXTURE/MOCK 聚合验证,而不是“端到端”真实生产链;本 PR 是 Refs #102,我不要求为这个增量补付费模型或整个父 Issue 的 REAL-E2E。作者现有 Bun/OS、10/0 focused、108/0 research、全 workspace typecheck 可保留。当前 head Actions 为 action_required,不是质量失败。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants