feat(research): 增加针对研报全部 Claims 的端到端验证聚合器 - #155
Open
Theater-ahyeon wants to merge 1 commit into
Open
Theater-ahyeon wants to merge 1 commit into
Theater-ahyeon wants to merge 1 commit into
Conversation
helsome
requested changes
Sep 22, 2026
helsome
left a comment
Owner
There was a problem hiding this comment.
这个聚合器切片有价值,但当前实现正好在 #102 最严格的 identity 边界上有两个具体 blocker:
claimId = claim-${idx}取决于遍历/插入顺序;在报告前面新增一个无关 claim 就会让后续 claim 全部换 ID,不满足 #102 的 stable claim IDs。- evidence id 直接用
runId,缺失时用evidence-${existing.length + 1}。前者是运行相关 id、后者是局部序号,都不是可解析到具体 source span / structured fact 的稳定 evidence identity;而 #104/#105 正在提供 canonical Source/Evidence/Claim bundle,继续单独生成这一套会形成冲突契约。
请最小化修正为复用实际统一证据 bundle 中的稳定 claim/evidence IDs(或在该依赖尚未进入 main 前把本 PR 明确 stack 在对应契约之上),并补一个 focused 回归:插入/重排无关 section/claim 后原有 claimId 不变,且返回的 evidenceIds 能 resolve 到同一 evidence item/source。当前测试使用 mock judge + 手工 report,应标为 FIXTURE/MOCK 聚合验证,而不是“端到端”真实生产链;本 PR 是 Refs #102,我不要求为这个增量补付费模型或整个父 Issue 的 REAL-E2E。作者现有 Bun/OS、10/0 focused、108/0 research、全 workspace typecheck 可保留。当前 head Actions 为 action_required,不是质量失败。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
改了什么
Refs #102(增量切片:研报端到端 Claim 验证聚合器)
按 Issue #102 的验收标准,Deep Research 的最终报告必须能够对其承载的所有事实与因果论断进行机器可读的核验与分类(supported / contradicted / insufficient_evidence):
packages/shared/src/research/claim-verifier.ts:ReportClaimVerificationSummary接口,规范化单份研报所有 Claim 的核验统计(reportId、verifiedAt、totalClaims、supported、contradicted、insufficientEvidence、results);verifyReportClaims(report, verifier, signal):自动扫描研报所有 sections 的evidence(EvidenceRef),提取原子 claim 并去重,调用底层ClaimVerifier对每个独立结论进行逐一核验,输出结构化统计摘要。packages/shared/src/research/claim-verifier.test.ts:packages/shared/src/research/index.ts:verifyReportClaims及对应接口类型。为什么要改
Issue #102 明确要求:
此前
claim-verifier.ts仅实现了单条ClaimVerificationInput的核验核心,缺少针对整份ResearchReport各 section 证据的批量解析、去重与端到端判定聚合函数。本 PR 完成了这一串联增量,让生产研报可以一键生成全篇核验摘要。兼容性
ResearchReport的基础结构与持久化格式。验证
环境:Bun 1.4.2 / Windows 11 (10.0.26200)