Skip to content
Open
85 changes: 85 additions & 0 deletions docs/evidence-contract.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# 统一证据契约(Source → Evidence ↔ Claim)

对应 Issue #100。Folio 此前并行演化了多套证据抽象:Copilot 的结构化金融事实
(`FinancialEvidenceEnvelope`)、Deep Research 报告的论点引用(`EvidenceRef`)、
网页/新闻(`NewsItem`)。核心风险不是功能缺失,而是**语义碎片化**——同一事实或
来源因来自不同子系统而获得不同的身份与元数据。本契约把它们统一到一条
**Source → Evidence ↔ Claim → Answer/Report** 关系链上。

## 设计原则

- **契约整合,不是新证据框架**。现有类型仍是各自生产方的权威表示,本契约是
附加的、可投影的统一视图;不删除领域专属字段,不要求一次迁移。
- **增量演化**。现有持久化记录保持可读;投影函数只读输入、从不修改,
不做大规模重命名迁移。
- **身份确定性**。所有 ID 由 sha256 确定性派生(截断 24 位,风格与
`financial-evidence` 的 `fe_` 一致),不引入随机性,保证
「组装 → 持久化 → 重新加载」全程稳定。

## 关系模型

```
EvidenceSource ──< EvidenceItem >── EvidenceClaim ──> Answer / ResearchReport
(来源) (证据) (论点)
```

- 一个来源可产出多条证据;一条证据必须属于恰好一个来源。
- 论点与证据是**多对多**:一个论点可引用多条证据,一条证据可支撑多个论点。
实现上由 Claim 单向持有 `evidenceIds[]`,Evidence 不反向命名 Claim。

## 代码位置

| 层 | 文件 | 内容 |
|----|------|------|
| core | `packages/core/src/evidence-contract.ts` | 契约类型与 schema 版本 |
| shared | `packages/shared/src/evidence/contract.ts` | 投影函数 + bundle 组装 + 序列化 |
| docs | 本文件 | 身份与生命周期语义 |

## 身份语义

| ID | 派生输入 | 语义后果 |
|----|----------|----------|
| `sourceId` | kind + origin(publisher / canonicalUrl / query) | 同一文档或同一查询被再次观察仍是**同一来源**;不含检索时间 |
| `evidenceId` | sourceId + kind + 内容 + retrievedAt | 同一事实**稍后再次观察**是同源新证据(observation),保留各自 provenance |
| `claimId` | statement + instrumentId 作用域 | 不同 run 产出**相同表述的论点**共享一个 claimId,在 bundle 中合并并并集其 evidence |

哈希输入统一经 `stableJson`(键排序)序列化,身份永不依赖对象键序。

## 投影函数

| 函数 | 输入(现有类型) | 输出 |
|------|------------------|------|
| `projectFinancialEvidence` | `FinancialEvidenceEnvelope[]` | 每个信封 → 1 个 `structured_finance` 来源 + 每个 value 一条 `structured_value` 证据(保留 metric/unit/currency/period/asOf/originalValue 金融语义) |
| `projectEvidenceRefs` | `EvidenceRef[]` | 每条引用 → 1 个 `tool` 来源 + 1 条 `tool_result` 证据 + 1 条 `unverified` 论点 |
| `projectNewsItems` | `NewsItem[]` | 每条新闻 → 1 个 `news` 来源(canonicalUrl=原文 URL)+ 1 条 `text_excerpt` 证据 |
| `projectTextEvidence` | 通用文档输入 | 1 个 `filing`/`web`/`news`/`other` 来源 + 1 条 `text_excerpt` 证据(保留 excerpt/location,支持 authority 元数据) |

组装与守卫:`buildEvidenceBundle`(按 id 去重合并、claim 并集 evidenceIds)、
`isEvidenceBundle`、`serializeEvidenceBundle` / `parseEvidenceBundle`(往返稳定,
未知 schema 版本返回 undefined)。

## 明确约定

- **结构化金融数据不伪造 URL**。`structured_finance` 来源没有公开文档,
`canonicalUrl` 恒为空;其身份由 publisher + dataset + query 承担。
缺 URL 是有语义的,不是缺失。
- **authority 元数据只在实际已知时填写**(如监管备案 =
`{ primary: true, sourceClass: 'regulator' }`),投影从不猜测。
- **冲突与不可用是显式状态**(`availability: 'conflicted' | 'unavailable'`),
不允许静默丢弃。
- **验证状态**首版恒为 `unverified`;claim 级验证(#13)与来源漂移检测(#20)
是独立关注点,接入时只需更新 `verification` / `verifiedBy`,契约不变。
- 不为此引入图数据库;不要求各来源类型字段完全一致,领域专属元数据放在
`providerMeta` / `provenance` 嵌套扩展中原样保留。

## 消费方

- **Source Inspector(#30)与引用检查器**可直接消费 bundle 投影:来源类别、
URL、摘录、溯源一应俱全,无需再各自解析三套原始类型。
- **评测(#14/#15)**可基于 claimId/evidenceId 统计引用覆盖率。

## 集成示例

`packages/shared/src/evidence/contract.test.ts` 的
`mixed-source integration` 用例演示了同一 bundle 同时携带结构化金融证据、
tool 论点证据、新闻摘录与监管备案摘录,并验证序列化重载后完全一致。
155 changes: 155 additions & 0 deletions packages/core/src/evidence-contract.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
/**
* Unified evidence contract — Source → Evidence ↔ Claim → Answer/Report.
*
* Folio evolved several parallel evidence shapes: FinancialEvidenceEnvelope
* for Copilot tool facts, EvidenceRef for research-report claims, NewsItem
* for web/news. This module defines ONE versioned contract they all project
* into without deleting their domain-specific fields. It is a contract
* integration, not a new evidence framework: existing types stay authoritative
* for their producers, and projections are additive. See
* docs/evidence-contract.md for the identity and lifecycle semantics.
*/

export const EVIDENCE_CONTRACT_SCHEMA_VERSION = 'folio-evidence-contract/v1' as const;

/** Where a source came from. */
export type EvidenceSourceKind =
| 'structured_finance'
| 'filing'
| 'news'
| 'web'
| 'tool'
| 'other';

/** Coarse source class for downstream authority ranking; ranking itself is out of scope. */
export type EvidenceSourceClass =
| 'regulator'
| 'exchange'
| 'issuer'
| 'press'
| 'aggregator'
| 'tool'
| 'other';

export interface EvidenceSource {
/** Stable identity, derived deterministically from kind + origin + canonicalUrl/query. */
sourceId: string;
kind: EvidenceSourceKind;
/** Publisher / provider identity, e.g. `longbridge`, `reuters`, `sec.gov`. */
publisher?: string;
/**
* Canonical public URL when the source has one. Never synthesized for
* structured finance data — absence of a URL is meaningful there; its
* identity is publisher + dataset + query instead.
*/
canonicalUrl?: string;
/** Epoch ms at which this source was retrieved by Folio. */
retrievedAt: number;
/** Epoch ms the source itself was published, when known. */
publishedAt?: number;
/** Lightweight authority metadata; only set when actually known, never guessed. */
authority?: {
/** True for regulator / exchange / issuer primary sources. */
primary: boolean;
sourceClass: EvidenceSourceClass;
};
/** Domain-specific provider metadata preserved verbatim as a nested extension. */
providerMeta?: Record<string, unknown>;
}

export type EvidenceItemKind =
| 'structured_value'
| 'text_excerpt'
| 'table_row'
| 'tool_result';

export type EvidenceAvailability = 'available' | 'conflicted' | 'unavailable';

/** Financial semantics retained when projecting structured finance facts. */
export interface FinancialEvidenceSemantics {
/** Canonical instrument id this fact belongs to. */
instrumentId?: string;
/** Stable field identity, e.g. `lastPrice` or `bars.0.close`. */
metric: string;
value: unknown;
/** Provider-side value before normalization, when tracked. */
originalValue?: unknown;
unit?: string;
currency?: string;
period?: string;
asOf?: number;
}

/** Document semantics retained when projecting text / web / filing evidence. */
export interface DocumentEvidenceExcerpt {
text: string;
/** Where the excerpt lives in the source, e.g. `paragraph.3` or `row.12`. */
location?: string;
}

export interface EvidenceFreshness {
retrievedAt: number;
/** Market-time of the fact itself, when known. */
asOf?: number;
stale: boolean;
}

export interface EvidenceItem {
/** Stable identity, derived deterministically from sourceId + kind + content. */
evidenceId: string;
/** The source this evidence was extracted from. */
sourceId: string;
kind: EvidenceItemKind;
/** Present when the evidence carries financial semantics. */
financial?: FinancialEvidenceSemantics;
/** Present when the evidence carries document/text semantics. */
excerpt?: DocumentEvidenceExcerpt;
freshness: EvidenceFreshness;
/** Conflicts and unavailability are explicit states, never silently dropped. */
availability: EvidenceAvailability;
/** Origin provenance preserved from the producing subsystem. */
provenance: {
runId?: string;
toolCallId?: string;
toolName?: string;
capabilityId?: string;
provider?: string;
/** Scoping instrument for non-financial evidence (news about a listing). */
instrumentId?: string;
/**
* Original producer-side evidence record id (e.g. the `fe_` envelope id of
* a Copilot turn), so pre-contract citations can be joined to contract
* items deterministically.
*/
envelopeId?: string;
};
}

export type ClaimVerificationStatus =
| 'unverified'
| 'supported'
| 'contradicted'
| 'insufficient_evidence';

export interface EvidenceClaim {
/** Stable identity, derived deterministically from the statement + scope. */
claimId: string;
/** The claim statement, e.g. "NVDA valuation is expensive". */
statement: string;
/**
* Evidence backing this claim — many-to-many. One claim may cite many
* items and one item may back many claims: claims reference evidence
* one-way, evidence never names its claims.
*/
evidenceIds: string[];
verification: ClaimVerificationStatus;
/** Which verifier (and version) produced the status, when verified. */
verifiedBy?: string;
}

export interface EvidenceBundle {
schemaVersion: typeof EVIDENCE_CONTRACT_SCHEMA_VERSION;
sources: EvidenceSource[];
evidence: EvidenceItem[];
claims: EvidenceClaim[];
}
1 change: 1 addition & 0 deletions packages/core/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -600,3 +600,4 @@ export * from './trace-projection.ts';
export * from './instrument.ts';
export * from './instrument-catalog.ts';
export * from './financial-evidence.ts';
export * from './evidence-contract.ts';
137 changes: 137 additions & 0 deletions packages/shared/src/evidence/answer-trace.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
import { describe, expect, it } from 'bun:test';
import type { FinancialEvidenceEnvelope } from '@finagent/core';
import { ANSWER_TRACE_SCHEMA_VERSION, buildAnswerEvidenceTrace } from './answer-trace.ts';
import { buildEvidenceBundle, projectFinancialEvidence } from './contract.ts';

function envelope(id: string, overrides: Partial<FinancialEvidenceEnvelope> = {}): FinancialEvidenceEnvelope {
return {
schemaVersion: 'financial-evidence/v1',
normalizationVersion: 'folio-normalization/v1',
id,
sessionId: 'session-1',
runId: 'run-1',
toolCallId: `call-${id}`,
toolName: 'get_quote',
kind: 'quote',
instrumentId: 'NVDA.US',
capabilityId: 'market.quote',
provider: 'longbridge',
query: { symbol: 'NVDA.US' },
values: [{ metric: 'lastPrice', originalValue: '210.50', normalizedValue: 210.5, currency: 'USD' }],
retrievedAt: 1000,
asOf: 900,
stale: false,
cacheHit: false,
resultSnapshot: { lastPrice: 210.5 },
resultHash: 'sha256:abc',
lineage: [],
...overrides,
};
}

const METRIC_BLOCK = [
'```folio-block',
JSON.stringify({
version: 1,
type: 'metric_grid',
title: 'NVDA quote',
metrics: [{ label: 'Last', value: 210.5, unit: 'price', currency: 'USD', evidenceIds: ['fe_env1'] }],
evidenceIds: ['fe_env1', 'fe_missing'],
}),
'```',
].join('\n');

const ANSWER = `Intro paragraph referencing the data.

\`\`\`json
{"type":"metric_grid","note":"plain code fence stays text"}
\`\`\`

${METRIC_BLOCK}`;

describe('buildAnswerEvidenceTrace', () => {
it('maps block citations to contract evidence and records explicit gaps', () => {
const trace = buildAnswerEvidenceTrace({
sessionId: 'session-1',
runId: 'run-1',
question: 'What is NVDA trading at?',
answer: ANSWER,
financialEvidence: [envelope('fe_env1')],
});

expect(trace.schemaVersion).toBe(ANSWER_TRACE_SCHEMA_VERSION);
expect(trace.question).toBe('What is NVDA trading at?');
expect(trace.citations).toHaveLength(1);
const citation = trace.citations[0]!;
expect(citation.blockType).toBe('metric_grid');
expect(citation.citedEvidenceIds).toEqual(['fe_env1', 'fe_missing']);
expect(citation.unmappedEvidenceIds).toEqual(['fe_missing']);
expect(citation.mappedEvidenceIds.length).toBeGreaterThan(0);
for (const evidenceId of citation.mappedEvidenceIds) {
const item = trace.bundle.evidence.find((candidate) => candidate.evidenceId === evidenceId);
expect(item?.provenance.envelopeId).toBe('fe_env1');
}
});

it('builds the bundle from the turn envelopes with structured_finance sources', () => {
const trace = buildAnswerEvidenceTrace({
sessionId: 'session-1',
runId: 'run-1',
question: 'q',
answer: ANSWER,
financialEvidence: [envelope('fe_env1'), envelope('fe_env2', { toolName: 'get_financials', kind: 'fundamental', capabilityId: 'company.financials' })],
});
const expected = buildEvidenceBundle(projectFinancialEvidence([
envelope('fe_env1'),
envelope('fe_env2', { toolName: 'get_financials', kind: 'fundamental', capabilityId: 'company.financials' }),
]));
expect(trace.bundle).toEqual(expected);
expect(trace.bundle.sources.every((source) => source.kind === 'structured_finance' && source.canonicalUrl === undefined)).toBe(true);
});

it('is deterministic for identical turns', () => {
const input = {
sessionId: 'session-1',
runId: 'run-1',
question: 'q',
answer: ANSWER,
financialEvidence: [envelope('fe_env1')],
};
expect(buildAnswerEvidenceTrace(input)).toEqual(buildAnswerEvidenceTrace(input));
});

it('reports a citation with an empty mapping when the turn has no envelopes', () => {
const trace = buildAnswerEvidenceTrace({
sessionId: 'session-1',
runId: 'run-1',
question: 'q',
answer: ANSWER,
financialEvidence: [],
});
expect(trace.bundle.evidence).toHaveLength(0);
expect(trace.citations[0]!.mappedEvidenceIds).toEqual([]);
expect(trace.citations[0]!.unmappedEvidenceIds).toEqual(['fe_env1', 'fe_missing']);
});

it('ignores unclosed fences and invalid block payloads', () => {
const streaming = 'Partial answer\n\n```folio-block\n{"version":1,"type":"metric_grid"';
const unclosed = buildAnswerEvidenceTrace({
sessionId: 'session-1',
runId: 'run-1',
question: 'q',
answer: streaming,
financialEvidence: [envelope('fe_env1')],
});
expect(unclosed.citations).toHaveLength(0);

const invalid = `Bad block.\n\n\`\`\`folio-block\n{"nope":true}\n\`\`\``;
const invalidTrace = buildAnswerEvidenceTrace({
sessionId: 'session-1',
runId: 'run-1',
question: 'q',
answer: invalid,
financialEvidence: [envelope('fe_env1')],
});
expect(invalidTrace.citations).toHaveLength(0);
});
});
Loading