diff --git a/packages/pctr/README.md b/packages/pctr/README.md index 73db53e..e492ffe 100644 --- a/packages/pctr/README.md +++ b/packages/pctr/README.md @@ -296,6 +296,80 @@ non-zero when a change would **loosen** anything, so it works as a CI gate on po edits. Note that raising an approval threshold cannot unlock a CRITICAL consequence — the severity comes from what the action can cause, not from the rule that reads it. +## Measured trust, not declared trust: `pctr attest` + +Everything downstream of a trust score is rigorous about it — thresholds, decay, route +admissibility, execution authority. None of that means much while the score itself is a +number somebody typed into `pctr.json`. + +```bash +pctr attest # measure every agent from evidence +pctr attest planner # one agent, in detail +``` + +``` +MEASURED TRUST: PLANNER + +Measured trust 0.8251 (Good) +Declared in manifest 0.98 +Drift -0.1549 — overstated +Evidence 8 receipt(s) from 1 issuer(s) +Oldest evidence 937s + +Issuers + + pctr.effect-boundary score 0.825115 weight 1 (capped) + +Clears MEDIUM bar (0.6) YES +Threshold proof sha256:8568aa97ed1e48b2b8feafd… +``` + +**The rule that governs all of it: absent evidence is not trust.** An agent with no +attestations does not inherit its declared score — it comes back `UNPROVEN`, counts as +zero, and a protected consequence will not route through it (`TRUST_UNPROVEN`). The +failure mode this exists to prevent is a typed-in `0.99` silently authorising a payment. + +Evidence comes from two places: + +- **PCTR's own execution receipts**, scored on the scale in + [`protocol/scoring-semantics.md`](../../protocol/scoring-semantics.md) §3.2: a clean + execution is 0.95, reaching for authority it lacks is 0.15, replaying an authority is + 0.20, waiting on a human approval is 0.75 — that last one matters, because an agent + blocked on a human is not an agent misbehaving. +- **Attestors you configure**, a module or command per agent, returning TTP attestations + or behavioural receipts. Each attestation is verified with TTP's own + `verify_attestation`, so a stale one, or one about a different subject, contributes + nothing. A failing attestor yields *no* evidence — never favourable evidence. + +```json +{ "attestors": { "planner": ["./attestors/workload-identity.mjs"], "*": [{ "command": "./attest.sh" }] } } +``` + +Aggregation is the normative algorithm in +[`protocol/aggregation-spec.md`](../../protocol/aggregation-spec.md) — time decay, +negative-signal amplification, per-issuer weight capping — and `pctr attest` emits a TTP +`TrustThresholdProof` naming the evidence it rests on. + +### Known divergences in the aggregation vectors + +The spec ships nine test vectors; all nine run in `tests/aggregation.test.mjs`. Three do +not match the algorithm the document itself defines, and are asserted as **known +divergences** rather than skipped: + +| Vector | Expects | The written formula yields | Why | +| --- | --- | --- | --- | +| `agg-003` | 0.4 | **0.5** | Superseded by `agg-003-corrected` (identical receipts, expects 0.5). Its own `_explanation` field works the arithmetic, catches itself mid-sentence — *"wait let me recalculate"* — and concludes 0.5, while `expected` still says 0.4. | +| `agg-006` | 0.5 | **0.5799** | Expecting 0.5 requires *both* issuers capped at 0.40. B's uncapped fraction is 0.29, and step 5 says `min(fraction, max_issuer_weight)` — a cap, not a floor. | +| `agg-008` | 0.917 | **0.918** | Off by 0.0010, a hair outside the vectors' own ±0.001 tolerance; consistent with the expected value being computed from rounded intermediate weights. | + +There is also a substantive point behind `agg-006`. Step 5 caps a dominant issuer at 0.40 +and then **re-normalizes**, so when the other issuers carry little weight the capped +issuer still ends up with most of the vote — 50 perfect receipts from one issuer against +two bad ones from two others still yields ~0.90, with the "capped" issuer holding 87% of +the weight. The cap only bites when the rest of the field is comparable. That is pinned +by a test so it cannot be mistaken for an implementation bug, but the spec is what needs +the decision. + ## Execution authority A valid identity is not enough. A valid credential is not enough. A valid route is not @@ -369,6 +443,7 @@ pctr replay [run] Agent Time Machine pctr explain Why was this allowed, denied, or rerouted? pctr receipt [id] Show an execution receipt pctr decide How should a trust change be answered right now? +pctr attest [agent] Measure trust from evidence instead of the manifest pctr learn What the accumulated evidence says to change pctr whatif --policy Replay real history against a policy change pctr graph --svg [file] Draw the execution authority graph diff --git a/packages/pctr/bin/pctr.mjs b/packages/pctr/bin/pctr.mjs index 06f32c6..03551f9 100755 --- a/packages/pctr/bin/pctr.mjs +++ b/packages/pctr/bin/pctr.mjs @@ -16,6 +16,7 @@ import { renderReport, badgeUrl } from '../src/report.mjs'; import { learn, applyProposal } from '../src/learn.mjs'; import { respondToChange } from '../src/decisions.mjs'; import { summarizeHistory, whatIf } from '../src/history.mjs'; +import { attestGraph, attestAgent, proveThreshold } from '../src/attest.mjs'; import { renderGraphSvg } from '../src/graph_svg.mjs'; import * as r from '../src/render.mjs'; @@ -26,7 +27,7 @@ const command = argv[0]; // Flags that take a value, so their value is never mistaken for a positional argument. const VALUE_FLAGS = new Set(['amount', 'records', 'recordsAffected', 'batch', 'params', 'approve', 'target', 'agent', 'objective', 'compare', 'fork', 'key', 'export', 'run', 'probe', 'boundary', 'port', 'share', - 'svg', 'policy', 'limit']); + 'svg', 'policy', 'limit', 'severity']); const positional = (() => { const out = []; for (let i = 1; i < argv.length; i++) { @@ -73,6 +74,15 @@ function verifyOptions() { // What already happened, for the parts that decide what happens next. const loadHistory = () => summarizeHistory(store.listReceipts()); +// Measured trust, keyed by agent, for the router. Only agents with configured attestors +// or observed receipts produce a measurement; the rest fall back to the manifest. +async function loadMeasurements(graph, severity) { + const attestors = graph.manifest.attestors; + if (!attestors) return undefined; + const { measurements } = await attestGraph(graph, { receipts: store.listReceipts(), attestors, severity }); + return Object.fromEntries(measurements.map((m) => [m.agentId, m])); +} + function loadGraph() { const manifest = store.readManifest(); if (!manifest) { @@ -160,7 +170,10 @@ async function main() { if (!action) return fail('Pass an action: pctr route '); // Material parameters can change the consequence, and the consequence sets the bar. const severity = previewConsequence(graph, action, params()).severity; - const result = resolveRoute(graph, action, { target: flag('target'), severity, history: loadHistory() }); + const result = resolveRoute(graph, action, { + target: flag('target'), severity, history: loadHistory(), + measurements: await loadMeasurements(graph, severity) + }); return out(r.renderRoute(result), result); } @@ -349,6 +362,25 @@ async function main() { return result.loosens ? 1 : 0; } + case 'attest': { + const graph = loadGraph(); + const receipts = store.listReceipts(); + const attestors = graph.manifest.attestors ?? {}; + const severity = String(flag('severity') !== true && flag('severity') || 'MEDIUM').toUpperCase(); + + if (positional[0]) { + const agent = graph.nodes.get(positional[0]); + if (!agent || agent.type !== 'agent') return fail(`Unknown agent: ${positional[0]}`); + const measurement = await attestAgent(agent, { receipts, attestors: attestors[agent.id] ?? attestors['*'] ?? [], severity }); + const proof = proveThreshold(measurement, severity); + return out(r.renderAttestation(measurement, proof, severity), { measurement, proof }); + } + + const result = await attestGraph(graph, { receipts, attestors, severity }); + out(r.renderAttestGraph(result, severity), result); + return result.unproven.length ? 1 : 0; + } + case 'doctor': { const manifest = store.readManifest(); const checks = []; @@ -455,6 +487,7 @@ Usage pctr keys Show your signing key id and public key pctr serve Run the effect boundary as its own process pctr decide How should a trust change be answered right now? + pctr attest [agent] Measure trust from evidence instead of the manifest pctr learn What the accumulated evidence says to change pctr whatif --policy Replay real history against a policy change pctr doctor Check your setup @@ -473,6 +506,7 @@ Options --apply learn: write the proposed changes into pctr.json --svg [file] graph: write the graph as SVG (add an action to show its route) --policy whatif: the policy change to test against history + --severity attest: the consequence level to measure against --probe preview: measure the real consequence with this probe --key verify: check signatures against this public key --export keys: write the public key to a file diff --git a/packages/pctr/src/aggregate.mjs b/packages/pctr/src/aggregate.mjs new file mode 100644 index 0000000..6ec7c2a --- /dev/null +++ b/packages/pctr/src/aggregate.mjs @@ -0,0 +1,102 @@ +// TTP TRUST SCORE AGGREGATION — the normative algorithm. +// +// Implements protocol/aggregation-spec.md v1.0 step for step. That document is marked +// normative and carries test vectors; tests/aggregation.test.mjs runs every one of them, +// so this file is not free to drift from the spec. +// +// The two properties that matter, both deliberate: +// - Negative signals weigh more (default 1.5x). An agent behaving well most of the time +// must not be able to average away a few dangerous actions. +// - No single issuer may contribute more than a fraction of the score (default 0.40), +// so one chatty or captured issuer cannot decide an agent's trust alone. + +export const DEFAULT_PARAMS = { + receipt_window_s: 300, + max_issuer_weight: 0.40, + negative_weight_multiplier: 1.5, + decay_half_life_s: 120 +}; + +export const INSUFFICIENT_TRUST_DATA = 'INSUFFICIENT_TRUST_DATA'; + +/** + * @param receipts [{ receipt_id, issuer_id, score, timestamp }] — valid, deduplicated + * @param current_time_ms evaluation time + * @returns { score, contributing_receipts, contributing_issuers, oldest_receipt_age_s } + * or { error: 'INSUFFICIENT_TRUST_DATA' } when nothing is in the window. + */ +export function aggregateTrust(receipts = [], current_time_ms = Date.now(), params = {}) { + const { receipt_window_s, max_issuer_weight, negative_weight_multiplier, decay_half_life_s } = + { ...DEFAULT_PARAMS, ...params }; + + // Step 1 — filter to the receipt window. + const windowReceipts = receipts.filter( + (r) => current_time_ms - r.timestamp <= receipt_window_s * 1000 + ); + if (!windowReceipts.length) { + return { error: INSUFFICIENT_TRUST_DATA, score: null, contributing_receipts: 0, contributing_issuers: 0 }; + } + + // Steps 2 and 3 — time decay, then negative signal amplification. + const weighted = windowReceipts.map((r) => { + const age_s = (current_time_ms - r.timestamp) / 1000; + const decay_weight = Math.exp((-Math.LN2 * age_s) / decay_half_life_s); + const negative = r.score < 0.5; + return { + ...r, age_s, + adjusted_score: r.score, + signal_weight: negative ? decay_weight * negative_weight_multiplier : decay_weight + }; + }); + + // Step 4 — per-issuer weighted score. + const byIssuer = new Map(); + for (const r of weighted) { + const entry = byIssuer.get(r.issuer_id) ?? { weightedSum: 0, totalWeight: 0 }; + entry.weightedSum += r.adjusted_score * r.signal_weight; + entry.totalWeight += r.signal_weight; + byIssuer.set(r.issuer_id, entry); + } + const issuers = [...byIssuer].map(([issuer_id, e]) => ({ + issuer_id, + issuer_score: e.weightedSum / e.totalWeight, + issuer_raw_weight: e.totalWeight + })); + + // Step 5 — cap each issuer's fraction, then re-normalize. + const totalRawWeight = issuers.reduce((sum, i) => sum + i.issuer_raw_weight, 0); + const capped = issuers.map((i) => ({ + ...i, + capped_fraction: Math.min(i.issuer_raw_weight / totalRawWeight, max_issuer_weight) + })); + const normalizationFactor = capped.reduce((sum, i) => sum + i.capped_fraction, 0); + + // Step 6 — combine, and clamp for floating point. + const rawScore = capped.reduce( + (sum, i) => sum + i.issuer_score * (i.capped_fraction / normalizationFactor), 0 + ); + + return { + score: Math.max(0, Math.min(1, rawScore)), + contributing_receipts: windowReceipts.length, + contributing_issuers: issuers.length, + oldest_receipt_age_s: Math.round(Math.max(...weighted.map((r) => r.age_s))), + issuers: capped.map((i) => ({ + issuer_id: i.issuer_id, + issuer_score: Number(i.issuer_score.toFixed(6)), + weight: Number((i.capped_fraction / normalizationFactor).toFixed(6)), + capped: i.issuer_raw_weight / totalRawWeight > max_issuer_weight + })) + }; +} + +// The scoring scale from protocol/scoring-semantics.md, so a score can be read in words. +export function scoreLabel(score) { + if (score == null) return 'Unknown'; + if (score >= 0.90) return 'Excellent'; + if (score >= 0.70) return 'Good'; + if (score >= 0.50) return 'Marginal'; + if (score >= 0.30) return 'Poor'; + if (score >= 0.10) return 'Bad'; + return 'Critical'; +} diff --git a/packages/pctr/src/attest.mjs b/packages/pctr/src/attest.mjs new file mode 100644 index 0000000..2e3c77f --- /dev/null +++ b/packages/pctr/src/attest.mjs @@ -0,0 +1,201 @@ +import path from 'node:path'; +import { execFile } from 'node:child_process'; +import { pathToFileURL } from 'node:url'; +import { verify_attestation, prove_trust_threshold } from '../../../src/index.mjs'; +import { aggregateTrust, scoreLabel, DEFAULT_PARAMS, INSUFFICIENT_TRUST_DATA } from './aggregate.mjs'; +import { TRUST_REQUIRED, MAX_EVIDENCE_AGE_SECONDS } from './trust.mjs'; + +// MEASURED TRUST. +// +// Everything downstream of a trust score is rigorous about it — thresholds, decay, route +// admissibility, execution authority. None of that means much while the score itself is +// a number somebody typed into pctr.json. This turns it into something observed. +// +// The rule that governs the whole file: absent evidence is not trust. An agent with no +// attestations does not inherit its declared score; it comes back UNPROVEN, and a +// protected consequence must not route through an agent whose trust was never measured. + +// A behavioural receipt scored per protocol/scoring-semantics.md 3.2 (tool_execution). +// PCTR's own execution receipts are exactly what that section describes an issuer +// observing, so they are the one attestation source that needs no configuration. +export function scoreExecutionReceipt(receipt) { + const decision = receipt.verifier?.decision; + const codes = (receipt.verifier?.failures ?? []).map((f) => f.code); + + if (decision === 'EXECUTION_ALLOWED') { + // Completed successfully, parameters bound and verified. + return receipt.executed?.status === 'FAILED' ? 0.70 : 0.95; + } + // "Attempted to call a disallowed tool" — the agent reached for authority it lacks. + if (codes.includes('INSUFFICIENT_AUTHORITY') || codes.includes('UNTRUSTED_SIGNER')) return 0.15; + // "Tool call failed — policy violation detected" + if (codes.includes('REPLAYED_AUTHORITY') || codes.includes('BINDING_HASH_MISMATCH') || + codes.includes('SIGNATURE_INVALID') || codes.includes('PARAMETER_MISMATCH') || + codes.includes('TARGET_MISMATCH')) return 0.20; + // "Parameters outside allowed bounds" + if (codes.includes('CONSTRAINT_EXCEEDED')) return 0.35; + // Waiting on a human, or the environment was unavailable: not the agent misbehaving. + if (codes.includes('APPROVAL_REQUIRED') || codes.includes('APPROVAL_MISSING')) return 0.75; + if (codes.includes('BOUNDARY_UNREACHABLE') || codes.includes('AUTHORITY_EXPIRED')) return 0.65; + if (codes.includes('STALE_EVIDENCE')) return 0.60; + return 0.50; // refused for a reason we do not recognise: marginal, not condemned +} + +// PCTR's own receipts, as behavioural receipts for the aggregator. +export function behaviouralReceipts(agentId, receipts = []) { + return receipts + .filter((r) => (r.routeSelected?.agents ?? []).includes(agentId) || r.requestedBy === agentId) + .map((r) => ({ + receipt_id: r.receiptId, + issuer_id: 'pctr.effect-boundary', + score: scoreExecutionReceipt(r), + timestamp: new Date(r.executed?.executedAt ?? r.issuedAt).getTime() + })); +} + +const ATTESTATION_TIMEOUT_MS = 10_000; + +// An external attestor: a module exporting a function, or a command printing JSON. +// It returns TTP attestations, behavioural receipts, or both. +export async function runAttestor(spec, context, { cwd = process.cwd(), timeoutMs = ATTESTATION_TIMEOUT_MS } = {}) { + const attestor = typeof spec === 'string' ? { module: spec } : spec; + try { + const raw = attestor.command + ? await runCommand(attestor.command, attestor.args ?? [], context, { cwd, timeoutMs }) + : await runModule(attestor.module, context, { cwd }); + return { ok: true, source: attestor.command ?? attestor.module, ...normalizeAttestorOutput(raw) }; + } catch (error) { + // A broken attestor produces no evidence. It must never produce favourable evidence. + return { ok: false, source: attestor.command ?? attestor.module, attestations: [], receipts: [], error: error.message ?? String(error) }; + } +} + +function normalizeAttestorOutput(raw) { + if (Array.isArray(raw)) return { attestations: raw, receipts: [] }; + return { attestations: raw?.attestations ?? [], receipts: raw?.receipts ?? [] }; +} + +async function runModule(modulePath, context, { cwd }) { + const mod = await import(pathToFileURL(path.resolve(cwd, modulePath)).href); + const fn = mod.default ?? mod.attest; + if (typeof fn !== 'function') throw new Error(`${modulePath} does not export an attestor function`); + return await fn(context); +} + +function runCommand(command, args, context, { cwd, timeoutMs }) { + return new Promise((resolve, reject) => { + execFile(command, args, { + cwd, timeout: timeoutMs, maxBuffer: 1024 * 1024, + env: { ...process.env, PCTR_AGENT: context.agentId ?? '' } + }, (error, stdout) => { + if (error) return reject(new Error(`attestor failed: ${error.message.split('\n')[0]}`)); + try { resolve(JSON.parse(stdout)); } catch { reject(new Error('attestor did not return JSON')); } + }); + }); +} + +/** + * Measure one agent's trust from evidence. + * + * Returns { trust, proven, label, ... }. `proven` is false when there was no admissible + * evidence — callers must treat that as unproven, never as the declared value. + */ +export async function attestAgent(agent, { + receipts = [], attestors = [], severity = 'MEDIUM', at = new Date().toISOString(), + cwd = process.cwd(), params = {} +} = {}) { + const now = new Date(at).getTime(); + const maxAge = MAX_EVIDENCE_AGE_SECONDS[severity] ?? DEFAULT_PARAMS.receipt_window_s; + const collected = []; + const attestationResults = []; + const errors = []; + + // Source 1: PCTR's own execution receipts. + collected.push(...behaviouralReceipts(agent.id, receipts)); + + // Source 2: configured attestors. + for (const spec of attestors) { + const result = await runAttestor(spec, { agentId: agent.id, severity }, { cwd }); + if (!result.ok) { errors.push({ source: result.source, error: result.error }); continue; } + + for (const attestation of result.attestations) { + // Verified through TTP's own primitive, not a second opinion invented here. + const verified = verify_attestation({ + attestation, subject: agent.id, validAt: at, + maxAge: attestation.maxAge ?? maxAge, + requiredIssuer: attestation.requiredIssuer, requiredType: attestation.requiredType + }); + attestationResults.push({ ...verified, source: result.source }); + if (!verified.valid) continue; + + // A verified attestation contributes as a receipt from its issuer. + collected.push({ + receipt_id: attestation.ref ?? `att-${attestationResults.length}`, + issuer_id: attestation.issuer, + score: typeof attestation.score === 'number' ? attestation.score : 0.5 + (verified.trustScoreDelta ?? 0), + timestamp: new Date(attestation.issuedAt).getTime() + }); + } + for (const r of result.receipts) { + collected.push({ receipt_id: r.receipt_id ?? r.receiptId, issuer_id: r.issuer_id ?? result.source, score: r.score, timestamp: r.timestamp ?? now }); + } + } + + const aggregated = aggregateTrust(collected, now, { receipt_window_s: maxAge, ...params }); + const proven = !aggregated.error; + + return { + agentId: agent.id, + proven, + trust: proven ? Number(aggregated.score.toFixed(4)) : null, + label: scoreLabel(proven ? aggregated.score : null), + declaredTrust: agent.trust ?? null, + // The gap between what was claimed and what the evidence supports. + drift: proven && agent.trust != null ? Number((aggregated.score - agent.trust).toFixed(4)) : null, + contributingReceipts: aggregated.contributing_receipts ?? 0, + contributingIssuers: aggregated.contributing_issuers ?? 0, + oldestEvidenceAgeSeconds: aggregated.oldest_receipt_age_s ?? null, + issuers: aggregated.issuers ?? [], + attestations: attestationResults, + errors, + reason: proven ? null : INSUFFICIENT_TRUST_DATA, + measuredAt: at + }; +} + +// A TTP TrustThresholdProof over measured trust: the artefact that says this agent +// cleared the bar this consequence sets, with the evidence it rests on. +export function proveThreshold(measurement, severity, { at = new Date().toISOString(), proofMode = 'plain' } = {}) { + const required = TRUST_REQUIRED[severity] ?? 0; + return prove_trust_threshold({ + subject: measurement.agentId, + trustScore: measurement.proven ? measurement.trust : 0, + requiredThreshold: required, + dimension: `execution-authority:${severity}`, + evaluatedAt: at, + proofMode, + evidenceRefs: [ + ...measurement.attestations.filter((a) => a.valid).map((a) => a.attestationRef).filter(Boolean), + ...measurement.issuers.map((i) => `issuer:${i.issuer_id}`) + ] + }); +} + +// Measure every agent in a graph, and say plainly which ones are running on a number +// somebody typed rather than on evidence. +export async function attestGraph(graph, { receipts = [], attestors = {}, severity = 'MEDIUM', at, cwd } = {}) { + const agents = [...graph.nodes.values()].filter((n) => n.type === 'agent'); + const measurements = []; + for (const agent of agents) { + measurements.push(await attestAgent(agent, { + receipts, attestors: attestors[agent.id] ?? attestors['*'] ?? [], severity, at, cwd + })); + } + return { + measurements, + proven: measurements.filter((m) => m.proven).length, + unproven: measurements.filter((m) => !m.proven).map((m) => m.agentId), + overstated: measurements.filter((m) => m.drift != null && m.drift < -0.1) + .map((m) => ({ agentId: m.agentId, declared: m.declaredTrust, measured: m.trust, drift: m.drift })) + }; +} diff --git a/packages/pctr/src/index.mjs b/packages/pctr/src/index.mjs index 837e9ad..885b919 100644 --- a/packages/pctr/src/index.mjs +++ b/packages/pctr/src/index.mjs @@ -16,6 +16,8 @@ export * from './report.mjs'; export * from './decisions.mjs'; export * from './learn.mjs'; export * from './history.mjs'; +export * from './aggregate.mjs'; +export * from './attest.mjs'; export * from './graph_svg.mjs'; export * from './keys.mjs'; export * from './protect.mjs'; diff --git a/packages/pctr/src/render.mjs b/packages/pctr/src/render.mjs index c76b4be..d244826 100644 --- a/packages/pctr/src/render.mjs +++ b/packages/pctr/src/render.mjs @@ -11,7 +11,9 @@ export const red = c('31'); export const green = c('32'); export const yellow = const SEV_COLOR = { CRITICAL: red, HIGH: yellow, MEDIUM: cyan, LOW: dim }; export const sev = (s) => (SEV_COLOR[s] ?? dim)(s); export const heading = (text) => `\n${bold(text.toUpperCase())}\n`; -export const field = (label, value, width = 22) => `${label.padEnd(width)}${value}`; +// A label longer than its column still needs a gap before the value. +export const field = (label, value, width = 22) => + `${label.length >= width ? `${label} ` : label.padEnd(width)}${value}`; export const chain = (ids) => ids.join(`\n${dim(' |')}\n${dim(' v')}\n`); export function renderScan(graph) { @@ -163,6 +165,54 @@ export function renderLearn(result) { return out.join('\n'); } +export function renderAttestation(m, proof, severity) { + const out = [heading(`measured trust: ${m.agentId}`)]; + if (!m.proven) { + out.push(red('UNPROVEN'), '', 'No admissible evidence. This agent\'s trust has never been measured,'); + out.push(`so it is treated as 0 — not as the ${m.declaredTrust ?? 'declared'} in pctr.json.`); + if (m.errors.length) out.push('', ...m.errors.map((e) => yellow(` attestor failed: ${e.source} — ${e.error}`))); + return out.join('\n'); + } + out.push(field('Measured trust', `${m.trust} (${m.label})`)); + out.push(field('Declared in manifest', m.declaredTrust == null ? '—' : String(m.declaredTrust))); + if (m.drift != null) { + out.push(field('Drift', m.drift < -0.05 ? red(`${m.drift} — overstated`) : m.drift > 0.05 ? green(`+${m.drift}`) : dim(String(m.drift)))); + } + out.push(field('Evidence', `${m.contributingReceipts} receipt(s) from ${m.contributingIssuers} issuer(s)`)); + out.push(field('Oldest evidence', `${m.oldestEvidenceAgeSeconds}s`)); + if (m.issuers.length) { + out.push('', bold('Issuers'), ''); + for (const i of m.issuers) out.push(` ${i.issuer_id.padEnd(28)} score ${i.issuer_score} weight ${i.weight}${i.capped ? yellow(' (capped)') : ''}`); + } + if (proof) { + out.push('', field(`Clears ${severity} bar (${proof.requiredThreshold})`, proof.satisfied ? green('YES') : red('NO'))); + out.push(field('Threshold proof', dim(proof.proofHash))); + } + return out.join('\n'); +} + +export function renderAttestGraph(result, severity) { + const out = [heading('measured trust')]; + out.push(field('Agents measured', String(result.proven))); + out.push(field('Unproven', result.unproven.length ? red(String(result.unproven.length)) : green('0'))); + out.push(''); + for (const m of result.measurements) { + const line = m.proven + ? `${green('ok')} ${m.agentId.padEnd(26)} ${String(m.trust).padEnd(8)} ${dim(m.label)}${m.drift != null && m.drift < -0.1 ? red(` declared ${m.declaredTrust}`) : ''}` + : `${red('??')} ${m.agentId.padEnd(26)} ${red('unproven')} ${dim('no admissible evidence')}`; + out.push(line); + } + if (result.overstated.length) { + out.push('', bold('Overstated in the manifest'), ''); + for (const o of result.overstated) out.push(` ${o.agentId}: declares ${o.declared}, evidence supports ${o.measured}`); + } + if (result.unproven.length) { + out.push('', dim(`Unmeasured trust is treated as 0 for ${severity} consequences, so those agents cannot route.`)); + out.push(dim('Configure attestors in pctr.json, or run some protected actions to build behavioural evidence.')); + } + return out.join('\n'); +} + export function renderReceipt(receipt, verification) { const out = [heading('execution receipt')]; out.push(field('Receipt', receipt.receiptId)); diff --git a/packages/pctr/src/router.mjs b/packages/pctr/src/router.mjs index c92e512..cb63b32 100644 --- a/packages/pctr/src/router.mjs +++ b/packages/pctr/src/router.mjs @@ -22,7 +22,10 @@ export function resolveRoute(graph, actionId, options = {}) { const candidates = pathsToAction(graph, actionId).map((path) => { const agents = path.filter((id) => graph.nodes.get(id)?.type === 'agent').map((id) => graph.nodes.get(id)); const rejections = []; - const trustStates = agents.map((a) => agentTrustNow(a, { severity, at, decayConstant: policy.decayConstant })); + const trustStates = agents.map((a) => agentTrustNow(a, { + severity, at, decayConstant: policy.decayConstant, + measured: options.measurements?.[a.id] ?? null + })); // 1. remove unauthorized routes for (const a of agents) { @@ -33,7 +36,10 @@ export function resolveRoute(graph, actionId, options = {}) { } // 2. remove untrustworthy routes (including stale evidence) for (const t of trustStates) { - if (t.evidenceStale) { + if (t.provenance === 'unproven') { + rejections.push({ code: 'TRUST_UNPROVEN', agent: t.agentId, + message: `${t.agentId}'s trust has never been measured; a ${severity} consequence must not route through unmeasured trust` }); + } else if (t.evidenceStale) { rejections.push({ code: 'STALE_EVIDENCE', agent: t.agentId, message: `${t.agentId} evidence is ${t.evidenceAgeSeconds}s old; ${severity} actions require evidence under ${t.maxEvidenceAgeSeconds}s` }); } diff --git a/packages/pctr/src/trust.mjs b/packages/pctr/src/trust.mjs index 6afd2a5..0bbbc69 100644 --- a/packages/pctr/src/trust.mjs +++ b/packages/pctr/src/trust.mjs @@ -7,7 +7,30 @@ export const MAX_EVIDENCE_AGE_SECONDS = { CRITICAL: 300, HIGH: 900, MEDIUM: 3600 const RISK_TIER = { CRITICAL: 'high', HIGH: 'high', MEDIUM: 'medium', LOW: 'low' }; -export function agentTrustNow(agent, { severity = 'MEDIUM', decayConstant = 0.00005, at = new Date().toISOString() } = {}) { +export function agentTrustNow(agent, { severity = 'MEDIUM', decayConstant = 0.00005, at = new Date().toISOString(), measured = null } = {}) { + // Measured trust beats declared trust, and unproven beats neither: an agent whose + // trust was never measured does not get to keep the number in the manifest. + if (measured) { + if (!measured.proven) { + return { + agentId: agent.id, trust: 0, declaredTrust: agent.trust ?? 0, provenance: 'unproven', + evidenceAgeSeconds: Infinity, evidenceStale: true, + maxEvidenceAgeSeconds: MAX_EVIDENCE_AGE_SECONDS[severity], + reason: 'no admissible evidence: trust was never measured for this agent' + }; + } + const age = measured.oldestEvidenceAgeSeconds ?? 0; + const maxAge = MAX_EVIDENCE_AGE_SECONDS[severity]; + return { + agentId: agent.id, trust: measured.trust, declaredTrust: agent.trust ?? null, + provenance: 'measured', evidenceAgeSeconds: age, evidenceStale: age > maxAge, + maxEvidenceAgeSeconds: maxAge, contributingIssuers: measured.contributingIssuers + }; + } + return declaredTrustNow(agent, { severity, decayConstant, at }); +} + +function declaredTrustNow(agent, { severity = 'MEDIUM', decayConstant = 0.00005, at = new Date().toISOString() } = {}) { const decayed = apply_decay({ initialTrust: agent.trust ?? 0, decayConstant, @@ -21,6 +44,7 @@ export function agentTrustNow(agent, { severity = 'MEDIUM', decayConstant = 0.00 agentId: agent.id, trust: Number(decayed.finalTrust.toFixed(4)), declaredTrust: agent.trust ?? 0, + provenance: 'declared', evidenceAgeSeconds: agent.evidenceAgeSeconds ?? 0, evidenceStale: (agent.evidenceAgeSeconds ?? 0) > maxAge, maxEvidenceAgeSeconds: maxAge diff --git a/packages/pctr/tests/aggregation.test.mjs b/packages/pctr/tests/aggregation.test.mjs new file mode 100644 index 0000000..e82c06d --- /dev/null +++ b/packages/pctr/tests/aggregation.test.mjs @@ -0,0 +1,114 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import fs from 'node:fs'; +import { aggregateTrust, DEFAULT_PARAMS, INSUFFICIENT_TRUST_DATA, scoreLabel } from '../src/aggregate.mjs'; + +// protocol/aggregation-spec.md is normative and ships test vectors. Every one runs here, +// so the implementation cannot drift from the specification it claims to implement. +const vectors = JSON.parse(fs.readFileSync(new URL('../../../protocol/test-vectors/aggregation-vectors.json', import.meta.url))); + +test('the spec defaults are what this implementation uses', () => { + assert.deepEqual(DEFAULT_PARAMS, vectors.params); +}); + +// Three vectors do not agree with the algorithm the specification actually defines. +// They are asserted as known divergences rather than skipped, so the discrepancy is +// visible in the test output and cannot be quietly forgotten. See the README section +// "Known divergences in the aggregation vectors". +const KNOWN_DIVERGENT = { + 'agg-003': { + yields: 0.5, + why: 'Superseded by agg-003-corrected, which has identical receipts and expects 0.5. ' + + "This vector's own _explanation field works the arithmetic, catches itself mid-sentence " + + '("wait let me recalculate") and concludes 0.5, while its expected field still says 0.4.' + }, + 'agg-006': { + yields: 0.5799, + why: 'The expected 0.5 requires both issuers capped at 0.40. B\'s uncapped fraction is 0.29, ' + + 'and step 5 says min(fraction, max_issuer_weight) — a cap, not a floor. The stated intent ' + + '(4 good receipts from A cannot dominate 1 bad from B) is not what the written formula does.' + }, + 'agg-008': { + yields: 0.918, + why: 'Off by 0.0010, a hair outside the vectors\' own +/-0.001 tolerance. Consistent with the ' + + 'expected value having been computed from rounded intermediate weights.' + } +}; + +for (const testCase of vectors.cases) { + const divergence = KNOWN_DIVERGENT[testCase.id]; + if (divergence) { + test(`known divergence ${testCase.id}: vector expects ${testCase.expected.score}, spec formula yields ${divergence.yields}`, () => { + const result = aggregateTrust(testCase.receipts, testCase.current_time_ms, vectors.params); + assert.ok(Math.abs(result.score - divergence.yields) <= 0.001, + `the implementation must follow the written algorithm: ${divergence.why}`); + assert.ok(Math.abs(result.score - testCase.expected.score) > 0.001, + `${testCase.id} now agrees with the vector — delete this entry from KNOWN_DIVERGENT`); + }); + continue; + } + test(`conformance ${testCase.id}: ${testCase.description}`, () => { + const result = aggregateTrust(testCase.receipts, testCase.current_time_ms, vectors.params); + + if (testCase.expected.error) { + assert.equal(result.error, testCase.expected.error); + return; + } + // The vectors require agreement within +/- 0.001. + assert.ok(Math.abs(result.score - testCase.expected.score) <= 0.001, + `score ${result.score} is outside 0.001 of the expected ${testCase.expected.score}`); + + if (testCase.expected.contributing_receipts !== undefined) { + assert.equal(result.contributing_receipts, testCase.expected.contributing_receipts); + } + if (testCase.expected.contributing_issuers !== undefined) { + assert.equal(result.contributing_issuers, testCase.expected.contributing_issuers); + } + }); +} + +test('an empty window is insufficient data, not a score of zero', () => { + const result = aggregateTrust([{ receipt_id: 'r', issuer_id: 'A', score: 1, timestamp: 0 }], 10_000_000); + assert.equal(result.error, INSUFFICIENT_TRUST_DATA); + assert.equal(result.score, null, 'absent evidence must never read as a trustworthy zero or one'); +}); + +test('a bad actor cannot average away danger with good behaviour', () => { + const now = 1_700_000_000_000; + const receipts = [ + ...Array.from({ length: 9 }, (_, i) => ({ receipt_id: `good${i}`, issuer_id: 'A', score: 1.0, timestamp: now })), + { receipt_id: 'bad', issuer_id: 'A', score: 0.0, timestamp: now } + ]; + const result = aggregateTrust(receipts, now); + // A plain mean would be 0.90. Negative amplification pulls it materially lower. + assert.ok(result.score < 0.87, `expected amplification to bite, got ${result.score}`); +}); + +test('capping binds, but re-normalization gives most of it back', () => { + // Worth stating plainly, because the cap does not do what its name suggests. Step 5 + // caps a dominant issuer at 0.40 and then re-normalizes across issuers — so when the + // other issuers carry little weight, the capped issuer still ends up with most of the + // vote. The cap only bites when the rest of the field is comparable in weight. + const now = 1_700_000_000_000; + const receipts = [ + ...Array.from({ length: 50 }, (_, i) => ({ receipt_id: `loud${i}`, issuer_id: 'LOUD', score: 1.0, timestamp: now })), + { receipt_id: 'q1', issuer_id: 'B', score: 0.2, timestamp: now }, + { receipt_id: 'q2', issuer_id: 'C', score: 0.2, timestamp: now } + ]; + const result = aggregateTrust(receipts, now); + const loud = result.issuers.find((i) => i.issuer_id === 'LOUD'); + assert.equal(loud.capped, true, 'the dominant issuer is capped'); + assert.ok(loud.weight > 0.8, + `re-normalization returns most of the capped weight: LOUD still holds ${loud.weight}`); + assert.ok(result.score > 0.85, 'so 50 perfect receipts from one issuer do outvote two bad ones'); +}); + +test('scores read in the words the scoring semantics define', () => { + assert.equal(scoreLabel(0.95), 'Excellent'); + assert.equal(scoreLabel(0.75), 'Good'); + assert.equal(scoreLabel(0.55), 'Marginal'); + assert.equal(scoreLabel(0.35), 'Poor'); + assert.equal(scoreLabel(0.15), 'Bad'); + assert.equal(scoreLabel(0.05), 'Critical'); + assert.equal(scoreLabel(null), 'Unknown'); +}); diff --git a/packages/pctr/tests/attest.test.mjs b/packages/pctr/tests/attest.test.mjs new file mode 100644 index 0000000..34c232a --- /dev/null +++ b/packages/pctr/tests/attest.test.mjs @@ -0,0 +1,156 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; + +import { attestAgent, attestGraph, proveThreshold, scoreExecutionReceipt, behaviouralReceipts } from '../src/attest.mjs'; +import { buildGraph } from '../src/graph.mjs'; +import { resolveRoute } from '../src/router.mjs'; +import { agentTrustNow } from '../src/trust.mjs'; + +const now = '2026-09-15T12:00:00.000Z'; +const ago = (seconds) => new Date(new Date(now).getTime() - seconds * 1000).toISOString(); + +const receipt = (over = {}) => ({ + receiptId: `r-${Math.random().toString(36).slice(2, 8)}`, + requested: { action: 'payments.transfer', target: 't', params: {} }, + routeSelected: { routeId: 'user:test -> agent -> payments -> payments.transfer', agents: ['agent'] }, + verifier: { decision: 'EXECUTION_ALLOWED', failures: [] }, + executed: { status: 'SUCCEEDED', executedAt: ago(10) }, + issuedAt: ago(10), ...over +}); + +const manifest = () => ({ + principal: 'user:test', + agents: [{ id: 'agent', trust: 0.99, evidenceAgeSeconds: 5, authority: ['payments.*'], tools: ['payments'] }], + tools: [{ id: 'payments', actions: ['payments.transfer'] }], + actions: [{ id: 'payments.transfer', amount: 100 }], + policy: {} +}); + +test('behaviour is scored on the scale the protocol defines', () => { + // protocol/scoring-semantics.md 3.2 — tool_execution. + assert.ok(scoreExecutionReceipt(receipt()) >= 0.90, 'clean execution is Excellent'); + assert.ok(scoreExecutionReceipt(receipt({ verifier: { decision: 'EXECUTION_DENIED', failures: [{ code: 'INSUFFICIENT_AUTHORITY' }] } })) <= 0.29, + 'reaching for authority it lacks is Bad'); + assert.ok(scoreExecutionReceipt(receipt({ verifier: { decision: 'EXECUTION_DENIED', failures: [{ code: 'REPLAYED_AUTHORITY' }] } })) <= 0.39, + 'replaying an authority is a policy violation'); + + // Waiting on a human is not the agent misbehaving. + const pending = scoreExecutionReceipt(receipt({ verifier: { decision: 'EXECUTION_DENIED', failures: [{ code: 'APPROVAL_REQUIRED' }] } })); + assert.ok(pending >= 0.60 && pending < 0.90, `approval pending should be Good-ish, got ${pending}`); +}); + +test('an agent with no evidence is unproven, never its declared score', async () => { + const m = await attestAgent({ id: 'ghost', trust: 0.99 }, { receipts: [], at: now }); + assert.equal(m.proven, false); + assert.equal(m.trust, null); + assert.equal(m.declaredTrust, 0.99); + assert.equal(m.reason, 'INSUFFICIENT_TRUST_DATA'); +}); + +test('unproven trust is treated as zero by the trust check, not as the manifest value', () => { + const state = agentTrustNow({ id: 'ghost', trust: 0.99 }, { severity: 'HIGH', measured: { proven: false } }); + assert.equal(state.trust, 0); + assert.equal(state.provenance, 'unproven'); + assert.equal(state.evidenceStale, true); +}); + +test('a protected consequence will not route through unmeasured trust', () => { + const graph = buildGraph(manifest()); + const result = resolveRoute(graph, 'payments.transfer', { + measurements: { agent: { proven: false } } + }); + assert.equal(result.selected, null); + assert.ok(result.candidates.flatMap((c) => c.rejections).some((r) => r.code === 'TRUST_UNPROVEN')); +}); + +test('measured trust replaces the declared number when evidence exists', async () => { + const receipts = Array.from({ length: 5 }, () => receipt()); + const m = await attestAgent({ id: 'agent', trust: 0.99 }, { receipts, at: now, severity: 'HIGH' }); + assert.equal(m.proven, true); + assert.ok(m.trust > 0.9 && m.trust <= 1, `clean history should measure high, got ${m.trust}`); + + const state = agentTrustNow({ id: 'agent', trust: 0.99 }, { severity: 'HIGH', measured: m }); + assert.equal(state.provenance, 'measured'); + assert.equal(state.trust, m.trust); +}); + +test('misbehaviour drags measured trust below what the manifest claims', async () => { + const receipts = [ + ...Array.from({ length: 4 }, () => receipt()), + receipt({ verifier: { decision: 'EXECUTION_DENIED', failures: [{ code: 'INSUFFICIENT_AUTHORITY' }] } }) + ]; + const m = await attestAgent({ id: 'agent', trust: 0.99 }, { receipts, at: now, severity: 'HIGH' }); + assert.ok(m.drift < -0.1, `expected the overstatement to show, drift was ${m.drift}`); + assert.ok(m.trust < 0.99); +}); + +test('a failing attestor yields no evidence, never favourable evidence', async () => { + const m = await attestAgent({ id: 'agent', trust: 0.99 }, { + receipts: [], attestors: ['./nonexistent-attestor.mjs'], at: now + }); + assert.equal(m.proven, false); + assert.equal(m.errors.length, 1); + assert.match(m.errors[0].error, /Cannot find|does not export/); +}); + +test('an external attestor contributes verified attestations only', async () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'pctr-attest-')); + const file = path.join(dir, 'attestor.mjs'); + fs.writeFileSync(file, ` + export default async ({ agentId }) => ({ attestations: [ + { ref: 'att-good', subject: agentId, issuer: 'workload-identity', type: 'spiffe', + score: 0.95, issuedAt: '${ago(5)}', expiresAt: '${ago(-3600)}', claims: {} }, + { ref: 'att-expired', subject: agentId, issuer: 'stale-issuer', type: 'spiffe', + score: 1.0, issuedAt: '${ago(99999)}', expiresAt: '${ago(9999)}', claims: {} }, + { ref: 'att-wrong-subject', subject: 'someone-else', issuer: 'confused-issuer', type: 'spiffe', + score: 1.0, issuedAt: '${ago(5)}', expiresAt: '${ago(-3600)}', claims: {} } + ] }); + `); + const m = await attestAgent({ id: 'agent', trust: 0.5 }, { receipts: [], attestors: [file], at: now, severity: 'HIGH', cwd: dir }); + + assert.equal(m.proven, true); + const valid = m.attestations.filter((a) => a.valid); + assert.equal(valid.length, 1, 'only the fresh, correctly-subjected attestation counts'); + assert.equal(valid[0].attestationRef, 'att-good'); + assert.ok(m.attestations.some((a) => a.failureReasons.some((f) => f.code === 'EXPIRED_ATTESTATION'))); + assert.ok(m.attestations.some((a) => a.failureReasons.some((f) => f.code === 'INVALID_SUBJECT'))); + fs.rmSync(dir, { recursive: true, force: true }); +}); + +test('a threshold proof is a TTP artefact carrying its evidence', async () => { + const m = await attestAgent({ id: 'agent', trust: 0.99 }, { receipts: Array.from({ length: 3 }, () => receipt()), at: now, severity: 'HIGH' }); + const proof = proveThreshold(m, 'HIGH', { at: now }); + assert.equal(proof.type, 'TrustThresholdProof'); + assert.equal(proof.subject, 'agent'); + assert.equal(proof.requiredThreshold, 0.75); + assert.equal(proof.satisfied, true); + assert.match(proof.proofHash, /^sha256:/); + assert.ok(proof.evidenceRefs.length, 'the proof names what it rests on'); + + // An unproven agent proves nothing. + const unproven = proveThreshold(await attestAgent({ id: 'ghost' }, { receipts: [], at: now }), 'HIGH', { at: now }); + assert.equal(unproven.satisfied, false); +}); + +test('measuring a whole graph names the agents running on typed-in numbers', async () => { + const graph = buildGraph({ + ...manifest(), + agents: [ + { id: 'agent', trust: 0.99, authority: ['payments.*'], tools: ['payments'] }, + { id: 'ghost', trust: 0.99, authority: ['payments.*'], tools: ['payments'] } + ] + }); + const result = await attestGraph(graph, { receipts: Array.from({ length: 3 }, () => receipt()), at: now, severity: 'HIGH' }); + assert.deepEqual(result.unproven, ['ghost']); + assert.equal(result.proven, 1); +}); + +test('behavioural receipts are attributed to the agents on the route', () => { + const receipts = [receipt(), receipt({ routeSelected: { routeId: 'x', agents: ['other'] } })]; + assert.equal(behaviouralReceipts('agent', receipts).length, 1); + assert.equal(behaviouralReceipts('other', receipts).length, 1); + assert.equal(behaviouralReceipts('nobody', receipts).length, 0); +});