feat: connector verification loop — evidence claims, run-and-watch, scenario record/replay

[tnunamak]

Settled state — exact head f6a1eea43 (33 commits), substantive suites green head-exact (polyfill-connectors and reference-implementation via dispatched runs after a push-trigger drop; prior content-identical head 92f6ea8f4 was 11/11 green including the full suite). Based on codex/uat-health-candidate-0812 (#125 train); retargets to main when #125 merges.

What this delivers

The connector verification loop: evidence-claims vocabulary (docs/reference/connector-evidence-claims.md), run-and-watch (bin/connector-dev.ts, pdpp.run-summary/1), scenario record/strict-replay (pdpp.connector-scenario/1, bin/scenario-record.ts / bin/scenario-verify.ts), fixture provenance labels + enforcement, connector-init scaffolding, observe-schema observation reports, generate-validators spike, and workflow docs. Proven end-to-end on the unmodified oura and spotify connectors against synthetic in-test providers (synthetic-spike evidence class by the tools' own grounding rules — the first non_loopback_contact_observed capture requires a machine with real accounts).

Verification at head

• Focused harness batch: 233 tests, 231 pass, 2 environment-gated skips (unprivileged user namespaces unavailable on the host — the isolation capability detector reports this honestly).
• Both real-connector spikes green under full wire-strictness.
• Package tsc, biome, conformance gates green; exact-head GitHub suite fully green.

Claim eligibility (all conditions independent, each with its own limitation string and negative test)

Canonical recorded_replay requires: registered connector identity · capture-time declaration digest · capture-time source digest · current declaration digest computed · current source digest computed · supported environment driver on every run · protocol trace on every run · OS-namespace descendant isolation · no unsupported evidence surface (ASSISTANCE) · driver evidence (recorded-http: ≥1 recorded interaction; consumption enforced by the matcher). Anything less: diagnostic_replay: PASS / recorded_replay: WITHHELD / named limitations, plus machine-readable claim: and scenario status: candidate oracle lines.

Wire and trace contract

Untrusted subprocess output is validated before normalization: unknown message types reject on record and verify sides; RECORD requires stream/key/object-data/emitted_at with the op directive validated, captured, and compared (delete↔upsert mutations fail); INTERACTION prompts compare field-exactly (kind/message/canonical schema/timeout; request_id volatile); RECORD ops are mandatory in the v1 format and STATE has symmetric wire validation. TRACE_POLICY dispositions all 13 runtime message kinds under a satisfies clause (compile-time exhaustive); tracked kinds get field-level dispositions (compared / digested / excluded-volatile) with continuation evidence via the runtime's own validator (parity-tested on 16 curated malformed cases). Evidence labels state observed facts only: non_loopback_contact_observed until provider-authority policy exists; author_live withheld by all tooling; secrets redact by default (--persist-otp opt-in).

Review history

Seven independent external review rounds plus two internal adversarial passes; every P0–P2 across all rounds repaired in-branch with per-finding disposition comments. Every strictness increase caught at least one real pre-existing bug (test-order dependence, silent trace field drops, JSON-path rooting mismatch, untracked fixture labels).

Honestly deferred (per the reviews' own disposition tables)

Provider-authority policy (labels already narrowed so nothing overclaims meanwhile) · browser HAR replay driver · gmail IMAP seedable fake · subprocess-output driver (slackdump/gmcli class) · overlap/dedup/idempotence scenario fixtures · built-package digest binding (publication pipeline) · response-body field-level secret handling (scenarios remain local-only).

Assisted-by: AI

[tnunamak]

Closure packet — final delta (head f6a1eea43 = a124a79a1 + an empty CI-retrigger commit; 33 commits).

Round-8 disposition: P1 (legacy ops-less scenarios could earn the canonical claim) → closed via the review's blessed simpler option: ops is mandatory in the v1 format — validateScenario rejects missing (missing_ops), misaligned (ops_length_mismatch), and invalid (invalid_op_literal) entries with named reasons; verifyStreamOps always compares; the legacy tier is deleted, not gated (unmerged format, local-only scenarios, no real legacy corpus). P2 (STATE lacked wire validation) → closed symmetric with RECORD/INTERACTION: assertValidStateMessage requires a nonempty stream (grounded in every real emission site, stated as grounded rather than invented — the wire type alone doesn't annotate it) and cursor property presence via Object.hasOwn (value opacity preserved); malformed STATE fails recording and replay on both paths through the single shared projection.

Pre-submission internal audit (the reviewer's own closure matrix applied to the delta before this push): one P2 found and fixed (a stale comment still describing ops as backward-compatible-optional), one P3 hardened (verify's emit now enforces STATE stream validity itself, so a future RunCollector bypassing the shared projection cannot silently reopen the gap). All other surfaces — vacuous-truth/empty-set on the new rules, writer-path completeness, STATE symmetry incl. state threading, claim/doc consistency, validator bypass routes, negative-control coverage — clean bill, with evidence per surface.

Verification: 233-test focused batch (231 pass, 2 environment-gated skips), both real-connector spikes green under mandatory ops. Exact-head workflow runs (manually dispatched after a push-trigger drop; results are head-exact): polyfill-connectors https://github.com/PDP-Connect/pdpp/actions/runs/31829065426 ✓, reference-implementation https://github.com/PDP-Connect/pdpp/actions/runs/31829081713 ✓, docker-images in progress at packet time (passed on the content-identical prior head 92f6ea8f4, as did REUSE).

Deferred list unchanged. Ready for final closure review.

Assisted-by: AI

[tnunamak]

Post-approval cosmetic delta: ec1397fb6 adds quiet: true to the three scenario CLIs' dotenvConfig calls, suppressing only the dotenv library's promotional tip line from tool output. No behavioral change; env loading identical; tests green. Approved head f6a1eea43 is otherwise unchanged.

Assisted-by: AI