From 014f69ba5c4df50599209f783cebc97c46353ed4 Mon Sep 17 00:00:00 2001 From: zackbart Date: Tue, 1 Sep 2026 12:06:12 -0400 Subject: [PATCH] Validate maintained provider contracts without credentials (#521) (#520) --- AGENTS.md | 11 +- CHANGELOG.md | 42 ++ README.md | 6 +- documentation/cloudflare.md | 62 ++- documentation/connectors.md | 28 +- documentation/linear.md | 9 + documentation/mixpanel.md | 16 +- documentation/notion.md | 61 ++- documentation/operations.md | 19 +- documentation/provider-conventions.md | 108 +++-- documentation/revenuecat.md | 24 +- documentation/stripe.md | 7 + documentation/upgrading.md | 10 +- documentation/vercel.md | 93 +++- package-lock.json | 4 +- package.json | 3 +- records/provider-audit.md | 57 ++- scripts/drift-check.mjs | 660 ++++++++++++++------------ scripts/drift/hosted-auth.d.mts | 4 - scripts/drift/hosted-auth.mjs | 19 - scripts/drift/vercel-endpoints.json | 2 +- src/catalog-drift.ts | 6 +- src/providers/cloudflare.ts | 143 +++++- src/providers/notion.ts | 160 ++++++- src/providers/revenuecat.ts | 11 + src/providers/stripe.ts | 7 + src/providers/vercel.ts | 183 ++++++- src/version.ts | 2 +- templates/node/package.json | 2 +- test/catalog-drift.test.ts | 32 ++ test/cloudflare-provider.test.ts | 60 ++- test/drift-check.test.ts | 248 ++++++++-- test/notion-provider.test.ts | 73 +++ test/revenuecat-provider.test.ts | 10 +- test/vercel-provider.test.ts | 129 +++++ 35 files changed, 1726 insertions(+), 585 deletions(-) delete mode 100644 scripts/drift/hosted-auth.d.mts delete mode 100644 scripts/drift/hosted-auth.mjs diff --git a/AGENTS.md b/AGENTS.md index 42370fd3..3796c964 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -122,11 +122,12 @@ dead, and neither is a state to leave the repository in. - **CHANGELOG.** Each release opens with a narrative paragraph — what this release is, what breaks, what a deployment can ignore — then `### Added` / `### Changed` / `### Fixed`. -- **Provider drift.** Before tagging a release, run `npm run drift:check` with - local provider credentials exported. It diffs each hosted-MCP catalog against - its vetted manifest and each hand-written provider's touched endpoints against - the provider's published OpenAPI document. Findings are read by a human and - become GitHub issues — no credential goes near CI, and nothing files itself. +- **Provider drift.** Run `npm run providers:check` for the credential-free + public check across every maintained provider. It covers official MCP + documentation and OpenAPI contracts and never reads a provider credential. + Remote MCP schemas remain owned by the live `tools/list` response rather than + a vendored copy. Findings are read by a human and become GitHub issues; + nothing files itself. See [`documentation/provider-conventions.md`](./documentation/provider-conventions.md#the-maintainer-run-drift-check). - **Releases.** `npm run release:check`, tag `v` matching diff --git a/CHANGELOG.md b/CHANGELOG.md index 07578b32..70f60a1a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,48 @@ All notable changes to this package are documented here. +## 0.22.2 — 2026-09-01 + +This patch makes the maintained-provider release check credential-free and +adds explicit official-MCP alternatives beside the existing Cloudflare, +Notion, and Vercel API connections. Existing declarations still select the +hand-written API interface by default. Linear, Mixpanel, RevenueCat, and Stripe +remain MCP-only, and deployments that do not select a new interface need no +configuration or credential change. + +### Added + +- **Selectable API and MCP interfaces.** Cloudflare, Notion, and Vercel accept + `surface: "mcp"` for their official hosted server while preserving the API + default. Each MCP wrapper supplies release-reviewed safety classifications, + keeps unknown tools fail-closed, and leaves provider descriptions and input + and output schemas untouched. +- **Credential-free provider validation.** `npm run providers:check` compares + every maintained provider with official public evidence: touched OpenAPI + operations for the three hand-written interfaces, and endpoint, OAuth, and + documented tool inventories for the seven hosted interfaces. It reads no + provider credential and generates no runtime tool. + +### Changed + +- **Public evidence replaces authenticated drift setup.** The lower-level + `drift:check` command now defaults to the same specification and documentation + checks; the credential helper and `--hosted` mode are removed. Live + `tools/list` responses remain authoritative for MCP schemas. +- **Hosted safety manifests refreshed.** Stripe and RevenueCat classifications + now cover their current official tool references. Vercel URL fetching is + approval-gated because an application GET route is not guaranteed to be + observational. + +### Fixed + +- **Reviewed Vercel event drift.** Build-log projection accepts the expanded + deployment-event variants, including events without a published timestamp, + and the touched-endpoint digest records the reviewed contract (#520). +- **Schema ownership stays intact.** Deterministic coverage now proves the + safety wrapper preserves each provider description, input schema, and output + schema by reference instead of substituting a vendored definition (#521). + ## 0.22.1 — 2026-08-31 This patch adds a maintained Vercel connection over the public REST API. It is diff --git a/README.md b/README.md index 92c24c53..f44e0d21 100644 --- a/README.md +++ b/README.md @@ -63,8 +63,10 @@ Fifty issues in, one small object out. Your context window notices. - **Wrap any HTTP API by hand.** A few lines per tool. No OpenAPI conversion — generated tool sprawl is the problem, not the fix. - **Use maintained connections** for Cloudflare, Linear, Mixpanel, Notion, - RevenueCat, Stripe, and Vercel — known endpoints, auth defaults, and vetted - read/write classifications, imported one at a time. + RevenueCat, Stripe, and Vercel: known endpoints, auth defaults, and vetted + read/write classifications, imported one at a time. Cloudflare, Notion, and + Vercel each let the deployment choose their hand-written API interface or + official hosted MCP. - **Let the agent work in code.** Search, chain, filter, join, and reduce inside the sandbox instead of round-tripping every call through the model. - **Teach undeclared result shapes by using them.** Successful read-only calls diff --git a/documentation/cloudflare.md b/documentation/cloudflare.md index ee3eaf71..6c0756d1 100644 --- a/documentation/cloudflare.md +++ b/documentation/cloudflare.md @@ -1,20 +1,20 @@ # Cloudflare prebuilt connection Import `cloudflare()` independently from -`@zackbart/connecta/providers/cloudflare`. It is a deliberate, hand-written -surface over Cloudflare's v4 REST API. Fifty-one tools combine ergonomic, -fully described operations for common work with three guarded escape hatches -for the rest of Cloudflare's fast-moving control plane. Reads, JSON mutations, -and raw/multipart uploads remain separate so safety routing does not depend on -an agent-supplied HTTP method. The connection keeps lean projections, typed -failures, and a rate-limit budget matching the documented one. It adds no -provider dependency, imports nothing outside Connecta, and is not reachable -from Connecta's root entry. +`@zackbart/connecta/providers/cloudflare`. The deployment chooses one of two +interfaces. The default is a deliberate, hand-written surface over +Cloudflare's v4 REST API. Its fifty-one tools combine ergonomic, fully +described operations for common work with three guarded escape hatches for the +rest of Cloudflare's fast-moving control plane. The other choice is +Cloudflare's official whole-API hosted MCP, which exposes `search` and +`execute` with live provider-owned schemas. Both are ordinary connectors and +neither is reachable from Connecta's root entry. ```ts import { cloudflare } from "@zackbart/connecta/providers/cloudflare"; const edge = cloudflare("cloudflare_prod", { + surface: "api", // optional; this is the backward-compatible default title: "Production edge", purpose: "DNS and cache administration for the production estate", zoneId: "0a1b2c3d4e5f60718293a4b5c6d7e8f9", @@ -23,6 +23,15 @@ const edge = cloudflare("cloudflare_prod", { }); ``` +Use Cloudflare's hosted code-mode interface instead: + +```ts +const wholeApi = cloudflare("cloudflare_mcp", { + surface: "mcp", + purpose: "Cloudflare administration outside the curated REST workflows", +}); +``` + Use the legacy user-scoped Global API Key when an existing deployment needs it: ```ts @@ -41,6 +50,29 @@ between a production and a staging instance needs to know which one answers the question. Account `instructions` are appended to the maintained guide and cannot change the connector's safety classification. +## Choosing an interface + +Use the API interface when its projected named tools cover the work. Connecta +owns those schemas, projections, typed errors, pagination, and the split +between read-only and mutating escape hatches. + +Use `surface: "mcp"` when broad product coverage matters more than projected +results. Cloudflare's official server covers more than 2,500 API endpoints +through two code-mode tools. `search` reads the OpenAPI document and is +read-only. `execute` can run a program containing any authorized HTTP method, +so Connecta always routes it through approval. A program that happens to use +only GET cannot be proven observational from the tool schema. + +The MCP catalog and schemas come from the live server. The release manifest +classifies the two known names but does not replace their schemas. OAuth is the +default. A headless deployment may instead pass `auth` with a scoped API token. +The credential remains the provider-side permission boundary either way. The +MCP interface accepts `callAdmission` when the deployment has a concurrency or +call-rate requirement; it does not reuse the API interface's REST-wide budget. + +The remaining sections document the hand-written API interface. MCP tool +arguments and results are intentionally read from the live server instead. + ## No SDK, on purpose Cloudflare publishes an official `cloudflare` npm SDK, and this connection does @@ -61,7 +93,7 @@ claim: the `cloudflare` package must not appear in `dependencies`, `peerDependencies`, or `devDependencies`, and every import in the provider must be relative. -## Credentials +## API credentials The default credential is a scoped Cloudflare API token, sent as `Authorization: Bearer `. Create it under My Profile → API Tokens → @@ -133,7 +165,7 @@ an empty `accountId` would fall back to the default again. A deployment that wants zones from one account passes `accountId` explicitly, and the property says so. -## Tools +## API tools The named surface covers workflows that benefit most from concise schemas and projections: @@ -423,6 +455,14 @@ not by Connecta. `maxConcurrency` is the bound that actually protects a shared credential, because a single `execute_code` program can fan out far faster than the window notices. +## Contract checks + +`npm run providers:check` compares the 49 fixed REST endpoints with +Cloudflare's published OpenAPI document and the two MCP names, endpoint, and +OAuth support with Cloudflare's official MCP page. It needs no Cloudflare +credential. The MCP schemas are not vendored or reconstructed: the live +`tools/list` response remains the contract agents receive. + ## Conventions This connection is audited against diff --git a/documentation/connectors.md b/documentation/connectors.md index 71acf90c..f5edbc1f 100644 --- a/documentation/connectors.md +++ b/documentation/connectors.md @@ -54,11 +54,29 @@ const analytics = mixpanel("product_analytics", { }); ``` -The constructor may use `remoteMcp()` or `api()` internally. Callers should not -need to care which transport gives the better agent-facing surface, and the -choice does not grant the connection different runtime privileges. Two -instances of the same provider are isolated in exactly the same way as two -hand-written connectors with different ids. +The constructor may use `remoteMcp()` or `api()` internally. When a provider's +official MCP and HTTP API expose materially different capabilities or schema +ownership, the constructor may offer an explicit deployment-time surface +choice. It must document the difference, keep a backward-compatible default, +and never let an agent switch surfaces during a run. The choice grants no +different runtime privileges. Two instances of the same provider are isolated +in exactly the same way as two hand-written connectors with different ids. + +That choice exists only when the two interfaces are genuinely different: + +| Provider | Maintained interfaces | Why | +| --- | --- | --- | +| Cloudflare | API and MCP | The API interface has 48 projected named tools plus three safety-split hatches. The official MCP compresses more than 2,500 endpoints into `search` and approval-gated `execute`. | +| Notion | API and MCP | The API interface has stable lean projections. The official MCP adds Notion-owned live schemas, workspace search, files, views, agents, and sessions. | +| Vercel | API and MCP | The API interface has projected deployment operations. The official MCP owns a broader, independently changing catalog. | +| Linear | MCP | Vendoring its GraphQL API would create a second schema system rather than a distinct maintained interface. | +| Mixpanel | MCP | Its hosted service already joins several Mixpanel APIs; flattening those APIs would recreate the catalog problem. | +| RevenueCat | MCP | Its official server is generated from API v2, so a second wrapper would duplicate the same contract. | +| Stripe | MCP | Its official server already exposes both named workflows and supported API methods. A second raw API interface would duplicate it. | + +This is not a requirement that every provider have two labels. A second choice +must change capability, result shape, or schema ownership enough to justify a +second contract. Otherwise it only gives agents two names for the same thing. A prebuilt connection's vetted annotations fill in downstream silence and otherwise preserve explicit annotations. This includes an explicit diff --git a/documentation/linear.md b/documentation/linear.md index c993dd13..afc4a60f 100644 --- a/documentation/linear.md +++ b/documentation/linear.md @@ -176,6 +176,15 @@ A budget-only rule needs no queue. If you add `maxConcurrency` you are asking for a queue, and the admission controller then requires the rest of the queue settings at construction. +## Public contract check + +`npm run drift:check -- --docs --provider linear` checks Linear's official MCP +setup page for the read-write endpoint and OAuth support without using a +credential. Linear does not publish an exact tool inventory there, so the +command says `setup metadata only` and makes no claim about names or schemas. +At runtime the live `tools/list` response remains the schema authority and is +passed through without a vendored replacement. + ## Conventions This connection is audited against diff --git a/documentation/mixpanel.md b/documentation/mixpanel.md index eb94cddd..616d1aef 100644 --- a/documentation/mixpanel.md +++ b/documentation/mixpanel.md @@ -85,9 +85,11 @@ A read-only live audit on 2026-08-13 confirmed all three refusals against the US hosted endpoint. They are reported upstream as [`mixpanel/mixpanel-headless#202`](https://github.com/mixpanel/mixpanel-headless/issues/202). The vetted catalog records current schema digests for all 64 tools, -so a later schema correction or regression appears by tool name in the -maintainer drift check. The guide can then shrink when the downstream schema -becomes complete; Connecta does not absorb the defect permanently. +so a later schema correction or regression increments runtime drift when an +ordinary catalog refresh observes it. The live definition is still served +unchanged. The credential-free provider check does not depend on those digests. +The guide can shrink when the downstream schema becomes complete; Connecta does +not absorb the defect permanently. The wrapper classifies the documented observational tools as reads and the documented create, update, edit, merge, dismiss, duplicate, and delete tools as @@ -142,6 +144,14 @@ for a queue, and the admission controller then requires the rest of the queue settings at construction. Discovery traffic is outside connector call admission either way and still needs restrained use. +## Public contract check + +`npm run drift:check -- --docs --provider mixpanel` compares Mixpanel's +official Available Tools table with the vetted manifest and checks all three +regional endpoints plus OAuth support. The current table lists 63 tools. It +omits `Fill-Event-Metadata`, which remains classified from the last +authenticated review and is reported as `not documented`, not silently removed. + ## Conventions This connection is audited against diff --git a/documentation/notion.md b/documentation/notion.md index 3bdcb0a4..4499cc87 100644 --- a/documentation/notion.md +++ b/documentation/notion.md @@ -1,39 +1,60 @@ # Notion prebuilt connection Import `notion()` independently from `@zackbart/connecta/providers/notion`. It -is a hand-written `api()` surface over Notion's public REST API — fifteen -deliberate tools, lean projections of Notion's famously bloated payloads, typed -failures, a rate budget matched to the documented limit, and a required usage -guide. It adds no provider dependency, imports no `node:` builtin, and is not +offers two deployment-time interfaces. The default is a hand-written `api()` +surface over Notion's public REST API: fifteen deliberate tools, lean +projections of Notion's famously bloated payloads, typed failures, a rate +budget matched to the documented limit, and a required usage guide. The other +choice is Notion's official hosted MCP with live provider-owned schemas and a +broader workspace, files, views, agents, and sessions catalog. Neither is reachable from Connecta's root entry. ```ts import { notion } from "@zackbart/connecta/providers/notion"; const wiki = notion("engineering_wiki", { + surface: "api", // optional; this is the backward-compatible default title: "Engineering wiki", purpose: "Runbooks, specs, and on-call notes for the platform team", instructions: "Prefer the Runbooks database; specs live under Projects.", }); ``` +Use Notion's hosted MCP instead: + +```ts +const workspace = notion("notion_mcp", { + surface: "mcp", + purpose: "Workspace search, files, views, and asynchronous agent sessions", +}); +``` + The `id` owns the ordinary connector namespaces; use a different id for every Notion workspace. `purpose` is required because an agent choosing between two instances needs to know which workspace answers the question. Workspace `instructions` are appended to the maintained guide and cannot change the connector's safety classification. -## Why this one is `api()` and not `remoteMcp()` +## Choosing an interface -Notion publishes an MCP server, but the interesting problem here is not -transport — it is shape. A single Notion page returns every property as a +Use the API interface for its compact, stable projections. A single Notion +page returns every property as a discriminated wrapper object, every string as an array of rich-text runs each carrying its own annotations block, and every user reference as a nested object. A twenty-five row database query is tens of kilobytes of structure around a few hundred bytes of meaning. Hand-writing the surface is what makes the projections possible, and the projections are the point. -## Authentication +Use `surface: "mcp"` for Notion's wider official capabilities, including +connected-source search, attachments, saved views, Notion Skills, agents, and +asynchronous sessions. Tool names and schemas come from the live server. +Connecta preserves them and only fills in release-reviewed safety annotations +when Notion is silent. OAuth is the hosted server's authentication contract. +Do not apply the hand-written REST schemas to similarly named MCP tools. The +MCP interface accepts `callAdmission` for an operator-supplied runtime policy; +it does not assume the REST interface's endpoint budget describes MCP traffic. + +## API authentication One operator-managed credential: an internal integration token from [notion.so/profile/integrations](https://www.notion.so/profile/integrations). @@ -52,6 +73,10 @@ Two Notion-specific facts decide whether a working token is enough: cheapest call that proves a token is live — and reports the workspace it authenticated into. +The MCP interface uses Notion OAuth instead of the integration-token form. An +`auth_required` failure means the grant is absent or expired and must be +completed again through `authorize_connector`. + ## The pinned API version The connection pins `Notion-Version: 2026-03-11` and offers no override. That @@ -69,7 +94,10 @@ type's payload rather than switching exhaustively. A property type that ships after this release degrades to its raw value, and a block type that does keeps its payload under `raw`; neither vanishes. -## Tools +The remaining sections document the hand-written API interface. MCP tool +arguments and results are intentionally read from the live server instead. + +## API tools Ten reads, all annotated `readOnlyHint: true`: @@ -229,13 +257,14 @@ Cursors are opaque. Notion's own versioning page is explicit that they may change in length, format, and structure at any time and must be passed back verbatim — never parsed, validated, or constructed. -## What this connection does not do +## What the API interface does not do No file uploads, no database or data-source creation, no schema editing, no block updates or deletes, no page moves. Those are all real Notion endpoints and all deliberately absent: this is a deliberate tool surface, not a mirror of -the API. Anything missing is reachable through a custom `api()` connector -beside this one, which remains a first-class path. +the API. Some are present on Notion's hosted MCP interface. Anything still +missing is reachable through a custom `api()` connector beside this one, which +remains a first-class path. The 2026-03-11 contract also offers more fields on create and update. They were reviewed after the 0.17.0 drift check and remain deliberately absent: @@ -263,6 +292,14 @@ The usage guide says it too, because an agent that assumes a hatch exists spends a search proving it does not: absent from the tool list means absent from this connection, not hidden behind a generic call. +## Contract checks + +`npm run providers:check` compares the 14 fixed REST endpoints with Notion's +published OpenAPI document and the 34 MCP names, endpoint, and OAuth support +with Notion's official pages. It needs no Notion credential. The MCP schemas +are not vendored or reconstructed: the live `tools/list` response remains the +contract agents receive. + ## Conventions This connection is audited against diff --git a/documentation/operations.md b/documentation/operations.md index 88bfde30..04fd63c6 100644 --- a/documentation/operations.md +++ b/documentation/operations.md @@ -194,10 +194,15 @@ Two more runners are deliberately outside `check`: - `npm run test:browser` — Playwright against a real headless Chromium (`npm run test:browser:install` once). It covers the embedded bundle without adding a browser download to the CI release check. -- `npm run drift:check` — the maintainer-run provider drift check, with local - provider credentials exported. No credential goes near CI and nothing files - itself; findings are read by a human and become issues +- `npm run drift:check` — the lower-level maintainer provider contract check. + It reads public MCP references and OpenAPI documents only. No provider + credential is read and nothing files itself; findings are read by a human + and become issues ([provider conventions](./provider-conventions.md#the-maintainer-run-drift-check)). +- `npm run providers:check` — the normal provider check across every maintained + provider: official MCP documentation plus the OpenAPI contracts for + hand-written HTTP connections. It uses the network, so it stays outside the + deterministic `check` chain. - `npm run load:admission` — the opt-in capacity matrix and soak ([request admission](./request-admission.md#measuring-capacity)). @@ -234,7 +239,7 @@ in. | `catalog.test.ts` | lexical ranking and the compact schema renderer — `const`, `allOf` beside siblings, `$ref`, the depth limit, per-schema caching, and 2020-12 keyword compatibility | | `clerk.test.ts` | protected-resource metadata, the browser sign-in config, OAuth and session tokens, cached best-effort activity labels with their caps, the hand-applied `azp` rejection, and the `allowedDomains` allowlist including every lookalike that must not be repaired into a match | | `cloudflare-access-auth.test.ts` | trusted `ctx.access` human and service identities, absent/error fail-closed behavior, service-token MCP admission without operator mutation, human same-origin mutation, and the Clerk-to-ambient shell switch | -| `cloudflare-provider.test.ts` | `cloudflare()` construction, tool surface, current R2 and KV jurisdictions, useful output declarations, request building, projections including additive provider fields, typed failures, and credential test | +| `cloudflare-provider.test.ts` | `cloudflare()` API and MCP construction, the code-mode safety manifest, API tool surface, current R2 and KV jurisdictions, useful output declarations, request building, projections including additive provider fields, typed failures, and credential test | | `code-first-surface.test.ts` | the seven-tool surface itself — an executor required, every removed option and top-level tool refused, compact always-loaded routing pinned below 1,000 characters, complete on-demand usage served, and `connecta.ui` findable before connector search | | `codemode-compat.test.ts` | the `Executor` seam staying structurally compatible with `@cloudflare/codemode`'s `DynamicWorkerExecutor`, enforced by `tsc` | | `config.test.ts` | the grouped `ConnectaConfig` boundary — each group forwarding to its internals, malformed admission bounds failing construction, and unknown own-properties rejected by their complete path before construction does work | @@ -254,7 +259,7 @@ in. | `meta-tools-search.test.ts` | registry-backed discovery: bounded search with page and address maxima, compact and JSON schemas with constraints, typed describe recovery and suggestions, and structured-result compatibility | | `meta-tools.test.ts` | the remaining registry-backed meta-tools: the complete on-demand usage skill, connector-guide selection and summary bounds, stored-credential drift, catalog health, authorization, probe timeouts, and unavailable or unknown browse recovery | | `mixpanel-provider.test.ts` | the Mixpanel proxy, its conditional-input guide, destructive metadata fill, and complete 64-tool schema-digest manifest | -| `notion-provider.test.ts` | Notion's deliberate tool surface, including declined expanded page inputs, request construction, lean projections, both pagination conventions, error mapping, and writes | +| `notion-provider.test.ts` | Notion's API and MCP construction, the hosted safety manifest and drift behavior, the deliberate REST surface including declined expanded page inputs, request construction, lean projections, both pagination conventions, error mapping, and writes | | `operator-boundary.test.ts` | the operator row of the decisions table, after every mutation route: authentication material managed without moving a declared structure, and the one honest exception — a credential write making a remote catalog appear, which is discovery arriving, not an operator editing the deployment | | `operator-store.test.ts` | `src/operator-ui/app/store.ts` against a fake browser: the Clerk listener, ambient Access requests without a browser-readable token, `gate()`, the generation fence, and the request path | | `provider-conventions.test.ts` | the conventions a test can hold: hand-written providers refusing schemas they cannot enforce (H5), their compact discovery schemas staying complete (H7), Cloudflare stating its second pagination convention in the schema (H10), and Notion saying it has no escape hatch (H14) | @@ -274,7 +279,7 @@ in. | `ui-credentials.test.ts` | credential-management routes: save, test, delete, validation, authentication, same-origin checks, and multi-field credential shapes | | `ui.test.ts` | the server shell and remaining `/ui/*` routes: gated `/ui/data` with broken-connector isolation and registry-owned catalog-observation containment, plus the URL safety gates | | `validate.test.ts` | `validateToolInput()` — a returned (not thrown) `invalid_args` naming the path, `additionalProperties: false` enforcement, per-schema validator caching, and an unusable schema passed through with one warning | -| `vercel-provider.test.ts` | `vercel()` construction, team scoping, project and deployment projections, finite build and runtime logs, value-safe environment variables, domains, lifecycle writes, REST hatches, typed failures, and credential test | +| `vercel-provider.test.ts` | `vercel()` API and MCP construction, MCP inventory classification, team scoping, project and deployment projections, finite build and runtime logs, value-safe environment variables, domains, lifecycle writes, REST hatches, typed failures, and credential test | ### Node-bound (`NODE_ONLY_SUITES`) @@ -286,7 +291,7 @@ justification for *not* re-running it in workerd, so "it was easier" is not one. | `deployment-shapes.test.ts` | the Worker as the only example with a loader-only sandbox, its agent instructions and setup guide pinning Claude and both ChatGPT Managed OAuth callback forms, one Node template that is also its own container, the same source running locally and in the container, the Node template's pinned esbuild install-script approval, the full operator surface in both, a template that cannot start on its own `.env.example`, a Worker README naming every optional peer its entrypoint imports, and the initializer's `.gitignore` staying in step | walks the template and example trees with Node filesystem APIs | | `doc-links.test.ts` | the documentation checker itself — local file and fragment resolution, repository URLs resolved back to the checkout, duplicate heading slugs, fenced-code exclusion, and useful failures | spawns the Node checker against filesystem fixtures | | `doctor-cli.test.ts` | `connecta doctor`'s executor line and credentials end to end — the sandbox the deployment reports is the one named, an unidentifiable executor gets an executor-neutral line, a hostile name is bounded, and a complete Cloudflare Access service-token pair is accepted while a partial pair is refused | spawns the CLI against a Node HTTP deployment over real sockets | -| `drift-check.test.ts` | the maintainer drift checker — hosted-provider credential framing, recorded touched endpoints, a quiet revision bump, clear failures for an unavailable spec/manifest/credential, `$ref` traversal, and one well-formed row per endpoint | spawns the Node checker against filesystem fixtures | +| `drift-check.test.ts` | the credential-free maintainer drift checker: recorded touched endpoints, heading, table, and inline MCP inventories, setup-only providers, live-schema ownership, a quiet revision bump, clear failures for unavailable inputs, `$ref` traversal, and one well-formed row per endpoint | spawns the checker against filesystem fixtures | | `file-storage.test.ts` | `fileStorage()` across instances, logical TTL plus physical pruning without clobbering a newer value, and corrupt-file quarantine | exercises the Node filesystem storage adapter | | `guest-api-contract-quickjs.test.ts` | the shared guest-contract cases on the real QuickJS executor, including identical caught failure codes and inline describe recovery, its exact absent globals, and blocked runtime imports | runs the contract cases on the Node QuickJS executor | | `node.test.ts` | the `listen()` adapter propagating an HTTP client disconnect through the Web `Request` and the MCP handler into a program's connector call, releasing both admission permits | exercises the Node HTTP adapter over real TCP sockets | diff --git a/documentation/provider-conventions.md b/documentation/provider-conventions.md index c22db2a4..2cf46d69 100644 --- a/documentation/provider-conventions.md +++ b/documentation/provider-conventions.md @@ -7,11 +7,13 @@ writes the judgment down so it can be argued with, audited, and reused. There are two genuinely different provider shapes, and one convention set cannot honestly cover both: -- **Hand-written HTTP providers** — `api()` surfaces where Connecta owns every - tool name, schema, projection, and error. Today: Cloudflare, Notion, Vercel. -- **Hosted-MCP proxies** — `remoteMcp()` wrappers around a server somebody else +- **Hand-written HTTP providers**: `api()` surfaces where Connecta owns every + tool name, schema, projection, and error. Today: Cloudflare, Notion, and the + Vercel API surface. +- **Hosted-MCP proxies**: `remoteMcp()` wrappers around a server somebody else operates, where the names, schemas, results, and error prose arrive as they - are. Today: Linear, Stripe, Mixpanel, RevenueCat. + are. Today: Cloudflare, Linear, Stripe, Mixpanel, Notion, RevenueCat, and + Vercel MCP interfaces. The governing principle for every convention below is the same: **keep the model that interacts with connecta as efficient as possible.** A convention @@ -85,8 +87,8 @@ other source with no description or no `inputSchema`. ## Hand-written HTTP providers Connecta owns the whole surface here, which means every miss is ours. These -apply to `api()`-based prebuilt connections (Cloudflare, Notion, Vercel) and are the -bar any future one is written to. +apply to `api()`-based prebuilt connections (Cloudflare, Notion, and Vercel's +API surface) and are the bar any future one is written to. None of them asks an author to re-derive transport safety. URL confinement, query and body construction, `ctx.signal`, redirect refusal, credential @@ -513,15 +515,12 @@ credential probe, and no automatic issue filing. Proactive credential liveness stays removed ([#179](https://github.com/zackbart/connecta/issues/179)); this is the shape that does not become it. -**What a manifest holds.** Every tool name a release reviewed, the verdict it -reviewed it as (`read-only`, `additive`, `destructive`), and — where a release -actually read them — a digest of that tool's input and output schemas. Today -three of the four proxies ship names and verdicts and no digests, because no -release has read a live schema and written it down, and an invented digest reports a -change that never happened. `npm run drift:check -- --record` reads them from a -live catalog and prints the block a release pastes in; until a release does, -a manifest without digests counts no schema changes, which is the honest answer -rather than a silent zero. +**What a manifest holds.** Every tool name a release reviewed and its verdict +(`read-only`, `additive`, or `destructive`). A manifest may also carry a schema +digest from a prior review, but that digest is runtime drift evidence, not a +schema Connecta serves. The credential-free provider check neither requires nor +updates schema digests. The live `tools/list` definition always remains the +agent-facing contract. **What it counts.** Four categories, and only counts: @@ -547,8 +546,9 @@ One activity event per *change* in the counts — an identical report every TTL is a heartbeat, not news — carrying the connector id and four integers. The event type has nowhere to put a tool name, a schema, an argument, a result, or downstream error prose, which is the same construction guarantee the tool-call -event makes. Which tool drifted is deliberately absent from the runtime: it is -answered by the maintainer-run check, with a live catalog in front of it. +event makes. Which account-specific tool drifted is deliberately absent from +this payload-free runtime report. The public check can name documented drift; +it cannot recover a plan-gated name the provider does not publish. **How far an observation reaches.** One runtime, and no further. The observation lives in the isolate or process that served the refresh; unlike the @@ -558,8 +558,9 @@ the instance that took the request: on Workers a `connecta doctor` run will usually land on an isolate that has served no refresh and print nothing, and behind more than one process it is a coin flip. Read an empty report as *this runtime has observed nothing*, never as *nothing drifted* — the durable record -of a finding is the activity event a sink already stored, and naming the tool -is still the maintainer-run check's job. +of a finding is the activity event a sink already stored. The public check can +name a documented change. Naming an unpublished, account-only tool is outside a +credential-free checker. **What a finding obliges.** A contradicted vetted verdict — the downstream calling a release-reviewed destructive tool `readOnlyHint: true`, or a vetted @@ -570,33 +571,43 @@ whether or not anybody noticed it arrived. ## The maintainer-run drift check -`npm run drift:check` is the other half +`npm run providers:check` is the other half ([#351](https://github.com/zackbart/connecta/issues/351)): a human at a laptop, -before a release, with local credentials and the published specifications in -front of them. It lives in +before a release, using only published specifications and documentation. It +lives in [`scripts/drift-check.mjs`](https://github.com/zackbart/connecta/blob/main/scripts/drift-check.mjs) and ships nowhere — `scripts/` is outside the package, no runtime module imports it, and nothing it reads becomes a runtime input. -**Hosted-MCP catalogs.** `--hosted` lists each proxy's live catalog with the -maintainer's own key and diffs it against the same `vettedCatalog()` manifest -the connector classifies from, reporting tools *by name*: added, no longer -served, annotation conflicts with what the downstream actually claimed, and — -once a manifest records schema digests — which tool's schemas moved. The names -live here rather than in the runtime because the runtime's counts are -payload-free by construction, and a name has no reader there anyway. It then -compares its own totals against `detectCatalogDrift()`: two readings of one -manifest that disagree mean one of them is lying, which is worth failing over. -One credential per provider comes from the environment — -`CONNECTA_DRIFT_LINEAR_KEY`, `CONNECTA_DRIFT_STRIPE_KEY`, -`CONNECTA_DRIFT_MIXPANEL_KEY`, `CONNECTA_DRIFT_REVENUECAT_KEY` — and a missing -or dead one stops the run with a message naming it rather than reporting an -empty catalog as mass removal. Linear, bare Stripe, and RevenueCat `sk_` values -use their documented bearer or Basic framing. -Mixpanel's beta service-account form is provider-specific: -`user:secret` becomes `Bearer Basic `, exactly as its MCP -documentation requires. A value that already includes whitespace is treated -as a complete Authorization value and passes through unchanged. +**Published MCP references.** `--docs` checks +the official setup page for each maintained hosted MCP connection, including +the endpoint and OAuth support. Cloudflare, Stripe, Mixpanel, Notion, +RevenueCat, and Vercel also publish structured tool inventories. The checker +compares their documented names with the same vetted manifests the wrappers +use. Linear's setup page does not enumerate tools, so its result says that only +setup metadata was checked. + +A documented addition with no classification is a finding. A classified tool +missing from public docs is printed but does not fail the run, because account +scope, staged rollout, and documentation lag can all hide a tool that an +earlier release reviewed. RevenueCat's `render-paywall-screenshot` is the odd +one out: the official inventory names it but leaves its Access column blank. +The checker reports the reviewed exception and Connecta keeps it fail-closed. + +`npm run providers:check` runs this documentation check for every maintained +hosted MCP connection and the touched-endpoint OpenAPI check for every +hand-written HTTP connection. It never reads a provider credential. The network +keeps it outside `npm run check`; a provider outage must not make the +deterministic test suite flaky. + +MCP schema handling needs a different assertion. Connecta does not vendor or +reconstruct those schemas, so there is no static MCP schema snapshot for this +command to compare. The live `tools/list` response is passed through at runtime, +and the deterministic `catalog-drift.test.ts` suite pins that the vetted wrapper +changes safety annotations only while preserving the provider input schema, +output schema, and description. A parameter table in a docs page is never +promoted into a runtime schema. The report states this directly instead of +printing a misleading authenticated-schema requirement. **Touched endpoints.** A hand-written provider is written against a published OpenAPI document and calls a few dozen of its operations, so @@ -630,13 +641,16 @@ H11's business, mapped from the status. `--record` rewrites the manifests from the documents on hand; run it when a finding has been reviewed, and read the diff before committing it. -Narrowing is checked against the half being run: `--specs --provider linear` -and `--hosted --provider notion` exit 2 rather than checking nothing and -reporting no drift, because a false green from a plausible typo is the one -failure mode a release-time exit code cannot afford. +Narrowing is checked against the part being run: `--specs --provider linear` +and `--specs --provider stripe` exit 2 rather than checking nothing and +reporting no drift. `npm run providers:check -- --provider vercel` checks the +REST OpenAPI contract and public MCP documentation without credentials. +`npm run drift:check` is the lower-level equivalent with the same +credential-free default plus fixture and recording flags. -**What it never does.** No downstream credential reaches CI. No scheduled job, -no background traffic in a deployment, no automatic issue filing. A finding is +**What it never does.** No provider credential is read at all. There is no +scheduled job, background traffic in a deployment, or automatic issue filing. +Public docs do not generate tools or replace the live MCP catalog. A finding is read by a human and becomes a GitHub issue they wrote, because the decision a finding needs — the provider moved this endpoint, or connecta has to stop calling it — is not one a diff can make. Published specifications remain drift diff --git a/documentation/revenuecat.md b/documentation/revenuecat.md index 44c8ee97..40826bd3 100644 --- a/documentation/revenuecat.md +++ b/documentation/revenuecat.md @@ -218,13 +218,10 @@ benchmarks, experiments, virtual currencies, and the account-billing tools are the usual absentees — so search this connector for what it actually exposes rather than assuming a documented tool is here. -**No schema digests are recorded.** No release has read RevenueCat's live -schemas and written them down; that needs a live project and a maintainer's own -key. The manifest therefore ships names and verdicts only, and the drift check -honestly counts zero schema changes rather than reporting an invented one. -`npm run drift:check -- --record` reads them from a live catalog and prints the -block a release pastes in -([#351](https://github.com/zackbart/connecta/issues/351)). +**No schemas are vendored.** The manifest ships names and safety verdicts only. +The live `tools/list` response remains the schema agents receive; Connecta does +not replace it with a snapshot or require a maintainer credential to validate +one. ## Rate limits @@ -287,12 +284,19 @@ still needs restrained use. additions that catalog serves, including `get-refund-request-preferences`, but cannot prove a globally documented tool was removed. The manifest stays a superset because plan, platform, and credential scope hide tools. -- **No complete schema set has been recorded**, which is why the manifest - carries no digests. The review read the new live schemas, but its scoped - catalog omitted many classified writes. +- **No complete schema set is vendored.** The live review read the new schemas, + but its scoped catalog omitted many classified writes. Runtime schemas still + come from the server. - **Whether `render-paywall-screenshot` mutates anything.** It has no access column, and guessing is exactly what P5 exists to prevent. +`npm run drift:check -- --docs --provider revenuecat` checks the official setup +page and its 105-row tool reference without a credential. It compares the +documented names with the release-reviewed classifications. The screenshot +tool's blank Access column is a manually reviewed exception: the checker +reports it separately and Connecta keeps it fail-closed. The check reads names, +not live schemas or machine-interpreted access verdicts. + ## Conventions This connection is audited against diff --git a/documentation/stripe.md b/documentation/stripe.md index e5f11def..f8cd6aba 100644 --- a/documentation/stripe.md +++ b/documentation/stripe.md @@ -247,6 +247,13 @@ about rather than read: tools' arguments is not documented. The guide states them because an agent that ignores them is wrong either way. +`npm run drift:check -- --docs --provider stripe` compares Stripe's official +16-tool table with the vetted manifest and checks the hosted endpoint plus OAuth +support. `list_available_accounts_or_orgs` and `manage_stripe_accounts` remain +classified from an authenticated review even though the current public table +does not list them. The command prints that discrepancy without treating docs +silence as proof that a live account no longer serves them. + ## Conventions This connection is audited against diff --git a/documentation/upgrading.md b/documentation/upgrading.md index 5b96501d..6ccddb04 100644 --- a/documentation/upgrading.md +++ b/documentation/upgrading.md @@ -57,7 +57,7 @@ exist so far: | --- | --- | --- | | **pre-template** | before 0.10.2 | no `connecta init` existed; hand-written, or copied from the retired `examples/node` | | **A** | 0.10.2 – 0.15.1 | `.env.example`, `.gitignore`, `AGENTS.md`, `CLAUDE.md`, `README.md`, `package.json`, `src/index.ts`, `tsconfig.json` | -| **B** | 0.16.0 – 0.22.1 | adds `.dockerignore`, `Dockerfile`, `docker-compose.yml`, and `src/file-activity.ts`; `src/index.ts` grows the four commented operator blocks; `.env.example` ships `CONNECTA_TOKEN=` empty | +| **B** | 0.16.0 – 0.22.2 | adds `.dockerignore`, `Dockerfile`, `docker-compose.yml`, and `src/file-activity.ts`; `src/index.ts` grows the four commented operator blocks; `.env.example` ships `CONNECTA_TOKEN=` empty | Generation A is a decade in template years and identifying it precisely does not matter, because you are about to reconstruct it exactly rather than guess @@ -106,7 +106,7 @@ know what to preserve, once to know what to re-verify at the end. ### Bump the pin and install ```sh -npm pkg set dependencies.@zackbart/connecta=0.22.1 +npm pkg set dependencies.@zackbart/connecta=0.22.2 npm install ``` @@ -130,7 +130,7 @@ Generate the *current* template beside the base you already made, into the same `$SCRATCH`: ```sh -(cd "$SCRATCH" && npx @zackbart/connecta@0.22.1 init current) +(cd "$SCRATCH" && npx @zackbart/connecta@0.22.2 init current) ``` You now have a three-way merge with a real base: `$SCRATCH/base` is what this @@ -186,7 +186,7 @@ A deployment older than 0.10.2 has no base to diff against. Do not try to manufacture one. Instead: 1. `SCRATCH=$(mktemp -d)`, then - `(cd "$SCRATCH" && npx @zackbart/connecta@0.22.1 init current)` — there is no + `(cd "$SCRATCH" && npx @zackbart/connecta@0.22.2 init current)` — there is no `base` leg here, only the current template to read from. 2. Copy `$SCRATCH/current` into the deployment file by file, **skipping `src/index.ts`**. @@ -207,7 +207,7 @@ first, so cross them bottom-up: start at the oldest one still above this deployment's pin and work back up the page, because each boundary assumes the older ones are already done. -### 0.21.2 → 0.22.1 +### 0.21.2 → 0.22.2 Connector and user policy remain config-as-code. If `identity.connectorAccess` is configured, every interactive human may now manage the authentication of diff --git a/documentation/vercel.md b/documentation/vercel.md index 6980e488..0160b8d3 100644 --- a/documentation/vercel.md +++ b/documentation/vercel.md @@ -1,44 +1,58 @@ # Vercel Import `vercel()` independently from `@zackbart/connecta/providers/vercel`. -It is a hand-written `api()` connection over Vercel's public REST API. The -connection owns 18 named operations and three provider-relative REST hatches. -It adds no provider dependency, imports no `node:` builtin, and is not reachable -from Connecta's root entry. +It offers two explicit provider surfaces: + +| `surface` | Contract owner | Authentication | Best fit | +| --- | --- | --- | --- | +| `"api"` or omitted | Connecta's hand-written schemas and projections | Vercel access token | Stable named operations, guarded REST hatches, environment variables, domains, and deployment lifecycle | +| `"mcp"` | Vercel's live MCP catalog and schemas | OAuth | Vercel's newest agent tools, documentation search, observability, purchases, toolbar comments, and direct deployment | + +The choice belongs to deployment configuration. It never changes during an +agent run, and neither surface receives different Connecta privileges. Use two +connector ids when a deployment needs both. ```ts import { vercel } from "@zackbart/connecta/providers/vercel"; -const hosting = vercel("hosting", { +const hostingApi = vercel("hosting_api", { + surface: "api", purpose: "Production web applications for the product team", teamId: "team_1a2b3c4d5e6f7g8h9i0j1k2l", }); + +const hostingMcp = vercel("hosting_mcp", { + surface: "mcp", + purpose: "Production diagnosis and Vercel-native agent workflows", +}); ``` +Omitting `surface` still selects `"api"`, so existing configurations do not +change behavior. + +## API surface + +The API surface is a hand-written `api()` connection over Vercel's public REST +API. It owns 18 named operations and three provider-relative REST hatches. It +adds no provider dependency, imports no `node:` builtin, and is not reachable +from Connecta's root entry. + The deployment stores one Vercel access token in Connecta's credential vault. Create the token in Vercel Account Settings under Tokens. Scope it to the personal account or team this connection needs and give it an expiration date. The operator UI's Test action calls `GET /v2/user` and reports the authenticated username, email, name, or id. Connecta never probes it in the background. -## Why this uses REST instead of Vercel MCP - -Vercel MCP provides useful project, deployment, and log reads, but it does not -cover the public API. This connection keeps those common reads and adds project -domains, value-safe environment-variable management, deployment promotion and -deletion, and direct access to versioned REST endpoints. The three hatches mean -a newly published Vercel endpoint does not require a Connecta release before an -agent can use it. - -This is still authored rather than generated. No OpenAPI document creates tools -at runtime. The named operations are reviewed, projected, classified, and -tested by hand. The published OpenAPI document is used only by -`npm run drift:check` to compare the 19 named endpoints this connection calls. +This surface is authored rather than generated. No OpenAPI document creates +tools at runtime. The named operations are reviewed, projected, classified, +and tested by hand. The published OpenAPI document is used only by +`npm run drift:check` to compare the 19 named endpoints this surface calls. ## Configuration ```ts vercel("hosting", { + surface: "api", title: "Production hosting", authScope: "shared", purpose: "Customer-facing sites owned by Platform", @@ -69,7 +83,35 @@ from replacing `Authorization`, passes `ctx.signal`, and stops reading at 8 MiB. The runtime-log read also returns at most 500 rows and stops a stream that stays open past 10 seconds. HTTP is accepted only for a loopback test double. -## Named tools +## MCP surface + +The MCP surface proxies Vercel's official hosted endpoint at +`https://mcp.vercel.com` and uses OAuth. Connecta does not copy Vercel's schemas +into this repository. Each complete `tools/list` response is preserved as the +live catalog, cached under the connector id, and invalidated when authorization +changes. + +Connecta does vendor one thing: the 32 tool names and their release-reviewed +read, additive-write, or destructive-write verdicts from Vercel's +[official MCP tool reference](https://vercel.com/docs/agent-resources/vercel-mcp/tools). +Those verdicts fill in missing annotations and keep reviewed destructive tools +off the read-only execution path. A new tool remains fail-closed until a release +classifies it. The provider check needs no schema digest because Connecta does +not serve a schema snapshot. + +This separation matters when diagnosing a schema mismatch: + +- On `surface: "api"`, the schema in `src/providers/vercel.ts` is Connecta's + contract and its tests must prove the behavior. +- On `surface: "mcp"`, Vercel's live `tools/list` response is the contract. + Connecta does not rewrite it. `search_tools` with compact schemas is a lossy + preview; use `describe` with JSON format when the exact schema matters. + +[integrations.sh's Vercel record](https://integrations.sh/vercel.com/) remains +useful discovery evidence. The contract check uses Vercel's official setup and +tool pages instead. Neither source replaces the live MCP schemas. + +## API named tools | Tool | What it does | | --- | --- | @@ -188,7 +230,12 @@ the REST hatches share one guarded transport. The trade is API drift, handled explicitly. `scripts/drift/vercel-endpoints.json` records the method, versioned path, specification revision, and request/response -digest for every fixed endpoint. Before a release, `npm run drift:check` compares -those rows with Vercel's published OpenAPI document at -`https://openapi.vercel.sh/`. The hatches are intentionally absent from that -list because their endpoint is chosen by deployment code at call time. +digest for every fixed endpoint. Before a release, +`npm run providers:check -- --provider vercel` compares those rows with +Vercel's published OpenAPI document at `https://openapi.vercel.sh/`, compares +the vendored MCP inventory with Vercel's official tool reference, and checks +the endpoint and OAuth support in Vercel's setup page. It states that live MCP +schemas are not vendored and that the live `tools/list` response remains their +runtime authority. The API hatches are absent from the endpoint manifest +because deployment code chooses their path at call time. No Vercel credential +is needed for this check. diff --git a/package-lock.json b/package-lock.json index e2dc1ffe..5221d9d0 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "@zackbart/connecta", - "version": "0.22.1", + "version": "0.22.2", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@zackbart/connecta", - "version": "0.22.1", + "version": "0.22.2", "license": "MIT", "dependencies": { "@cfworker/json-schema": "^4.1.1", diff --git a/package.json b/package.json index 35ab5a85..5c53704e 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@zackbart/connecta", - "version": "0.22.1", + "version": "0.22.2", "type": "module", "sideEffects": false, "description": "One MCP to rule them all — a single MCP endpoint aggregating many downstream connectors behind a code-first surface of seven meta-tools.", @@ -114,6 +114,7 @@ "check:security": "npm audit --omit=dev --audit-level=moderate", "check:package": "node scripts/check-package.mjs", "load:admission": "npm run build && node eval/admission-load.mjs", + "providers:check": "tsx scripts/drift-check.mjs --specs --docs", "drift:check": "tsx scripts/drift-check.mjs", "check": "npm run check:docs && npm run check:operator-ui && npm run check:lint && npm run check:unused && npm run typecheck && npm run test && npm run build && npm run check:examples", "release:check": "npm run check && npm run check:security && npm run check:package", diff --git a/records/provider-audit.md b/records/provider-audit.md index 1edf5c47..9db3a81d 100644 --- a/records/provider-audit.md +++ b/records/provider-audit.md @@ -1,11 +1,11 @@ # Provider audit [`provider-conventions.md`](../documentation/provider-conventions.md) wrote the bar down. This -document runs it against the seven maintained prebuilt connections and returns a -verdict for every applicable convention: **meets**, **misses** (with the fix), -or **n/a** (with the reason). A convention is never quietly skipped, and an -accepted miss is recorded as a provider-specific exception with its argument -rather than left blank. +document runs it against the seven maintained providers and their ten selectable +interfaces, then returns a verdict for every applicable convention: **meets**, +**misses** (with the fix), or **n/a** (with the reason). A convention is never +quietly skipped, and an accepted miss is recorded as a provider-specific +exception with its argument rather than left blank. Hand-written HTTP providers are audited against H1–H14; hosted-MCP proxies against P1–P13. Applying a hand-written convention to a proxy is a category @@ -120,6 +120,29 @@ v9. The reviewed 19 fixed operations now live in `scripts/drift/vercel-endpoints.json`; the dynamic hatches do not pretend to have a fixed endpoint manifest. +## Cloudflare, Notion, and Vercel — hosted-MCP alternatives + +These three providers also expose an explicitly selected hosted-MCP interface. +They share the proxy boundary without pretending their catalogs are the same: +the provider owns every name, description, schema, and result, while Connecta +adds only the release-reviewed safety verdict and routing guide. + +| Convention | Verdict | Notes | +| --- | --- | --- | +| P1 add, never rewrite | meets | `withVettedCatalog()` preserves every live tool field and changes only missing or contradicted safety annotations; deterministic coverage pins the description and both schemas by reference | +| P2 identity | meets | all three require a non-empty `purpose`; `instructions` are appended to the guide and cannot alter classification | +| P3 routing fact | meets | default titles name the provider and `(MCP)` interface; each guide opens with the deployment purpose and official-interface boundary | +| P4 endpoint default | n/a, recorded | each provider publishes one whole-interface endpoint, so there is no regional or access-mode endpoint choice to default; selecting `surface: "mcp"` is itself explicit | +| P5 classification | meets | every officially documented name is reviewed; unlisted names fail closed, and Vercel URL fetching is destructive because an application GET route may execute stateful code | +| P6 catalog varies | meets | the guides say permissions, account features, plans, clients, and provider rollout can change the live catalog independently of Connecta | +| P7 reduction advice | meets | all three guides tell agents to narrow at the provider call and reduce again inside `execute_code`; the summaries identify the cross-tool routing facts | +| P8 identity resolution | meets | Cloudflare searches the OpenAPI document before execute; Notion and Vercel name the discovery tools and opaque ids their subsequent calls require | +| P9 authentication | meets | all require HTTPS and default to OAuth; Cloudflare additionally accepts a scoped bearer token for a headless deployment | +| P10 no credential test | meets | construction performs no downstream request and the wrappers expose no credential-test hook | +| P11 transport vs tool error | meets | inherited from `remoteMcp()`; provider argument, permission, plan, and rollout errors remain provider-owned prose | +| P12 admission budget | meets | no guessed provider-wide quota is hardcoded; each MCP option accepts an operator-supplied per-runtime admission policy | +| P13 drift visible | meets | module-level vetted catalogs feed runtime count-only drift and the credential-free public-docs check; live `tools/list` remains the schema authority | + ## Linear — hosted-MCP proxy | Convention | Verdict | Notes | @@ -181,8 +204,8 @@ Linear's reasoning. ## RevenueCat — hosted-MCP proxy Written after the conventions existed, so it has no misses to record, only two -places where the honest answer departs from the obvious one. Ninety-six -documented tools, ninety-five classified, one deliberately not. +places where the honest answer departs from the obvious one. One hundred five +documented tools, 104 classified, one deliberately not. | Convention | Verdict | Notes | | --- | --- | --- | @@ -190,7 +213,7 @@ documented tools, ninety-five classified, one deliberately not. | P2 identity | meets | required `purpose` (blank throws), `instructions` appended under `## Project instructions`, and appended text cannot reach the classification | | P3 routing fact | meets, with the fact split in two | the routing fact is scope, and it has two halves. The *shape* — one project versus every project the account can reach — is knowable at construction and rides the default title (`RevenueCat (single project)` versus `RevenueCat`). *Which* project a key opens is not knowable without calling something, which P10 forbids, so it rides the guide's first line and the declared summary, built from the operator's `purpose`. That makes this the one maintained proxy with a purpose-bearing summary rather than a static one, and the reason is P3's own cost: two `sk_` connectors share a title, an endpoint, and a catalog, so a static summary would leave them indistinguishable in the only field search returns | | P4 endpoint default | n/a — one endpoint, and the scope rides the credential | RevenueCat publishes a single MCP endpoint, so there is nothing to select between. The scope difference comes from the credential shape itself, which the constructor reads rather than asks for: `auth.type === "headers"` *is* the single-project declaration. There is no mode to default and no mode to contradict, so the P4 machinery Stripe needs has nothing to do here | -| P5 classification | meets | 51 reads, 15 additive writes, 29 destructive writes named; `get-refund-request-preferences` is the added read. `render-paywall-screenshot` remains unclassified because RevenueCat's reference gives it no access column; its current explicit read-only annotation is preserved, and silence fails closed. Nine borderline verdicts are argued beside the rows they decide, and asserted in the suite so a silent flip fails | +| P5 classification | meets | 51 reads, 17 additive writes, and 36 destructive writes named. `render-paywall-screenshot` remains unclassified because RevenueCat's reference gives it no access column; its current explicit read-only annotation is preserved, and silence fails closed. Borderline verdicts are argued beside the rows they decide and asserted in the suite so a silent flip fails | | P6 catalog varies | meets | the guide names paywall AI editing, benchmarks, experiments, virtual currencies, and account billing as the plan-, platform-, and beta-gated areas where absence is expected, and separately names the unclassified tool so its approval prompt does not read as a bug | | P7 reduction advice | meets | structured guide, declared summary, cursor-then-reduce advice aimed at the two objects that are actually large here (customers and their event history). `required` stays unset: the project-resolution sequence is worth reading before a run, not before every call | | P8 identity resolution | meets | the guide names the whole chain — `list-projects` for the `project_id` every project-scoped call takes, then `list-apps`, `list-products`, `list-entitlements`, `list-offerings`, `list-paywalls`, `list-audiences`, and `list-customers` for the ids their `get-`, `update-`, `archive-`, and `delete-` counterparts expect — and says a plausible-looking id belongs to another project or to nobody. For OAuth it also says to stop and ask when more than one project fits | @@ -198,22 +221,26 @@ documented tools, ninety-five classified, one deliberately not. | P10 no credential test | meets | no `credential`, `testCredential`, or `testCredentials`. This is also where the constructor's most tempting option was refused: a `project?: string` checked against `list-projects` at construction is a credential test wearing a configuration hat, so the operator's stated purpose carries the claim and the agent confirms it on first use. There is no recognizable-credential contradiction to throw on either — an `sk_` key encodes no project — so the construction-time half of P10 has nothing to check here, exactly as it has nothing to check for Mixpanel's region | | P11 transport vs tool error | meets | inherited whole from `remoteMcp()`; the wrapper adds no error handling and reads no downstream prose. The guide says a rejected argument, a permission gap, and a plan restriction all arrive in RevenueCat's own words | | P12 admission budget | meets, by declining a number that exists | RevenueCat does publish limits, which is why this row needed an argument rather than a shrug. It meters per domain — 480/min for customer information, virtual currencies, and refunds; 60 for project configuration and audiences; 25 for charts and metrics — and a `ConnectorCallAdmissionPolicy` carries exactly one rule. Picking 25 throttles a customer read loop to a nineteenth of its allowance; picking 480 leaves a chart sweep unprotected; neither is the provider's limit. The metering scope repeats the point: developer-level keys are metered per developer, which a per-runtime counter cannot approximate. So the guide states RevenueCat's own numbers and the `429` / `Retry-After` / `backoff_ms` signals, and `callAdmission` stays an operator option with a documented example | -| P13 drift visible | meets | both lists are module-level constants in one file and *are* the manifest the wrapper classifies from, compared against the live catalog on every refresh ([#343](https://github.com/zackbart/connecta/issues/343)). The maintainer-run check accepts `revenuecat` with `CONNECTA_DRIFT_REVENUECAT_KEY`. No schema digests are recorded, and the manifest says so rather than shipping invented ones | +| P13 drift visible | meets | both lists are module-level constants in one file and *are* the manifest the wrapper classifies from, compared against the live catalog on every refresh ([#343](https://github.com/zackbart/connecta/issues/343)). The credential-free provider check compares them with RevenueCat's official 105-row reference. No schema is vendored; the live definition remains the one agents receive | ## Scoreboard -| Provider | Meets | Missed and fixed | Recorded exception | Open | +| Provider interface | Meets | Missed and fixed | Recorded exception | Open | | --- | --- | --- | --- | --- | -| Cloudflare | 9 | 5 | H5 hatch request parts | — | -| Notion | 10 | 4 | H5 exclusive parent | — | +| Cloudflare API | 9 | 5 | H5 hatch request parts | — | +| Cloudflare MCP | 12 | 0 | P4 n/a (one endpoint) | — | +| Notion API | 10 | 4 | H5 exclusive parent | — | +| Notion MCP | 12 | 0 | P4 n/a (one endpoint) | — | | Linear | 11 | 2 | P4 departs from the letter | — | | Stripe | 10 | 3 | — | — | | Mixpanel | 7 | 5 | P10 half n/a | — | | RevenueCat | 12 | 0 | P4 n/a (one endpoint); P3 met with a purpose-bearing summary | — | -| Vercel | 14 | 0 | — | — | +| Vercel API | 14 | 0 | — | — | +| Vercel MCP | 12 | 0 | P4 n/a (one endpoint) | — | -Nineteen misses, nineteen fixes, six recorded exceptions, one judgment left to -the issue that owns it. The pattern in the misses is worth naming: sixteen of +The original seven interfaces recorded nineteen misses and nineteen fixes. The +three hosted alternatives add no misses and record their one-endpoint P4 cases +instead of inventing a choice. The pattern in the original misses is worth naming: sixteen of the nineteen are a guide, a title, or a schema description failing to *say* something the implementation already did correctly. Only three changed what a provider does — Notion refusing an unevaluable schema, Linear requiring an diff --git a/scripts/drift-check.mjs b/scripts/drift-check.mjs index 18468904..18b6e128 100644 --- a/scripts/drift-check.mjs +++ b/scripts/drift-check.mjs @@ -1,23 +1,20 @@ // Maintainer tooling, not deployment runtime. Nothing here ships: `scripts/` // is outside the package `files`, and no runtime module imports it. // -// Two halves, both release-time and both human-triggered: +// Two parts, both credential-free, release-time, and human-triggered: // -// - **Hosted-MCP catalogs.** The runtime check (#343) rides a refresh a -// deployment already asked for and produces four counts *by construction* — -// it has nowhere to put a tool name, so no surface it reaches has to remember -// to strip one. That guarantee is worth keeping, so the names live here -// instead: a maintainer with local credentials in front of a live workspace -// is the one audience that can act on them. This half compares against the -// same `vettedCatalog()` manifest the connector classifies from, and then -// cross-checks its own totals against `detectCatalogDrift()` — two readings -// of one manifest that disagree would mean one of them is lying. // - **Touched endpoints.** Hand-written HTTP providers are written against a // published OpenAPI document, and only against the handful of operations they // actually call. The committed manifests in `scripts/drift/` record that // handful — method, path, the spec revision a release reviewed it at, and a // digest of the request/response contract at that revision — so a check can // report the endpoints connecta touches without reading the other 2,000. +// - **Published MCP references.** When a provider publishes a tool reference, +// compare its documented inventory and public connection metadata with the +// maintained wrapper without needing account credentials. This catches an +// unclassified tool before a live workspace is available. Remote MCP schemas +// are never vendored here: the provider's live `tools/list` response remains +// the runtime authority, and tests pin that passthrough. // // Published specifications are drift evidence and nothing else. Nothing here // generates a tool, and no runtime module reads a spec — schema ingestion stays @@ -26,7 +23,6 @@ import { createHash } from "node:crypto"; import { readFile, writeFile } from "node:fs/promises"; import { dirname, resolve as resolvePath } from "node:path"; import { fileURLToPath } from "node:url"; -import { hostedAuthorizationHeader } from "./drift/hosted-auth.mjs"; const repositoryRoot = resolvePath( dirname(fileURLToPath(import.meta.url)), @@ -34,28 +30,77 @@ const repositoryRoot = resolvePath( ); const defaultManifestDirectory = resolvePath(repositoryRoot, "scripts/drift"); -/** Hosted-MCP proxies: a live catalog, read with the maintainer's own key. */ -const HOSTED_PROVIDERS = ["linear", "stripe", "mixpanel", "revenuecat"]; /** Hand-written HTTP providers: a published specification, read as evidence. */ const SPEC_PROVIDERS = ["cloudflare", "notion", "vercel"]; - -/** Where each hosted provider's credential comes from, and what it is. */ -const HOSTED_CREDENTIALS = { +/** Hosted MCP providers with official public documentation we can read. */ +const DOCS_PROVIDERS = [ + "cloudflare", + "linear", + "stripe", + "mixpanel", + "notion", + "revenuecat", + "vercel", +]; + +const DOCUMENTED_MCP = { + cloudflare: { + setup: + "https://developers.cloudflare.com/agents/model-context-protocol/cloudflare/servers-for-cloudflare/index.md", + inventory: { + url: "https://developers.cloudflare.com/agents/model-context-protocol/cloudflare/servers-for-cloudflare/index.md", + format: "inline-calls", + start: "## Cloudflare API MCP server", + end: "### Connect to the Cloudflare API MCP server", + }, + }, linear: { - variable: "CONNECTA_DRIFT_LINEAR_KEY", - hint: "a Linear personal API key", + setup: "https://linear.app/docs/mcp.md", + inventory: undefined, }, stripe: { - variable: "CONNECTA_DRIFT_STRIPE_KEY", - hint: "a Stripe restricted API key", + setup: "https://docs.stripe.com/mcp.md", + inventory: { + url: "https://docs.stripe.com/mcp.md", + format: "table", + start: "## Tools", + end: "### Supported API methods", + }, }, mixpanel: { - variable: "CONNECTA_DRIFT_MIXPANEL_KEY", - hint: "a Mixpanel service account as user:secret", + setup: "https://docs.mixpanel.com/docs/features/mcp.md", + inventory: { + url: "https://docs.mixpanel.com/docs/features/mcp.md", + format: "table", + start: "## Available Tools", + end: "## MCP Server URLs", + }, + }, + notion: { + setup: + "https://developers.notion.com/guides/mcp/get-started-with-mcp.md", + inventory: { + url: "https://developers.notion.com/guides/mcp/mcp-supported-tools.md", + format: "inline", + prefix: "notion-", + }, }, revenuecat: { - variable: "CONNECTA_DRIFT_REVENUECAT_KEY", - hint: "a RevenueCat API v2 secret key", + setup: "https://www.revenuecat.com/docs/tools/mcp/setup.md", + inventory: { + url: "https://www.revenuecat.com/docs/tools/mcp/tools-reference.md", + format: "table", + // The reference publishes this name with a blank Access column. A + // release cannot infer read or write from its verb, so it stays closed. + acknowledgedUnclassified: new Set(["render-paywall-screenshot"]), + }, + }, + vercel: { + setup: "https://vercel.com/docs/agent-resources/vercel-mcp.md", + inventory: { + url: "https://vercel.com/docs/agent-resources/vercel-mcp/tools.md", + format: "headings", + }, }, }; @@ -84,24 +129,17 @@ function usage(message) { [ "usage: npm run drift:check -- [options]", "", - " --hosted only diff hosted-MCP catalogs (needs credentials)", " --specs only compare touched endpoints with published specs", + " --docs only compare public MCP docs and connection metadata", " --provider limit to one provider (repeatable)", " --spec = read a provider's published spec from here", + " --tool-reference =", + " read its published MCP tool reference here", + " --setup-reference =", + " read its official MCP setup documentation here", " --manifest-dir touched-endpoint manifests (default scripts/drift)", - " --record rewrite the touched-endpoint manifests from the", - " specs on hand, and print hosted schema digests", + " --record rewrite touched-endpoint manifests from the specs", " --json print the report as JSON", - "", - "Credentials are read from the environment, one per hosted provider:", - ...Object.entries(HOSTED_CREDENTIALS).map( - ([provider, { variable, hint }]) => - ` ${variable} ${hint} (${provider})`, - ), - "", - "A value containing a space is sent as the Authorization header verbatim,", - "one containing a colon as Basic credentials (Mixpanel: Bearer Basic),", - "and anything else as a Bearer token.", ].join("\n"), ); process.exit(2); @@ -109,10 +147,12 @@ function usage(message) { function parseArguments(argv) { const options = { - hosted: false, specs: false, + docs: false, providers: [], specSources: new Map(), + toolReferenceSources: new Map(), + setupReferenceSources: new Map(), manifestDirectory: defaultManifestDirectory, record: false, json: false, @@ -125,42 +165,74 @@ function parseArguments(argv) { index += 1; return value; }; - if (argument === "--hosted") options.hosted = true; - else if (argument === "--specs") options.specs = true; + if (argument === "--specs") options.specs = true; + else if (argument === "--docs") options.docs = true; else if (argument === "--record") options.record = true; else if (argument === "--json") options.json = true; else if (argument === "--provider") options.providers.push(next()); else if (argument === "--manifest-dir") options.manifestDirectory = resolvePath(next()); - else if (argument === "--spec") { + else if ( + argument === "--spec" || + argument === "--tool-reference" || + argument === "--setup-reference" + ) { const value = next(); const separator = value.indexOf("="); - if (separator < 1) usage("--spec expects ="); - options.specSources.set( - value.slice(0, separator), - value.slice(separator + 1), - ); + if (separator < 1) usage(`${argument} expects =`); + const target = + argument === "--spec" + ? options.specSources + : argument === "--tool-reference" + ? options.toolReferenceSources + : options.setupReferenceSources; + target.set(value.slice(0, separator), value.slice(separator + 1)); } else usage(`unknown argument: ${argument}`); } - // Neither half named means both — a release checks the whole surface. - if (!options.hosted && !options.specs) { - options.hosted = true; + // No part named means both: a release checks the whole provider surface. + if (!options.specs && !options.docs) { options.specs = true; + options.docs = true; + } + if (options.specSources.size > 0 && !options.specs) { + usage("--spec requires --specs when a check mode is selected explicitly"); + } + if ( + (options.toolReferenceSources.size > 0 || + options.setupReferenceSources.size > 0) && + !options.docs + ) { + usage( + "--tool-reference and --setup-reference require --docs when a check mode is selected explicitly", + ); } - const known = new Set([...HOSTED_PROVIDERS, ...SPEC_PROVIDERS]); + const known = new Set([...SPEC_PROVIDERS, ...DOCS_PROVIDERS]); // A provider only one half checks, named alongside the other half, would // narrow the run to nothing — and a check whose whole value is its exit code // must not print "no drift" for a run that looked at nothing. const selectable = new Set([ - ...(options.hosted ? HOSTED_PROVIDERS : []), ...(options.specs ? SPEC_PROVIDERS : []), + ...(options.docs ? DOCS_PROVIDERS : []), ]); - for (const provider of [...options.providers, ...options.specSources.keys()]) { + const requestedProviders = [ + ...options.providers, + ...options.specSources.keys(), + ...options.toolReferenceSources.keys(), + ...options.setupReferenceSources.keys(), + ]; + for (const provider of requestedProviders) { if (!known.has(provider)) usage(`unknown provider: ${provider}`); if (!selectable.has(provider)) { - const half = HOSTED_PROVIDERS.includes(provider) ? "--hosted" : "--specs"; + const modes = [ + ...(SPEC_PROVIDERS.includes(provider) ? ["--specs"] : []), + ...(DOCS_PROVIDERS.includes(provider) ? ["--docs"] : []), + ]; + const availability = + modes.length === 1 + ? `only checked by ${modes[0]}` + : `checked by ${modes.join(" or ")}`; usage( - `${provider} is only checked by ${half}, which this run did not select — ` + + `${provider} is ${availability}, which this run did not select. ` + "that combination would check nothing.", ); } @@ -176,209 +248,6 @@ function selected(options, providers) { /** A fatal condition a maintainer can fix, reported without a stack trace. */ class UnavailableError extends Error {} -// --------------------------------------------------------------------------- -// Hosted-MCP catalogs -// --------------------------------------------------------------------------- - -function maintainerContext() { - const store = new Map(); - return { - baseUrl: "https://drift-check.invalid", - storage: { - async get(key) { - return store.get(key) ?? null; - }, - async set(key, value) { - store.set(key, value); - }, - async delete(key) { - store.delete(key); - }, - async list(prefix) { - return [...store.keys()].filter((key) => key.startsWith(prefix)).sort(); - }, - }, - logger: { - debug() {}, - info() {}, - warn(...args) { - console.warn(...args); - }, - error(...args) { - console.error(...args); - }, - }, - }; -} - -/** - * Load the provider modules from source. - * - * They are TypeScript, so this half runs under `tsx` — which is what - * `npm run drift:check` does. The import is lazy so the specification half - * still runs on plain `node`, and the failure says which of the two happened. - */ -async function loadHostedProviders() { - try { - const [drift, remote, linear, stripe, mixpanel, revenuecat] = - await Promise.all([ - import("../src/catalog-drift.ts"), - import("../src/connectors/remote-mcp.ts"), - import("../src/providers/linear.ts"), - import("../src/providers/stripe.ts"), - import("../src/providers/mixpanel.ts"), - import("../src/providers/revenuecat.ts"), - ]); - return { - detectCatalogDrift: drift.detectCatalogDrift, - vettedSchemaDigest: drift.vettedSchemaDigest, - remoteMcp: remote.remoteMcp, - endpoints: { - // The manifest classifies the whole catalog, so the check reads the - // whole catalog: Linear's read-only endpoint would report every write - // it deliberately cannot serve as a name no longer served. - linear: linear.LINEAR_MCP_ENDPOINTS["read-write"], - stripe: stripe.STRIPE_MCP_ENDPOINT, - // Region shapes residency, not the catalog; US is the provider default. - mixpanel: mixpanel.MIXPANEL_MCP_ENDPOINTS.us, - revenuecat: revenuecat.REVENUECAT_MCP_ENDPOINT, - }, - catalogs: { - linear: linear.LINEAR_VETTED_CATALOG, - stripe: stripe.STRIPE_VETTED_CATALOG, - mixpanel: mixpanel.MIXPANEL_VETTED_CATALOG, - // Names and verdicts only: no release has read a live RevenueCat - // schema, so this manifest records no digests and the check honestly - // counts zero schema changes rather than reporting an invented one. - revenuecat: revenuecat.REVENUECAT_VETTED_CATALOG, - }, - }; - } catch (error) { - throw new UnavailableError( - "could not load the provider manifests from TypeScript source — run " + - "this half through `npm run drift:check`, which uses tsx " + - `(${error instanceof Error ? error.message : String(error)})`, - ); - } -} - -/** The downstream's own annotation, as the report prints it. */ -function downstreamClaim(tool) { - const annotations = tool.annotations ?? {}; - const claims = []; - if (annotations.readOnlyHint !== undefined) - claims.push(`readOnlyHint: ${annotations.readOnlyHint}`); - if (annotations.destructiveHint !== undefined) - claims.push(`destructiveHint: ${annotations.destructiveHint}`); - return claims.length ? claims.join(", ") : "no annotations"; -} - -/** - * Whether the downstream explicitly contradicts a vetted verdict. - * - * The same rule `detectCatalogDrift()` counts with, restated here because the - * runtime one deliberately cannot return a name. The totals are compared - * afterwards, which is what keeps this restatement from drifting on its own. - */ -function contradicts(verdict, tool) { - const annotations = tool.annotations ?? {}; - if (verdict === "read-only") { - return ( - annotations.readOnlyHint === false || annotations.destructiveHint === true - ); - } - return annotations.readOnlyHint === true; -} - -async function checkHostedProvider(provider, runtime, options) { - const credential = HOSTED_CREDENTIALS[provider]; - const secret = process.env[credential.variable]; - if (!secret || !secret.trim()) { - throw new UnavailableError( - `${credential.variable} is not set — this check needs ${credential.hint}. ` + - "Narrow the run with --provider or --specs if that is deliberate.", - ); - } - const connector = runtime.remoteMcp(`drift-${provider}`, { - url: runtime.endpoints[provider], - auth: { - type: "headers", - headers: { Authorization: hostedAuthorizationHeader(provider, secret) }, - }, - requireHttps: true, - }); - const ctx = maintainerContext(); - let tools; - try { - tools = await connector.listTools(ctx); - } catch (error) { - throw new UnavailableError( - `could not list ${provider}'s catalog with ${credential.variable}: ` + - (error instanceof Error ? error.message : String(error)), - ); - } finally { - await connector.closeScope?.(ctx); - } - - const catalog = runtime.catalogs[provider]; - const served = new Set(tools.map((tool) => tool.name)); - const added = []; - const annotationConflicts = []; - const schemaChanges = []; - const digests = {}; - for (const tool of [...tools].sort((a, b) => (a.name < b.name ? -1 : 1))) { - const record = catalog.tools.get(tool.name); - if (!record) { - added.push({ tool: tool.name, downstream: downstreamClaim(tool) }); - continue; - } - if (contradicts(record.verdict, tool)) { - annotationConflicts.push({ - tool: tool.name, - vetted: record.verdict, - downstream: downstreamClaim(tool), - }); - } - const digest = await runtime.vettedSchemaDigest(tool); - digests[tool.name] = digest; - if (record.schemaDigest !== undefined && record.schemaDigest !== digest) { - schemaChanges.push({ tool: tool.name, recorded: record.schemaDigest }); - } - } - const removed = [...catalog.tools.keys()] - .filter((name) => !served.has(name)) - .sort(); - - // Two readings of one manifest. If they disagree, one of them is wrong, and - // finding that out here is cheaper than trusting either. - const counts = await runtime.detectCatalogDrift(catalog, tools); - const mine = { - unclassifiedTools: added.length, - unservedTools: removed.length, - annotationConflicts: annotationConflicts.length, - schemaChanges: schemaChanges.length, - }; - const disagreement = Object.keys(mine).filter( - (key) => mine[key] !== counts[key], - ); - - const recorded = [...catalog.tools.values()].some( - (record) => record.schemaDigest !== undefined, - ); - return { - provider, - endpoint: runtime.endpoints[provider], - tools: tools.length, - added, - removed, - annotationConflicts, - schemaChanges, - schemaDigestsRecorded: recorded, - disagreement, - ...(options.record ? { digests } : {}), - }; -} - // --------------------------------------------------------------------------- // Touched endpoints // --------------------------------------------------------------------------- @@ -594,49 +463,197 @@ async function recordManifest(path, manifest, recorded) { } // --------------------------------------------------------------------------- -// Reporting +// Published MCP references // --------------------------------------------------------------------------- -function printHosted(result) { - const header = `${result.provider} — ${result.tools} tools at ${result.endpoint}`; - console.log(header); - for (const { tool, downstream } of result.added) { - console.log(` added ${tool} (${downstream})`); +async function loadPublished(provider, label, source) { + let text; + try { + if (/^https?:\/\//.test(source)) { + const response = await fetch(source); + if (!response.ok) { + throw new Error(`HTTP ${response.status}`); + } + text = await response.text(); + } else { + text = await readFile(resolvePath(source), "utf8"); + } + } catch (error) { + throw new UnavailableError( + `could not read ${provider}'s ${label} from ${source}: ` + + (error instanceof Error ? error.message : String(error)), + ); } - for (const tool of result.removed) { - console.log(` removed ${tool}`); + return text; +} + +async function loadDocumentedProviders() { + try { + const [cloudflare, linear, stripe, mixpanel, notion, revenuecat, vercel] = await Promise.all([ + import("../src/providers/cloudflare.ts"), + import("../src/providers/linear.ts"), + import("../src/providers/stripe.ts"), + import("../src/providers/mixpanel.ts"), + import("../src/providers/notion.ts"), + import("../src/providers/revenuecat.ts"), + import("../src/providers/vercel.ts"), + ]); + return { + cloudflare: { + endpoints: [cloudflare.CLOUDFLARE_MCP_ENDPOINT], + catalog: cloudflare.CLOUDFLARE_MCP_VETTED_CATALOG, + }, + linear: { + endpoints: [linear.LINEAR_MCP_ENDPOINTS["read-write"]], + catalog: linear.LINEAR_VETTED_CATALOG, + }, + stripe: { + endpoints: [stripe.STRIPE_MCP_ENDPOINT], + catalog: stripe.STRIPE_VETTED_CATALOG, + }, + mixpanel: { + endpoints: Object.values(mixpanel.MIXPANEL_MCP_ENDPOINTS), + catalog: mixpanel.MIXPANEL_VETTED_CATALOG, + }, + notion: { + endpoints: [notion.NOTION_MCP_ENDPOINT], + catalog: notion.NOTION_MCP_VETTED_CATALOG, + }, + revenuecat: { + endpoints: [revenuecat.REVENUECAT_MCP_ENDPOINT], + catalog: revenuecat.REVENUECAT_VETTED_CATALOG, + }, + vercel: { + endpoints: [vercel.VERCEL_MCP_ENDPOINT], + catalog: vercel.VERCEL_MCP_VETTED_CATALOG, + }, + }; + } catch (error) { + throw new UnavailableError( + "could not load documented provider contracts from TypeScript source; " + + "run this through `npm run drift:check`, which uses tsx " + + `(${error instanceof Error ? error.message : String(error)})`, + ); } - for (const { tool, vetted, downstream } of result.annotationConflicts) { - console.log(` conflict ${tool} — vetted ${vetted}, downstream ${downstream}`); +} + +function documentedSection(markdown, inventory) { + const start = inventory.start ? markdown.indexOf(inventory.start) : 0; + if (start < 0) return ""; + const afterStart = markdown.slice(start + (inventory.start?.length ?? 0)); + if (!inventory.end) return afterStart; + const end = afterStart.indexOf(inventory.end); + return end < 0 ? afterStart : afterStart.slice(0, end); +} + +/** Exact tool names from a provider's documented inventory section. */ +function documentedToolNames(markdown, inventory) { + const section = documentedSection(markdown, inventory); + const candidate = /^[A-Za-z](?:[A-Za-z0-9_-]*[A-Za-z0-9])?$/; + if (inventory.format === "headings") { + return [...section.matchAll(/^### ([^\r\n]+)$/gm)] + .map((match) => match[1].replaceAll("\\_", "_").trim()) + .filter((name) => candidate.test(name)) + .sort(); } - for (const { tool, recorded } of result.schemaChanges) { - console.log(` schema ${tool} — no longer ${recorded}`); + if (inventory.format === "inline" || inventory.format === "inline-calls") { + const names = [...section.matchAll(/`([^`]+)`/g)] + .map((match) => match[1].trim()) + .map((name) => + inventory.format === "inline-calls" ? name.replace(/\(\)$/, "") : name, + ) + .filter((name) => candidate.test(name)) + .filter((name) => + inventory.prefix === undefined ? true : name.startsWith(inventory.prefix), + ); + return [...new Set(names)].sort(); } - if (!result.schemaDigestsRecorded) { - console.log( - " schema not comparable — this manifest records no digests; " + - "rerun with --record to print the block to paste into vettedCatalog()", - ); + const names = []; + for (const line of section.split("\n")) { + if (!line.startsWith("|")) continue; + const name = [...line.matchAll(/`([^`]+)`/g)] + .map((match) => match[1]) + .find((value) => candidate.test(value)); + if (name !== undefined) names.push(name); } - if (result.disagreement.length > 0) { - console.log( - ` BUG this check and detectCatalogDrift() disagree on: ${result.disagreement.join(", ")}`, + return [...new Set(names)].sort(); +} + +async function checkDocumentedProvider(provider, runtime, options) { + const defaults = DOCUMENTED_MCP[provider]; + const sources = { + setup: options.setupReferenceSources.get(provider) ?? defaults.setup, + tools: + options.toolReferenceSources.get(provider) ?? defaults.inventory?.url, + }; + const [setup, markdown] = await Promise.all([ + loadPublished(provider, "official MCP setup reference", sources.setup), + sources.tools === undefined + ? Promise.resolve(undefined) + : loadPublished(provider, "MCP tool reference", sources.tools), + ]); + const documented = + markdown === undefined + ? undefined + : documentedToolNames(markdown, defaults.inventory); + if (documented !== undefined && documented.length === 0) { + throw new UnavailableError( + `${provider}'s MCP tool reference contained no recognizable tool names`, ); } - if (result.digests) { - console.log(` schemaDigests: ${JSON.stringify(result.digests, null, 2)}`); + const reviewed = [...runtime.catalog.tools.keys()].sort(); + const reviewedSet = new Set(reviewed); + const documentedSet = new Set(documented ?? []); + const acknowledged = defaults.inventory?.acknowledgedUnclassified ?? new Set(); + const added = (documented ?? []).filter( + (name) => !reviewedSet.has(name) && !acknowledged.has(name), + ); + const intentionallyUnclassified = (documented ?? []).filter( + (name) => !reviewedSet.has(name) && acknowledged.has(name), + ); + const removed = + documented === undefined + ? [] + : reviewed.filter((name) => !documentedSet.has(name)); + + const findings = []; + for (const endpoint of runtime.endpoints) { + if ( + !setup.includes(endpoint) && + !setup.includes(endpoint.replace(/\/$/, "")) + ) { + findings.push({ + kind: "mcp-endpoint", + detail: `official setup documentation does not name Connecta's endpoint ${endpoint}`, + }); + } } - if ( - result.added.length + - result.removed.length + - result.annotationConflicts.length + - result.schemaChanges.length === - 0 - ) { - console.log(" no drift"); + if (!setup.toLowerCase().includes("oauth")) { + findings.push({ + kind: "mcp-auth", + detail: "official setup documentation does not mention OAuth", + }); } + + return { + provider, + toolReference: sources.tools, + setupReference: sources.setup, + inventoryChecked: documented !== undefined, + documentedTools: documented?.length, + added, + removed, + intentionallyUnclassified, + findings, + schemaAuthority: "live-tools-list", + schemasVendored: false, + }; } +// --------------------------------------------------------------------------- +// Reporting +// --------------------------------------------------------------------------- + function printSpec(result, recorded) { console.log( `${result.provider} — ${result.findings.length ? `${result.findings.length} finding(s)` : "no drift"} across ${ @@ -651,23 +668,50 @@ function printSpec(result, recorded) { if (recorded) console.log(` recorded ${recorded}`); } +function printDocs(result) { + console.log( + result.inventoryChecked + ? `${result.provider} MCP docs: ${result.documentedTools} documented tools` + : `${result.provider} MCP docs: setup metadata only; no official tool inventory`, + ); + for (const tool of result.added) { + console.log(` unclassified ${tool}`); + } + for (const tool of result.removed) { + console.log(` not documented ${tool} (kept from release review)`); + } + for (const tool of result.intentionallyUnclassified) { + console.log(` fail-closed ${tool} (official access class is blank)`); + } + for (const finding of result.findings) { + console.log(` ${finding.kind.padEnd(16)} ${finding.detail}`); + } + if ( + result.added.length + result.findings.length === 0 + ) { + console.log( + result.inventoryChecked + ? " documented additions are classified; connection metadata matches" + : " connection metadata matches", + ); + } + console.log( + " schemas live tools/list remains authoritative; no MCP schema is vendored", + ); +} + function findingCount(report) { let total = 0; - for (const result of report.hosted) { - total += - result.added.length + - result.removed.length + - result.annotationConflicts.length + - result.schemaChanges.length + - result.disagreement.length; - } for (const result of report.specs) total += result.findings.length; + for (const result of report.docs) { + total += result.added.length + result.findings.length; + } return total; } async function main() { const options = parseArguments(process.argv.slice(2)); - const report = { hosted: [], specs: [] }; + const report = { specs: [], docs: [] }; if (options.specs) { for (const provider of selected(options, SPEC_PROVIDERS)) { @@ -686,25 +730,13 @@ async function main() { } } - if (options.hosted) { - const providers = selected(options, HOSTED_PROVIDERS); + if (options.docs) { + const providers = selected(options, DOCS_PROVIDERS); if (providers.length > 0) { - const missing = providers.filter( - (provider) => !process.env[HOSTED_CREDENTIALS[provider].variable]?.trim(), - ); - if (missing.length === providers.length && missing.length > 1) { - // All of them at once is a maintainer who exported nothing, not one - // stale key. Say so in a single message rather than one per provider. - throw new UnavailableError( - `no hosted credentials are set (${missing - .map((provider) => HOSTED_CREDENTIALS[provider].variable) - .join(", ")}). Run with --specs to check published specifications alone.`, - ); - } - const runtime = await loadHostedProviders(); + const runtimes = await loadDocumentedProviders(); for (const provider of providers) { - report.hosted.push( - await checkHostedProvider(provider, runtime, options), + report.docs.push( + await checkDocumentedProvider(provider, runtimes[provider], options), ); } } @@ -717,7 +749,7 @@ async function main() { for (const result of report.specs) { printSpec(result, result.recordedTo); } - for (const result of report.hosted) printHosted(result); + for (const result of report.docs) printDocs(result); console.log( findings === 0 ? "\nNo drift against the reviewed manifests." diff --git a/scripts/drift/hosted-auth.d.mts b/scripts/drift/hosted-auth.d.mts deleted file mode 100644 index b60d3e40..00000000 --- a/scripts/drift/hosted-auth.d.mts +++ /dev/null @@ -1,4 +0,0 @@ -export function hostedAuthorizationHeader( - provider: string, - value: string, -): string; diff --git a/scripts/drift/hosted-auth.mjs b/scripts/drift/hosted-auth.mjs deleted file mode 100644 index 82a0a2cb..00000000 --- a/scripts/drift/hosted-auth.mjs +++ /dev/null @@ -1,19 +0,0 @@ -/** - * Frame one hosted-provider credential as its Authorization header. - * - * Mixpanel's beta MCP service-account scheme is deliberately not ordinary - * HTTP Basic: the provider requires `Bearer Basic `. - * The other hosted providers use their ordinary bearer or Basic forms — - * RevenueCat's API v2 secret key is a plain `Bearer sk_…`, which is what a - * colon-free, whitespace-free value already becomes, so it needs no shaping - * of its own. - */ -export function hostedAuthorizationHeader(provider, value) { - const credential = value.trim(); - if (/\s/.test(credential)) return credential; - if (credential.includes(":")) { - const encoded = Buffer.from(credential, "utf8").toString("base64"); - return provider === "mixpanel" ? `Bearer Basic ${encoded}` : `Basic ${encoded}`; - } - return `Bearer ${credential}`; -} diff --git a/scripts/drift/vercel-endpoints.json b/scripts/drift/vercel-endpoints.json index f494524e..39f56c18 100644 --- a/scripts/drift/vercel-endpoints.json +++ b/scripts/drift/vercel-endpoints.json @@ -44,7 +44,7 @@ "method": "GET", "path": "/v3/deployments/{idOrUrl}/events", "specRevision": "0.0.1", - "contract": "sha256:62dfec8f83e91c31b548a0a34dbf1b5d92a30c6931b366cd2828786c8c95fdde" + "contract": "sha256:fbeedd856cfdb853fdcd969ce973c7a899f064d59625dfba9e19ba4ac4eabf03" }, { "method": "GET", diff --git a/src/catalog-drift.ts b/src/catalog-drift.ts index f94ca4a1..55cc0446 100644 --- a/src/catalog-drift.ts +++ b/src/catalog-drift.ts @@ -65,9 +65,9 @@ interface VettedToolRecord { * Digest of the input and output schemas that release read, or undefined * when no release has recorded them. Undefined is not "unchanged": a * manifest with no digest cannot report a schema change, and says so by - * counting none. `npm run drift:check -- --record` reads them from a live - * catalog and prints the block a release pastes in - * ([#351](https://github.com/zackbart/connecta/issues/351)). + * counting none. The credential-free provider check does not create or + * update schema digests; the live `tools/list` response remains the schema + * agents receive ([#351](https://github.com/zackbart/connecta/issues/351)). */ schemaDigest?: string; } diff --git a/src/providers/cloudflare.ts b/src/providers/cloudflare.ts index 2fa86b39..cf2ee95d 100644 --- a/src/providers/cloudflare.ts +++ b/src/providers/cloudflare.ts @@ -1,5 +1,11 @@ /** See documentation/cloudflare.md#no-sdk-on-purpose. */ -import { api, type ApiTool } from "../connectors/api.js"; +import { api, defined, type ApiTool } from "../connectors/api.js"; +import { + remoteMcp, + withCredentialDefaults, + type RemoteMcpAuth, +} from "../connectors/remote-mcp.js"; +import { vettedCatalog, withVettedCatalog } from "../catalog-drift.js"; import { guardedFetch, retryAfterMs, @@ -17,6 +23,8 @@ import type { /** Cloudflare's v4 REST base. Override only for a proxy or a test double. */ export const CLOUDFLARE_API_BASE = "https://api.cloudflare.com/client/v4"; +/** Cloudflare's official whole-API hosted MCP endpoint. */ +export const CLOUDFLARE_MCP_ENDPOINT = "https://mcp.cloudflare.com/mcp"; /** Authentication schemes accepted by Cloudflare's v4 API. */ export type CloudflareAuthentication = "apiToken" | "globalApiKey"; @@ -58,13 +66,22 @@ export const CLOUDFLARE_CONTENT_DNS_RECORD_TYPES = [ "TXT", ] as const; -export interface CloudflareOptions { - /** Human-readable display name; defaults to "Cloudflare". */ +interface CloudflareCommonOptions { + /** Human-readable display name; defaults identify the selected interface. */ title?: string; /** Downstream auth ownership. Defaults to one shared deployment grant. */ authScope?: "shared" | "personal"; /** Which account/estate this connection administers, and for whom. */ purpose: string; + /** Account-specific conventions appended to the maintained provider guide. */ + instructions?: string; + /** Connector-specific inline result limit; omit to inherit the deployment. */ + maxResultBytes?: number; +} + +export interface CloudflareApiOptions extends CloudflareCommonOptions { + /** Omit for backward compatibility; the hand-written API interface is default. */ + surface?: "api"; /** * Default account id for account-scoped tools. When set, `accountId` becomes * an optional argument; when omitted, agents must pass one and can find it @@ -83,14 +100,26 @@ export interface CloudflareOptions { authentication?: CloudflareAuthentication; /** Credential presentation override; credentials are always operator-managed. */ credential?: ConnectorCredentialConfig; - /** Account-specific conventions appended to the maintained provider guide. */ - instructions?: string; - /** Connector-specific inline result limit; omit to inherit the deployment. */ - maxResultBytes?: number; /** Simultaneous downstream calls. Defaults to 6. */ maxConcurrency?: number; } +export interface CloudflareMcpOptions extends CloudflareCommonOptions { + surface: "mcp"; + /** OAuth by default, or a scoped API token for a headless deployment. */ + auth?: RemoteMcpAuth; + /** Optional per-runtime downstream call-admission policy. */ + callAdmission?: ConnectorCallAdmissionPolicy; +} + +/** Backward-compatible API options; existing consumers may extend this interface. */ +export interface CloudflareOptions extends CloudflareApiOptions {} + +/** Select one Cloudflare interface when deployment configuration constructs it. */ +export type CloudflareConnectionOptions = + | CloudflareOptions + | CloudflareMcpOptions; + /** See documentation/cloudflare.md#rate-limits. */ function admissionPolicy(maxConcurrency: number): ConnectorCallAdmissionPolicy { return { @@ -3543,7 +3572,7 @@ function buildTools( return tools; } -function usageGuide( +function apiUsageGuide( purpose: string, scope: Scoping, instructions: string | undefined, @@ -3582,12 +3611,82 @@ ${ }`; } -/** A maintained Cloudflare REST API connection. */ -export function cloudflare(id: string, options: CloudflareOptions): Connector { - const purpose = options.purpose.trim(); - if (!purpose) { - throw new Error("cloudflare() requires a non-empty account purpose."); - } +export const CLOUDFLARE_MCP_VETTED_CATALOG = vettedCatalog({ + reads: new Set(["search"]), + // `execute` can send any method to more than 2,500 API endpoints. Its input + // schema cannot prove a particular program is observational, so it stays on + // the approval path even when that program happens to issue only GETs. + writes: new Map([["execute", "destructive"]]), +}); + +function mcpUsageGuide( + purpose: string, + instructions: string | undefined, +): string { + const accountInstructions = instructions?.trim(); + return `# Cloudflare MCP usage + +Official whole-API MCP interface: ${purpose} + +- The catalog contains \`search\` and \`execute\`. Search runs code against + Cloudflare's OpenAPI document. Execute runs code that may call any authorized + Cloudflare API endpoint. +- Use \`search\` to find the exact method, path, and fields before writing an + execute program. Do not guess an endpoint from product naming. +- Connecta routes every \`execute\` call through approval because the tool can + mix GET, POST, PUT, PATCH, and DELETE requests inside one program. The MCP + schema cannot establish that arbitrary code is read-only. +- Keep returned values small. Filter and project inside the Cloudflare MCP + program, then reduce again inside Connecta's \`execute_code\` when several + calls must be joined. +- OAuth and API-token permissions remain the provider-side boundary. An + \`auth_required\` failure needs authorization or a token with the required + Cloudflare permission. +${ + accountInstructions + ? `\n## Account instructions\n\n${accountInstructions}\n` + : "" + }`; +} + +function cloudflareMcp( + id: string, + purpose: string, + options: CloudflareMcpOptions, +): Connector { + const connector = remoteMcp(id, { + url: CLOUDFLARE_MCP_ENDPOINT, + ...defined({ + authScope: options.authScope, + callAdmission: options.callAdmission, + maxResultBytes: options.maxResultBytes, + }), + title: options.title ?? "Cloudflare (MCP)", + description: `Cloudflare's official whole-API MCP interface: ${purpose}`, + auth: withCredentialDefaults(options.auth ?? { type: "oauth" }, { + credential: { + label: "Cloudflare API token", + description: + "A scoped Cloudflare API token. Connecta sends it as a bearer token to mcp.cloudflare.com and stores it encrypted.", + placeholder: "Paste Cloudflare API token", + }, + }), + requireHttps: true, + usageGuide: { + content: mcpUsageGuide(purpose, options.instructions), + summary: + "Official whole-API MCP. Search the OpenAPI document, then approve each mixed-method execute program.", + required: true, + }, + }); + return withVettedCatalog(connector, CLOUDFLARE_MCP_VETTED_CATALOG); +} + +function cloudflareApi( + id: string, + purpose: string, + options: CloudflareApiOptions, +): Connector { const maxConcurrency = options.maxConcurrency ?? 6; if (!Number.isInteger(maxConcurrency) || maxConcurrency < 1) { throw new Error("cloudflare() maxConcurrency must be a positive integer."); @@ -3613,7 +3712,7 @@ export function cloudflare(id: string, options: CloudflareOptions): Connector { credential: credentialConfig(authentication, options.credential), callAdmission: admissionPolicy(maxConcurrency), usageGuide: { - content: usageGuide(purpose, scope, options.instructions, authentication), + content: apiUsageGuide(purpose, scope, options.instructions, authentication), // Explicit rather than derived: the first content line is the zone // scoping rule, which varies per deployment and reads as an instruction // rather than as the routing fact a browsing agent needs. @@ -3676,3 +3775,17 @@ export function cloudflare(id: string, options: CloudflareOptions): Connector { }), }); } + +/** A maintained Cloudflare connection using the selected provider interface. */ +export function cloudflare( + id: string, + options: CloudflareConnectionOptions, +): Connector { + const purpose = options.purpose.trim(); + if (!purpose) { + throw new Error("cloudflare() requires a non-empty account purpose."); + } + return options.surface === "mcp" + ? cloudflareMcp(id, purpose, options) + : cloudflareApi(id, purpose, options); +} diff --git a/src/providers/notion.ts b/src/providers/notion.ts index 21791658..8ed022cf 100644 --- a/src/providers/notion.ts +++ b/src/providers/notion.ts @@ -1,4 +1,6 @@ -import { api, type ApiTool } from "../connectors/api.js"; +import { api, defined, type ApiTool } from "../connectors/api.js"; +import { remoteMcp } from "../connectors/remote-mcp.js"; +import { vettedCatalog, withVettedCatalog } from "../catalog-drift.js"; import { guardedFetch, retryAfterMs, @@ -14,6 +16,8 @@ import type { /** Notion's REST origin. Every tool below speaks to exactly this host. */ export const NOTION_API_BASE_URL = "https://api.notion.com"; +/** Notion's official hosted MCP endpoint. */ +export const NOTION_MCP_ENDPOINT = "https://mcp.notion.com/mcp"; /** See documentation/notion.md#the-pinned-api-version. */ export const NOTION_API_VERSION = "2026-03-11"; @@ -53,8 +57,8 @@ const NOTION_ADMISSION: ConnectorCallAdmissionPolicy = { ], }; -export interface NotionOptions { - /** Human-readable display name; defaults to "Notion". */ +interface NotionCommonOptions { + /** Human-readable display name; defaults identify the selected interface. */ title?: string; /** Downstream auth ownership. Defaults to one shared deployment grant. */ authScope?: "shared" | "personal"; @@ -62,6 +66,14 @@ export interface NotionOptions { purpose: string; /** Workspace-specific conventions appended to the maintained provider guide. */ instructions?: string; + /** Connector-specific inline result limit; omit to inherit the deployment. */ + maxResultBytes?: number; +} + +/** Connecta's maintained hand-written Notion REST interface. */ +export interface NotionApiOptions extends NotionCommonOptions { + /** Omit for backward compatibility; the hand-written API interface is default. */ + surface?: "api"; /** Operator-facing label for the integration token. */ credentialLabel?: string; /** @@ -69,10 +81,21 @@ export interface NotionOptions { * Defaults to 25; Notion's maximum is 100. */ defaultPageSize?: number; - /** Connector-specific inline result limit; omit to inherit the deployment. */ - maxResultBytes?: number; } +/** Notion's official hosted MCP interface, authenticated through OAuth. */ +export interface NotionMcpOptions extends NotionCommonOptions { + surface: "mcp"; + /** Optional per-runtime downstream call-admission policy. */ + callAdmission?: ConnectorCallAdmissionPolicy; +} + +/** Backward-compatible API options; existing consumers may extend this interface. */ +export interface NotionOptions extends NotionApiOptions {} + +/** Select one Notion interface when deployment configuration constructs it. */ +export type NotionConnectionOptions = NotionOptions | NotionMcpOptions; + // --------------------------------------------------------------------------- // Transport and typed failures // --------------------------------------------------------------------------- @@ -1636,7 +1659,7 @@ function buildTools(defaultPageSize: number): ApiTool[] { // --------------------------------------------------------------------------- /** See documentation/notion.md#databases-contain-data-sources. */ -function usageGuide(purpose: string, instructions: string | undefined): string { +function apiUsageGuide(purpose: string, instructions: string | undefined): string { const accountInstructions = instructions?.trim(); return `# Notion usage @@ -1736,12 +1759,112 @@ ${ }`; } -/** A maintained Notion connection over the public REST API. */ -export function notion(id: string, options: NotionOptions): Connector { - const purpose = options.purpose.trim(); - if (!purpose) { - throw new Error("notion() requires a non-empty workspace purpose."); - } +/** Release-reviewed Notion MCP inventory and safety verdicts. */ +export const NOTION_MCP_VETTED_CATALOG = vettedCatalog({ + reads: new Set([ + "notion-search", + "notion-search-skills", + "notion-fetch", + "notion-download-attachment", + "notion-query-data-sources", + "notion-query-meeting-notes", + "notion-search-agents", + "notion-list-agents", + "notion-query-sessions", + "notion-search-sessions", + "notion-get-session-status", + "notion-wait-session", + "notion-list-session-events", + "notion-read-session-event", + "notion-get-comments", + "notion-get-teams", + "notion-get-users", + "notion-get-async-task", + ]), + writes: new Map([ + ["notion-create-file-upload", "additive"], + ["notion-create-attachment", "additive"], + ["notion-create-pages", "additive"], + ["notion-duplicate-page", "additive"], + ["notion-create-database", "additive"], + ["notion-create-folder", "additive"], + ["notion-create-view", "additive"], + ["notion-spawn-session", "additive"], + ["notion-send-message-to-session", "additive"], + ["notion-create-comment", "additive"], + ["notion-update-page", "destructive"], + ["notion-convert-page-to-skill", "destructive"], + ["notion-move-pages", "destructive"], + ["notion-update-data-source", "destructive"], + ["notion-update-view", "destructive"], + ["notion-stop-session", "destructive"], + ]), +}); + +function mcpUsageGuide( + purpose: string, + instructions: string | undefined, +): string { + const accountInstructions = instructions?.trim(); + return `# Notion MCP usage + +Official MCP interface: tool names, descriptions, argument schemas, and result +schemas come from Notion's live server. Connecta preserves that catalog and +only fills in release-reviewed safety annotations when Notion leaves them out. + +Workspace purpose: ${purpose} + +- Discover the live catalog before assuming a tool exists. Notion can gate + tools by workspace, account, client, and rollout independently of Connecta. +- Start broad discovery with \`notion-search\`, then use \`notion-fetch\` on + the exact page, database, data source, or object before changing it. +- Use the live input schema as the contract. Notion owns these MCP schemas; + the REST schemas in Connecta's API interface do not apply to MCP tools with + similar names. +- Session and agent tools can launch asynchronous work. Read session state and + events before sending another message, waiting, or stopping a session. +- File uploads and attachment tools create durable workspace state. Keep + downloads inside the requested task and do not expose signed attachment URLs. +- An \`auth_required\` failure means this connector's OAuth grant is missing or + expired. Run \`authorize_connector\` for this connector id, then retry. +${ + accountInstructions + ? `\n## Workspace instructions\n\n${accountInstructions}\n` + : "" + }`; +} + +function notionMcp( + id: string, + purpose: string, + options: NotionMcpOptions, +): Connector { + const connector = remoteMcp(id, { + url: NOTION_MCP_ENDPOINT, + ...defined({ + authScope: options.authScope, + callAdmission: options.callAdmission, + maxResultBytes: options.maxResultBytes, + }), + title: options.title ?? "Notion (MCP)", + description: `Notion's official hosted MCP interface: ${purpose}`, + auth: { type: "oauth" }, + requireHttps: true, + usageGuide: { + content: mcpUsageGuide(purpose, options.instructions), + summary: + "Official MCP. Live Notion schemas, object discovery, sessions, agents, attachments, and OAuth ownership.", + required: true, + }, + }); + return withVettedCatalog(connector, NOTION_MCP_VETTED_CATALOG); +} + +function notionApi( + id: string, + purpose: string, + options: NotionApiOptions, +): Connector { const defaultPageSize = options.defaultPageSize ?? DEFAULT_PAGE_SIZE; if ( !Number.isInteger(defaultPageSize) || @@ -1785,7 +1908,7 @@ export function notion(id: string, options: NotionOptions): Connector { }, callAdmission: NOTION_ADMISSION, usageGuide: { - content: usageGuide(purpose, options.instructions), + content: apiUsageGuide(purpose, options.instructions), summary: "Database-to-data-source lookup, property write rules, lean-vs-raw results, and Notion's overloaded 403/404.", required: true, @@ -1796,3 +1919,14 @@ export function notion(id: string, options: NotionOptions): Connector { : {}), }); } + +/** A maintained Notion connection using the selected provider interface. */ +export function notion(id: string, options: NotionConnectionOptions): Connector { + const purpose = options.purpose.trim(); + if (!purpose) { + throw new Error("notion() requires a non-empty workspace purpose."); + } + return options.surface === "mcp" + ? notionMcp(id, purpose, options) + : notionApi(id, purpose, options); +} diff --git a/src/providers/revenuecat.ts b/src/providers/revenuecat.ts index 6f628ed6..52271bf6 100644 --- a/src/providers/revenuecat.ts +++ b/src/providers/revenuecat.ts @@ -145,10 +145,21 @@ const WRITE_TOOLS: ReadonlyMap = new Map([ ["update-audience", "destructive"], // Paywalls + ["attach-offering-to-paywall", "destructive"], + ["detach-offering-from-paywall", "destructive"], ["duplicate-paywall", "additive"], ["publish-paywall", "destructive"], ["unpublish-paywall", "destructive"], + // Offerings and experiments added to the published reference after #512. + ["duplicate-offering", "additive"], + ["create-experiment", "additive"], + ["pause-experiment", "destructive"], + ["resume-experiment", "destructive"], + ["start-experiment", "destructive"], + ["stop-experiment", "destructive"], + ["update-experiment", "destructive"], + // Customers and subscriptions ["assign-customer-offering", "destructive"], ["grant-customer-entitlement", "destructive"], diff --git a/src/providers/stripe.ts b/src/providers/stripe.ts index 908c71e5..187a7377 100644 --- a/src/providers/stripe.ts +++ b/src/providers/stripe.ts @@ -89,7 +89,12 @@ const READ_ONLY_TOOLS = new Set([ "stripe_api_search", "stripe_api_details", "stripe_api_read", + "get_stripe_account_info", "get_balance_summary", + "list_metrics", + "explain_metric", + "metric_drilldown", + "show_metric_app", "list_available_accounts_or_orgs", "manage_stripe_accounts", "search_stripe_documentation", @@ -98,8 +103,10 @@ const READ_ONLY_TOOLS = new Set([ /** Reviewed writes, including mixed read/create tools: `documentation/stripe.md`. */ const WRITE_TOOLS: ReadonlyMap = new Map([ ["stripe_api_write", "destructive"], + ["create_refund", "destructive"], ["stripe_implementation_planner", "additive"], ["stripe_analytics", "additive"], + ["stripe_report", "additive"], ["send_stripe_mcp_feedback", "additive"], ]); diff --git a/src/providers/vercel.ts b/src/providers/vercel.ts index 3b8efe77..deac50d2 100644 --- a/src/providers/vercel.ts +++ b/src/providers/vercel.ts @@ -1,5 +1,7 @@ /** See documentation/vercel.md#no-sdk-on-purpose. */ -import { api, type ApiTool } from "../connectors/api.js"; +import { api, defined, type ApiTool } from "../connectors/api.js"; +import { remoteMcp } from "../connectors/remote-mcp.js"; +import { vettedCatalog, withVettedCatalog } from "../catalog-drift.js"; import { guardedFetch, retryAfterMs, @@ -17,6 +19,8 @@ import type { /** Vercel's public REST origin. Override only for a proxy or test double. */ export const VERCEL_API_BASE_URL = "https://api.vercel.com"; +/** Vercel's official hosted MCP endpoint. */ +export const VERCEL_MCP_ENDPOINT = "https://mcp.vercel.com"; const MAX_PAGE_SIZE = 100; const DEFAULT_PAGE_SIZE = 20; @@ -25,27 +29,44 @@ const MAX_RUNTIME_LOG_ROWS = 500; const DEFAULT_RUNTIME_LOG_ROWS = 100; const RUNTIME_LOG_TIMEOUT_MS = 10_000; -export interface VercelOptions { - /** Human-readable display name; defaults to "Vercel". */ +interface VercelCommonOptions { + /** Human-readable display name; defaults identify the selected surface. */ title?: string; /** Downstream auth ownership. Defaults to one shared deployment grant. */ authScope?: "shared" | "personal"; /** Which Vercel account or team this connection operates, and for whom. */ purpose: string; - /** Default team id for scoped calls. Omit to use the token's personal account. */ - teamId?: string; /** Account-specific conventions appended to the maintained provider guide. */ instructions?: string; - /** API base override for a proxy or test double. */ - baseUrl?: string; - /** Default page size for list tools. Defaults to 20; Vercel's local cap is 100. */ - defaultPageSize?: number; /** Optional per-runtime downstream call-admission policy. */ callAdmission?: ConnectorCallAdmissionPolicy; /** Connector-specific inline result limit; omit to inherit the deployment. */ maxResultBytes?: number; } +/** Connecta's maintained hand-written Vercel REST surface. */ +export interface VercelApiOptions extends VercelCommonOptions { + /** Omit for backward compatibility; the hand-written API surface is the default. */ + surface?: "api"; + /** Default team id for scoped calls. Omit to use the token's personal account. */ + teamId?: string; + /** API base override for a proxy or test double. */ + baseUrl?: string; + /** Default page size for list tools. Defaults to 20; Vercel's local cap is 100. */ + defaultPageSize?: number; +} + +/** Vercel's official hosted MCP surface, authenticated through OAuth. */ +export interface VercelMcpOptions extends VercelCommonOptions { + surface: "mcp"; +} + +/** Backward-compatible API options; existing consumers may extend this interface. */ +export interface VercelOptions extends VercelApiOptions {} + +/** Select one Vercel surface when deployment configuration constructs it. */ +export type VercelConnectionOptions = VercelOptions | VercelMcpOptions; + type JsonRecord = Record; function asRecord(value: unknown): JsonRecord { @@ -962,7 +983,7 @@ function tools( type: { type: "string" }, createdAt: { type: "number" }, message: { type: "string" }, payload: { type: "object" }, }, - required: ["type", "createdAt"], + required: ["type"], }, }, }, @@ -986,7 +1007,8 @@ function tools( const event = asRecord(value); const eventPayload = asRecord(event["payload"]); return compact({ - type: event["type"], createdAt: event["created"] ?? event["date"], + type: event["type"] ?? "unknown", + createdAt: event["created"] ?? event["date"], message: eventPayload["text"] ?? eventPayload["message"], payload: Object.keys(eventPayload).length === 0 ? undefined : eventPayload, }); @@ -1335,7 +1357,7 @@ function tools( ]; } -function usageGuide( +function apiUsageGuide( purpose: string, teamId: string | undefined, instructions: string | undefined, @@ -1394,12 +1416,124 @@ ${ }`; } -/** A maintained Vercel connection over the public REST API. */ -export function vercel(id: string, options: VercelOptions): Connector { - const purpose = options.purpose.trim(); - if (!purpose) { - throw new Error("vercel() requires a non-empty account purpose."); - } +/** Reads reviewed against Vercel's official MCP tool reference. */ +const MCP_READ_ONLY_TOOLS = new Set([ + "search_vercel_documentation", + "list_teams", + "list_projects", + "get_project", + "list_deployments", + "get_deployment", + "get_deployment_build_logs", + "get_runtime_logs", + "get_runtime_errors", + "get_web_analytics", + "list_agent_run_projects", + "list_agent_runs", + "get_agent_run", + "get_agent_run_trace", + "check_domain_availability_and_price", + "get_purchase_quote", + "get_domain_order", + "list_toolbar_threads", + "get_toolbar_thread", + // Returns CLI guidance. Any later CLI execution is outside this MCP call. + "use_vercel_cli", +]); + +/** Writes reviewed against Vercel's official MCP tool reference. */ +const MCP_WRITE_TOOLS: ReadonlyMap = + new Map([ + // These only append state. + ["reply_to_toolbar_thread", "additive"], + ["add_toolbar_reaction", "additive"], + // Deploying to production, billing, access grants, imports, and edits can + // all change existing state, even where the provider uses a create verb. + ["deploy_to_vercel", "destructive"], + ["buy_pro", "destructive"], + ["buy_credits", "destructive"], + ["buy_addon", "destructive"], + ["buy_domain", "destructive"], + ["get_access_to_vercel_url", "destructive"], + // A GET against application code is not guaranteed to be observational. + ["web_fetch_vercel_url", "destructive"], + ["import-claude-design-from-url", "destructive"], + ["change_toolbar_thread_resolve_status", "destructive"], + ["edit_toolbar_message", "destructive"], + ]); + +/** Release-reviewed Vercel MCP inventory and safety verdicts. */ +export const VERCEL_MCP_VETTED_CATALOG = vettedCatalog({ + reads: MCP_READ_ONLY_TOOLS, + writes: MCP_WRITE_TOOLS, +}); + +function mcpUsageGuide( + purpose: string, + instructions: string | undefined, +): string { + const accountInstructions = instructions?.trim(); + return `# Vercel MCP usage + +Official MCP surface: tool names, descriptions, argument schemas, and result +schemas come from Vercel's live server. Connecta preserves that catalog and +only fills in release-reviewed safety annotations when Vercel leaves them out. + +Account purpose: ${purpose} + +- Discover the live catalog before assuming a tool exists. Vercel can change + the surface independently of a Connecta release, and account features may + affect what the authorization can reach. +- Resolve team, project, deployment, run, thread, and order ids with the list + and get tools. Do not guess opaque ids. +- Diagnose deployments with \`get_deployment\`, then build logs, runtime error + clusters, and runtime logs. Narrow time windows before raising result limits. +- Purchase tools change billing. Read a quote first and carry its price, + idempotency key, and requested term into the confirmed purchase unchanged. +- \`get_access_to_vercel_url\` creates a temporary access grant. Treat the URL + it returns as a credential and do not expose it outside the requested task. +- \`deploy_to_vercel\` and \`import-claude-design-from-url\` can create or update + live projects. Read the target and deployment mode before approving them. +- An \`auth_required\` failure means this connector's OAuth grant is missing or + expired. Run \`authorize_connector\` for this connector id, then retry. +${ + accountInstructions + ? `\n## Account instructions\n\n${accountInstructions}\n` + : "" + }`; +} + +function vercelMcp( + id: string, + purpose: string, + options: VercelMcpOptions, +): Connector { + const connector = remoteMcp(id, { + url: VERCEL_MCP_ENDPOINT, + ...defined({ + authScope: options.authScope, + callAdmission: options.callAdmission, + maxResultBytes: options.maxResultBytes, + }), + title: options.title ?? "Vercel (MCP)", + description: `Vercel's official hosted MCP surface: ${purpose}`, + auth: { type: "oauth" }, + requireHttps: true, + usageGuide: { + content: mcpUsageGuide(purpose, options.instructions), + summary: + "Official MCP. Live Vercel schemas, id resolution, deployment diagnosis, purchases, and access grants.", + required: true, + }, + }); + return withVettedCatalog(connector, VERCEL_MCP_VETTED_CATALOG); +} + +function vercelApi( + id: string, + purpose: string, + options: VercelApiOptions, +): Connector { const defaultPageSize = options.defaultPageSize ?? DEFAULT_PAGE_SIZE; if ( !Number.isInteger(defaultPageSize) || @@ -1441,7 +1575,7 @@ export function vercel(id: string, options: VercelOptions): Connector { } }, usageGuide: { - content: usageGuide(purpose, teamId, options.instructions), + content: apiUsageGuide(purpose, teamId, options.instructions), summary: "Team scoping, deployment diagnosis, value-safe environment variables, REST hatches, and cursor pagination.", required: true, @@ -1455,3 +1589,14 @@ export function vercel(id: string, options: VercelOptions): Connector { : {}), }); } + +/** A maintained Vercel connection using the selected provider surface. */ +export function vercel(id: string, options: VercelConnectionOptions): Connector { + const purpose = options.purpose.trim(); + if (!purpose) { + throw new Error("vercel() requires a non-empty account purpose."); + } + return options.surface === "mcp" + ? vercelMcp(id, purpose, options) + : vercelApi(id, purpose, options); +} diff --git a/src/version.ts b/src/version.ts index de20c29a..bf4b3e13 100644 --- a/src/version.ts +++ b/src/version.ts @@ -4,4 +4,4 @@ * a bump that forgets this file fails the build rather than shipping a stale * version to `/health` and to downstream MCP handshakes. */ -export const CONNECTA_VERSION = "0.22.1"; +export const CONNECTA_VERSION = "0.22.2"; diff --git a/templates/node/package.json b/templates/node/package.json index ca29919d..95a51236 100644 --- a/templates/node/package.json +++ b/templates/node/package.json @@ -15,7 +15,7 @@ "typecheck": "tsc --noEmit" }, "dependencies": { - "@zackbart/connecta": "0.22.1", + "@zackbart/connecta": "0.22.2", "quickjs-emscripten": "0.32.0" }, "devDependencies": { diff --git a/test/catalog-drift.test.ts b/test/catalog-drift.test.ts index deca626e..3c836d92 100644 --- a/test/catalog-drift.test.ts +++ b/test/catalog-drift.test.ts @@ -199,6 +199,38 @@ describe("detectCatalogDrift()", () => { }); describe("withVettedCatalog()", () => { + it("preserves the downstream MCP schemas byte-for-byte in memory", async () => { + const inputSchema = { + type: "object", + properties: { + issueId: { type: "string", pattern: "^ISSUE-[0-9]+$" }, + }, + required: ["issueId"], + additionalProperties: false, + } as const; + const outputSchema = { + type: "object", + properties: { state: { enum: ["open", "closed"] } }, + required: ["state"], + } as const; + const definition: ToolDef = { + name: "get_issue", + description: "Provider-owned definition", + inputSchema, + outputSchema, + }; + const { connector } = proxy("linear_test", () => [definition]); + const [served] = await connector.listTools(context); + + expect(served?.inputSchema).toBe(inputSchema); + expect(served?.outputSchema).toBe(outputSchema); + expect(served?.description).toBe(definition.description); + expect(served?.annotations).toEqual({ + readOnlyHint: true, + destructiveHint: false, + }); + }); + it("classifies exactly as the provider lists say", async () => { const { connector } = proxy("linear_test", () => [ ...currentCatalog(), diff --git a/test/cloudflare-provider.test.ts b/test/cloudflare-provider.test.ts index 110e5d9d..bc3ac00f 100644 --- a/test/cloudflare-provider.test.ts +++ b/test/cloudflare-provider.test.ts @@ -2,10 +2,25 @@ // is the request it builds and the result it projects. `fetch` is stubbed for // the whole file; nothing here reaches the network. import { afterEach, beforeEach, describe, expect, it, it as test, vi } from "vitest"; +import type { ToolDef } from "../src/types.js"; +import { mockRemoteMcp } from "./fixtures/hosted-provider.js"; + +const mcpMocks = vi.hoisted(() => ({ + listTools: vi.fn<() => Promise>(), + remoteMcp: vi.fn(), +})); + +vi.mock("../src/connectors/remote-mcp.js", async (importOriginal) => ({ + ...(await importOriginal()), + remoteMcp: mcpMocks.remoteMcp, +})); + import { CLOUDFLARE_API_BASE, CLOUDFLARE_CONTENT_DNS_RECORD_TYPES, CLOUDFLARE_DNS_RECORD_TYPES, + CLOUDFLARE_MCP_ENDPOINT, + CLOUDFLARE_MCP_VETTED_CATALOG, cloudflare, } from "../src/providers/cloudflare.js"; import { ConnectorCallError } from "../src/errors.js"; @@ -15,7 +30,6 @@ import type { Connector, ConnectorContext, ConnectorUsageGuide, - ToolDef, } from "../src/types.js"; const TOKEN = "cf-token"; @@ -98,6 +112,7 @@ const realFetch = globalThis.fetch; beforeEach(() => { calls = []; + mockRemoteMcp(mcpMocks); }); afterEach(() => { @@ -153,6 +168,49 @@ describe("cloudflare() construction", () => { }); }); + it("binds the explicit MCP interface to Cloudflare's whole-API endpoint", () => { + const callAdmission = { rules: [{ maxConcurrency: 2 }] }; + const connector = connection({ surface: "mcp", callAdmission }); + expect(mcpMocks.remoteMcp).toHaveBeenCalledWith( + "edge", + expect.objectContaining({ + url: CLOUDFLARE_MCP_ENDPOINT, + title: "Cloudflare (MCP)", + auth: { type: "oauth" }, + callAdmission, + requireHttps: true, + }), + ); + expect(connector.kind).toBe("mcp"); + expect(structuredGuide(connector).content).toContain("search"); + expect(structuredGuide(connector).content).toContain("approval"); + expect(calls).toEqual([]); + }); + + it("classifies Cloudflare's two code-mode tools fail-closed", async () => { + expect([...CLOUDFLARE_MCP_VETTED_CATALOG.tools.entries()]).toEqual([ + ["search", { verdict: "read-only" }], + ["execute", { verdict: "destructive" }], + ]); + mcpMocks.listTools.mockResolvedValue([ + { name: "search" }, + { name: "execute", annotations: { readOnlyHint: true } }, + { name: "new-cloudflare-tool" }, + ]); + const tools = await connection({ surface: "mcp" }).listTools( + contextWithToken(), + ); + expect(tools[0]?.annotations).toEqual({ + readOnlyHint: true, + destructiveHint: false, + }); + expect(tools[1]?.annotations).toEqual({ + readOnlyHint: false, + destructiveHint: true, + }); + expect(tools[2]?.annotations).toEqual({ readOnlyHint: false }); + }); + it("declares the two fields required by legacy Global API Key authentication", () => { const connector = connection({ authentication: "globalApiKey" }); expect(connector.credential).toMatchObject({ diff --git a/test/drift-check.test.ts b/test/drift-check.test.ts index b3c61854..cfc3bca9 100644 --- a/test/drift-check.test.ts +++ b/test/drift-check.test.ts @@ -4,11 +4,28 @@ import { tmpdir } from "node:os"; import { join } from "node:path"; import { fileURLToPath } from "node:url"; import { afterEach, describe, expect, it } from "vitest"; -import { hostedAuthorizationHeader } from "../scripts/drift/hosted-auth.mjs"; +import { + CLOUDFLARE_MCP_ENDPOINT, + CLOUDFLARE_MCP_VETTED_CATALOG, +} from "../src/providers/cloudflare.js"; +import { LINEAR_MCP_ENDPOINTS } from "../src/providers/linear.js"; +import { + NOTION_MCP_ENDPOINT, + NOTION_MCP_VETTED_CATALOG, +} from "../src/providers/notion.js"; +import { + STRIPE_MCP_ENDPOINT, + STRIPE_VETTED_CATALOG, +} from "../src/providers/stripe.js"; +import { + VERCEL_MCP_ENDPOINT, + VERCEL_MCP_VETTED_CATALOG, +} from "../src/providers/vercel.js"; const checker = fileURLToPath( new URL("../scripts/drift-check.mjs", import.meta.url), ); +const tsx = fileURLToPath(new URL("../node_modules/tsx/dist/cli.mjs", import.meta.url)); const manifestDirectory = fileURLToPath(new URL("../scripts/drift", import.meta.url)); const temporary: string[] = []; @@ -116,7 +133,6 @@ function run( directory: string, providers: string[], extra: string[] = [], - environment: Record = {}, ) { const result = spawnSync( process.execPath, @@ -133,16 +149,7 @@ function run( ]), ...extra, ], - { - encoding: "utf8", - env: { - ...process.env, - CONNECTA_DRIFT_LINEAR_KEY: undefined, - CONNECTA_DRIFT_STRIPE_KEY: undefined, - CONNECTA_DRIFT_MIXPANEL_KEY: undefined, - ...environment, - } as NodeJS.ProcessEnv, - }, + { encoding: "utf8" }, ); return { status: result.status, output: `${result.stdout}${result.stderr}` }; } @@ -152,6 +159,52 @@ function findings(output: string, provider: string): Finding[] { return report.specs.find((entry: any) => entry.provider === provider).findings; } +async function documentedVercelWorkspace(): Promise<{ + directory: string; + toolReference: string; + setupReference: string; +}> { + const directory = await mkdtemp(join(tmpdir(), "connecta-drift-docs-")); + temporary.push(directory); + const toolReference = join(directory, "vercel-tools.md"); + const setupReference = join(directory, "vercel-setup.md"); + const headings = [...VERCEL_MCP_VETTED_CATALOG.tools.keys()] + .sort() + .map((name) => `### ${name.replaceAll("_", "\\_")}`) + .join("\n\n"); + await writeFile(toolReference, `# Vercel tools\n\n${headings}\n`); + await writeFile( + setupReference, + `# Vercel MCP setup\n\nEndpoint: ${VERCEL_MCP_ENDPOINT}\n\nOAuth is required.\n`, + ); + return { directory, toolReference, setupReference }; +} + +function runDocumented( + provider: string, + toolReference: string, + setupReference: string, +) { + const result = spawnSync( + process.execPath, + [ + tsx, + checker, + "--docs", + "--provider", + provider, + ...(toolReference + ? ["--tool-reference", `${provider}=${toolReference}`] + : []), + "--setup-reference", + `${provider}=${setupReference}`, + "--json", + ], + { encoding: "utf8" }, + ); + return { status: result.status, output: `${result.stdout}${result.stderr}` }; +} + afterEach(async () => { await Promise.all( temporary.splice(0).map((path) => rm(path, { recursive: true, force: true })), @@ -159,21 +212,6 @@ afterEach(async () => { }); describe("maintainer drift check", () => { - it("frames each hosted provider's documented credential shape", () => { - expect(hostedAuthorizationHeader("linear", "lin_api_key")).toBe( - "Bearer lin_api_key", - ); - expect(hostedAuthorizationHeader("stripe", "user:secret")).toBe( - `Basic ${Buffer.from("user:secret").toString("base64")}`, - ); - expect(hostedAuthorizationHeader("mixpanel", "user:secret")).toBe( - `Bearer Basic ${Buffer.from("user:secret").toString("base64")}`, - ); - expect(hostedAuthorizationHeader("mixpanel", "Bearer Basic encoded")).toBe( - "Bearer Basic encoded", - ); - }); - it("records the touched endpoints and then reports no drift against them", async () => { const { directory } = await workspace(["cloudflare", "notion"]); const recorded = run(directory, ["cloudflare", "notion"], ["--record"]); @@ -319,19 +357,13 @@ describe("maintainer drift check", () => { ); }); - it("fails clearly when no local hosted credential is exported", async () => { + it("has no credentialed hosted mode", () => { const result = spawnSync(process.execPath, [checker, "--hosted"], { encoding: "utf8", - env: { - ...process.env, - CONNECTA_DRIFT_LINEAR_KEY: undefined, - CONNECTA_DRIFT_STRIPE_KEY: undefined, - CONNECTA_DRIFT_MIXPANEL_KEY: undefined, - } as NodeJS.ProcessEnv, }); expect(result.status).toBe(2); expect(`${result.stdout}${result.stderr}`).toContain( - "no hosted credentials are set", + "unknown argument: --hosted", ); }); @@ -457,10 +489,10 @@ describe("maintainer drift check", () => { }); it.each([ - ["--specs", "linear", "--hosted"], - ["--hosted", "notion", "--specs"], + ["--specs", "linear", "--docs"], + ["--specs", "stripe", "--docs"], ])( - "refuses %s narrowed to %s, which only %s checks", + "refuses %s narrowed to %s, which %s checks", async (half, provider, other) => { const result = spawnSync( process.execPath, @@ -470,12 +502,146 @@ describe("maintainer drift check", () => { // Silently checking nothing and exiting 0 is the wrong failure mode for a // command whose whole value is its exit code. expect(result.status).toBe(2); - expect(`${result.stdout}${result.stderr}`).toContain( - `${provider} is only checked by ${other}`, - ); + const output = `${result.stdout}${result.stderr}`; + expect(output).toContain(`${provider} is `); + expect(output).toContain(other); + expect(output).toContain("which this run did not select"); }, ); + it("checks Vercel's public MCP inventory while naming live schema ownership", async () => { + const { toolReference, setupReference } = + await documentedVercelWorkspace(); + const clean = runDocumented("vercel", toolReference, setupReference); + expect(clean.status).toBe(0); + const cleanReport = JSON.parse(clean.output).docs[0]; + expect(cleanReport).toMatchObject({ + provider: "vercel", + documentedTools: 32, + added: [], + removed: [], + findings: [], + schemaAuthority: "live-tools-list", + schemasVendored: false, + }); + + await writeFile( + toolReference, + `${await readFile(toolReference, "utf8")}\n### new\\_vercel\\_tool\n`, + ); + const drifted = runDocumented("vercel", toolReference, setupReference); + expect(drifted.status).toBe(1); + expect(JSON.parse(drifted.output).docs[0].added).toEqual([ + "new_vercel_tool", + ]); + }); + + it("reads table inventories and treats documented additions as findings", async () => { + const directory = await mkdtemp(join(tmpdir(), "connecta-drift-table-")); + temporary.push(directory); + const reference = join(directory, "stripe.md"); + const rows = [...STRIPE_VETTED_CATALOG.tools.keys()] + .sort() + .map((name) => `| Account | \`${name}\` | Fixture |`) + .join("\n"); + await writeFile( + reference, + `# Stripe MCP\n\n${STRIPE_MCP_ENDPOINT}\n\nOAuth\n\n## Tools\n\n| Resource | Tool | Description |\n| --- | --- | --- |\n${rows}\n\n### Supported API methods\n`, + ); + const clean = runDocumented("stripe", reference, reference); + expect(clean.status).toBe(0); + expect(JSON.parse(clean.output).docs[0]).toMatchObject({ + inventoryChecked: true, + added: [], + schemaAuthority: "live-tools-list", + schemasVendored: false, + }); + + await writeFile( + reference, + `${await readFile(reference, "utf8")}\n| Other | \`new_stripe_tool\` | New |\n`, + ); + // The row landed after the configured section boundary, so move the + // boundary too. This proves the parser checks the named section only. + expect(runDocumented("stripe", reference, reference).status).toBe(0); + const content = await readFile(reference, "utf8"); + await writeFile( + reference, + content.replace( + "### Supported API methods", + "| Other | `new_stripe_tool` | New |\n\n### Supported API methods", + ), + ); + const drifted = runDocumented("stripe", reference, reference); + expect(drifted.status).toBe(1); + expect(JSON.parse(drifted.output).docs[0].added).toContain( + "new_stripe_tool", + ); + }); + + it("reads inline names from Cloudflare and Notion's official doc shapes", async () => { + const directory = await mkdtemp(join(tmpdir(), "connecta-drift-inline-")); + temporary.push(directory); + + const cloudflare = join(directory, "cloudflare.md"); + await writeFile( + cloudflare, + `# Cloudflare MCP\n\nOAuth\n\n## Cloudflare API MCP server\n\nTwo tools: \`search()\` and \`execute()\`.\n\n### Connect to the Cloudflare API MCP server\n\n${CLOUDFLARE_MCP_ENDPOINT}\n`, + ); + const cloudflareResult = runDocumented( + "cloudflare", + cloudflare, + cloudflare, + ); + expect(cloudflareResult.status).toBe(0); + expect(JSON.parse(cloudflareResult.output).docs[0]).toMatchObject({ + documentedTools: CLOUDFLARE_MCP_VETTED_CATALOG.tools.size, + added: [], + findings: [], + }); + + const notionTools = join(directory, "notion-tools.md"); + const notionSetup = join(directory, "notion-setup.md"); + await writeFile( + notionTools, + [...NOTION_MCP_VETTED_CATALOG.tools.keys()] + .sort() + .map((name) => `\`${name}\``) + .join("\n\n"), + ); + await writeFile( + notionSetup, + `# Notion MCP\n\n${NOTION_MCP_ENDPOINT}\n\nOAuth setup.\n`, + ); + const notionResult = runDocumented("notion", notionTools, notionSetup); + expect(notionResult.status).toBe(0); + expect(JSON.parse(notionResult.output).docs[0]).toMatchObject({ + documentedTools: NOTION_MCP_VETTED_CATALOG.tools.size, + added: [], + findings: [], + }); + }); + + it("checks endpoint and OAuth docs when a provider publishes no tool inventory", async () => { + const directory = await mkdtemp(join(tmpdir(), "connecta-drift-setup-")); + temporary.push(directory); + const setup = join(directory, "linear.md"); + await writeFile( + setup, + `# Linear MCP\n\n${LINEAR_MCP_ENDPOINTS["read-write"]}\n\nOAuth setup.\n`, + ); + const result = runDocumented("linear", "", setup); + expect(result.status).toBe(0); + expect(JSON.parse(result.output).docs[0]).toMatchObject({ + inventoryChecked: false, + added: [], + removed: [], + findings: [], + schemaAuthority: "live-tools-list", + schemasVendored: false, + }); + }); + it("commits one well-formed row per touched endpoint", async () => { for (const provider of ["cloudflare", "notion"]) { const manifest = await committed(provider); diff --git a/test/notion-provider.test.ts b/test/notion-provider.test.ts index 1b131281..7e8a99af 100644 --- a/test/notion-provider.test.ts +++ b/test/notion-provider.test.ts @@ -1,8 +1,26 @@ import { afterEach, beforeEach, describe, expect, it, it as test, vi } from "vitest"; import { ConnectorCallError } from "../src/errors.js"; +import type { ToolDef } from "../src/types.js"; +import { + itClassifiesLikeARelease, + mockRemoteMcp, +} from "./fixtures/hosted-provider.js"; + +const mcpMocks = vi.hoisted(() => ({ + listTools: vi.fn<() => Promise>(), + remoteMcp: vi.fn(), +})); + +vi.mock("../src/connectors/remote-mcp.js", async (importOriginal) => ({ + ...(await importOriginal()), + remoteMcp: mcpMocks.remoteMcp, +})); + import { NOTION_API_BASE_URL, NOTION_API_VERSION, + NOTION_MCP_ENDPOINT, + NOTION_MCP_VETTED_CATALOG, notion, } from "../src/providers/notion.js"; import { isExplicitlyReadOnly } from "../src/tool-safety.js"; @@ -39,6 +57,7 @@ const realFetch = globalThis.fetch; beforeEach(() => { calls.length = 0; queued = []; + mockRemoteMcp(mcpMocks); globalThis.fetch = vi.fn(async (input: any, init: any = {}) => { calls.push({ url: String(input), @@ -299,6 +318,60 @@ describe("notion() tool surface", () => { }); }); +describe("notion() MCP interface", () => { + function mcp(): Connector { + return notion("workspace_mcp", { + surface: "mcp", + purpose: "Team knowledge base", + }); + } + + it("binds the explicit MCP interface to Notion's OAuth endpoint", () => { + const callAdmission = { rules: [{ maxConcurrency: 2 }] }; + const connector = notion("workspace_mcp", { + surface: "mcp", + purpose: "Team knowledge base", + callAdmission, + }); + expect(mcpMocks.remoteMcp).toHaveBeenCalledWith( + "workspace_mcp", + expect.objectContaining({ + url: NOTION_MCP_ENDPOINT, + title: "Notion (MCP)", + auth: { type: "oauth" }, + callAdmission, + requireHttps: true, + }), + ); + expect(connector.kind).toBe("mcp"); + expect(connector.credential).toBeUndefined(); + expect((connector.usageGuide as { content: string }).content).toContain( + "live server", + ); + expect(globalThis.fetch).not.toHaveBeenCalled(); + }); + + it("classifies every tool in Notion's published MCP reference", () => { + const counts = { "read-only": 0, additive: 0, destructive: 0 }; + for (const { verdict } of NOTION_MCP_VETTED_CATALOG.tools.values()) { + counts[verdict] += 1; + } + expect(NOTION_MCP_VETTED_CATALOG.tools.size).toBe(34); + expect(counts).toEqual({ + "read-only": 18, + additive: 10, + destructive: 6, + }); + }); + + itClassifiesLikeARelease(mcp, mcpMocks, { + read: ["notion-search", "notion-fetch", "notion-get-users"], + write: "notion-create-comment", + destructive: "notion-update-page", + unknown: ["notion-new-tool", "notion-new-read", "notion-new-write"], + }); +}); + describe("notion() request construction", () => { it("sends the pinned API version and the operator's bearer token", async () => { queue({ body: PAGE_FIXTURE }); diff --git a/test/revenuecat-provider.test.ts b/test/revenuecat-provider.test.ts index 5cb9b462..f6b573fc 100644 --- a/test/revenuecat-provider.test.ts +++ b/test/revenuecat-provider.test.ts @@ -296,14 +296,14 @@ describe("revenuecat()", () => { const verdicts = REVENUECAT_VETTED_CATALOG.tools; const counts = { "read-only": 0, additive: 0, destructive: 0 }; for (const { verdict } of verdicts.values()) counts[verdict] += 1; - // The 2026-08-30 reading of RevenueCat's tool reference: 96 tools, of - // which 95 carry an access column. + // The 2026-09-01 reading of RevenueCat's tool reference: 105 tools, of + // which 104 carry an access column. expect(counts).toEqual({ "read-only": 51, - additive: 15, - destructive: 29, + additive: 17, + destructive: 36, }); - expect(verdicts.size).toBe(95); + expect(verdicts.size).toBe(104); expect(verdicts.get("get-refund-request-preferences")?.verdict).toBe( "read-only", ); diff --git a/test/vercel-provider.test.ts b/test/vercel-provider.test.ts index e86868a5..26bb0802 100644 --- a/test/vercel-provider.test.ts +++ b/test/vercel-provider.test.ts @@ -2,8 +2,26 @@ // the requests, projections, secret handling, and typed failures we own. import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import { ConnectorCallError } from "../src/errors.js"; +import type { ToolDef } from "../src/types.js"; +import { + itClassifiesLikeARelease, + mockRemoteMcp, +} from "./fixtures/hosted-provider.js"; + +const mcpMocks = vi.hoisted(() => ({ + listTools: vi.fn<() => Promise>(), + remoteMcp: vi.fn(), +})); + +vi.mock("../src/connectors/remote-mcp.js", async (importOriginal) => ({ + ...(await importOriginal()), + remoteMcp: mcpMocks.remoteMcp, +})); + import { VERCEL_API_BASE_URL, + VERCEL_MCP_ENDPOINT, + VERCEL_MCP_VETTED_CATALOG, vercel, } from "../src/providers/vercel.js"; import { memoryStorage } from "../src/storage/memory.js"; @@ -40,6 +58,7 @@ function queue(...items: StubResponse[]): void { beforeEach(() => { responses = []; calls.length = 0; + mockRemoteMcp(mcpMocks); globalThis.fetch = vi.fn(async (input: unknown, init: RequestInit = {}) => { const text = responses[0]?.text ?? JSON.stringify(responses[0]?.body ?? {}); @@ -156,6 +175,74 @@ describe("vercel() construction", () => { }); }); +describe("vercel() MCP surface", () => { + function mcp(): Connector { + return vercel("hosting_mcp", { + surface: "mcp", + purpose: "Production deployment diagnosis", + }); + } + + it("binds the explicit MCP surface to Vercel's OAuth endpoint", () => { + const callAdmission = { rules: [{ maxConcurrency: 2 }] }; + const connector = vercel("hosting_mcp", { + surface: "mcp", + purpose: "Production deployment diagnosis", + callAdmission, + }); + expect(mcpMocks.remoteMcp).toHaveBeenCalledWith( + "hosting_mcp", + expect.objectContaining({ + url: VERCEL_MCP_ENDPOINT, + title: "Vercel (MCP)", + auth: { type: "oauth" }, + callAdmission, + requireHttps: true, + }), + ); + expect(connector.kind).toBe("mcp"); + expect(connector.credential).toBeUndefined(); + expect(guide(connector).content).toContain("live server"); + expect(guide(connector).content).toContain("Purchase tools change billing"); + expect(globalThis.fetch).not.toHaveBeenCalled(); + }); + + it("classifies every tool in Vercel's published MCP reference", () => { + const counts = { "read-only": 0, additive: 0, destructive: 0 }; + for (const { verdict } of VERCEL_MCP_VETTED_CATALOG.tools.values()) { + counts[verdict] += 1; + } + expect(VERCEL_MCP_VETTED_CATALOG.tools.size).toBe(32); + expect(counts).toEqual({ + "read-only": 20, + additive: 2, + destructive: 10, + }); + expect( + VERCEL_MCP_VETTED_CATALOG.tools.get("get_purchase_quote")?.verdict, + ).toBe("read-only"); + expect( + VERCEL_MCP_VETTED_CATALOG.tools.get("deploy_to_vercel")?.verdict, + ).toBe("destructive"); + expect( + VERCEL_MCP_VETTED_CATALOG.tools.get("web_fetch_vercel_url")?.verdict, + ).toBe("destructive"); + expect( + VERCEL_MCP_VETTED_CATALOG.tools.get("reply_to_toolbar_thread")?.verdict, + ).toBe("additive"); + for (const record of VERCEL_MCP_VETTED_CATALOG.tools.values()) { + expect(record.schemaDigest).toBeUndefined(); + } + }); + + itClassifiesLikeARelease(mcp, mcpMocks, { + read: ["list_projects", "get_project", "get_deployment"], + write: "reply_to_toolbar_thread", + destructive: "buy_domain", + unknown: ["new_vercel_tool", "peek_at_new_thing", "wreck_new_thing"], + }); +}); + describe("Vercel reads and projections", () => { it("lists teams with opaque cursor pagination", async () => { queue({ @@ -376,6 +463,48 @@ describe("Vercel reads and projections", () => { }); }); + it("accepts the expanded build-event variants without inventing a schema", async () => { + queue({ + body: [ + { + type: "stdout", + created: 100, + payload: { + text: "Building", + deploymentId: "dpl_1", + info: { type: "build", name: "web", serviceName: "frontend" }, + }, + }, + { + type: "alias-assigned", + date: 101, + deploymentId: "dpl_1", + alias: ["app.example.com"], + aliasError: null, + aliasWarning: null, + }, + {}, + ], + }); + const result = await call(connection(), "get_build_logs", { + deploymentId: "dpl_1", + }); + expect(result.events).toEqual([ + { + type: "stdout", + createdAt: 100, + message: "Building", + payload: { + text: "Building", + deploymentId: "dpl_1", + info: { type: "build", name: "web", serviceName: "frontend" }, + }, + }, + { type: "alias-assigned", createdAt: 101 }, + { type: "unknown" }, + ]); + }); + it("parses runtime stream JSON under either content type and caps rows", async () => { queue({ text: