Skip to content

feat(mcp): let clients cache tools/list for an hour and share it across users - #245

Open
mattzcarey wants to merge 1 commit into
mainfrom
feat/tools-list-cache-hint
Open

mattzcarey wants to merge 1 commit into
mainfrom
feat/tools-list-cache-hint

Conversation

@mattzcarey

Copy link
Copy Markdown
Contributor

MCP 2026-07-28 requires ttlMs and cacheScope on tools/list (caching). We sent the SDK default, ttlMs: 0 and private, so no client ever reused a list. Since #235 every caller gets the same tool metadata, and the spec calls public "appropriate for lists of tools ... when they are identical for all users."

  • tools/list returns ttlMs: 3600000, cacheScope: 'public' on both tool surfaces. The lists change only on deploy or the daily spec refresh, and a warm isolate already keeps the spec artifacts for an hour.
  • ?codemode=false now reports serverInfo.name: 'cloudflare-api-endpoints' instead of cloudflare-api. Clients key cached lists by name@version (the TypeScript SDK client does), and both surfaces reported the same identity with different tools. With the hint alone, a client sharing one cache store across both URLs served the Code Mode tools to ?codemode=false. Metrics still report cloudflare-api for both.
  • server/discover keeps the SDK default. 2025-era responses carry no cache fields, so clients on that protocol see no change. ChatGPT's published-app snapshot is unaffected either way.
  • Tool metadata must stay caller-independent. The "tool metadata is identical for every user" tests in tests/non-codemode.test.ts already enforce that, and AGENTS.md now says why.
const server = new McpServer(codemode ? SERVER_INFO : ENDPOINT_TOOLS_SERVER_INFO, {
  cacheHints: { 'tools/list': { ttlMs: TOOL_LIST_TTL_MS, cacheScope: 'public' } }
})

The new client test runs three TypeScript SDK clients on one InMemoryResponseCacheStore, the way a gateway would:

sequenceDiagram
  participant A as Alice, /mcp
  participant B as Bob, /mcp
  participant E as Bob, /mcp?codemode=false
  participant W as Worker
  A->>W: tools/list
  W-->>A: docs, search, execute (public, 1h)
  B->>B: tools/list served from Alice's entry
  E->>W: tools/list (cloudflare-api-endpoints, cache miss)
  W-->>E: docs, get_accounts_workers_scripts
Loading

Touches the same createServer block as #244; the two merge cleanly.

…ss users

MCP 2026-07-28 requires ttlMs and cacheScope on tools/list. We sent the
SDK default, ttlMs: 0 and private, so no client ever reused a list. Since
#235 every caller gets the same tool metadata, which is what public means,
so tools/list now returns ttlMs: 3600000 and cacheScope: 'public' on both
tool surfaces. An hour matches how long a warm isolate keeps the spec
artifacts; the lists change only on deploy or the daily spec refresh.

Clients key cached lists by server name@version, as the TypeScript SDK
client does, and both surfaces reported cloudflare-api. With the hint
alone, a client sharing one cache store across /mcp and
/mcp?codemode=false served the Code Mode tools to the endpoint surface.
The endpoint surface now reports cloudflare-api-endpoints. Metrics still
report cloudflare-api for both.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant