diff --git a/docs/adding-tools.md b/docs/adding-tools.md index aa8be1e59..ae9dbc646 100644 --- a/docs/adding-tools.md +++ b/docs/adding-tools.md @@ -6,10 +6,10 @@ Step-by-step guide for adding new tools to the Sentry MCP server. Not every tool is exposed to every consumer. We rely on several mechanisms to keep the active tool set manageable: -- **Catalog by default** — Most tools are searchable/executable through `search_tools` + `execute_tool` automatically when experimental mode is enabled. Search uses the tool's existing name and description. +- **Catalog by default** — Most tools are searchable/executable through `search_tools` + `execute_tool` automatically. Search uses the tool's existing name and description. - **Catalog registry** — `packages/mcp-core/src/tools/catalog/index.ts` lists ordinary Sentry operation tools. The catalog directory is intentionally flat: one tool entry per file. - **Special tools** — Wrapper/gateway tools (`search_tools`, `execute_tool`, `use_sentry`) live in `packages/mcp-core/src/tools/special/`. They still use the same tool types, but they are kept out of the ordinary catalog. -- **Central direct exposure policy** — `packages/mcp-core/src/tools/surfaces.ts` lists the catalog tools that are also exposed directly through MCP `tools/list`. Stable and experimental modes have separate direct surfaces; experimental mode intentionally keeps more long-tail tools catalog-only because `search_tools` + `execute_tool` are available there. +- **Central direct exposure policy** — `packages/mcp-core/src/tools/surfaces.ts` lists the catalog tools that are also exposed directly through MCP `tools/list`. The direct surface intentionally keeps long-tail tools catalog-only because `search_tools` + `execute_tool` are available as primary primitives. - **`requiredCapabilities`** — Tools declare which project capabilities they need (e.g. `profiles`, `replays`, `traces`). If the upstream project doesn't have a capability enabled, the tool is automatically hidden. - **`experimental` / `hideInExperimentalMode`** — Feature flags for tools that are being tested or replaced. - **Skills & constraints** — The server filters tools based on granted skills and org/project constraints. @@ -30,9 +30,9 @@ Before adding a new tool, consider if it could be: After creating a tool module, add it to `packages/mcp-core/src/tools/catalog/index.ts`. Then update `packages/mcp-core/src/tools/surfaces.ts` only when it should be directly exposed through MCP `tools/list`: -- Add high-frequency, foundational stable tools to `TOP_LEVEL_TOOL_NAMES`. -- Add only the most essential experimental direct tools to `EXPERIMENTAL_TOP_LEVEL_TOOL_NAMES`. Tools omitted from this list remain available through `search_tools` and `execute_tool` after the normal skill, constraint, experimental, and capability filters pass. -- Leave long-tail tools out of the experimental direct surface unless there is a clear reason they need to be visible without discovery. The catalog gateway tools themselves are experimental for now. +- Add only high-frequency, foundational tools to `TOP_LEVEL_TOOL_NAMES`. +- Leave long-tail tools out of the direct surface unless there is a clear reason they need to be visible without discovery. Tools omitted from this list remain available through `search_tools` and `execute_tool` after the normal skill, constraint, experimental, and capability filters pass. +- Keep `EXPERIMENTAL_TOP_LEVEL_TOOL_NAMES` aligned with `TOP_LEVEL_TOOL_NAMES` unless a future experimental mode intentionally needs a different direct surface. - Keep private implementation helpers as plain modules/functions instead of MCP tools. Do not add search-only summaries or catalog-only schemas. `search_tools` indexes the existing tool name and description. diff --git a/docs/architecture.md b/docs/architecture.md index 3fc325f59..91c85fcdf 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -206,15 +206,14 @@ Each tool follows a consistent structure: - Sentry MCP must stay under 25 tools (target: ~20) - Consolidate functionality where possible - Consider parameter variants over new tools -- Long-tail operations can live in the tool catalog and, in experimental mode, - be reached through `search_tools` and `execute_tool` instead of being exposed - directly in `tools/list` +- Long-tail operations can live in the tool catalog and be reached through + `search_tools` and `execute_tool` instead of being exposed directly in + `tools/list` **Tool Catalog and Direct Exposure:** Tool modules define the operation itself. Ordinary tools are catalog-eligible by -default and can be reached through `search_tools` and `execute_tool` when the -catalog gateway is enabled in experimental mode. +default and can be reached through `search_tools` and `execute_tool`. Ordinary operation modules live as flat files under `packages/mcp-core/src/tools/catalog/` and are registered in `packages/mcp-core/src/tools/catalog/index.ts`. @@ -227,8 +226,7 @@ search, schema, and execution helpers. Wrapper/gateway tools such as `packages/mcp-core/src/tools/surfaces.ts` only centralizes the subsets of catalog tools that should also be exposed directly through MCP `tools/list`. -Stable mode keeps a broader direct surface for compatibility. Experimental mode -keeps a smaller direct surface because long-tail operations can be discovered +The direct surface stays small because long-tail operations can be discovered with `search_tools` and invoked with `execute_tool`. The same availability filters (skills, constraints, experimental mode, and required capabilities) apply before either direct registration or catalog diff --git a/docs/claude-code-plugin.md b/docs/claude-code-plugin.md index fa3d98062..7eae0c49c 100644 --- a/docs/claude-code-plugin.md +++ b/docs/claude-code-plugin.md @@ -10,8 +10,8 @@ Two variants are published: | Plugin | MCP URL | Purpose | |--------|---------|---------| -| `sentry-mcp` | `https://mcp.sentry.dev/mcp` | Stable tool defaults | -| `sentry-mcp-experimental` | `https://mcp.sentry.dev/mcp?experimental=1` | Forward-looking tool variants and experimental features | +| `sentry-mcp` | `https://mcp.sentry.dev/mcp` | Default catalog gateway surface | +| `sentry-mcp-experimental` | `https://mcp.sentry.dev/mcp?experimental=1` | Forward-looking feature flags | ## Directory Layout @@ -73,7 +73,7 @@ This script: 1. Imports all tools from `packages/mcp-core/src/tools/index.ts` 2. Imports all skills from `packages/mcp-core/src/skills.ts` 3. Writes `toolDefinitions.json` and `skillDefinitions.json` to `packages/mcp-core/src/` -4. Updates `allowedTools` in both `plugins/sentry-mcp/agents/sentry-mcp.md` and `plugins/sentry-mcp-experimental/agents/sentry-mcp.md`, using the stable direct surface for the stable plugin and the experimental direct surface for the experimental plugin +4. Updates `allowedTools` in both `plugins/sentry-mcp/agents/sentry-mcp.md` and `plugins/sentry-mcp-experimental/agents/sentry-mcp.md`, using the direct surface for each plugin mode The script runs automatically as a `prebuild` and `pretest` hook in `packages/mcp-core/package.json`. Run it explicitly after: - Adding, removing, or renaming tools diff --git a/docs/security.md b/docs/security.md index 4e885b491..65d22d1e7 100644 --- a/docs/security.md +++ b/docs/security.md @@ -118,7 +118,7 @@ The remote deployment intentionally separates trust boundaries: 2. **Authorization is layered**: Sentry scopes, MCP skills, and MCP resource constraints each narrow access differently. 3. **Each session can be path-scoped**: `/mcp/:org` and `/mcp/:org/:project` produce tokens that only work for that scoped MCP URL. 4. **Refresh does not widen access**: MCP refresh reuses the same stored grant props and does not ask Sentry for broader permissions. -5. **Stale or invalid grants fail closed**: legacy grants, missing props, or rejected upstream tokens are revoked or require re-authentication. +5. **Stale or invalid grants fail closed**: missing props, invalid skills, or rejected upstream tokens are revoked or require re-authentication. Already-issued Cloudflare grants with the old `preprod` skill are treated as `inspect`. ## OAuth Architecture diff --git a/packages/mcp-cloudflare/src/client/components/chat/chat.tsx b/packages/mcp-cloudflare/src/client/components/chat/chat.tsx index 50064f83c..b8f67270d 100644 --- a/packages/mcp-cloudflare/src/client/components/chat/chat.tsx +++ b/packages/mcp-cloudflare/src/client/components/chat/chat.tsx @@ -398,7 +398,7 @@ Try asking me things like: title={ endpointMode === "agent" ? "Agent mode: Only use_sentry tool (click to switch to standard)" - : "Standard mode: All 19 tools available (click to switch to agent)" + : "Standard mode: Direct tools plus searchable catalog (click to switch to agent)" } className={`shadow-lg max-xl:order-2 rounded-xl backdrop-blur ${ endpointMode === "agent" ? "ring-4 ring-violet-300/50" : "ring-0" diff --git a/packages/mcp-cloudflare/src/server/lib/mcp-handler.test.ts b/packages/mcp-cloudflare/src/server/lib/mcp-handler.test.ts index 7895f13a8..b38f48755 100644 --- a/packages/mcp-cloudflare/src/server/lib/mcp-handler.test.ts +++ b/packages/mcp-cloudflare/src/server/lib/mcp-handler.test.ts @@ -213,6 +213,22 @@ describe("MCP Handler", () => { expect(await response.text()).toContain("No valid skills"); }); + it("accepts legacy preprod grants as inspect", async () => { + const request = createMcpRequest("tools/list"); + const ctx = createMcpContext({ grantedSkills: ["preprod"] }); + + const response = await mcpHandler.fetch!(request, createTestEnv(), ctx); + + expect(response.status).toBe(200); + const body = await parseSSEResponse<{ + result?: { tools: Array<{ name: string }> }; + }>(response); + const toolNames = body.result?.tools.map((tool) => tool.name) ?? []; + + expect(toolNames).toContain("search_events"); + expect(toolNames).not.toContain("search_docs"); + }); + it("applies the authenticated MCP rate limit per user", async () => { const mockUserRateLimiter = { limit: vi.fn().mockResolvedValue({ success: true }), diff --git a/packages/mcp-cloudflare/src/server/lib/mcp-handler.ts b/packages/mcp-cloudflare/src/server/lib/mcp-handler.ts index 75a52a386..a5645b6ca 100644 --- a/packages/mcp-cloudflare/src/server/lib/mcp-handler.ts +++ b/packages/mcp-cloudflare/src/server/lib/mcp-handler.ts @@ -233,9 +233,11 @@ const mcpHandler: ExportedHandler = { ); } - const { valid: validSkills, invalid: invalidSkills } = parseSkills( - rawProps.grantedSkills as string[], + const grantedSkills = (rawProps.grantedSkills as string[]).map((skill) => + skill === "preprod" ? "inspect" : skill, ); + const { valid: validSkills, invalid: invalidSkills } = + parseSkills(grantedSkills); if (invalidSkills.length > 0) { logWarn("Ignoring invalid skills from OAuth provider", { diff --git a/packages/mcp-cloudflare/src/server/oauth/callback.test.ts b/packages/mcp-cloudflare/src/server/oauth/callback.test.ts index 8cddf42a4..fbde23637 100644 --- a/packages/mcp-cloudflare/src/server/oauth/callback.test.ts +++ b/packages/mcp-cloudflare/src/server/oauth/callback.test.ts @@ -87,6 +87,7 @@ async function createClient(testEnv: Env, name = "Test Client") { async function createSignedCallbackState( clientId: string, resource?: string, + skills: string[] = ["inspect"], ): Promise { const now = Date.now(); const payload: OAuthState = { @@ -94,7 +95,7 @@ async function createSignedCallbackState( clientId, redirectUri: REDIRECT_URI, scope: ["org:read"], - skills: ["inspect"], + skills, ...(resource ? { resource } : {}), }, iat: now, @@ -427,6 +428,29 @@ describe("oauth callback routes", () => { ); }); + it("rejects callback with legacy preprod skill selection", async () => { + const testEnv = createTestEnv(); + const oauthApp = createTestApp(); + const client = await createClient(testEnv); + const cookie = await approveClient(oauthApp, testEnv, client.clientId); + const state = await createSignedCallbackState( + client.clientId, + undefined, + ["preprod"], + ); + + const response = await callCallback(oauthApp, testEnv, { + state, + code: "test-code", + cookie, + }); + + expect(response.status).toBe(400); + expect(await response.text()).toContain( + "You must select at least one valid permission", + ); + }); + it("rejects callback with cookie for different client", async () => { const testEnv = createTestEnv(); const oauthApp = createTestApp(); diff --git a/packages/mcp-core/scripts/generate-definitions.ts b/packages/mcp-core/scripts/generate-definitions.ts index 060c40ef8..d4c70aa0d 100644 --- a/packages/mcp-core/scripts/generate-definitions.ts +++ b/packages/mcp-core/scripts/generate-definitions.ts @@ -191,6 +191,29 @@ async function generateSkillDefinitions() { throw new Error("Failed to import tools from src/tools/index.ts"); } + const defaultVisibleEntries = Object.entries(toolsDefault).flatMap( + ([key, tool]): Array<[string, DefinitionTool]> => { + if (!tool || typeof tool !== "object") { + throw new Error(`Invalid tool: ${key}`); + } + + const t = tool as DefinitionTool; + if ( + surfacesModule.isWrapperToolName(t.name) || + !toolTypesModule.isToolVisibleInMode(t, false) + ) { + return []; + } + + return [[key, t]]; + }, + ); + const defaultDirectToolNames = toolNamesFromEntries( + defaultVisibleEntries.filter(([, tool]) => + surfacesModule.isTopLevelToolName(tool.name, false), + ), + ); + // Build tools array for each skill const skillsWithTools = skills.map((skill) => { const skillTools: Array<{ @@ -217,6 +240,10 @@ async function generateSkillDefinitions() { return [toolKey, t.name]; }), ); + const availableToolNames = new Set([ + ...skillToolNames, + ...defaultDirectToolNames, + ]); for (const [, tool] of skillToolEntries) { const t = tool as DefinitionTool; @@ -224,8 +251,8 @@ async function generateSkillDefinitions() { name: t.name, description: toolTypesModule.resolveDescription(t.description, { experimentalMode: false, - availableToolNames: skillToolNames, - directToolNames: skillToolNames, + availableToolNames, + directToolNames: defaultDirectToolNames, }), requiredScopes: Array.isArray(t.requiredScopes) ? t.requiredScopes : [], }); diff --git a/packages/mcp-core/src/internal/formatting.test.ts b/packages/mcp-core/src/internal/formatting.test.ts index 9e7ab5648..c9def26da 100644 --- a/packages/mcp-core/src/internal/formatting.test.ts +++ b/packages/mcp-core/src/internal/formatting.test.ts @@ -191,7 +191,7 @@ describe("formatIssueOutput", () => { }); expect(output).toContain( - "- Full distributed trace and span tree: Use the Sentry tool `get_sentry_resource(resourceType='trace', organizationSlug='sentry-mcp-evals', resourceId='3032af8bcdfe4423b937fc5c041d5d82')`", + "- Full distributed trace and span tree: Use the Sentry tool `get_sentry_resource`", ); expect(output).toContain( "- Related span search: Related span search is not available in this session", diff --git a/packages/mcp-core/src/internal/tool-helpers/tool-call-formatting.test.ts b/packages/mcp-core/src/internal/tool-helpers/tool-call-formatting.test.ts index ca14b55fb..0955de386 100644 --- a/packages/mcp-core/src/internal/tool-helpers/tool-call-formatting.test.ts +++ b/packages/mcp-core/src/internal/tool-helpers/tool-call-formatting.test.ts @@ -15,12 +15,10 @@ describe("tool call formatting", () => { }, experimentalMode: true, }), - ).toBe( - "Use the Sentry tool `search_events(organizationSlug='my-org', query='level:error')`", - ); + ).toBe("Use the Sentry tool `search_events`"); }); - it("formats catalog guidance when a tool is not top-level in the current mode", () => { + it("formats tool-name guidance when a tool is not top-level in the current mode", () => { expect( formatToolCallInstruction({ toolName: "get_doc", @@ -29,12 +27,10 @@ describe("tool call formatting", () => { }, experimentalMode: true, }), - ).toBe( - 'Use the Sentry tool `get_doc`: search `search_tools(query=\'get_doc\')`, then call `execute_tool` with name `get_doc` and arguments `{"path":"/platforms/javascript/guides/nextjs.md"}`', - ); + ).toBe("Use the Sentry tool `get_doc`"); }); - it("formats purpose text before catalog gateway steps", () => { + it("formats purpose text in tool-name guidance", () => { expect( formatToolCallInstruction({ toolName: "find_releases", @@ -50,7 +46,7 @@ describe("tool call formatting", () => { purpose: "to list releases and their details", }), ).toBe( - 'Use the Sentry tool `find_releases` to list releases and their details: search `search_tools(query=\'find_releases\')`, then call `execute_tool` with name `find_releases` and arguments `{"organizationSlug":"my-org"}`', + "Use the Sentry tool `find_releases` to list releases and their details", ); }); @@ -66,7 +62,7 @@ describe("tool call formatting", () => { ).toBe("Release listing is not available"); }); - it("does not format non-top-level stable tools as direct calls", () => { + it("formats tool-name guidance for non-top-level tools in default mode", () => { expect( formatToolCallInstruction({ toolName: "get_snapshot_image", @@ -77,9 +73,7 @@ describe("tool call formatting", () => { }, experimentalMode: false, }), - ).toBe( - "The Sentry tool `get_snapshot_image` is not available in this session", - ); + ).toBe("Use the Sentry tool `get_snapshot_image`"); }); it("uses fallback guidance for unavailable tools", () => { @@ -100,7 +94,7 @@ describe("tool call formatting", () => { ).toBe("Release listing is not available"); }); - it("uses catalog guidance only when the target tool is available", () => { + it("uses tool-name guidance only when the target tool is available", () => { expect( formatToolCallInstruction({ toolName: "find_releases", @@ -114,9 +108,7 @@ describe("tool call formatting", () => { "execute_tool", ]), }), - ).toBe( - 'Use the Sentry tool `find_releases`: search `search_tools(query=\'find_releases\')`, then call `execute_tool` with name `find_releases` and arguments `{"organizationSlug":"my-org"}`', - ); + ).toBe("Use the Sentry tool `find_releases`"); }); it("escapes arguments in direct call examples", () => { diff --git a/packages/mcp-core/src/internal/tool-helpers/tool-call-formatting.ts b/packages/mcp-core/src/internal/tool-helpers/tool-call-formatting.ts index 15552b092..8a8fbceff 100644 --- a/packages/mcp-core/src/internal/tool-helpers/tool-call-formatting.ts +++ b/packages/mcp-core/src/internal/tool-helpers/tool-call-formatting.ts @@ -26,10 +26,6 @@ function formatArguments(args: Record): string { .join(", "); } -function formatArgumentsJson(args: Record): string { - return JSON.stringify(args); -} - function formatPurpose(purpose: string | undefined): string { return purpose ? ` ${purpose}` : ""; } @@ -62,11 +58,14 @@ export function formatToolCall({ return `${toolName}(${formattedArgs})`; } +/** + * Formats user-facing tool guidance as a direct call, catalog gateway call, or + * fallback message based on the current session's available/direct tools. + */ export function formatToolCallInstruction({ toolName, - arguments: args = {}, + arguments: _args = {}, experimentalMode, - searchQuery = toolName, purpose, availableToolNames, directToolNames, @@ -75,7 +74,6 @@ export function formatToolCallInstruction({ toolName: string; arguments?: Record; experimentalMode: boolean; - searchQuery?: string; purpose?: string; availableToolNames?: ReadonlySet; directToolNames?: ReadonlySet; @@ -87,28 +85,17 @@ export function formatToolCallInstruction({ targetAvailable && isDirectTool(toolName, experimentalMode, directToolNames) ) { - return `Use the Sentry tool \`${formatToolCall({ - toolName, - arguments: args, - })}\`${formatPurpose(purpose)}`; + return `Use the Sentry tool \`${toolName}\`${formatPurpose(purpose)}`; } const catalogGatewayAvailable = - experimentalMode && isToolAvailable("search_tools", availableToolNames) && isToolAvailable("execute_tool", availableToolNames) && isDirectTool("search_tools", experimentalMode, directToolNames) && isDirectTool("execute_tool", experimentalMode, directToolNames); if (targetAvailable && catalogGatewayAvailable) { - return [ - `Use the Sentry tool \`${toolName}\`${formatPurpose(purpose)}:`, - `search \`${formatToolCall({ - toolName: "search_tools", - arguments: { query: searchQuery }, - })}\`,`, - `then call \`execute_tool\` with name \`${toolName}\` and arguments \`${formatArgumentsJson(args)}\``, - ].join(" "); + return `Use the Sentry tool \`${toolName}\`${formatPurpose(purpose)}`; } return ( diff --git a/packages/mcp-core/src/schema.ts b/packages/mcp-core/src/schema.ts index 52893d1fa..77d91dd04 100644 --- a/packages/mcp-core/src/schema.ts +++ b/packages/mcp-core/src/schema.ts @@ -24,7 +24,7 @@ export const ParamTeamSlug = z .trim() .superRefine(validateSlug) .describe( - "The team's slug. You can find a list of existing teams in an organization using the `find_teams()` tool.", + "The team's slug. You can find a list of existing teams in an organization with the Sentry tool `find_teams`.", ); export const ParamProjectSlug = z diff --git a/packages/mcp-core/src/server.test.ts b/packages/mcp-core/src/server.test.ts index 1f708414a..a9e094d58 100644 --- a/packages/mcp-core/src/server.test.ts +++ b/packages/mcp-core/src/server.test.ts @@ -3,12 +3,7 @@ import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js"; import { setUser } from "@sentry/core"; import { beforeEach, describe, expect, it, vi } from "vitest"; import { buildServer } from "./server"; -import tools from "./tools"; -import { - CATALOG_INFRASTRUCTURE_TOOL_NAMES, - getTopLevelToolNames, -} from "./tools/surfaces"; -import { type ToolConfig, isToolVisibleInMode } from "./tools/types"; +import type { ToolConfig } from "./tools/types"; import type { ServerContext } from "./types"; // Mock the Sentry core module @@ -96,26 +91,22 @@ function getStructuredContent>( return structuredContent as T; } -const DOC_TOOL_NAMES = new Set(["search_docs", "get_doc"]); -const CATALOG_GATEWAY_TOOL_NAMES = new Set( - CATALOG_INFRASTRUCTURE_TOOL_NAMES, -); - -function getExpectedTopLevelToolNames({ - docs = false, - experimental = false, -}: { - docs?: boolean; - experimental?: boolean; -} = {}) { - return [...getTopLevelToolNames({ experimentalMode: experimental })] - .filter((toolName) => docs || !DOC_TOOL_NAMES.has(toolName)) - .filter((toolName) => isToolVisibleInMode(tools[toolName], experimental)) - .filter( - (toolName) => experimental || !CATALOG_GATEWAY_TOOL_NAMES.has(toolName), - ) - .sort(); -} +const DEFAULT_DIRECT_TOOL_NAMES = [ + "analyze_issue_with_seer", + "execute_tool", + "find_organizations", + "find_projects", + "get_sentry_resource", + "search_events", + "search_issues", + "search_tools", + "update_issue", + "whoami", +].sort(); +const DEFAULT_DIRECT_TOOL_NAMES_WITH_DOCS = [ + ...DEFAULT_DIRECT_TOOL_NAMES, + "search_docs", +].sort(); describe("buildServer", () => { beforeEach(() => { @@ -262,7 +253,7 @@ describe("buildServer", () => { }); }); - describe("capability-based tool filtering (experimental)", () => { + describe("capability-based tool filtering", () => { it("hides tools when project lacks required capabilities", () => { const server = buildServer({ experimentalMode: true, @@ -409,7 +400,7 @@ describe("buildServer", () => { expect(toolNames).not.toContain("multi_cap_tool"); }); - it("does not filter by capabilities when experimentalMode is false", () => { + it("hides tools with unmet capabilities when experimentalMode is false", () => { const server = buildServer({ experimentalMode: false, context: { @@ -434,8 +425,7 @@ describe("buildServer", () => { }); const toolNames = getRegisteredToolNames(server); - // All tools should be visible when experimentalMode is false - expect(toolNames).toContain("tool_with_caps"); + expect(toolNames).not.toContain("tool_with_caps"); expect(toolNames).toContain("tool_without_caps"); }); }); @@ -580,12 +570,12 @@ describe("buildServer", () => { expect(toolNames).not.toContain("get_snapshot"); expect(toolNames).not.toContain("get_snapshot_image"); expect(toolNames).not.toContain("get_snapshot_details"); - expect(toolNames).not.toContain("search_tools"); - expect(toolNames).not.toContain("execute_tool"); + expect(toolNames).toContain("search_tools"); + expect(toolNames).toContain("execute_tool"); expect(toolNames.length).toBeGreaterThan(0); }); - it("filters experimental default tools when experimentalMode is false", () => { + it("includes catalog gateway tools when experimentalMode is false", () => { const server = buildServer({ context: baseContext, experimentalMode: false, @@ -597,8 +587,8 @@ describe("buildServer", () => { expect(toolNames).toContain("get_sentry_resource"); expect(toolNames).not.toContain("get_issue_details"); expect(toolNames).not.toContain("get_trace_details"); - expect(toolNames).not.toContain("search_tools"); - expect(toolNames).not.toContain("execute_tool"); + expect(toolNames).toContain("search_tools"); + expect(toolNames).toContain("execute_tool"); }); it("includes all default tools when experimentalMode is true", () => { @@ -639,11 +629,10 @@ describe("buildServer", () => { const registeredTools = await listRegisteredTools(server); const toolNames = registeredTools.map((tool) => tool.name).sort(); - const expectedToolNames = getExpectedTopLevelToolNames(); - expect(toolNames).toEqual(expectedToolNames); - expect(toolNames).not.toContain("search_tools"); - expect(toolNames).not.toContain("execute_tool"); + expect(toolNames).toEqual(DEFAULT_DIRECT_TOOL_NAMES); + expect(toolNames).toContain("search_tools"); + expect(toolNames).toContain("execute_tool"); expect(toolNames).not.toContain("use_sentry"); expect(toolNames).not.toContain("search_docs"); expect(toolNames).not.toContain("get_doc"); @@ -669,35 +658,38 @@ describe("buildServer", () => { const registeredTools = await listRegisteredTools(server); const toolNames = registeredTools.map((tool) => tool.name).sort(); - expect(toolNames).toEqual(getExpectedTopLevelToolNames({ docs: true })); + expect(toolNames).toEqual(DEFAULT_DIRECT_TOOL_NAMES_WITH_DOCS); expect(toolNames).toContain("search_docs"); - expect(toolNames).toContain("get_doc"); - expect(toolNames).not.toContain("search_tools"); - expect(toolNames).not.toContain("execute_tool"); + expect(toolNames).not.toContain("get_doc"); + expect(toolNames).toContain("search_tools"); + expect(toolNames).toContain("execute_tool"); }); - it("discloses catalog gateway tools through MCP tools/list in experimental mode", async () => { - const server = buildServer({ + it("keeps the same direct tool surface in experimental mode", async () => { + const defaultServer = buildServer({ + context: baseContext, + }); + const experimentalServer = buildServer({ context: baseContext, experimentalMode: true, }); - const registeredTools = await listRegisteredTools(server); - const toolNames = registeredTools.map((tool) => tool.name).sort(); - - expect(toolNames).toEqual( - getExpectedTopLevelToolNames({ experimental: true }), - ); - expect(toolNames).toContain("search_tools"); - expect(toolNames).toContain("execute_tool"); + const defaultToolNames = (await listRegisteredTools(defaultServer)) + .map((tool) => tool.name) + .sort(); + const experimentalToolNames = ( + await listRegisteredTools(experimentalServer) + ) + .map((tool) => tool.name) + .sort(); + + expect(experimentalToolNames).toEqual(defaultToolNames); + expect(experimentalToolNames).toEqual(DEFAULT_DIRECT_TOOL_NAMES); }); - it("does not advertise stable-only hidden snapshot image calls in stable descriptions", async () => { + it("advertises Sentry tool guidance for snapshot image tools", async () => { const server = buildServer({ - context: { - ...baseContext, - grantedSkills: new Set([...baseContext.grantedSkills!, "preprod"]), - }, + context: baseContext, }); const registeredTools = await listRegisteredTools(server); @@ -706,10 +698,7 @@ describe("buildServer", () => { ); expect(getSentryResource?.description).toContain( - "Full-resolution snapshot image bytes are not available in this session", - ); - expect(getSentryResource?.description).not.toContain( - "get_snapshot_image(", + "Use the Sentry tool `get_snapshot_image` for full-resolution image bytes", ); }); @@ -731,15 +720,11 @@ describe("buildServer", () => { "- **Find releases**: Release listing is not available in this session", ); expect(text).not.toContain("find_releases("); - expect(text).not.toContain( - "search `search_tools(query='find_releases')`", - ); }); - it("keeps long-tail tools catalog-only in experimental mode", async () => { + it("keeps long-tail tools catalog-only by default", async () => { const server = buildServer({ context: baseContext, - experimentalMode: true, }); const toolNames = getRegisteredToolNames(server); @@ -772,18 +757,20 @@ describe("buildServer", () => { expect(registeredTools.map((tool) => tool.name)).toEqual(["use_sentry"]); }); - it("keeps preprod tools catalog-only while enforcing their skill gate", async () => { - const withoutPreprod = buildServer({ - context: baseContext, - experimentalMode: true, + it("keeps snapshot tools catalog-only while enforcing the inspect skill gate", async () => { + const withoutInspect = buildServer({ + context: { + ...baseContext, + grantedSkills: new Set(["triage"]), + }, }); - const withoutPreprodToolNames = getRegisteredToolNames(withoutPreprod); - expect(withoutPreprodToolNames).not.toContain("get_snapshot"); - expect(withoutPreprodToolNames).not.toContain("get_snapshot_image"); - expect(withoutPreprodToolNames).not.toContain("get_latest_base_snapshot"); + const withoutInspectToolNames = getRegisteredToolNames(withoutInspect); + expect(withoutInspectToolNames).not.toContain("get_snapshot"); + expect(withoutInspectToolNames).not.toContain("get_snapshot_image"); + expect(withoutInspectToolNames).not.toContain("get_latest_base_snapshot"); const hiddenResult = await callRegisteredTool( - withoutPreprod, + withoutInspect, "search_tools", { query: "snapshot", @@ -797,26 +784,16 @@ describe("buildServer", () => { "get_snapshot", ); - const withPreprod = buildServer({ - context: { - ...baseContext, - grantedSkills: new Set([ - "inspect", - "triage", - "project-management", - "seer", - "preprod", - ]), - }, - experimentalMode: true, + const withInspect = buildServer({ + context: baseContext, }); - const withPreprodToolNames = getRegisteredToolNames(withPreprod); - expect(withPreprodToolNames).not.toContain("get_snapshot"); - expect(withPreprodToolNames).not.toContain("get_snapshot_image"); - expect(withPreprodToolNames).not.toContain("get_latest_base_snapshot"); + const withInspectToolNames = getRegisteredToolNames(withInspect); + expect(withInspectToolNames).not.toContain("get_snapshot"); + expect(withInspectToolNames).not.toContain("get_snapshot_image"); + expect(withInspectToolNames).not.toContain("get_latest_base_snapshot"); const visibleResult = await callRegisteredTool( - withPreprod, + withInspect, "search_tools", { query: "snapshot", @@ -856,7 +833,6 @@ describe("buildServer", () => { it("exposes catalog tools with conservative safety annotations", async () => { const server = buildServer({ context: baseContext, - experimentalMode: true, }); const registeredTools = await listRegisteredTools(server); @@ -901,7 +877,6 @@ describe("buildServer", () => { projectSlug: null, }, }, - experimentalMode: true, }); const result = await callRegisteredTool(server, "search_tools", { @@ -950,7 +925,6 @@ describe("buildServer", () => { regionUrl: "https://us.sentry.io", }, }, - experimentalMode: true, }); const result = await callRegisteredTool(server, "search_tools", { @@ -984,7 +958,6 @@ describe("buildServer", () => { it("search_tools includes catalog-only tools that are not directly registered", async () => { const server = buildServer({ context: baseContext, - experimentalMode: true, }); const toolNames = getRegisteredToolNames(server); @@ -1003,10 +976,50 @@ describe("buildServer", () => { ); }); + it("search_tools and execute_tool enforce project capabilities by default", async () => { + const server = buildServer({ + context: { + ...baseContext, + constraints: { + organizationSlug: "sentry-mcp-evals", + projectSlug: "cloudflare-mcp", + projectCapabilities: { + profiles: false, + replays: false, + logs: true, + traces: true, + }, + }, + }, + }); + + const result = await callRegisteredTool(server, "search_tools", { + query: "replay details", + limit: 10, + }); + const payload = getStructuredContent<{ + results: Array<{ name: string }>; + }>(result); + + expect(payload.results.map((tool) => tool.name)).not.toContain( + "get_replay_details", + ); + const executeResult = await callRegisteredTool(server, "execute_tool", { + name: "get_replay_details", + arguments: { + replayId: "7e07485f12f9416b8b1426260799b51f", + }, + }); + + expect(executeResult).toMatchObject({ isError: true }); + expect(getTextContent(executeResult)).toContain( + 'Tool "get_replay_details" is not available in this session', + ); + }); + it("execute_tool dispatches to an available tool", async () => { const server = buildServer({ context: baseContext, - experimentalMode: true, }); const result = await callRegisteredTool(server, "execute_tool", { @@ -1020,7 +1033,6 @@ describe("buildServer", () => { it("execute_tool dispatches to a catalog-only tool", async () => { const server = buildServer({ context: baseContext, - experimentalMode: true, }); const toolNames = getRegisteredToolNames(server); @@ -1049,7 +1061,6 @@ describe("buildServer", () => { regionUrl: "https://us.sentry.io", }, }, - experimentalMode: true, }); const result = await callRegisteredTool(server, "execute_tool", { @@ -1064,13 +1075,25 @@ describe("buildServer", () => { ); }); - it("exposes get_profile_details safety annotations through tool metadata", async () => { + it("exposes catalog-only tool safety annotations through search_tools", async () => { const server = buildServer({ context: baseContext, }); - const registeredTools = await listRegisteredTools(server); - const getProfileDetailsTool = registeredTools.find( + const registeredToolNames = getRegisteredToolNames(server); + expect(registeredToolNames).not.toContain("get_profile_details"); + + const result = await callRegisteredTool(server, "search_tools", { + query: "profile details", + limit: 5, + }); + const payload = getStructuredContent<{ + results: Array<{ + name: string; + annotations: Record; + }>; + }>(result); + const getProfileDetailsTool = payload.results.find( (tool) => tool.name === "get_profile_details", ); @@ -1083,7 +1106,7 @@ describe("buildServer", () => { }); }); - it("removes constrained organizationSlug from get_replay_details schema", () => { + it("removes constrained organizationSlug from catalog tool schemas", async () => { const server = buildServer({ context: { ...baseContext, @@ -1094,30 +1117,24 @@ describe("buildServer", () => { }, }); - const replayTool = ( - server as unknown as { - _registeredTools?: Record< - string, - { - inputSchema?: { - shape?: Record; - _def?: { shape?: () => Record }; - }; - } - >; - } - )._registeredTools?.get_replay_details; - const inputSchema = replayTool?.inputSchema; - const shape = - typeof inputSchema?.shape === "object" - ? inputSchema.shape - : (inputSchema?._def?.shape?.() ?? {}); - - expect(Object.keys(shape).sort()).toEqual([ - "regionUrl", - "replayId", - "replayUrl", - ]); + const result = await callRegisteredTool(server, "search_tools", { + query: "replay details", + limit: 1, + }); + const payload = getStructuredContent<{ + results: Array<{ + name: string; + inputSchema: { + properties?: Record; + }; + }>; + }>(result); + const replayTool = payload.results[0]; + + expect(replayTool?.name).toBe("get_replay_details"); + expect( + Object.keys(replayTool?.inputSchema.properties ?? {}).sort(), + ).toEqual(["regionUrl", "replayId", "replayUrl"]); }); }); }); diff --git a/packages/mcp-core/src/skillDefinitions.json b/packages/mcp-core/src/skillDefinitions.json index cf033c35b..65b727bbd 100644 --- a/packages/mcp-core/src/skillDefinitions.json +++ b/packages/mcp-core/src/skillDefinitions.json @@ -5,7 +5,7 @@ "description": "Search for errors, analyze traces, and explore event details", "defaultEnabled": true, "order": 1, - "toolCount": 19, + "toolCount": 22, "tools": [ { "name": "find_organizations", @@ -52,6 +52,11 @@ "description": "Get tag value distribution for a specific Sentry issue.\n\nUse this tool when you need to:\n- Understand how an issue is distributed across different tag values\n- Get aggregate counts of unique tag values (e.g., 'how many unique URLs are affected')\n- Analyze which browsers, environments, or URLs are most impacted by an issue\n- View the tag distributions page data programmatically\n\nCommon tag keys:\n- `url`: Request URLs affected by the issue\n- `browser`: Browser types and versions\n- `browser.name`: Browser names only\n- `os`: Operating systems\n- `environment`: Deployment environments (production, staging, etc.)\n- `release`: Software releases\n- `device`: Device types\n- `user`: Affected users\n\n\n### Get URL distribution for an issue\n```\nget_issue_tag_values(organizationSlug='my-organization', issueId='PROJECT-123', tagKey='url')\n```\n\n### Get browser distribution using issue URL\n```\nget_issue_tag_values(issueUrl='https://sentry.io/issues/PROJECT-123/', tagKey='browser')\n```\n\n### Get environment distribution\n```\nget_issue_tag_values(organizationSlug='my-organization', issueId='PROJECT-123', tagKey='environment')\n```\n\n\n\n- If user provides a Sentry URL, pass the ENTIRE URL to issueUrl parameter unchanged\n- Common tag keys: url, browser, browser.name, os, environment, release, device, user\n- Tag keys are case-sensitive\n", "requiredScopes": ["event:read"] }, + { + "name": "get_latest_base_snapshot", + "description": "Get the latest UI screenshots/images for an app from the preprod snapshot system.\n\nThis is the primary tool for retrieving app screenshots — not search_events or search_issues.\n\nUse this tool when you need to:\n- Get screenshots, screens, golden images, or reference images for an app\n- Find what the current UI looks like (latest screenshots from the main/default branch)\n- List available snapshots or browse images before requesting specific ones\n- Look up dark mode, light mode, or other variant screenshots\n- Understand what baseline images exist when investigating snapshot test or visual regression CI failures\n\nThe appId parameter is the app identifier (e.g. 'sentry-frontend', 'com.emergetools.hackernews').\nReturns compact image metadata (display_name, image_file_name, group, description) for every image.\n\n\n### Get the latest screenshots for an app\n\n```\nget_latest_base_snapshot(organizationSlug=\"sentry\", appId=\"sentry-frontend\", project=\"frontend\")\n```\n\n### Get the latest screenshots for a specific branch\n\n```\nget_latest_base_snapshot(organizationSlug=\"sentry\", appId=\"sentry-frontend\", project=\"frontend\", branch=\"main\")\n```\n\n\n\n- The response includes compact metadata per image. Scan the list to find images matching what you need (e.g. filter by group or name containing 'button').\n- To view a specific image, use get_sentry_resource(url='?selectedSnapshot=').\n- If you need to investigate a specific snapshot comparison, use get_sentry_resource with the snapshot URL.\n", + "requiredScopes": ["project:read"] + }, { "name": "get_profile", "description": "Analyze CPU profiling data to identify performance bottlenecks and detect regressions.\n\nUSE THIS TOOL WHEN:\n- User asks why a specific endpoint/transaction is slow\n- User wants to understand where CPU time is spent\n- User asks about performance bottlenecks\n- User wants to compare performance between time periods\n- User shares a Sentry profile URL\n\nRETURNS:\n- Hot paths (call stacks consuming the most CPU time)\n- Performance percentiles (p75, p95, p99) for each function\n- User code vs library code breakdown\n- Actionable recommendations for optimization\n- Regression analysis when comparing periods\n\n\n### Analyze from URL (with transaction name)\n```\nget_profile(\n profileUrl='https://my-org.sentry.io/explore/profiling/profile/backend/flamegraph/?profilerId=abc123',\n transactionName='/api/users'\n)\n```\n\n### Analyze by transaction name\n```\nget_profile(\n organizationSlug='my-org',\n transactionName='/api/users',\n projectSlugOrId='backend'\n)\n```\n\n### Compare performance between periods\n```\nget_profile(\n organizationSlug='my-org',\n transactionName='/api/users',\n projectSlugOrId='backend',\n statsPeriod='7d',\n compareAgainstPeriod='14d'\n)\n```\n\n\n\n- Use `focusOnUserCode: true` (default) to filter out library code\n- High p99 relative to p75 indicates inconsistent performance\n- Use compareAgainstPeriod to detect regressions over time\n- Transaction names are case-sensitive\n", @@ -74,9 +79,19 @@ }, { "name": "get_sentry_resource", - "description": "Fetch a Sentry resource by URL, or by resourceType plus resourceId.\nPass a Sentry URL directly when possible; the resource type is auto-detected.\n\nSupports issues, events, traces, spans, AI conversations, breadcrumbs, replays, preprod snapshots, and snapshot images.\nTrace lookups return a condensed overview by default.\n\nAI Conversations: A conversation is a set of spans sharing the same gen_ai.conversation.id. Use resourceType='ai_conversation' with a conversation ID to fetch all spans for that conversation. To discover or list conversation IDs, use search_events with dataset='spans' and query='has:gen_ai.conversation.id'. Conversations are NOT issues — do not use search_issues for conversation queries.\n\nFor preprod snapshot URLs (matching 'sentry.io/preprod/snapshots/'):\n- Without ?selectedSnapshot=: returns the snapshot diff summary (changed, added, removed images)\n- With ?selectedSnapshot=: returns the image preview and metadata. Full-resolution snapshot image bytes are not available in this session.\n\nResource IDs:\n- span: :\n- snapshot: \n- snapshotImage: :\n\n\nget_sentry_resource(url='https://sentry.io/issues/PROJECT-123/')\nget_sentry_resource(resourceType='issue', organizationSlug='my-org', resourceId='PROJECT-123')\nget_sentry_resource(resourceType='span', organizationSlug='my-org', resourceId=':')\nget_sentry_resource(resourceType='ai_conversation', organizationSlug='my-org', resourceId='conversation-123')\nget_sentry_resource(url='https://sentry.sentry.io/preprod/snapshots/123/')\nget_sentry_resource(url='https://sentry.sentry.io/preprod/snapshots/123/?selectedSnapshot=login_screen.png')\n", + "description": "Fetch a Sentry resource by URL, or by resourceType plus resourceId.\nPass a Sentry URL directly when possible; the resource type is auto-detected.\n\nSupports issues, events, traces, spans, AI conversations, breadcrumbs, replays, preprod snapshots, and snapshot images.\nTrace lookups return a condensed overview by default.\n\nAI Conversations: A conversation is a set of spans sharing the same gen_ai.conversation.id. Use resourceType='ai_conversation' with a conversation ID to fetch all spans for that conversation. To discover or list conversation IDs, use search_events with dataset='spans' and query='has:gen_ai.conversation.id'. Conversations are NOT issues — do not use search_issues for conversation queries.\n\nFor preprod snapshot URLs (matching 'sentry.io/preprod/snapshots/'):\n- Without ?selectedSnapshot=: returns the snapshot diff summary (changed, added, removed images)\n- With ?selectedSnapshot=: returns the image preview and metadata. Use the Sentry tool `get_snapshot_image` for full-resolution image bytes.\n\nResource IDs:\n- span: :\n- snapshot: \n- snapshotImage: :\n\n\nget_sentry_resource(url='https://sentry.io/issues/PROJECT-123/')\nget_sentry_resource(resourceType='issue', organizationSlug='my-org', resourceId='PROJECT-123')\nget_sentry_resource(resourceType='span', organizationSlug='my-org', resourceId=':')\nget_sentry_resource(resourceType='ai_conversation', organizationSlug='my-org', resourceId='conversation-123')\nget_sentry_resource(url='https://sentry.sentry.io/preprod/snapshots/123/')\nget_sentry_resource(url='https://sentry.sentry.io/preprod/snapshots/123/?selectedSnapshot=login_screen.png')\n", "requiredScopes": ["event:read", "project:read"] }, + { + "name": "get_snapshot", + "description": "Get a preprod snapshot comparison summary, including metadata, counts, and changed image sections.\n\nUse this tool when you need to:\n- Investigate a failed snapshot test from CI\n- Review what changed in a specific preprod snapshot\n- Browse snapshot image file names before viewing a specific image\n\nPass organizationSlug and snapshotId. Use get_sentry_resource for snapshot URLs.\nCompact output is returned by default. Set showUnmodified=true to list unchanged and skipped images separately.\n\n\n### Browse a snapshot\n\n```\nget_snapshot(organizationSlug=\"sentry\", snapshotId=\"231949\")\n```\n\n### Include unchanged and skipped images\n\n```\nget_snapshot(organizationSlug=\"sentry\", snapshotId=\"231949\", showUnmodified=true)\n```\n\n\n\n- Use the Sentry tool `get_snapshot_image` to view a specific image preview or full-resolution image bytes.\n- Use get_sentry_resource when starting from a Sentry snapshot URL.\n- The diff percent field shows what percentage of pixels changed (0-100).\n- showUnmodified=true is useful when a diff snapshot has no changed image sections.\n", + "requiredScopes": ["project:read"] + }, + { + "name": "get_snapshot_image", + "description": "Get metadata and image content for one image in a preprod snapshot.\n\nUse this tool when you need to:\n- View the current, previous, or diff image for a snapshot entry\n- Inspect metadata and context for a specific snapshot image\n- Fetch full-resolution image bytes when preview images are insufficient; full-resolution images can substantially increase response size and token usage\n\nPass organizationSlug, snapshotId, and imageIdentifier. Use get_sentry_resource for snapshot URLs.\nPreview images are returned by default; set imageResolution=full for original bytes when needed, but expect higher response size and token usage.\n\n\n### View a preview image\n\n```\nget_snapshot_image(organizationSlug=\"sentry\", snapshotId=\"231949\", imageIdentifier=\"login_screen.png\")\n```\n\n### Fetch original full-resolution bytes\n\n```\nget_snapshot_image(organizationSlug=\"sentry\", snapshotId=\"231949\", imageIdentifier=\"login_screen.png\", imageResolution=\"full\")\n```\n\n\n\n- Use get_snapshot first if you need to discover available image file names.\n- Use get_sentry_resource when starting from a Sentry snapshot URL.\n- imageIdentifier values may include slashes; pass the full image_file_name exactly as shown by get_snapshot.\n", + "requiredScopes": ["project:read"] + }, { "name": "get_trace_details", "description": "Get detailed information about a specific Sentry trace by ID.\n\nUSE THIS TOOL WHEN USERS:\n- Provide a specific trace ID (e.g., 'a4d1aae7216b47ff8117cf4e09ce9d0a')\n- Ask to 'show me trace [TRACE-ID]', 'explain trace [TRACE-ID]'\n- Want high-level overview and link to view trace details in Sentry\n- Need trace statistics and span breakdown\n- Want an overview first, then a guided pivot into additional spans or events\n\nDO NOT USE for:\n- General searching for traces (use search_events with trace queries)\n- Complete span enumeration or branch-by-branch reconstruction (use search_events scoped to the trace)\n\nTRIGGER PATTERNS:\n- 'Show me trace abc123' → use get_trace_details\n- 'Explain trace a4d1aae7216b47ff8117cf4e09ce9d0a' → use get_trace_details\n- 'What is trace [trace-id]' → use get_trace_details\n\n\n### Get trace overview\n```\nget_trace_details(organizationSlug='my-organization', traceId='a4d1aae7216b47ff8117cf4e09ce9d0a')\n```\n\n### Focus a single span\n```\nget_trace_details(organizationSlug='my-organization', traceId='a4d1aae7216b47ff8117cf4e09ce9d0a', spanId='aa8e7f3384ef4ff5')\n```\n\n\n\n- Trace IDs are 32-character hexadecimal strings\n- This returns a condensed trace overview, not a full span dump\n- Provide `spanId` to focus on a single span within the trace\n- If the response says it shows a subset of spans, use search_events to inspect the rest of the trace\n", @@ -184,7 +199,7 @@ }, { "name": "search_docs", - "description": "Search Sentry documentation for SDK setup, instrumentation, and configuration guidance.\n\nUse this tool when you need to:\n- Set up Sentry SDK or framework integrations (Django, Flask, Express, Next.js, etc.)\n- Configure features like performance monitoring, error sampling, or release tracking\n- Implement custom instrumentation (spans, transactions, breadcrumbs)\n- Configure data scrubbing, filtering, or sampling rules\n\nReturns snippets only. Use the Sentry tool `get_doc(path='...')` to fetch full documentation content.\n\n\n```\nsearch_docs(query='Django setup configuration SENTRY_DSN', guide='python/django')\nsearch_docs(query='source maps webpack upload', guide='javascript/nextjs')\n```\n\n\n\n- Use guide parameter to filter to specific technologies (e.g., 'javascript/nextjs')\n- Include specific feature names like 'beforeSend', 'tracesSampleRate', 'SENTRY_DSN'\n", + "description": "Search Sentry documentation for SDK setup, instrumentation, and configuration guidance.\n\nUse this tool when you need to:\n- Set up Sentry SDK or framework integrations (Django, Flask, Express, Next.js, etc.)\n- Configure features like performance monitoring, error sampling, or release tracking\n- Implement custom instrumentation (spans, transactions, breadcrumbs)\n- Configure data scrubbing, filtering, or sampling rules\n\nReturns snippets only. Use the Sentry tool `get_doc` to fetch full documentation content.\n\n\n```\nsearch_docs(query='Django setup configuration SENTRY_DSN', guide='python/django')\nsearch_docs(query='source maps webpack upload', guide='javascript/nextjs')\n```\n\n\n\n- Use guide parameter to filter to specific technologies (e.g., 'javascript/nextjs')\n- Include specific feature names like 'beforeSend', 'tracesSampleRate', 'SENTRY_DSN'\n", "requiredScopes": [] }, { @@ -323,50 +338,5 @@ "requiredScopes": [] } ] - }, - { - "id": "preprod", - "name": "Preprod Snapshots", - "description": "Inspect visual regression snapshot tests from CI — view changed images and diff masks", - "defaultEnabled": false, - "order": 6, - "toolCount": 7, - "tools": [ - { - "name": "find_organizations", - "description": "Find organizations that the user has access to in Sentry.\n\nUse this tool when you need to:\n- View organizations in Sentry\n- Find an organization's slug to aid other tool requests\n- Search for specific organizations by name or slug\n\nReturns up to 25 results. If you hit this limit, use the query parameter to narrow down results.", - "requiredScopes": ["org:read"] - }, - { - "name": "find_projects", - "description": "Find projects in Sentry.\n\nUse this tool when you need to:\n- View projects in a Sentry organization\n- Find a project's slug to aid other tool requests\n- Search for specific projects by name or slug\n\nReturns up to 25 results. If you hit this limit, use the query parameter to narrow down results.", - "requiredScopes": ["project:read"] - }, - { - "name": "get_latest_base_snapshot", - "description": "Get the latest UI screenshots/images for an app from the preprod snapshot system.\n\nThis is the primary tool for retrieving app screenshots — not search_events or search_issues.\n\nUse this tool when you need to:\n- Get screenshots, screens, golden images, or reference images for an app\n- Find what the current UI looks like (latest screenshots from the main/default branch)\n- List available snapshots or browse images before requesting specific ones\n- Look up dark mode, light mode, or other variant screenshots\n- Understand what baseline images exist when investigating snapshot test or visual regression CI failures\n\nThe appId parameter is the app identifier (e.g. 'sentry-frontend', 'com.emergetools.hackernews').\nReturns compact image metadata (display_name, image_file_name, group, description) for every image.\n\n\n### Get the latest screenshots for an app\n\n```\nget_latest_base_snapshot(organizationSlug=\"sentry\", appId=\"sentry-frontend\", project=\"frontend\")\n```\n\n### Get the latest screenshots for a specific branch\n\n```\nget_latest_base_snapshot(organizationSlug=\"sentry\", appId=\"sentry-frontend\", project=\"frontend\", branch=\"main\")\n```\n\n\n\n- The response includes compact metadata per image. Scan the list to find images matching what you need (e.g. filter by group or name containing 'button').\n- To view a specific image, use get_sentry_resource(url='?selectedSnapshot=').\n- If you need to investigate a specific snapshot comparison, use get_sentry_resource with the snapshot URL.\n", - "requiredScopes": ["project:read"] - }, - { - "name": "get_sentry_resource", - "description": "Fetch a Sentry resource by URL, or by resourceType plus resourceId.\nPass a Sentry URL directly when possible; the resource type is auto-detected.\n\nSupports issues, events, traces, spans, AI conversations, breadcrumbs, replays, preprod snapshots, and snapshot images.\nTrace lookups return a condensed overview by default.\n\nAI Conversations: A conversation is a set of spans sharing the same gen_ai.conversation.id. Use resourceType='ai_conversation' with a conversation ID to fetch all spans for that conversation. To discover or list conversation IDs, use search_events with dataset='spans' and query='has:gen_ai.conversation.id'. Conversations are NOT issues — do not use search_issues for conversation queries.\n\nFor preprod snapshot URLs (matching 'sentry.io/preprod/snapshots/'):\n- Without ?selectedSnapshot=: returns the snapshot diff summary (changed, added, removed images)\n- With ?selectedSnapshot=: returns the image preview and metadata. Use the Sentry tool `get_snapshot_image(organizationSlug='', snapshotId='', imageIdentifier='', imageResolution='full')` for full-resolution image bytes.\n\nResource IDs:\n- span: :\n- snapshot: \n- snapshotImage: :\n\n\nget_sentry_resource(url='https://sentry.io/issues/PROJECT-123/')\nget_sentry_resource(resourceType='issue', organizationSlug='my-org', resourceId='PROJECT-123')\nget_sentry_resource(resourceType='span', organizationSlug='my-org', resourceId=':')\nget_sentry_resource(resourceType='ai_conversation', organizationSlug='my-org', resourceId='conversation-123')\nget_sentry_resource(url='https://sentry.sentry.io/preprod/snapshots/123/')\nget_sentry_resource(url='https://sentry.sentry.io/preprod/snapshots/123/?selectedSnapshot=login_screen.png')\n", - "requiredScopes": ["event:read", "project:read"] - }, - { - "name": "get_snapshot", - "description": "Get a preprod snapshot comparison summary, including metadata, counts, and changed image sections.\n\nUse this tool when you need to:\n- Investigate a failed snapshot test from CI\n- Review what changed in a specific preprod snapshot\n- Browse snapshot image file names before viewing a specific image\n\nPass organizationSlug and snapshotId. Use get_sentry_resource for snapshot URLs.\nCompact output is returned by default. Set showUnmodified=true to list unchanged and skipped images separately.\n\n\n### Browse a snapshot\n\n```\nget_snapshot(organizationSlug=\"sentry\", snapshotId=\"231949\")\n```\n\n### Include unchanged and skipped images\n\n```\nget_snapshot(organizationSlug=\"sentry\", snapshotId=\"231949\", showUnmodified=true)\n```\n\n\n\n- Use the Sentry tool `get_snapshot_image(organizationSlug='', snapshotId='', imageIdentifier='')` to view a specific image preview or full-resolution image bytes.\n- Use get_sentry_resource when starting from a Sentry snapshot URL.\n- The diff percent field shows what percentage of pixels changed (0-100).\n- showUnmodified=true is useful when a diff snapshot has no changed image sections.\n", - "requiredScopes": ["project:read"] - }, - { - "name": "get_snapshot_image", - "description": "Get metadata and image content for one image in a preprod snapshot.\n\nUse this tool when you need to:\n- View the current, previous, or diff image for a snapshot entry\n- Inspect metadata and context for a specific snapshot image\n- Fetch full-resolution image bytes when preview images are insufficient; full-resolution images can substantially increase response size and token usage\n\nPass organizationSlug, snapshotId, and imageIdentifier. Use get_sentry_resource for snapshot URLs.\nPreview images are returned by default; set imageResolution=full for original bytes when needed, but expect higher response size and token usage.\n\n\n### View a preview image\n\n```\nget_snapshot_image(organizationSlug=\"sentry\", snapshotId=\"231949\", imageIdentifier=\"login_screen.png\")\n```\n\n### Fetch original full-resolution bytes\n\n```\nget_snapshot_image(organizationSlug=\"sentry\", snapshotId=\"231949\", imageIdentifier=\"login_screen.png\", imageResolution=\"full\")\n```\n\n\n\n- Use get_snapshot first if you need to discover available image file names.\n- Use get_sentry_resource when starting from a Sentry snapshot URL.\n- imageIdentifier values may include slashes; pass the full image_file_name exactly as shown by get_snapshot.\n", - "requiredScopes": ["project:read"] - }, - { - "name": "whoami", - "description": "Identify the authenticated user in Sentry.\n\nUse this tool when you need to:\n- Get the user's name and email address.", - "requiredScopes": [] - } - ] } ] diff --git a/packages/mcp-core/src/skills.test.ts b/packages/mcp-core/src/skills.test.ts index f0350d719..8fa185174 100644 --- a/packages/mcp-core/src/skills.test.ts +++ b/packages/mcp-core/src/skills.test.ts @@ -9,16 +9,13 @@ import { } from "./skills"; import skillDefinitions from "./skillDefinitions"; -function getGeneratedSkillToolDescription(skillId: string, toolName: string) { +function getGeneratedSkillToolNames(skillId: string) { const skill = skillDefinitions.find( (definition) => definition.id === skillId, ); expect(skill).toBeDefined(); - const tool = skill?.tools?.find((definition) => definition.name === toolName); - expect(tool).toBeDefined(); - - return tool?.description ?? ""; + return new Set(skill?.tools?.map((definition) => definition.name) ?? []); } describe("skills module", () => { @@ -29,6 +26,7 @@ describe("skills module", () => { expect(SKILLS["project-management"]).toBeDefined(); expect(SKILLS.seer).toBeDefined(); expect(SKILLS.docs).toBeDefined(); + expect("preprod" in SKILLS).toBe(false); }); it("includes metadata for each skill", () => { @@ -69,6 +67,8 @@ describe("skills module", () => { expect(isValidSkill("invalid")).toBe(false); expect(isValidSkill("")).toBe(false); expect(isValidSkill("INSPECT")).toBe(false); + expect(isValidSkill("preprod")).toBe(false); + expect(isValidSkill("toString")).toBe(false); }); }); @@ -124,6 +124,18 @@ describe("skills module", () => { expect(valid.size).toBe(2); expect(invalid.length).toBe(0); }); + + it("treats legacy preprod skill as invalid by default", () => { + const { valid, invalid } = parseSkills(["preprod"]); + expect(valid).toEqual(new Set()); + expect(invalid).toEqual(["preprod"]); + }); + + it("does not treat object prototype properties as skills", () => { + const { valid, invalid } = parseSkills(["toString"]); + expect(valid).toEqual(new Set()); + expect(invalid).toEqual(["toString"]); + }); }); describe("isEnabledBySkills", () => { @@ -155,30 +167,12 @@ describe("skills module", () => { }); describe("generated skill definitions", () => { - it("resolves dynamic descriptions with each skill's tool availability", () => { - const preprodDescription = getGeneratedSkillToolDescription( - "preprod", - "get_sentry_resource", - ); - const triageDescription = getGeneratedSkillToolDescription( - "triage", - "get_sentry_resource", - ); - - expect(preprodDescription).toContain( - "Use the Sentry tool `get_snapshot_image(", - ); - expect(preprodDescription).toContain("imageResolution='full'"); - expect(preprodDescription).not.toContain( - "Full-resolution snapshot image bytes are not available in this session", - ); + it("lists tools for each skill", () => { + const inspectToolNames = getGeneratedSkillToolNames("inspect"); + const triageToolNames = getGeneratedSkillToolNames("triage"); - expect(triageDescription).toContain( - "Full-resolution snapshot image bytes are not available in this session", - ); - expect(triageDescription).not.toContain( - "Use the Sentry tool `get_snapshot_image(", - ); + expect(inspectToolNames).toContain("get_snapshot_image"); + expect(triageToolNames).not.toContain("get_snapshot_image"); }); }); }); diff --git a/packages/mcp-core/src/skills.ts b/packages/mcp-core/src/skills.ts index 66cd5b771..e071c4bec 100644 --- a/packages/mcp-core/src/skills.ts +++ b/packages/mcp-core/src/skills.ts @@ -11,8 +11,7 @@ export type Skill = | "triage" | "project-management" | "seer" - | "docs" - | "preprod"; + | "docs"; // Central registry with metadata (used by OAuth UI) export interface SkillDefinition { @@ -61,14 +60,6 @@ export const SKILLS: Record = { defaultEnabled: false, order: 5, }, - preprod: { - id: "preprod", - name: "Preprod Snapshots", - description: - "Inspect visual regression snapshot tests from CI — view changed images and diff masks", - defaultEnabled: false, - order: 6, - }, }; // Sorted array for UI ordering @@ -121,7 +112,7 @@ export const DEFAULT_SKILLS: Skill[] = SKILLS_ARRAY.filter( // Validation export function isValidSkill(skill: string): skill is Skill { - return skill in SKILLS; + return Object.hasOwn(SKILLS, skill); } // Check if tool is enabled by granted skills (ANY match = enabled) diff --git a/packages/mcp-core/src/toolDefinitions.json b/packages/mcp-core/src/toolDefinitions.json index 8a9cb1ef2..823b32a3f 100644 --- a/packages/mcp-core/src/toolDefinitions.json +++ b/packages/mcp-core/src/toolDefinitions.json @@ -128,7 +128,7 @@ }, "requiredScopes": ["project:write"], "skills": ["project-management"], - "surface": "direct" + "surface": "catalog" }, { "name": "create_project", @@ -155,7 +155,7 @@ }, "teamSlug": { "type": "string", - "description": "The team's slug. You can find a list of existing teams in an organization using the `find_teams()` tool." + "description": "The team's slug. You can find a list of existing teams in an organization with the Sentry tool `find_teams`." }, "name": { "type": "string", @@ -193,7 +193,7 @@ }, "requiredScopes": ["project:write", "team:read", "org:read"], "skills": ["project-management"], - "surface": "direct" + "surface": "catalog" }, { "name": "create_team", @@ -229,6 +229,32 @@ }, "requiredScopes": ["team:write"], "skills": ["project-management"], + "surface": "catalog" + }, + { + "name": "execute_tool", + "description": "Execute an available Sentry MCP tool discovered through search_tools.\n\nUse this tool when you need to:\n- Call a Sentry operation returned by search_tools\n- Execute a tool by name using arguments that match its returned schema\n\n\nexecute_tool(name='find_projects', arguments={ organizationSlug: 'my-org' })\nexecute_tool(name='update_issue', arguments={ organizationSlug: 'my-org', issueId: 'PROJ-1', status: 'resolved' })\n\n\n\n- Use search_tools first if you are not sure which name or arguments to pass.\n- Arguments are validated against the target tool's schema before execution.\n- Active organization, project, and region constraints are injected automatically.\n", + "inputSchema": { + "type": "object", + "properties": { + "name": { + "type": "string", + "minLength": 1, + "description": "The name of the available tool to execute." + }, + "arguments": { + "type": "object", + "additionalProperties": {}, + "default": {}, + "description": "Arguments for the target tool, matching the schema returned by search_tools." + } + }, + "required": ["name"], + "additionalProperties": false, + "$schema": "http://json-schema.org/draft-07/schema#" + }, + "requiredScopes": [], + "skills": ["inspect", "seer", "docs", "triage", "project-management"], "surface": "direct" }, { @@ -265,7 +291,7 @@ }, "requiredScopes": ["project:read"], "skills": ["project-management"], - "surface": "direct" + "surface": "catalog" }, { "name": "find_organizations", @@ -291,14 +317,7 @@ "$schema": "http://json-schema.org/draft-07/schema#" }, "requiredScopes": ["org:read"], - "skills": [ - "inspect", - "seer", - "docs", - "triage", - "project-management", - "preprod" - ], + "skills": ["inspect", "seer", "docs", "triage", "project-management"], "surface": "direct" }, { @@ -343,14 +362,7 @@ "$schema": "http://json-schema.org/draft-07/schema#" }, "requiredScopes": ["project:read"], - "skills": [ - "inspect", - "seer", - "docs", - "triage", - "project-management", - "preprod" - ], + "skills": ["inspect", "seer", "docs", "triage", "project-management"], "surface": "direct" }, { @@ -409,7 +421,7 @@ }, "requiredScopes": ["project:read"], "skills": ["inspect"], - "surface": "direct" + "surface": "catalog" }, { "name": "find_teams", @@ -454,7 +466,7 @@ }, "requiredScopes": ["team:read"], "skills": ["inspect", "triage", "project-management"], - "surface": "direct" + "surface": "catalog" }, { "name": "get_ai_conversation_details", @@ -504,7 +516,7 @@ }, "requiredScopes": [], "skills": ["docs"], - "surface": "direct" + "surface": "catalog" }, { "name": "get_event_attachment", @@ -557,7 +569,7 @@ }, "requiredScopes": ["event:read"], "skills": ["inspect"], - "surface": "direct" + "surface": "catalog" }, { "name": "get_issue_activity", @@ -697,7 +709,7 @@ }, "requiredScopes": ["event:read"], "skills": ["inspect"], - "surface": "direct" + "surface": "catalog" }, { "name": "get_latest_base_snapshot", @@ -758,7 +770,7 @@ "$schema": "http://json-schema.org/draft-07/schema#" }, "requiredScopes": ["project:read"], - "skills": ["preprod"], + "skills": ["inspect"], "surface": "catalog" }, { @@ -885,7 +897,7 @@ }, "requiredScopes": ["event:read"], "skills": ["inspect"], - "surface": "direct" + "surface": "catalog" }, { "name": "get_release_details", @@ -996,11 +1008,11 @@ }, "requiredScopes": ["org:read", "project:read", "event:read"], "skills": ["inspect"], - "surface": "direct" + "surface": "catalog" }, { "name": "get_sentry_resource", - "description": "Fetch a Sentry resource by URL, or by resourceType plus resourceId.\nPass a Sentry URL directly when possible; the resource type is auto-detected.\n\nSupports issues, events, traces, spans, AI conversations, breadcrumbs, replays, preprod snapshots, and snapshot images.\nTrace lookups return a condensed overview by default.\n\nAI Conversations: A conversation is a set of spans sharing the same gen_ai.conversation.id. Use resourceType='ai_conversation' with a conversation ID to fetch all spans for that conversation. To discover or list conversation IDs, use search_events with dataset='spans' and query='has:gen_ai.conversation.id'. Conversations are NOT issues — do not use search_issues for conversation queries.\n\nFor preprod snapshot URLs (matching 'sentry.io/preprod/snapshots/'):\n- Without ?selectedSnapshot=: returns the snapshot diff summary (changed, added, removed images)\n- With ?selectedSnapshot=: returns the image preview and metadata. Full-resolution snapshot image bytes are not available in this session.\n\nResource IDs:\n- span: :\n- snapshot: \n- snapshotImage: :\n\n\nget_sentry_resource(url='https://sentry.io/issues/PROJECT-123/')\nget_sentry_resource(resourceType='issue', organizationSlug='my-org', resourceId='PROJECT-123')\nget_sentry_resource(resourceType='span', organizationSlug='my-org', resourceId=':')\nget_sentry_resource(resourceType='ai_conversation', organizationSlug='my-org', resourceId='conversation-123')\nget_sentry_resource(url='https://sentry.sentry.io/preprod/snapshots/123/')\nget_sentry_resource(url='https://sentry.sentry.io/preprod/snapshots/123/?selectedSnapshot=login_screen.png')\n", + "description": "Fetch a Sentry resource by URL, or by resourceType plus resourceId.\nPass a Sentry URL directly when possible; the resource type is auto-detected.\n\nSupports issues, events, traces, spans, AI conversations, breadcrumbs, replays, preprod snapshots, and snapshot images.\nTrace lookups return a condensed overview by default.\n\nAI Conversations: A conversation is a set of spans sharing the same gen_ai.conversation.id. Use resourceType='ai_conversation' with a conversation ID to fetch all spans for that conversation. To discover or list conversation IDs, use search_events with dataset='spans' and query='has:gen_ai.conversation.id'. Conversations are NOT issues — do not use search_issues for conversation queries.\n\nFor preprod snapshot URLs (matching 'sentry.io/preprod/snapshots/'):\n- Without ?selectedSnapshot=: returns the snapshot diff summary (changed, added, removed images)\n- With ?selectedSnapshot=: returns the image preview and metadata. Use the Sentry tool `get_snapshot_image` for full-resolution image bytes.\n\nResource IDs:\n- span: :\n- snapshot: \n- snapshotImage: :\n\n\nget_sentry_resource(url='https://sentry.io/issues/PROJECT-123/')\nget_sentry_resource(resourceType='issue', organizationSlug='my-org', resourceId='PROJECT-123')\nget_sentry_resource(resourceType='span', organizationSlug='my-org', resourceId=':')\nget_sentry_resource(resourceType='ai_conversation', organizationSlug='my-org', resourceId='conversation-123')\nget_sentry_resource(url='https://sentry.sentry.io/preprod/snapshots/123/')\nget_sentry_resource(url='https://sentry.sentry.io/preprod/snapshots/123/?selectedSnapshot=login_screen.png')\n", "inputSchema": { "type": "object", "properties": { @@ -1037,12 +1049,12 @@ "$schema": "http://json-schema.org/draft-07/schema#" }, "requiredScopes": ["event:read", "project:read"], - "skills": ["inspect", "triage", "seer", "preprod"], + "skills": ["inspect", "triage", "seer"], "surface": "direct" }, { "name": "get_snapshot", - "description": "Get a preprod snapshot comparison summary, including metadata, counts, and changed image sections.\n\nUse this tool when you need to:\n- Investigate a failed snapshot test from CI\n- Review what changed in a specific preprod snapshot\n- Browse snapshot image file names before viewing a specific image\n\nPass organizationSlug and snapshotId. Use get_sentry_resource for snapshot URLs.\nCompact output is returned by default. Set showUnmodified=true to list unchanged and skipped images separately.\n\n\n### Browse a snapshot\n\n```\nget_snapshot(organizationSlug=\"sentry\", snapshotId=\"231949\")\n```\n\n### Include unchanged and skipped images\n\n```\nget_snapshot(organizationSlug=\"sentry\", snapshotId=\"231949\", showUnmodified=true)\n```\n\n\n\n- Use the Sentry tool `get_sentry_resource(resourceType='snapshotImage', organizationSlug='', resourceId=':')` to view a specific image preview.\n- Use get_sentry_resource when starting from a Sentry snapshot URL.\n- The diff percent field shows what percentage of pixels changed (0-100).\n- showUnmodified=true is useful when a diff snapshot has no changed image sections.\n", + "description": "Get a preprod snapshot comparison summary, including metadata, counts, and changed image sections.\n\nUse this tool when you need to:\n- Investigate a failed snapshot test from CI\n- Review what changed in a specific preprod snapshot\n- Browse snapshot image file names before viewing a specific image\n\nPass organizationSlug and snapshotId. Use get_sentry_resource for snapshot URLs.\nCompact output is returned by default. Set showUnmodified=true to list unchanged and skipped images separately.\n\n\n### Browse a snapshot\n\n```\nget_snapshot(organizationSlug=\"sentry\", snapshotId=\"231949\")\n```\n\n### Include unchanged and skipped images\n\n```\nget_snapshot(organizationSlug=\"sentry\", snapshotId=\"231949\", showUnmodified=true)\n```\n\n\n\n- Use the Sentry tool `get_snapshot_image` to view a specific image preview or full-resolution image bytes.\n- Use get_sentry_resource when starting from a Sentry snapshot URL.\n- The diff percent field shows what percentage of pixels changed (0-100).\n- showUnmodified=true is useful when a diff snapshot has no changed image sections.\n", "inputSchema": { "type": "object", "properties": { @@ -1079,7 +1091,7 @@ "$schema": "http://json-schema.org/draft-07/schema#" }, "requiredScopes": ["project:read"], - "skills": ["preprod"], + "skills": ["inspect"], "surface": "catalog" }, { @@ -1127,7 +1139,7 @@ "$schema": "http://json-schema.org/draft-07/schema#" }, "requiredScopes": ["project:read"], - "skills": ["preprod"], + "skills": ["inspect"], "surface": "catalog" }, { @@ -1174,7 +1186,7 @@ }, { "name": "search_docs", - "description": "Search Sentry documentation for SDK setup, instrumentation, and configuration guidance.\n\nUse this tool when you need to:\n- Set up Sentry SDK or framework integrations (Django, Flask, Express, Next.js, etc.)\n- Configure features like performance monitoring, error sampling, or release tracking\n- Implement custom instrumentation (spans, transactions, breadcrumbs)\n- Configure data scrubbing, filtering, or sampling rules\n\nReturns snippets only. Use the Sentry tool `get_doc(path='...')` to fetch full documentation content.\n\n\n```\nsearch_docs(query='Django setup configuration SENTRY_DSN', guide='python/django')\nsearch_docs(query='source maps webpack upload', guide='javascript/nextjs')\n```\n\n\n\n- Use guide parameter to filter to specific technologies (e.g., 'javascript/nextjs')\n- Include specific feature names like 'beforeSend', 'tracesSampleRate', 'SENTRY_DSN'\n", + "description": "Search Sentry documentation for SDK setup, instrumentation, and configuration guidance.\n\nUse this tool when you need to:\n- Set up Sentry SDK or framework integrations (Django, Flask, Express, Next.js, etc.)\n- Configure features like performance monitoring, error sampling, or release tracking\n- Implement custom instrumentation (spans, transactions, breadcrumbs)\n- Configure data scrubbing, filtering, or sampling rules\n\nReturns snippets only. Use the Sentry tool `get_doc` to fetch full documentation content.\n\n\n```\nsearch_docs(query='Django setup configuration SENTRY_DSN', guide='python/django')\nsearch_docs(query='source maps webpack upload', guide='javascript/nextjs')\n```\n\n\n\n- Use guide parameter to filter to specific technologies (e.g., 'javascript/nextjs')\n- Include specific feature names like 'beforeSend', 'tracesSampleRate', 'SENTRY_DSN'\n", "inputSchema": { "type": "object", "properties": { @@ -1544,7 +1556,7 @@ }, "requiredScopes": ["event:read"], "skills": ["inspect", "triage"], - "surface": "direct" + "surface": "catalog" }, { "name": "search_issues", @@ -1613,6 +1625,40 @@ "skills": ["inspect", "triage", "seer"], "surface": "direct" }, + { + "name": "search_tools", + "description": "Search the available Sentry MCP tool catalog by name and description.\n\nMany Sentry operations are intentionally not exposed as top-level tools. Use this for any Sentry-related task when you do not see an obvious direct tool, including long-tail inspection, project management, documentation lookup, preprod snapshots, attachments, DSNs, releases, teams, and issue-specific pivots.\n\nUse this tool when you need to:\n- Find the right Sentry operation for a task\n- Discover catalog tools and their schemas for a task\n- Inspect the executable JSON input schema for an available tool\n\n\nsearch_tools(query='list projects')\nsearch_tools(query='update issue status')\nsearch_tools(query='find dsn', limit=5)\nsearch_tools(query='snapshot image')\n\n\n\n- Results only include tools available in the current session.\n- If a Sentry operation is not listed as a direct tool, search here before deciding it is unavailable.\n- Returned schemas already account for active organization, project, and region constraints.\n- Use the returned name and schema when executing a catalog result.\n- This tool returns structured JSON. Do not parse markdown from its text content.\n", + "inputSchema": { + "type": "object", + "properties": { + "query": { + "type": "string", + "minLength": 1, + "description": "Natural language keywords describing the Sentry operation, resource, or workflow to find." + }, + "limit": { + "anyOf": [ + { + "type": "integer", + "exclusiveMinimum": 0, + "maximum": 20 + }, + { + "type": "null" + } + ], + "default": 8, + "description": "Maximum number of matching tools to return, up to 20." + } + }, + "required": ["query"], + "additionalProperties": false, + "$schema": "http://json-schema.org/draft-07/schema#" + }, + "requiredScopes": [], + "skills": ["inspect", "seer", "docs", "triage", "project-management"], + "surface": "direct" + }, { "name": "update_issue", "description": "Update a Sentry issue's status or assignment.\n\nUse this to resolve, reopen, assign, or ignore an issue.\n\n\n```\nupdate_issue(organizationSlug='my-org', issueId='PROJECT-123', status='resolved')\nupdate_issue(organizationSlug='my-org', issueId='PROJECT-123', assignedTo='user:123456')\nupdate_issue(organizationSlug='my-org', issueId='PROJECT-123', status='ignored')\nupdate_issue(organizationSlug='my-org', issueId='PROJECT-123', status='ignored', ignoreMode='forever')\nupdate_issue(organizationSlug='my-org', issueId='PROJECT-123', status='ignored', ignoreMode='untilOccurrenceCount', ignoreCount=100, ignoreWindowMinutes=60)\nupdate_issue(organizationSlug='my-org', issueId='PROJECT-123', status='ignored', reason='Ignoring because this is expected noise from the staging deploy')\n```\n\n\n\n- Provide `issueUrl` or `organizationSlug` + `issueId`.\n- At least one of `status` or `assignedTo` is required.\n- `assignedTo` format: `user:ID` or `team:ID_OR_SLUG`.\n- Use `whoami` to find your user ID for self-assignment.\n- Status values: `resolved`, `resolvedInNextRelease`, `unresolved`, `ignored`.\n- `status='ignored'` defaults to `ignoreMode='untilEscalating'`.\n- Ignore modes: `untilEscalating`, `forever`, `forDuration`, `untilOccurrenceCount`, `untilUserCount`.\n- Matching ignore inputs are `ignoreDurationMinutes`, `ignoreCount` + optional `ignoreWindowMinutes`, or `ignoreUserCount` + optional `ignoreUserWindowMinutes`.\n- To switch an already ignored issue between `untilEscalating`, `forever`, and condition-based ignore modes, first set `status='unresolved'`, then ignore it again with the new rule.\n- `reason` is optional. When provided, it will be posted as a comment on the issue's activity feed explaining why the action was taken.\n", @@ -1780,7 +1826,7 @@ "anyOf": [ { "type": "string", - "description": "The team's slug. You can find a list of existing teams in an organization using the `find_teams()` tool." + "description": "The team's slug. You can find a list of existing teams in an organization with the Sentry tool `find_teams`." }, { "type": "null" @@ -1796,21 +1842,14 @@ }, "requiredScopes": ["project:write"], "skills": ["project-management"], - "surface": "direct" + "surface": "catalog" }, { "name": "whoami", "description": "Identify the authenticated user in Sentry.\n\nUse this tool when you need to:\n- Get the user's name and email address.", "inputSchema": {}, "requiredScopes": [], - "skills": [ - "inspect", - "seer", - "docs", - "triage", - "project-management", - "preprod" - ], + "skills": ["inspect", "seer", "docs", "triage", "project-management"], "surface": "direct" } ] diff --git a/packages/mcp-core/src/tools/catalog-runtime/availability.ts b/packages/mcp-core/src/tools/catalog-runtime/availability.ts index d8ece80ef..a18faff4e 100644 --- a/packages/mcp-core/src/tools/catalog-runtime/availability.ts +++ b/packages/mcp-core/src/tools/catalog-runtime/availability.ts @@ -66,14 +66,11 @@ function getToolPlacement({ function hasRequiredCapabilities({ tool, context, - experimentalMode, }: { tool: ToolConfig; context: ServerContext; - experimentalMode: boolean; }): boolean { if ( - !experimentalMode || !context.constraints.projectSlug || !context.constraints.projectCapabilities || !tool.requiredCapabilities?.length @@ -191,7 +188,7 @@ export function getAvailableTools({ continue; } - if (!hasRequiredCapabilities({ tool, context, experimentalMode })) { + if (!hasRequiredCapabilities({ tool, context })) { continue; } diff --git a/packages/mcp-core/src/tools/catalog/get-event-attachment.test.ts b/packages/mcp-core/src/tools/catalog/get-event-attachment.test.ts index 150777706..18485a467 100644 --- a/packages/mcp-core/src/tools/catalog/get-event-attachment.test.ts +++ b/packages/mcp-core/src/tools/catalog/get-event-attachment.test.ts @@ -39,7 +39,7 @@ describe("get_event_attachment", () => { **SHA1:** abc123def456 To download this attachment with the attachmentId provided: - Use the Sentry tool \`get_event_attachment(organizationSlug='sentry-mcp-evals', projectSlug='cloudflare-mcp', eventId='7ca573c0f4814912aaa9bdc77d1a7d51', attachmentId='123')\` + Use the Sentry tool \`get_event_attachment\` " `); diff --git a/packages/mcp-core/src/tools/catalog/get-issue-details.test.ts b/packages/mcp-core/src/tools/catalog/get-issue-details.test.ts index f486d8f1a..0fee94cbb 100644 --- a/packages/mcp-core/src/tools/catalog/get-issue-details.test.ts +++ b/packages/mcp-core/src/tools/catalog/get-issue-details.test.ts @@ -257,10 +257,10 @@ describe("get_issue_details", () => { - Commit message issue reference: \`Fixes CLOUDFLARE-MCP-41\` automatically closes the issue when the commit is merged. - The stacktrace includes first-party application code and third-party code. First-party frames are usually the best starting point for triage. - - Issue event search: Use the Sentry tool \`search_issue_events(organizationSlug='sentry-mcp-evals', issueId='CLOUDFLARE-MCP-41', query='your query')\` - - Full distributed trace and span tree: Use the Sentry tool \`get_sentry_resource(resourceType='trace', organizationSlug='sentry-mcp-evals', resourceId='3032af8bcdfe4423b937fc5c041d5d82')\` - - Related span search: Use the Sentry tool \`search_events(organizationSlug='sentry-mcp-evals', dataset='spans', query='trace:3032af8bcdfe4423b937fc5c041d5d82')\` - - Related log search: Use the Sentry tool \`search_events(organizationSlug='sentry-mcp-evals', dataset='logs', query='trace:3032af8bcdfe4423b937fc5c041d5d82')\` + - Issue event search: Use the Sentry tool \`search_issue_events\` + - Full distributed trace and span tree: Use the Sentry tool \`get_sentry_resource\` + - Related span search: Use the Sentry tool \`search_events\` + - Related log search: Use the Sentry tool \`search_events\` " `); }); @@ -413,7 +413,7 @@ describe("get_issue_details", () => { - https://sentry-mcp-evals.sentry.io/explore/replays/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/ - Use the Sentry tool \`get_replay_details(organizationSlug='sentry-mcp-evals', replayId='7e07485f12f9416b8b1426260799b51f')\` to inspect a replay in detail." + Use the Sentry tool \`get_replay_details\` to inspect a replay in detail." `); expect(result).not.toContain("**replayId**:"); }); @@ -516,10 +516,10 @@ describe("get_issue_details", () => { - Commit message issue reference: \`Fixes CLOUDFLARE-MCP-41\` automatically closes the issue when the commit is merged. - The stacktrace includes first-party application code and third-party code. First-party frames are usually the best starting point for triage. - - Issue event search: Use the Sentry tool \`search_issue_events(organizationSlug='sentry-mcp-evals', issueId='CLOUDFLARE-MCP-41', query='your query')\` - - Full distributed trace and span tree: Use the Sentry tool \`get_sentry_resource(resourceType='trace', organizationSlug='sentry-mcp-evals', resourceId='3032af8bcdfe4423b937fc5c041d5d82')\` - - Related span search: Use the Sentry tool \`search_events(organizationSlug='sentry-mcp-evals', dataset='spans', query='trace:3032af8bcdfe4423b937fc5c041d5d82')\` - - Related log search: Use the Sentry tool \`search_events(organizationSlug='sentry-mcp-evals', dataset='logs', query='trace:3032af8bcdfe4423b937fc5c041d5d82')\` + - Issue event search: Use the Sentry tool \`search_issue_events\` + - Full distributed trace and span tree: Use the Sentry tool \`get_sentry_resource\` + - Related span search: Use the Sentry tool \`search_events\` + - Related log search: Use the Sentry tool \`search_events\` " `); }); @@ -801,10 +801,10 @@ describe("get_issue_details", () => { - Commit message issue reference: \`Fixes CLOUDFLARE-MCP-41\` automatically closes the issue when the commit is merged. - The stacktrace includes first-party application code and third-party code. First-party frames are usually the best starting point for triage. - - Issue event search: Use the Sentry tool \`search_issue_events(organizationSlug='sentry-mcp-evals', issueId='CLOUDFLARE-MCP-41', query='your query')\` - - Full distributed trace and span tree: Use the Sentry tool \`get_sentry_resource(resourceType='trace', organizationSlug='sentry-mcp-evals', resourceId='3032af8bcdfe4423b937fc5c041d5d82')\` - - Related span search: Use the Sentry tool \`search_events(organizationSlug='sentry-mcp-evals', dataset='spans', query='trace:3032af8bcdfe4423b937fc5c041d5d82')\` - - Related log search: Use the Sentry tool \`search_events(organizationSlug='sentry-mcp-evals', dataset='logs', query='trace:3032af8bcdfe4423b937fc5c041d5d82')\` + - Issue event search: Use the Sentry tool \`search_issue_events\` + - Full distributed trace and span tree: Use the Sentry tool \`get_sentry_resource\` + - Related span search: Use the Sentry tool \`search_events\` + - Related log search: Use the Sentry tool \`search_events\` " `); }); @@ -1444,7 +1444,7 @@ describe("get_issue_details", () => { - Commit message issue reference: \`Fixes MCP-SERVER-EQE\` automatically closes the issue when the commit is merged. - The stacktrace includes first-party application code and third-party code. First-party frames are usually the best starting point for triage. - - Issue event search: Use the Sentry tool \`search_issue_events(organizationSlug='sentry-mcp-evals', issueId='MCP-SERVER-EQE', query='your query')\` + - Issue event search: Use the Sentry tool \`search_issue_events\` " `); }); diff --git a/packages/mcp-core/src/tools/catalog/get-latest-base-snapshot.ts b/packages/mcp-core/src/tools/catalog/get-latest-base-snapshot.ts index dd2041552..36cfd265f 100644 --- a/packages/mcp-core/src/tools/catalog/get-latest-base-snapshot.ts +++ b/packages/mcp-core/src/tools/catalog/get-latest-base-snapshot.ts @@ -9,7 +9,7 @@ import { renderSnapshotImageTreeSection } from "../support/snapshots/formatting" export default defineTool({ name: "get_latest_base_snapshot", - skills: ["preprod"], + skills: ["inspect"], requiredScopes: ["project:read"], description: [ "Get the latest UI screenshots/images for an app from the preprod snapshot system.", diff --git a/packages/mcp-core/src/tools/catalog/get-sentry-resource.test.ts b/packages/mcp-core/src/tools/catalog/get-sentry-resource.test.ts index 0ddf8d805..56ee1094b 100644 --- a/packages/mcp-core/src/tools/catalog/get-sentry-resource.test.ts +++ b/packages/mcp-core/src/tools/catalog/get-sentry-resource.test.ts @@ -338,9 +338,9 @@ describe("get_sentry_resource", () => { ## Next Steps - - **Search spans**: Use the Sentry tool \`search_events(organizationSlug='test-org', dataset='spans', query='trace:b4d1aae7216b47ff8117cf4e09ce9d0b')\` - - **Search errors**: Use the Sentry tool \`search_events(organizationSlug='test-org', dataset='errors', query='trace:b4d1aae7216b47ff8117cf4e09ce9d0b')\` - - **Search logs**: Use the Sentry tool \`search_events(organizationSlug='test-org', dataset='logs', query='trace:b4d1aae7216b47ff8117cf4e09ce9d0b')\`" + - **Search spans**: Use the Sentry tool \`search_events\` + - **Search errors**: Use the Sentry tool \`search_events\` + - **Search logs**: Use the Sentry tool \`search_events\`" `); }); @@ -489,7 +489,7 @@ describe("get_sentry_resource", () => { To get release information: - **View in Sentry**: [Open Release](https://my-org.sentry.io/releases/v1.2.3/) - - **Find releases**: Use the Sentry tool \`find_releases(organizationSlug='my-org')\` to list releases and their details + - **Find releases**: Use the Sentry tool \`find_releases\` to list releases and their details - **Search issues**: Use \`search_issues\` with query \`release:v1.2.3\` to find issues in this release" `); }); @@ -507,7 +507,7 @@ describe("get_sentry_resource", () => { To get release information: - **View in Sentry**: [Open Release](https://my-org.sentry.io/releases/backend%402024.01.15-abc123/) - - **Find releases**: Use the Sentry tool \`find_releases(organizationSlug='my-org')\` to list releases and their details + - **Find releases**: Use the Sentry tool \`find_releases\` to list releases and their details - **Search issues**: Use \`search_issues\` with query \`release:backend@2024.01.15-abc123\` to find issues in this release" `); }); @@ -844,7 +844,7 @@ describe("get_sentry_resource", () => { 'get_sentry_resource(resourceType="snapshotImage", resourceId="55:")', ); expect(result).toContain( - "- Full-resolution snapshot image bytes are not available in this session", + "- Use the Sentry tool `get_snapshot_image` to fetch original full-resolution image bytes", ); expect(result).not.toContain("?selectedSnapshot="); }); @@ -940,7 +940,7 @@ describe("get_sentry_resource", () => { expect(result[0]).toMatchObject({ type: "text", text: expect.stringContaining( - "Full-resolution snapshot image bytes are not available in this session", + "Use the Sentry tool `get_snapshot_image` for full-resolution image bytes", ), }); }); @@ -1014,7 +1014,7 @@ describe("get_sentry_resource", () => { expect(result[0]).toMatchObject({ type: "text", text: expect.stringContaining( - "- **Full Resolution**: Full-resolution snapshot image bytes are not available in this session", + "- **Full Resolution**: Use the Sentry tool `get_snapshot_image` for full-resolution image bytes", ), }); expect(result).toContainEqual( diff --git a/packages/mcp-core/src/tools/catalog/get-sentry-resource.ts b/packages/mcp-core/src/tools/catalog/get-sentry-resource.ts index d40e00e13..4c0be6e73 100644 --- a/packages/mcp-core/src/tools/catalog/get-sentry-resource.ts +++ b/packages/mcp-core/src/tools/catalog/get-sentry-resource.ts @@ -502,7 +502,7 @@ function generateUnsupportedResourceMessage( export default defineTool({ name: "get_sentry_resource", - skills: ["inspect", "triage", "seer", "preprod"], + skills: ["inspect", "triage", "seer"], requiredScopes: ["event:read", "project:read"], description: ({ experimentalMode, availableToolNames, directToolNames }) => { diff --git a/packages/mcp-core/src/tools/catalog/get-snapshot-image.test.ts b/packages/mcp-core/src/tools/catalog/get-snapshot-image.test.ts index 5a14ca5bd..f8a0d31ac 100644 --- a/packages/mcp-core/src/tools/catalog/get-snapshot-image.test.ts +++ b/packages/mcp-core/src/tools/catalog/get-snapshot-image.test.ts @@ -161,7 +161,7 @@ describe("get_snapshot_image", () => { expect(textParts[0]!.text).toContain("**Diff**: 12.5%"); expect(textParts[0]!.text).toContain("**Image Resolution**: preview"); expect(textParts[0]!.text).toContain( - "Use the Sentry tool `get_snapshot_image(organizationSlug='sentry', snapshotId='231949', imageIdentifier='login_screen.png', imageResolution='full')`", + "Use the Sentry tool `get_snapshot_image` for full-resolution image bytes", ); expect(textParts[0]!.text).toContain("### Context"); expect(textParts[0]!.text).toContain("- **device_name**: iPhone 16"); @@ -176,7 +176,7 @@ describe("get_snapshot_image", () => { - **Status**: changed - **Diff**: 12.5% - **Image Resolution**: preview - - **Full Resolution**: Use the Sentry tool \`get_snapshot_image(organizationSlug='sentry', snapshotId='231949', imageIdentifier='login_screen.png', imageResolution='full')\` for full-resolution image bytes + - **Full Resolution**: Use the Sentry tool \`get_snapshot_image\` for full-resolution image bytes - **Display Name**: login_screen.png - **Group**: auth - **File**: \`login_screen.png\` diff --git a/packages/mcp-core/src/tools/catalog/get-snapshot-image.ts b/packages/mcp-core/src/tools/catalog/get-snapshot-image.ts index 704a9b2c6..8bf3a8fad 100644 --- a/packages/mcp-core/src/tools/catalog/get-snapshot-image.ts +++ b/packages/mcp-core/src/tools/catalog/get-snapshot-image.ts @@ -9,7 +9,7 @@ import { fetchSnapshotImage } from "../support/snapshots/handlers"; export default defineTool({ name: "get_snapshot_image", - skills: ["preprod"], + skills: ["inspect"], requiredScopes: ["project:read"], description: [ "Get metadata and image content for one image in a preprod snapshot.", diff --git a/packages/mcp-core/src/tools/catalog/get-snapshot.test.ts b/packages/mcp-core/src/tools/catalog/get-snapshot.test.ts index eb75e0247..88e7c506f 100644 --- a/packages/mcp-core/src/tools/catalog/get-snapshot.test.ts +++ b/packages/mcp-core/src/tools/catalog/get-snapshot.test.ts @@ -153,8 +153,8 @@ describe("get_snapshot", () => { ## Next Steps - - Use the Sentry tool \`get_snapshot_image(organizationSlug='sentry', snapshotId='231949', imageIdentifier='')\` to view a specific image preview - - Use the Sentry tool \`get_snapshot_image(organizationSlug='sentry', snapshotId='231949', imageIdentifier='', imageResolution='full')\` to fetch original full-resolution image bytes" + - Use the Sentry tool \`get_snapshot_image\` to view a specific image preview + - Use the Sentry tool \`get_snapshot_image\` to fetch original full-resolution image bytes" `); expect(result).not.toContain("**Unchanged:**"); expect(result).not.toContain("skipped.png"); diff --git a/packages/mcp-core/src/tools/catalog/get-snapshot.ts b/packages/mcp-core/src/tools/catalog/get-snapshot.ts index 005e2f126..701078585 100644 --- a/packages/mcp-core/src/tools/catalog/get-snapshot.ts +++ b/packages/mcp-core/src/tools/catalog/get-snapshot.ts @@ -10,7 +10,7 @@ import { fetchSnapshotSummary } from "../support/snapshots/handlers"; export default defineTool({ name: "get_snapshot", - skills: ["preprod"], + skills: ["inspect"], requiredScopes: ["project:read"], description: ({ experimentalMode, availableToolNames, directToolNames }) => { const imageInstruction = formatToolCallInstruction({ @@ -23,8 +23,7 @@ export default defineTool({ experimentalMode, availableToolNames, directToolNames, - fallbackInstruction: - "Use the Sentry tool `get_sentry_resource(resourceType='snapshotImage', organizationSlug='', resourceId=':')`", + fallbackInstruction: "Use the Sentry tool `get_sentry_resource`", }); const imageInstructionSuffix = imageInstruction.includes( "get_snapshot_image", diff --git a/packages/mcp-core/src/tools/catalog/get-trace-details.test.ts b/packages/mcp-core/src/tools/catalog/get-trace-details.test.ts index bbe48766b..7a0637f0b 100644 --- a/packages/mcp-core/src/tools/catalog/get-trace-details.test.ts +++ b/packages/mcp-core/src/tools/catalog/get-trace-details.test.ts @@ -209,9 +209,9 @@ describe("get_trace_details", () => { ## Next Steps - - **Search spans**: Use the Sentry tool \`search_events(organizationSlug='sentry-mcp-evals', query='show more spans from trace a4d1aae7216b47ff8117cf4e09ce9d0a')\` - - **Search errors**: Use the Sentry tool \`search_events(organizationSlug='sentry-mcp-evals', query='show error events from trace a4d1aae7216b47ff8117cf4e09ce9d0a')\` - - **Search logs**: Use the Sentry tool \`search_events(organizationSlug='sentry-mcp-evals', query='show logs from trace a4d1aae7216b47ff8117cf4e09ce9d0a')\`" + - **Search spans**: Use the Sentry tool \`search_events\` + - **Search errors**: Use the Sentry tool \`search_events\` + - **Search logs**: Use the Sentry tool \`search_events\`" `); }); @@ -235,7 +235,7 @@ describe("get_trace_details", () => { ); expect(result).toContain("**Total Spans**: 112"); expect(result).toContain( - "**Search spans**: Use the Sentry tool `search_events(organizationSlug='sentry-mcp-evals', query='show more spans from trace a4d1aae7216b47ff8117cf4e09ce9d0a')`", + "**Search spans**: Use the Sentry tool `search_events`", ); }); @@ -260,7 +260,7 @@ describe("get_trace_details", () => { ); expect(result).toContain( - "**Search spans**: Use the Sentry tool `search_events(organizationSlug='sentry-mcp-evals', dataset='spans', query='trace:a4d1aae7216b47ff8117cf4e09ce9d0a')`", + "**Search spans**: Use the Sentry tool `search_events`", ); }); @@ -635,9 +635,9 @@ describe("get_trace_details", () => { ## Next Steps - - **Search spans**: Use the Sentry tool \`search_events(organizationSlug='sentry-mcp-evals', query='show more spans from trace b4d1aae7216b47ff8117cf4e09ce9d0b')\` - - **Search errors**: Use the Sentry tool \`search_events(organizationSlug='sentry-mcp-evals', query='show error events from trace b4d1aae7216b47ff8117cf4e09ce9d0b')\` - - **Search logs**: Use the Sentry tool \`search_events(organizationSlug='sentry-mcp-evals', query='show logs from trace b4d1aae7216b47ff8117cf4e09ce9d0b')\`" + - **Search spans**: Use the Sentry tool \`search_events\` + - **Search errors**: Use the Sentry tool \`search_events\` + - **Search logs**: Use the Sentry tool \`search_events\`" `); }); @@ -1358,9 +1358,9 @@ describe("get_trace_details", () => { ## Next Steps - - **Search spans**: Use the Sentry tool \`search_events(organizationSlug='sentry-mcp-evals', query='show sibling spans or the rest of trace b4d1aae7216b47ff8117cf4e09ce9d0b')\` - - **Search errors**: Use the Sentry tool \`search_events(organizationSlug='sentry-mcp-evals', query='show error events from trace b4d1aae7216b47ff8117cf4e09ce9d0b')\` - - **Search logs**: Use the Sentry tool \`search_events(organizationSlug='sentry-mcp-evals', query='show logs from trace b4d1aae7216b47ff8117cf4e09ce9d0b')\`" + - **Search spans**: Use the Sentry tool \`search_events\` + - **Search errors**: Use the Sentry tool \`search_events\` + - **Search logs**: Use the Sentry tool \`search_events\`" `); }); }); diff --git a/packages/mcp-core/src/tools/catalog/search-docs.test.ts b/packages/mcp-core/src/tools/catalog/search-docs.test.ts index f02f61c4b..ff295cb9f 100644 --- a/packages/mcp-core/src/tools/catalog/search-docs.test.ts +++ b/packages/mcp-core/src/tools/catalog/search-docs.test.ts @@ -28,7 +28,7 @@ describe("search_docs", () => { Found 2 matches - These are just snippets. Use the Sentry tool \`get_doc(path='...')\` to fetch the full content. + These are just snippets. Use the Sentry tool \`get_doc\` to fetch the full content. ## 1. https://docs.sentry.io/product/rate-limiting diff --git a/packages/mcp-core/src/tools/special/execute-tool.ts b/packages/mcp-core/src/tools/special/execute-tool.ts index 3b449e0bb..287678e19 100644 --- a/packages/mcp-core/src/tools/special/execute-tool.ts +++ b/packages/mcp-core/src/tools/special/execute-tool.ts @@ -14,7 +14,6 @@ export function createExecuteTool(getTools: () => ToolRegistry) { name: "execute_tool", skills: ALL_SKILLS, requiredScopes: [], - experimental: true, description: [ "Execute an available Sentry MCP tool discovered through search_tools.", "", diff --git a/packages/mcp-core/src/tools/special/search-tools.ts b/packages/mcp-core/src/tools/special/search-tools.ts index 542cf76c7..3a614a9e4 100644 --- a/packages/mcp-core/src/tools/special/search-tools.ts +++ b/packages/mcp-core/src/tools/special/search-tools.ts @@ -24,7 +24,7 @@ export const searchToolsOutputSchema = z.object({ inputSchema: z .record(z.unknown()) .describe( - "JSON Schema for the arguments to pass to execute_tool. Session-constrained parameters are omitted.", + "JSON Schema for the matching tool's arguments. Session-constrained parameters are omitted.", ), annotations: toolAnnotationsOutputSchema, }), @@ -50,15 +50,14 @@ export function createSearchToolsTool(getTools: () => ToolRegistry) { name: "search_tools", skills: ALL_SKILLS, requiredScopes: [], - experimental: true, description: [ "Search the available Sentry MCP tool catalog by name and description.", "", - "In experimental mode, many Sentry operations are intentionally not exposed as top-level tools. Use this for any Sentry-related task when you do not see an obvious direct tool, including long-tail inspection, project management, documentation lookup, preprod snapshots, attachments, DSNs, releases, teams, and issue-specific pivots.", + "Many Sentry operations are intentionally not exposed as top-level tools. Use this for any Sentry-related task when you do not see an obvious direct tool, including long-tail inspection, project management, documentation lookup, preprod snapshots, attachments, DSNs, releases, teams, and issue-specific pivots.", "", "Use this tool when you need to:", - "- Find the right Sentry operation before calling execute_tool", - "- Discover available Sentry tools for a task without scanning the top-level tool list", + "- Find the right Sentry operation for a task", + "- Discover catalog tools and their schemas for a task", "- Inspect the executable JSON input schema for an available tool", "", "", @@ -72,7 +71,7 @@ export function createSearchToolsTool(getTools: () => ToolRegistry) { "- Results only include tools available in the current session.", "- If a Sentry operation is not listed as a direct tool, search here before deciding it is unavailable.", "- Returned schemas already account for active organization, project, and region constraints.", - "- Call execute_tool with the returned name and arguments that match the returned schema.", + "- Use the returned name and schema when executing a catalog result.", "- This tool returns structured JSON. Do not parse markdown from its text content.", "", ].join("\n"), diff --git a/packages/mcp-core/src/tools/special/use-sentry/handler.test.ts b/packages/mcp-core/src/tools/special/use-sentry/handler.test.ts index e6593dfea..f6f205402 100644 --- a/packages/mcp-core/src/tools/special/use-sentry/handler.test.ts +++ b/packages/mcp-core/src/tools/special/use-sentry/handler.test.ts @@ -22,7 +22,6 @@ const ALL_SKILLS: Skill[] = [ "seer", "triage", "project-management", - "preprod", ]; const mockContext: ServerContext = { diff --git a/packages/mcp-core/src/tools/support/snapshots/handlers.ts b/packages/mcp-core/src/tools/support/snapshots/handlers.ts index a67809937..283ff5de4 100644 --- a/packages/mcp-core/src/tools/support/snapshots/handlers.ts +++ b/packages/mcp-core/src/tools/support/snapshots/handlers.ts @@ -9,10 +9,7 @@ import { blobToBase64, createImagePreview, } from "../../../internal/blob-utils"; -import { - formatToolCall, - formatToolCallInstruction, -} from "../../../internal/tool-helpers/tool-call-formatting"; +import { formatToolCallInstruction } from "../../../internal/tool-helpers/tool-call-formatting"; import { type SnapshotImageEntry, type SnapshotImageTreeItem, @@ -93,23 +90,8 @@ function fullResolutionHint({ })}`; } -function getSnapshotImagePreviewFallback({ - organizationSlug, - snapshotId, - imageIdentifier, -}: { - organizationSlug: string; - snapshotId: string; - imageIdentifier: string; -}): string { - return `Use the Sentry tool \`${formatToolCall({ - toolName: "get_sentry_resource", - arguments: { - organizationSlug, - resourceType: "snapshotImage", - resourceId: `${snapshotId}:${imageIdentifier}`, - }, - })}\``; +function getSnapshotImagePreviewFallback(): string { + return "Use the Sentry tool `get_sentry_resource`"; } function formatSnapshotImageFullResolutionStep({ @@ -552,11 +534,7 @@ export async function fetchSnapshotSummary( experimentalMode: options.experimentalMode ?? false, availableToolNames: options.availableToolNames, directToolNames: options.directToolNames, - fallbackInstruction: getSnapshotImagePreviewFallback({ - organizationSlug, - snapshotId, - imageIdentifier: "", - }), + fallbackInstruction: getSnapshotImagePreviewFallback(), })} to view a specific image preview\n${formatSnapshotImageFullResolutionStep( { organizationSlug, diff --git a/packages/mcp-core/src/tools/surfaces.ts b/packages/mcp-core/src/tools/surfaces.ts index d78446170..1fb3cfeb7 100644 --- a/packages/mcp-core/src/tools/surfaces.ts +++ b/packages/mcp-core/src/tools/surfaces.ts @@ -17,46 +17,21 @@ export const WRAPPER_TOOL_NAMES = ["use_sentry"] as const; export const TOP_LEVEL_TOOL_NAMES = [ "whoami", "find_organizations", - "find_teams", "find_projects", - "find_releases", - "get_issue_tag_values", - "get_replay_details", - "get_event_attachment", "update_issue", "search_events", - "create_team", - "create_project", - "update_project", - "create_dsn", - "find_dsns", "analyze_issue_with_seer", "search_docs", - "get_doc", "search_issues", - "search_issue_events", - "get_profile_details", "get_sentry_resource", ...CATALOG_INFRASTRUCTURE_TOOL_NAMES, ] as const; -export const EXPERIMENTAL_TOP_LEVEL_TOOL_NAMES = [ - "whoami", - "find_organizations", - "find_projects", - "update_issue", - "search_events", - "analyze_issue_with_seer", - "search_docs", - "search_issues", - "get_sentry_resource", - ...CATALOG_INFRASTRUCTURE_TOOL_NAMES, -] as const; +// The experimental direct surface is intentionally aligned with the default +// surface now that search_tools and execute_tool are primary primitives. +export const EXPERIMENTAL_TOP_LEVEL_TOOL_NAMES = TOP_LEVEL_TOOL_NAMES; const topLevelToolNames = new Set(TOP_LEVEL_TOOL_NAMES); -const experimentalTopLevelToolNames = new Set( - EXPERIMENTAL_TOP_LEVEL_TOOL_NAMES, -); const wrapperToolNames = new Set(WRAPPER_TOOL_NAMES); const catalogInfrastructureToolNames = new Set( CATALOG_INFRASTRUCTURE_TOOL_NAMES, @@ -79,21 +54,17 @@ export function isDefaultTopLevelToolName(toolName: string): boolean { export function isTopLevelToolName( toolName: string, - experimentalMode: boolean, + _experimentalMode: boolean, ): boolean { - return experimentalMode - ? experimentalTopLevelToolNames.has(toolName) - : topLevelToolNames.has(toolName); + return topLevelToolNames.has(toolName); } export function getTopLevelToolNames({ - experimentalMode, + experimentalMode: _experimentalMode, }: { experimentalMode: boolean; }): readonly TopLevelToolName[] { - return experimentalMode - ? EXPERIMENTAL_TOP_LEVEL_TOOL_NAMES - : TOP_LEVEL_TOOL_NAMES; + return TOP_LEVEL_TOOL_NAMES; } export function isWrapperToolName(toolName: string): boolean { diff --git a/packages/mcp-server-evals/src/evals/utils/mcpToolCallRunner.ts b/packages/mcp-server-evals/src/evals/utils/mcpToolCallRunner.ts index 2c674c9a7..5401d9575 100644 --- a/packages/mcp-server-evals/src/evals/utils/mcpToolCallRunner.ts +++ b/packages/mcp-server-evals/src/evals/utils/mcpToolCallRunner.ts @@ -40,8 +40,8 @@ export function McpToolCallTaskRunner( tools, system: [ "You are a Sentry assistant with access to Sentry MCP tools.", - "Use search_tools before execute_tool when the needed Sentry operation is not directly listed as a tool.", - "When search_tools returns a tool, call execute_tool with that returned tool name and arguments matching the returned schema.", + "Use search_tools only when you need to discover the right Sentry operation or inspect its schema.", + "When you already know the right Sentry tool name, use that tool directly through the available MCP tools.", ].join("\n"), prompt: input, stopWhen: stepCountIs(maxSteps), diff --git a/packages/mcp-server/src/cli/resolve.test.ts b/packages/mcp-server/src/cli/resolve.test.ts index 231ab301a..8d1148f57 100644 --- a/packages/mcp-server/src/cli/resolve.test.ts +++ b/packages/mcp-server/src/cli/resolve.test.ts @@ -159,6 +159,16 @@ describe("cli/finalize", () => { expect(cfg.finalSkills.has("docs")).toBe(false); }); + it("throws on legacy preprod skill in stdio", () => { + expect(() => + finalize({ + accessToken: "tok", + skills: "preprod", + unknownArgs: [], + }), + ).toThrow(/Invalid skills provided: preprod/); + }); + it("throws on empty skills after validation", () => { expect(() => finalize({ @@ -174,13 +184,13 @@ describe("cli/finalize", () => { accessToken: "tok", unknownArgs: [], }); - expect(cfg.finalSkills.size).toBe(6); + expect(cfg.finalSkills.size).toBe(5); expect(cfg.finalSkills.has("inspect")).toBe(true); expect(cfg.finalSkills.has("triage")).toBe(true); expect(cfg.finalSkills.has("project-management")).toBe(true); expect(cfg.finalSkills.has("seer")).toBe(true); expect(cfg.finalSkills.has("docs")).toBe(true); - expect(cfg.finalSkills.has("preprod")).toBe(true); + expect(cfg.finalSkills.has("preprod")).toBe(false); }); // --disable-skills tests @@ -191,12 +201,12 @@ describe("cli/finalize", () => { unknownArgs: [], }); expect(cfg.finalSkills.has("seer")).toBe(false); - expect(cfg.finalSkills.size).toBe(5); + expect(cfg.finalSkills.size).toBe(4); expect(cfg.finalSkills.has("inspect")).toBe(true); expect(cfg.finalSkills.has("triage")).toBe(true); expect(cfg.finalSkills.has("project-management")).toBe(true); expect(cfg.finalSkills.has("docs")).toBe(true); - expect(cfg.finalSkills.has("preprod")).toBe(true); + expect(cfg.finalSkills.has("preprod")).toBe(false); }); it("removes disabled skills when combined with --skills", () => { @@ -241,7 +251,7 @@ describe("cli/finalize", () => { }); expect(cfg.finalSkills.has("seer")).toBe(false); expect(cfg.finalSkills.has("docs")).toBe(false); - expect(cfg.finalSkills.size).toBe(4); + expect(cfg.finalSkills.size).toBe(3); }); it("silently ignores disabling a skill not in the active set", () => { diff --git a/plugins/sentry-mcp-experimental/agents/sentry-mcp.md b/plugins/sentry-mcp-experimental/agents/sentry-mcp.md index 14230b43d..069ff01b8 100644 --- a/plugins/sentry-mcp-experimental/agents/sentry-mcp.md +++ b/plugins/sentry-mcp-experimental/agents/sentry-mcp.md @@ -29,8 +29,8 @@ You are a Sentry expert. Investigate errors, analyze performance, and manage pro 1. Identify the user's intent and select the most appropriate tool by reading tool descriptions. 2. Pass Sentry URLs unchanged to `issueUrl` or `url` parameters — NEVER try to fetch Sentry URLs via HTTP directly, always use the MCP tools which handle authentication. -3. When you see a URL containing `sentry.io/preprod/snapshots/`, pass it unchanged to `get_sentry_resource`. When asked for app screenshots or images without a specific snapshot URL, use `search_tools` to find the right preprod snapshot tool, then call it through `execute_tool`. -4. For any Sentry-related operation that is not directly listed as a tool, treat it as a Sentry tool: use `search_tools` to find it, then call it through `execute_tool`. +3. When you see a URL containing `sentry.io/preprod/snapshots/`, pass it unchanged to `get_sentry_resource`. When asked for app screenshots or images without a specific snapshot URL, use the relevant preprod snapshot tool; use `search_tools` only if you need to discover which tool fits. +4. For any Sentry-related operation that is not directly listed as a tool, treat it as a Sentry tool. Use `search_tools` only when you need to discover the tool name or schema. 5. Interpret `org/project` notation as `organizationSlug/projectSlug`. 6. Chain multiple tool calls when a request requires it. 7. Present results directly — lead with actionable information. @@ -39,11 +39,11 @@ You are a Sentry expert. Investigate errors, analyze performance, and manage pro - `search_issues` returns grouped issue lists. `search_events` returns counts, aggregations, or individual event rows. - `get_sentry_resource` fetches a known issue, event, trace, span, replay, breadcrumbs, or generic Sentry resource from a URL or resource ID. It also routes supported profile URLs to profile details. `analyze_issue_with_seer` provides AI root cause analysis with code fixes. -- Snapshot tools such as `get_snapshot`, `get_snapshot_image`, and `get_latest_base_snapshot` are catalog tools. Discover them with `search_tools` and call them with `execute_tool`; do not call them directly. -- Use `get_snapshot` via `execute_tool` for a preprod snapshot diff summary from `organizationSlug` + `snapshotId`. For snapshot URLs, use `get_sentry_resource` instead. -- Use `get_snapshot_image` via `execute_tool` for metadata and preview/full image content for one snapshot image. Use the exact `image_file_name` from `get_snapshot` as `imageIdentifier`. -- When asked for screenshots, screens, golden images, reference images, dark/light mode visuals, or to list available snapshots for an app, use `get_latest_base_snapshot` via `execute_tool` with the `appId` parameter. This is not an event or issue search operation. -- `search_events` and `search_issues` accept `query` as natural language or direct Sentry search syntax; when an agent is configured, it repairs the query and related params before running. For issue-scoped event searches, use the Sentry tool `search_issue_events`: find it with `search_tools`, then call it with `execute_tool`. +- Snapshot tools such as `get_snapshot`, `get_snapshot_image`, and `get_latest_base_snapshot` are catalog tools. Use `search_tools` only when you need to inspect their schemas. +- Use `get_snapshot` for a preprod snapshot diff summary from `organizationSlug` + `snapshotId`. For snapshot URLs, use `get_sentry_resource` instead. +- Use `get_snapshot_image` for metadata and preview/full image content for one snapshot image. Use the exact `image_file_name` from `get_snapshot` as `imageIdentifier`. +- When asked for screenshots, screens, golden images, reference images, dark/light mode visuals, or to list available snapshots for an app, use `get_latest_base_snapshot` with the `appId` parameter. This is not an event or issue search operation. +- `search_events` and `search_issues` accept `query` as natural language or direct Sentry search syntax; when an agent is configured, it repairs the query and related params before running. For issue-scoped event searches, use `search_issue_events`. - AI conversations are spans grouped by `gen_ai.conversation.id` — they are NOT issues. Use `get_sentry_resource(resourceType='ai_conversation')` for a specific conversation, or `search_events` with `dataset='spans'` and `query='has:gen_ai.conversation.id'` to list them. - Trace responses from `get_sentry_resource` are condensed overviews by default. Use `resourceType='span'` with `resourceId=':'` or a trace URL with `?node=span-` to focus one span directly; otherwise, if the trace output says it shows a subset of spans and the user needs more detail, follow up with `search_events` on that trace. diff --git a/plugins/sentry-mcp/agents/sentry-mcp.md b/plugins/sentry-mcp/agents/sentry-mcp.md index bffbfc2f4..069ff01b8 100644 --- a/plugins/sentry-mcp/agents/sentry-mcp.md +++ b/plugins/sentry-mcp/agents/sentry-mcp.md @@ -2,33 +2,24 @@ name: sentry-mcp description: Sentry error tracking and performance monitoring agent. Use when the user asks about errors, exceptions, issues, stack traces, performance, - traces, releases, AI conversations, or provides a Sentry URL (including URLs - containing sentry.io/preprod/snapshots/). Handles searching, analyzing, - triaging, and managing Sentry resources. + traces, releases, snapshots, screenshots, visual regression, CI snapshot + failures, preprod checks, AI conversations, or provides a Sentry URL + (especially URLs containing sentry.io/preprod/snapshots/). Handles searching, + analyzing, triaging, and managing Sentry resources including preprod snapshot + inspection. mcpServers: - sentry allowedTools: - analyze_issue_with_seer - - create_dsn - - create_project - - create_team - - find_dsns + - execute_tool - find_organizations - find_projects - - find_releases - - find_teams - - get_doc - - get_event_attachment - - get_issue_tag_values - - get_profile_details - - get_replay_details - get_sentry_resource - search_docs - search_events - - search_issue_events - search_issues + - search_tools - update_issue - - update_project - whoami --- @@ -38,17 +29,21 @@ You are a Sentry expert. Investigate errors, analyze performance, and manage pro 1. Identify the user's intent and select the most appropriate tool by reading tool descriptions. 2. Pass Sentry URLs unchanged to `issueUrl` or `url` parameters — NEVER try to fetch Sentry URLs via HTTP directly, always use the MCP tools which handle authentication. -3. When you see a URL containing `sentry.io/preprod/snapshots/`, pass it unchanged to `get_sentry_resource`. If asked for app screenshots or images without a specific snapshot URL, explain that this stable direct-tool agent needs a snapshot URL for preprod image inspection. -4. Interpret `org/project` notation as `organizationSlug/projectSlug`. -5. Chain multiple tool calls when a request requires it. -6. Present results directly — lead with actionable information. +3. When you see a URL containing `sentry.io/preprod/snapshots/`, pass it unchanged to `get_sentry_resource`. When asked for app screenshots or images without a specific snapshot URL, use the relevant preprod snapshot tool; use `search_tools` only if you need to discover which tool fits. +4. For any Sentry-related operation that is not directly listed as a tool, treat it as a Sentry tool. Use `search_tools` only when you need to discover the tool name or schema. +5. Interpret `org/project` notation as `organizationSlug/projectSlug`. +6. Chain multiple tool calls when a request requires it. +7. Present results directly — lead with actionable information. ## Key Tool Distinctions - `search_issues` returns grouped issue lists. `search_events` returns counts, aggregations, or individual event rows. - `get_sentry_resource` fetches a known issue, event, trace, span, replay, breadcrumbs, or generic Sentry resource from a URL or resource ID. It also routes supported profile URLs to profile details. `analyze_issue_with_seer` provides AI root cause analysis with code fixes. -- `get_sentry_resource` handles preprod snapshot URLs, including selected snapshot image URLs. Catalog-only snapshot tools are not directly available in this stable plugin, so do not call them directly. -- `search_events`, `search_issues`, and `search_issue_events` accept `query` as natural language or direct Sentry search syntax; when an agent is configured, it repairs the query and related params before running. +- Snapshot tools such as `get_snapshot`, `get_snapshot_image`, and `get_latest_base_snapshot` are catalog tools. Use `search_tools` only when you need to inspect their schemas. +- Use `get_snapshot` for a preprod snapshot diff summary from `organizationSlug` + `snapshotId`. For snapshot URLs, use `get_sentry_resource` instead. +- Use `get_snapshot_image` for metadata and preview/full image content for one snapshot image. Use the exact `image_file_name` from `get_snapshot` as `imageIdentifier`. +- When asked for screenshots, screens, golden images, reference images, dark/light mode visuals, or to list available snapshots for an app, use `get_latest_base_snapshot` with the `appId` parameter. This is not an event or issue search operation. +- `search_events` and `search_issues` accept `query` as natural language or direct Sentry search syntax; when an agent is configured, it repairs the query and related params before running. For issue-scoped event searches, use `search_issue_events`. - AI conversations are spans grouped by `gen_ai.conversation.id` — they are NOT issues. Use `get_sentry_resource(resourceType='ai_conversation')` for a specific conversation, or `search_events` with `dataset='spans'` and `query='has:gen_ai.conversation.id'` to list them. - Trace responses from `get_sentry_resource` are condensed overviews by default. Use `resourceType='span'` with `resourceId=':'` or a trace URL with `?node=span-` to focus one span directly; otherwise, if the trace output says it shows a subset of spans and the user needs more detail, follow up with `search_events` on that trace.