Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions apps/server/src/mcp/browserTools.ts
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ export const BrowserSnapshotTool = readsOnly(
export const BrowserScreenshotTool = readsOnly(
Tool.make("browser_screenshot", {
description:
"See what the page in the Threadlines preview panel currently looks like. Use this for anything visual -- layout, spacing, colour, whether something is actually on screen -- and use browser_snapshot instead when you need to act on an element or read an error.",
"See what the page in the Threadlines browser panel currently looks like. Use this for anything visual -- layout, spacing, colour, whether something is actually on screen -- and use browser_snapshot instead when you need to act on an element or read an error.",
parameters: PreviewAutomationEmptyInputSchema,
success: PreviewAutomationScreenshotSchema,
failure: PreviewAutomationErrorSchema,
Expand All @@ -75,7 +75,7 @@ export const BrowserScreenshotTool = readsOnly(
export const BrowserStatusTool = readsOnly(
Tool.make("browser_status", {
description:
"Where the Threadlines preview panel is right now: its URL, title, size, and whether it is still loading. Cheap; use it to confirm a navigation landed rather than taking a whole snapshot.",
"Where the Threadlines browser panel is right now: its URL, title, size, and whether it is still loading. Cheap; use it to confirm a navigation landed rather than taking a whole snapshot.",
parameters: PreviewAutomationEmptyInputSchema,
success: PreviewAutomationStatusSchema,
failure: PreviewAutomationErrorSchema,
Expand All @@ -86,7 +86,7 @@ export const BrowserStatusTool = readsOnly(
export const BrowserTabsTool = readsOnly(
Tool.make("browser_tabs", {
description:
"Every page open in the Threadlines preview panel, including each stable tab id, with the user's visible tab and your pinned tab marked. Use a tab id on any browser tool to act there without changing what the user sees.",
"Every page open in the Threadlines browser panel, including each stable tab id, with the user's visible tab and your pinned tab marked. Use a tab id on any browser tool to act there without changing what the user sees.",
success: PreviewAutomationTabsSchema,
failure: PreviewAutomationErrorSchema,
dependencies,
Expand Down Expand Up @@ -129,7 +129,7 @@ export const BrowserSelectTabTool = changesThePage(
export const BrowserNavigateTool = changesThePage(
Tool.make("browser_navigate", {
description:
"Go to a URL in the Threadlines preview panel the user is watching. This replaces what they are looking at, so navigate when you need to be somewhere else, not to reset state you can reach by clicking. Local and private addresses always work, but a site outside the project's approved list needs the user's approval: if this is refused for that reason, wait for them to allow it in the browser panel and then navigate again.",
"Go to a URL in the Threadlines browser panel the user is watching. This replaces what they are looking at, so navigate when you need to be somewhere else, not to reset state you can reach by clicking. Local and private addresses always work, but a site outside the project's approved list needs the user's approval: if this is refused for that reason, wait for them to allow it in the browser panel and then navigate again.",
parameters: PreviewAutomationNavigateInputSchema,
success: PreviewAutomationStatusSchema,
failure: PreviewAutomationErrorSchema,
Expand Down
32 changes: 3 additions & 29 deletions apps/server/src/provider/CodexDeveloperInstructions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -133,32 +133,6 @@ Use the \`request_user_input\` tool only when it is listed in the available tool
In Default mode, strongly prefer making reasonable assumptions and executing the user's request rather than stopping to ask questions. If you absolutely must ask a question because the answer cannot be discovered from local context and a reasonable assumption would be risky, ask the user directly with a concise plain-text question. Never write a multiple choice question as a textual assistant message.
</collaboration_mode>`;

/**
* Which browser the user means.
*
* Codex ships a bundled browser plugin whose skill claims the phrase "in-app
* browser" as one of its own aliases, and instructs the model outright: do not
* use external MCP browser-control tools for this surface. Asked to click
* something "in our browser", the model followed that skill, found no ChatGPT
* in-app browser available, silently fell back to the user's real Chrome, and
* reported on a signed-in personal session -- while telling the user it was
* using the in-app browser. It was not being careless; it was obeying a skill
* that had claimed the words.
*
* A tool description cannot answer that: the model commits to the skill before
* it reads our tool list. Developer instructions can, because developer-role
* text outranks a skill file, which reaches the model as tool output.
*
* Deliberately narrow. It says which surface the user is looking at and how to
* reach it, and leaves Codex's plugins alone for what they are for -- asked for
* Chrome, the model should still drive Chrome.
*/
export const CODEX_PREVIEW_PANEL_DEVELOPER_INSTRUCTIONS = `<threadlines_preview_panel>
This session runs inside Threadlines. The user is looking at a live page in Threadlines' embedded preview panel.

When the user says "our browser", "the browser", "the in-app browser", or "the page", they mean that panel. They do not mean Chrome, and they do not mean Codex's own in-app browser; those are different surfaces and neither can see what the user is looking at.

The panel is reachable only through the MCP tools \`mcp__threadlines_browser__*\` (browser_snapshot, browser_click, browser_navigate, browser_type, and the rest). Start with \`mcp__threadlines_browser__browser_snapshot\`.

Do not use the Browser plugin, the Chrome plugin, or any other browser automation for the user's panel. Use those only when the user explicitly asks for Chrome or for a browser outside Threadlines.
</threadlines_preview_panel>`;
/** Which browser the user means, and why to reach for it. Shared with Claude;
* see previewPanelInstructions.ts for the reasoning. */
export { CODEX_PREVIEW_PANEL_DEVELOPER_INSTRUCTIONS } from "./previewPanelInstructions.ts";
13 changes: 12 additions & 1 deletion apps/server/src/provider/Layers/ClaudeAdapter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ import {
isLinkedWorktreeCheckout,
missingWorkingDirectoryDetail,
} from "../../vcs/CheckoutPresence.ts";
import { CLAUDE_PREVIEW_PANEL_INSTRUCTIONS } from "../previewPanelInstructions.ts";
import * as Cause from "effect/Cause";
import * as DateTime from "effect/DateTime";
import * as Duration from "effect/Duration";
Expand Down Expand Up @@ -6504,14 +6505,24 @@ export const makeClaudeAdapter = Effect.fn("makeClaudeAdapter")(function* (
type: "http",
url: mcpEndpointUrl(serverConfig.port),
headers: { Authorization: `Bearer ${browserCredential}` },
// In the prompt rather than deferred behind tool search. Deferred,
// the model sees eighteen tool names and none of the sentences in
// browserTools.ts, and reaches for whichever browser has a blurb in
// context instead. About 3k tokens, all in the cached prefix. The
// SDK waits for the server before the first turn; it is this
// process, so the wait is nothing.
alwaysLoad: true,
},
},
...(apiModelId ? { model: apiModelId } : {}),
pathToClaudeCodeExecutable: claudeBinaryPath,
systemPrompt: {
type: "preset",
preset: "claude_code",
...(runsInManagedWorktree ? { append: MANAGED_WORKTREE_INSTRUCTION } : {}),
append: [
CLAUDE_PREVIEW_PANEL_INSTRUCTIONS,
...(runsInManagedWorktree ? [MANAGED_WORKTREE_INSTRUCTION] : []),
].join("\n\n"),
},
settingSources: [...CLAUDE_SETTING_SOURCES],
// SDK 0.3.233 dropped the todo/task tools from the default tool
Expand Down
63 changes: 63 additions & 0 deletions apps/server/src/provider/previewPanelInstructions.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
/**
* The browser panel, introduced to a model.
*
* Two jobs, in this order. First, sell it: an agent that needs to look at a web
* page reaches for whatever browser it already knows -- Chrome DevTools,
* Playwright, a bundled browser plugin -- unless its instructions say the panel
* beside the chat is the one to use, and why. Second, disambiguate: Codex ships
* a browser plugin whose skill claims the phrase "in-app browser" as one of its
* own aliases and tells the model outright not to use external MCP browser
* tools for it. Asked to click something "in our browser", the model followed
* that skill, found no ChatGPT in-app browser, silently fell back to the user's
* real Chrome, and reported on a signed-in personal session -- while telling
* the user it was using the in-app browser. It was not being careless; it was
* obeying a skill that had claimed the words.
*
* A tool description can do neither job. The model commits to a skill before
* it reads the tool list, and Claude does not see MCP tool descriptions at all
* until it searches for them. System-level text can, because developer-role
* instructions outrank a skill file, which reaches the model as tool output.
*
* One body with the rivals named per provider, so both providers hear the same
* story about the same panel. Kept short on purpose: every line here competes
* with the user's actual task on every turn.
*/

interface PreviewPanelRivals {
/** What the user does not mean by "the in-app browser", for this provider. */
readonly notThePanel: string;
/** The browser tooling this provider would otherwise reach for. */
readonly doNotUse: string;
}

const RIVALS: Record<"claude" | "codex", PreviewPanelRivals> = {
claude: {
notThePanel: "Claude in Chrome",
doNotUse:
"Do not use Claude in Chrome, the Chrome DevTools MCP tools, Playwright, or any other browser automation for the user's panel.",
},
codex: {
notThePanel: "Codex's own in-app browser",
doNotUse:
"Do not use the Browser plugin, the Chrome plugin, or any other browser automation for the user's panel.",
},
};

export function buildPreviewPanelInstructions(provider: "claude" | "codex"): string {
const rivals = RIVALS[provider];
return `<threadlines_browser>
This session runs inside Threadlines, a desktop workspace for coding agents. A browser panel sits beside this chat. Use it for any work that involves looking at a web page: opening a local dev server, checking a layout or a style change, reproducing a UI bug, reading console errors, or filling in a form. Prefer it over Chrome, headless browsers, and other browser automation, because the user watches every step in the panel, sees where you click, and can mark up the page and send the marks back to you. A browser you open anywhere else shows them nothing.

The panel is reachable only through the MCP tools \`mcp__threadlines_browser__*\` (browser_snapshot, browser_screenshot, browser_navigate, browser_click, browser_type, and the rest). Start with \`mcp__threadlines_browser__browser_snapshot\`. You do not need to ask the user to open the panel: your first browser tool call opens it. If a call reports that no browser is connected, the user is not viewing this thread in the Threadlines desktop app; say so before reaching for any other browser.

When the user says "our browser", "the browser", "the in-app browser", "the preview", or "the page", they mean that panel. They do not mean Chrome, and they do not mean ${rivals.notThePanel}; those are different surfaces and none of them can see what the user is looking at.

${rivals.doNotUse} Use those only when the user explicitly asks for Chrome or for a browser outside Threadlines.
</threadlines_browser>`;
}

/** Appended to Claude's system prompt on every session. */
export const CLAUDE_PREVIEW_PANEL_INSTRUCTIONS = buildPreviewPanelInstructions("claude");

/** Appended to Codex's developer instructions on every turn. */
export const CODEX_PREVIEW_PANEL_DEVELOPER_INSTRUCTIONS = buildPreviewPanelInstructions("codex");
2 changes: 1 addition & 1 deletion packages/contracts/src/previewAutomation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ export class PreviewAutomationNoHostError extends Schema.TaggedError<PreviewAuto
{ threadId: ThreadId, operation: PreviewAutomationOperationSchema },
) {
override get message(): string {
return `No browser preview is open for this thread, so ${this.operation} has nothing to act on. Ask the user to open the browser panel.`;
return `No Threadlines desktop window is showing this thread, so ${this.operation} has nothing to act on. The browser panel opens itself once the user views this thread in the desktop app; ask them to do that rather than using another browser.`;
}
}

Expand Down
Loading