Skip to content

Replace builtin computer control with managed Cua MCP - #212

Merged
NarwhalChen merged 7 commits into
mainfrom
feat/cua-driver-mcp
Jul 31, 2026
Merged

Replace builtin computer control with managed Cua MCP#212
NarwhalChen merged 7 commits into
mainfrom
feat/cua-driver-mcp

Conversation

@NarwhalChen

@NarwhalChen NarwhalChen commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • remove the RobotJS-backed builtin:computer_control tool and its dedicated tests while retaining RobotJS and the shared robot wrapper for paste and other non-CUA behavior
  • register cua as a Convera-managed stdio MCP using cua-driver mcp, preserve only its user-controlled disabled state, and protect it from replacement or deletion
  • keep startup non-blocking when Cua Driver is absent and expose an actionable missing-binary error in MCP settings, with no fallback to the former RobotJS CUA
  • pass the managed Cua stdio configuration directly to Codex and Claude so they consume the original MCP tool schemas; Cua tools are not converted into Convera business-tool aliases
  • make global MCP initialization idempotent so starting a chat does not disconnect Cua during catalog construction
  • upgrade @modelcontextprotocol/sdk to 1.13.0, the minimum release that accepts the official driver's MCP protocol version 2025-06-18
  • add root AGENTS.md instructions requiring future agents to discover and run Convera's real Electron automation before declaring application behavior complete

The command and lifecycle follow the official Cua Driver references:

Verification

  • pnpm --filter @convera/app test — 21 files, 68 tests passed
  • pnpm --filter @convera/app lint — passed with 6 pre-existing hook warnings outside this change
  • pnpm --filter @convera/app exec tsc --noEmit --skipLibCheck — passed
  • pnpm --filter @convera/app automation:typecheck — passed
  • pnpm --filter @convera/app format — passed
  • pnpm --filter @convera/app automation:prepare — packaged Electron and prepared matching ChromeDriver
  • push hook Prettier, ESLint, and full Vitest suite — passed
  • official Cua Driver 0.14.1:
    • installed through the official installer; doctor passed
    • macOS Accessibility and Screen Recording permissions granted to the signed com.trycua.driver app
    • telemetry disabled
    • raw cua-driver mcp handshake succeeded and exposed 49 native tools
  • real packaged Electron E2E with an isolated authenticated Codex profile:
    • Convera started with the managed default cua server
    • inner Codex used only rendered native tools including mcp__cua__start_session, mcp__cua__launch_app, mcp__cua__list_windows, mcp__cua__get_window_state, mcp__cua__click, mcp__cua__press_key, and mcp__cua__end_session
    • those tools operated the real macOS Calculator and produced 7×8 = 56
    • Convera rendered the final marker CONVERA_NATIVE_CUA_56
    • an independent Cua AX-tree read confirmed expression 7×8 and result 56
  • missing-binary, managed deletion/replacement protection, user disable, default config, native-tool exposure, and removal of the legacy builtin are covered by tests
  • Codex and Claude adapter tests verify original managed stdio passthrough; no real Claude process was run

Remaining boundary

  • real macOS Codex-to-Cua interaction is verified; Windows and Linux were not exercised
  • Claude is covered at the adapter/catalog boundary only, as requested
  • application source typechecking passes with library declarations skipped; plain tsc --noEmit is still affected by existing third-party declaration errors
  • CI for the latest push should be reviewed before merge

This PR is intentionally left unmerged.

@vercel

vercel Bot commented Jul 30, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
foxyfox-docs Error Error Jul 31, 2026 7:53am
foxyfox-website Error Error Jul 31, 2026 7:53am

Request Review

@vercel
vercel Bot temporarily deployed to Preview – foxyfox-website July 30, 2026 15:25 Inactive
@vercel
vercel Bot temporarily deployed to Preview – foxyfox-docs July 30, 2026 15:25 Inactive
@vercel

vercel Bot commented Jul 30, 2026

Copy link
Copy Markdown

@NarwhalChen must be a member of the codefox-repo team on Vercel to deploy.
- Click here to add @NarwhalChen to the team.
- If you initiated this build, request access.

Learn more about collaboration on Vercel and other options here.

@vercel
vercel Bot temporarily deployed to Preview – foxyfox-docs July 30, 2026 16:05 Inactive
@vercel
vercel Bot temporarily deployed to Preview – foxyfox-website July 30, 2026 16:05 Inactive
# Conflicts:
#	packages/app/src/electron/ai/__tests__/codex-cli-mcp.test.ts
#	packages/app/src/electron/ai/provider-adapter.ts
#	packages/app/src/electron/ai/providers/claude-code.ts
#	packages/app/src/electron/ai/runtime.ts
#	packages/app/src/electron/main.ts
#	packages/app/src/electron/mcp/runtime-catalog.test.ts
#	packages/app/src/renderer/components/settings/pages/mcp-page.tsx
@NarwhalChen
NarwhalChen merged commit 541eceb into main Jul 31, 2026
3 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant