Skip to content

agent.md

Latest commit

 

History

History
260 lines (202 loc) · 19.6 KB

agent.md

File metadata and controls

260 lines (202 loc) · 19.6 KB
Raw

Agent and the MCP server

Agent is the design assistant built into Karbonized. You describe what you want and it edits the canvas with the same actions you use: it adds and updates blocks, aligns them, changes the background and exports images. It works with the model provider you choose.

The desktop app can also expose those actions as a local MCP server, so Claude Desktop, Claude Code, Cursor and other MCP clients can control Karbonized.

Using Agent

Open the panel with Ctrl/⌘ + L, the Agent button in the status bar, the AI menu in the menu bar or Show Agent in the command palette. It docks on the left edge of the window, opposite the properties panel. The layout button at the left of the status bar switches between the canvas alone, the properties panel, the agent, or both.

  • Type a request and press Enter (Shift + Enter for a new line).
  • Answers stream in. Every action shows up as a card; open it to see its arguments, its result and how long it took.
  • Stop ends the response at any point. Retry asks again from your last message.
  • The model picker under the message box switches between your providers.
  • New chat starts over; the history button reopens the last 30 chats. Chats are stored on your device.

When the model accepts images, Agent can look at a snapshot of the canvas to check its work.

Model providers

Open Agent settings (gear in the panel, or the command palette) and add a provider:

Provider Base URL Key Notes
Anthropic https://api.anthropic.com Required
OpenAI https://api.openai.com/v1 Required
Google Gemini https://generativelanguage.googleapis.com Required
OpenRouter https://openrouter.ai/api/v1 Required Any model on OpenRouter
Ollama http://localhost:11434/v1 No Local models
LM Studio http://localhost:1234/v1 No Local models
OpenAI-compatible your server Optional Any server that speaks the Chat Completions API

Browse lists the models of the provider, and Test connection checks the base URL and key. Pick a model that supports tool calling; small local models may struggle with multi-step edits.

Web version and CORS

The web app calls providers straight from the browser. Anthropic, OpenAI, Gemini and OpenRouter allow it. For local servers:

  • Ollama: allow the site with the OLLAMA_ORIGINS environment variable (set it to the address of the site, for example OLLAMA_ORIGINS=https://example.github.io) and restart Ollama.
  • LM Studio: turn on Enable CORS in the server settings.
  • Other servers: enable CORS on the server, or use the desktop app.

The desktop app sends provider requests from its main process, so CORS never applies there.

API keys

Keys never leave your device except in requests to their provider, and they are never logged.

  • Desktop app: keys are encrypted with the system keychain and only decrypted by the main process when it sends a request. The page cannot read them back, and a key only works with the base URL it was saved for. If you change the base URL, save the key again.
  • Web app: keys are stored in this browser (IndexedDB).

Undo

Everything Agent does in one response is one undo step: press Ctrl + Z on the canvas, or Undo changes under the response. Changes made by MCP clients undo one tool call at a time. Your own edits made while Agent works are never merged into its step.

Tools

Agent and the MCP server share the same tools:

Tool What it does
get_design_guide The design standards (sizes per platform, layout, type, color, HTML block rules, final checklist)
search_icons Icon names (Font Awesome and installed icon packs) for icon blocks and @type:icon variables
search_fonts Google Fonts families by name, kind and weights, with the weights each one has
get_brand_kit The brand kit: named colors, fonts, logos (without the images) and guidelines
update_brand_kit Create or change the brand kit: name, palette, fonts by role (checked against Google Fonts), logo names and uses, guidelines
save_brand_logo Add a logo to the brand kit from SVG markup (cleaned of scripts), a data:image/… URL or an image block of the canvas
add_brand_logo Place a logo of the brand kit as an image block, by id or variant, keeping its proportions
get_workspace Canvas size, background, selection and every block with its position, size and properties
create_workspace New project with a canvas size, opened in the editor
set_canvas_background Color, gradient, texture, wallpaper or dynamic background, blur and noise
set_canvas_size Resize the canvas
set_guides Place the guides blocks snap to, in canvas pixels
set_variables Create, fill or remove project variables (template slots that blocks show as {{name}})
list_block_types Block types, their properties and size limits, code themes
add_block Add a block (code, text, image, window, phone, shape, icon, QR, freehand stroke, HTML)
update_block Name, position, size, rotation, crop, visibility, lock and properties
delete_blocks Delete blocks
select_blocks Select blocks in the editor
align_blocks / distribute_blocks Align or space blocks; one block aligns to the canvas
reorder_block Bring to front, forward, backward, send to back
get_html_block / update_html_block Read or replace the HTML, CSS and JavaScript of an HTML block
list_components The component library: imported .kcomponent files and the starter pack
import_component Add a .kcomponent file (YAML) to the library
add_component Put a component from the library on the canvas, at its manifest size
export_component The .kcomponent file of a library component or of an HTML block
load_starter_pack Import the components that ship with Karbonized
get_canvas_snapshot PNG of the canvas (models with image input)
export_image Export PNG, JPEG or SVG without a dialog: to the export folder (desktop), as a download (web) or back to the caller
list_commands / run_command Editor commands: undo, duplicate, zoom, snapping, rulers, and the tools the user draws with (shape, brush, nodes, eraser, crop, pan)

Design standards

Both Agent and MCP clients design against one guide, src/lib/agent/core/design-guide.ts: canvas sizes per platform (and safe zones for stories), margins and an 8 px grid, a type scale for images seen on a phone, contrast and palette rules, when and how to use HTML blocks, and a checklist to run against get_canvas_snapshot before finishing. It is part of the Agent system prompt and of the MCP server instructions, and get_design_guide returns it for clients that ignore server instructions. Change the guide there; the prompt, the instructions and the tool follow.

The guide asks models to build a design block by block: the background with set_canvas_background, every headline and paragraph as a text block, and one HTML block per component (a stat tile, a card, a badge row, a chart), sized to its content.

Every HTML block a model writes has html, css and js (src/lib/agent/tools/html-contract.ts): the look as annotated :root variables in the CSS and the content (labels, values, list items, chart data) as // @var JS variables that the script writes into the markup, so the user edits both from the panel. add_block, update_block and update_html_block refuse code without them, with a script that declares a // @var again, or with an array or object value that is not JSON (an array must be a list of strings, which is what the panel edits), and say what to fix; they turn allow-scripts on for the block. They also answer with hints (src/lib/agent/tools/html-hints.ts) when an HTML block has too few variables, covers most of the canvas or holds paragraphs of text, so the model fixes it in the same turn.

Fonts: the guide sends models to search_fonts (the whole Google Fonts catalog, with the weights of each family), gives pairings by tone and asks for fontSource: "google". add_block, update_block and update_html_block answer with hints when a family is not in Google Fonts, is not set to load or lacks the weight (src/lib/agent/tools/font-hints.ts).

Models are asked to look at the canvas while they build, not only at the end. After a few changes without a get_canvas_snapshot call, the result of the next change carries a reminder to look (CHANGES_BEFORE_LOOK in src/lib/agent/tools/registry.ts); callers without image input never get it. The snapshot waits for pending fonts so it does not show the fallback.

Brand kit

The Brand kit tab of the properties panel (also File → Brand kit… or the command palette, which open it; outside the editor it opens in a dialog) holds the colors, fonts, logos and guidelines of the user's brand. The color picker shows the brand colors first and the font picker the brand fonts. Agent and MCP clients are told to call get_brand_kit before a new design and to follow it over the palettes of the design guide; add_brand_logo places a logo without sending the image through the model. When the user gives them their brand or asks for one, they save it with update_brand_kit and save_brand_logo (src/lib/agent/tools/brand.ts); those changes are saved at once, show in the tab, and are not part of the canvas undo. A kit can be exported and imported as a .kbrand file (JSON). It is stored in IndexedDB (src/stores/brand-store.ts, model in src/lib/brand/brand-kit.ts).

Templates and project variables

A project can hold variables (the Variables tab of the properties panel): named texts, long texts and dates. Any text, code, window, QR or HTML block that contains {{name}} shows the value instead; the block keeps the placeholder, so the same design becomes a template. Changing a value is one undo step.

That is how a model makes the next post without redesigning: get_workspace lists the variables, set_variables fills them, export_image saves the result. Unknown names stay visible as {{name}}; set_variables reports variables no block uses and references with no variable. Values in HTML blocks are escaped, and a date can be today. The code lives in src/lib/variables/variables.ts; blocks read it through useResolvedText.

What tools cannot do

run_command runs the commands that are safe without a person watching: edit.* and arrange.* (undoable document changes), view.* (viewport and editor chrome), tools.* (pick a tool, insert a block) and file.copy-image. The rest is left out on purpose:

  • Dialogs, saving and leaving the editor (file.*, help.*, block.*, and the gallery behind tools.components) need a person. export_image, export_component and list_components cover what a model needs from them.
  • Clearing the workspace has no undo step; delete_blocks does.

Picking a tool changes what the user's next drag does — a model cannot draw by itself. To put something on the canvas, add_block and add_component are the direct route: they take a position, a size and properties.

MCP server (desktop app)

  1. Turn on AI → MCP server in the menu bar, or open Agent settings → MCP server and turn on Allow other apps to control Karbonized. While it is on, the MCP indicator in the status bar shows its state and opens these settings.
  2. Pick your client and copy its configuration. It already contains the URL and your token.
  3. That's it: Karbonized does not have to be open. The bridge in the configuration starts it in the background (no window, a tray icon) when a client needs it; the app has to have run once with the server turned on, which is when it records how to start itself.

Running without a window. While the server is on, closing the window hides Karbonized in the tray instead of quitting, so clients keep working (Keep running when the window is closed, on by default). The tray icon opens the window again or quits. Start in the background when you log in (Windows and macOS) starts it hidden at login. Launching Karbonized again shows the running instance: only one runs at a time. Tool calls that arrive while the hidden window is still loading wait for it instead of failing.

Exports. export_image saves to the export folder (Pictures/Karbonized unless you pick another one in these settings or in the export dialog) under a free name and returns the path, without a dialog. destination: "return" sends the image back to the client instead, and "ask" opens a save dialog.

Claude Desktop

Claude Desktop starts MCP servers as local processes, so Karbonized ships a small bridge (mcp-stdio.cjs) that runs with the Karbonized executable itself; Node.js is not needed. The same bridge serves Claude Code and Cursor, and starts Karbonized when it is not running. Paste the configuration in Settings → Developer → Edit Config and restart Claude Desktop. It looks like this:

{
	"mcpServers": {
		"karbonized": {
			"command": "C:\\Users\\you\\AppData\\Local\\Programs\\Karbonized\\Karbonized.exe",
			"args": [
				"C:\\Users\\you\\AppData\\Local\\Programs\\Karbonized\\resources\\app.asar.unpacked\\dist-electron\\mcp-stdio.cjs"
			],
			"env": {
				"ELECTRON_RUN_AS_NODE": "1",
				"KARBONIZED_MCP_URL": "http://127.0.0.1:7824/mcp",
				"KARBONIZED_MCP_TOKEN": "<your token>"
			}
		}
	}
}

Claude Code

claude mcp add karbonized -e ELECTRON_RUN_AS_NODE=1 -e KARBONIZED_MCP_URL=http://127.0.0.1:7824/mcp -e KARBONIZED_MCP_TOKEN=<your token> -- <Karbonized executable> <path to mcp-stdio.cjs>

Copy the exact command from the settings, which fills in both paths. Clients that only speak HTTP can still use http://127.0.0.1:7824/mcp with the header Authorization: Bearer <your token>, but then Karbonized has to be running (turn on Start in the background when you log in).

Cursor

Add to ~/.cursor/mcp.json (or .cursor/mcp.json in a project):

The same command, args and env as for Claude Desktop (copy them from the settings).

Security

  • The server is off by default and only listens on 127.0.0.1.
  • Every request needs the token. Requests that come from web pages (a foreign Origin or Host) are refused.
  • Anyone with the token can edit your canvas: keep it private, and regenerate it in settings if it leaks (then update your clients).

Troubleshooting

  • "Add your API key": the active provider needs a key. On desktop, save it again after changing the base URL.
  • "Could not reach the provider": check the base URL and that local servers are running. On the web, see CORS.
  • The model answers but does nothing: choose a model with tool calling.
  • MCP client: "Could not reach Karbonized": open the app and turn the server on. "rejected the token": copy the configuration again.
  • MCP client: "No workspace is open": open a project, or let the client call create_workspace.
  • Port already in use: choose another port in settings and update your clients.

For contributors

  • src/lib/editor/actions.ts: editor actions with arguments (blocks, canvas, workspaces), all undoable.
  • src/lib/blocks/catalog.ts: block types, properties, defaults and size limits. Keep it in sync with the useControlState calls of each block.
  • src/lib/agent/tools/: tool definitions (zod schemas) and executeTool, shared by Agent and the MCP server.
  • src/lib/agent/core/: provider-neutral chat types, SSE parser, agent loop, errors and key redaction.
  • src/lib/agent/providers/: Anthropic, OpenAI Chat Completions and Gemini adapters, and the provider presets.
  • src/lib/agent/transport/: browser fetch and Electron main-process transports.
  • src-electron/agent/: key storage and provider requests in the main process.
  • src-electron/mcp/: MCP server and stdio bridge; the renderer side is src/lib/agent/mcp/renderer.ts.

To add a tool, define it with defineTool in src/lib/agent/tools/, add it to editorTools and write a test. Mutating tools must change the document synchronously in execute (it runs inside a history transaction) and can wait in settle.