Skip to content

Create Excalidraw Agent#2168

Draft
GeorgeNgMsft wants to merge 5 commits intomainfrom
dev/georgeng/excalidraw_agent
Draft

Create Excalidraw Agent#2168
GeorgeNgMsft wants to merge 5 commits intomainfrom
dev/georgeng/excalidraw_agent

Conversation

@GeorgeNgMsft
Copy link
Copy Markdown
Contributor

@GeorgeNgMsft GeorgeNgMsft commented Apr 8, 2026

Add Excalidraw Agent for AI-Powered Diagram Generation

Summary
Adds a new excalidraw TypeAgent that converts documentation, markdown, Visio XML exports, Mermaid
diagrams, and plain-text architecture descriptions into https://excalidraw.com/ diagram files using
AI. The agent integrates with the existing TypeAgent dispatcher and agent provider infrastructure.

What's Changed
New package: ts/packages/agents/excalidraw/

  • excalidrawActionSchema.ts — Defines two actions:
    • createDiagram — accepts sourceContent, sourceType (markdown | text | visio-xml | mermaid |
      architecture), optional diagramTitle, and optional outputPath
    • exportDiagram — writes provided Excalidraw JSON to a specified path
  • excalidrawActionHandler.ts — Calls the OpenAI chat model with a structured system prompt that
    instructs the LLM to output valid Excalidraw JSON (type, version, elements[], appState). Writes the
    resulting .excalidraw file to the user-specified path, or defaults to ~/Documents/ with a timestamped
    filename.
  • excalidrawManifest.json — Agent manifest for TypeAgent discovery

Agent registration: ts/packages/defaultAgentProvider/

  • Registered in config.json, config.all.json, and config.agent.json with execMode: "dispatcher"
  • Added excalidraw-agent as a workspace dependency

Usage Examples

▎ "Create an Excalidraw diagram from this architecture document"
▎ "Convert this Mermaid diagram to an Excalidraw file and save it to C:\diagrams\arch.excalidraw"
▎ "Turn this Visio XML into an Excalidraw diagram"

Testing

  • pnpm run build excalidraw-agent passes (114 tasks, tsc + asc compilation)
  • Schema compiles to excalidrawSchema.pas.json in dist/

Notes

  • Output defaults to ~/Documents/<title>_.excalidraw when no path is provided
    (cross-platform: works on Windows and Unix)
  • The LLM prompt includes explicit Excalidraw JSON structure guidance (elements, text binding,
    arrows, colors, layout) to maximize diagram quality

GeorgeNgMsft and others added 2 commits April 8, 2026 13:33
Implements a new TypeAgent agent (excalidraw-agent) that uses an LLM to
convert source materials (markdown, plain text, Visio XML, Mermaid, and
architecture descriptions) into valid Excalidraw JSON diagrams. Includes
createDiagram and exportDiagram actions, with output defaulting to the
user's Documents folder.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant