Request for Shaping Feedback: Addons for Agent Canvas UI

[DevinVinson]

Request for Shaping Feedback: Addons for Agent Canvas UI

I’ve been experimenting with local UI add-ons for Agent Canvas in this branch:

• Initial PoC pass: https://github.com/DevinVinson/agent-canvas/tree/dv/addon-support
• Updated to use .openhands files: https://github.com/DevinVinson/agent-canvas/tree/dv/addons-clean-v1
• Another branch with docker live reloading and a whole api route for addon refreshing: https://github.com/DevinVinson/agent-canvas/tree/dv/addons-clean-v1--with-docker-live-editing . This would support asking your agent to build addons and seeing them live.

This is not ready as a merge PR yet. I’m opening this first to get shaping feedback on the direction, the user experience, and where add-ons should live long term.

What The Prototype Explores

The prototype adds a trusted local, frontend-only add-on system for Agent Canvas.

The current implementation discovers add-ons from a repo-local folder:

addons/<addon-id>/.openhands/addon.json

Then npm run make-addons generates a registry that Agent Canvas uses to
include the add-ons in the frontend app.

The important thing about the current prototype: add-on files are compiled by Agent Canvas as part of the Vite/React build. When an add-on is already registered, Vite HMR works well for source changes, which makes local iteration feel good. Adding/removing an add-on or changing manifest metadata still requires regenerating the registry and restarting/rebuilding the UI.

Proof Of Concept Examples

1. Routed UI Add-on

The first use case is a normal add-on page that appears in the Agent Canvas sidebar and mounts under:

/addons/:addonId/*

Example manifest:

{
  "name": "agent-status",
  "title": "Agent Status",
  "frontend": {
    "entry": "src/index.tsx"
  },
  "sidebar": {
    "order": 350
  }
}

The add-on exports a register(api) function and returns a React component for
the route.

This proves that a local add-on can add a dedicated UI page without adding backend routes or changing agent-server behavior.

2. Style-only Shell Add-on

The second use case is a route-less add-on that customizes the Agent Canvas shell with CSS but does not add a sidebar icon or page.

Example manifest:

{
  "name": "canvas-polish",
  "title": "Canvas Polish",
  "frontend": {
    "route": false
  },
  "sidebar": {
    "visible": false
  },
  "styling": {
    "appCss": ["src/agent-canvas.css"]
  }
}

The CSS can scope to the Agent Canvas root:

[data-agent-server-ui][data-agent-canvas-addons~="canvas-polish"]
  > [data-theme] {
  --oh-color-primary: #7dd3fc;
}

This proves that an add-on can enhance or customize the whole UI without introducing a new route.

The Next Unclear Use Case: Route-less JavaScript

The third case I’d like feedback on is a UI add-on that needs JavaScript but still should not add a route.

Examples:

• global keyboard shortcuts
• small floating UI elements
• shell badges/status indicators
• command-palette entries, if we add shell slots later
• small app-wide UI behavior paired with CSS customization

I’m unsure which direction is better here.

Option A: Component-based Root Add-ons

A route-less add-on could still provide a normal .ts, .tsx, or .js entry:

{
  "name": "canvas-polish",
  "title": "Canvas Polish",
  "frontend": {
    "route": false,
    "entry": "src/index.tsx"
  },
  "sidebar": {
    "visible": false
  }
}

Then it could return a root-level React component:

export default function register(api) {
  return {
    AppComponent: CanvasPolishRoot,
  };
}

Agent Canvas would mount that component near the app root, outside the route outlet.

This keeps add-ons in the React/Vite/TypeScript world and preserves HMR in dev, but it still assumes Agent Canvas is compiling the add-on source.

Option B: Compiled Runtime Scripts

Another option is to support compiled JavaScript loaded on top of the running UI, for example an IIFE:

{
  "name": "canvas-polish",
  "title": "Canvas Polish",
  "frontend": {
    "route": false
  },
  "runtime": {
    "script": "dist/canvas-polish.iife.js",
    "style": "dist/canvas-polish.css"
  }
}

The script could register itself through a small global API.

This may fit future installs better because the running Agent Canvas app would not need to compile user source code. But it is less React-native, harder to typecheck, and opens more questions around cleanup, lifecycle, dependency duplication, and security.

Where Should Add-ons Live?

The prototype currently uses a repo-local addons/ folder because that was easiest for proving the concept.

For a real user-facing flow, I think local add-ons probably belong in:

~/.openhands/agent-canvas/addons/<addon-id>/

That would keep user customization with other local Agent Canvas/OpenHands data rather than mixing it into the Agent Canvas source checkout.

Possible layout:

~/.openhands/agent-canvas/addons/my-addon/
  .openhands/addon.json
  src/index.tsx
  src/styles.css

I’d like feedback on whether this is the right default.

Future Install Question

The current build-time model works naturally when developing Agent Canvas from source:

• npm run dev
• generate the add-on registry
• Vite compiles add-on source
• HMR works for already registered add-ons

But the future install story is less obvious.

If users install Agent Canvas from npm or run a prebuilt static package, then user-local .ts / .tsx add-ons cannot simply be compiled into the already-built app unless we add a specific addon development/build flow.

Possible directions:

1. Source/dev mode only:

   • source add-ons work when running npm run dev
   • good for contributors and local experimentation
   • not a complete installed-user story

2. Add-on dev mode:

   • support something like a dev-addons or
     ~/.openhands/agent-canvas/addons folder only in dev mode
   • HMR remains a local development feature
   • installed/static Agent Canvas would not promise source add-on compilation

3. Local add-on build/cache:

   • Agent Canvas could eventually provide a command that compiles user add-ons
     into a local cache under ~/.openhands/agent-canvas
   • more complex, but could support installed usage without committing add-on
     source into the main repo

4. Runtime compiled add-ons:

   • use precompiled JS/CSS assets loaded by the running app
   • possibly better for installed users
   • bigger API and security design

I’m not sure which of these should be the target yet.

Current Leaning

My current feelings on some of these:

Keep the first mergeable version focused on trusted local UI add-ons

For the first real PR, I think add-ons should be treated as local code that the user intentionally places on their own machine. They would not be sandboxed, reviewed, or installed from a marketplace.

Keep backend/agent runtime plugins out of scope

This proposal is only about the Agent Canvas browser UI. It should not add anything else.

Those may be useful someday, but they are a different design problem with a larger security and compatibility surface.

Support routed add-on pages and style-only shell add-ons

The first mergeable version should support the two use cases already proved in the prototype:

1. Routed add-on pages:

   • add a sidebar entry
   • mount under /addons/:addonId/*
   • render a React component returned by register(api)

2. Style-only shell add-ons:

   • add app-level CSS
   • do not add a route
   • do not add a sidebar icon
   • scope CSS through the Agent Canvas root marker

This keeps the first implementation understandable. Addons could be full single page apps with their own nav item and route which gives a lot of surface area when combined with the existing apis.

Move user add-ons toward ~/.openhands/agent-canvas/addons

The repo-local addons/ folder is useful for proving the concept, but it is probably not the right final location for user add-ons. Local user customization should likely live beside other Agent Canvas/OpenHands local state:

~/.openhands/agent-canvas/addons/<addon-id>/

This would keep user-installed add-ons out of the Agent Canvas source checkout and avoid making local add-on files show up as repo changes.

The open implementation question is how to do this cleanly while still letting Vite compile and watch those files. A polished version may need:

• a configurable add-on root
• Vite server.fs.allow support for the external directory
• a generated registry that does not commit machine-local absolute paths
• Docker path handling for the mounted ~/.openhands directory
• clear docs for source/dev mode versus installed/static mode

Preserve Vite HMR for already registered add-ons in dev mode

One nice property of the prototype is that once an add-on is registered, editing its source files can update through Vite HMR. That makes local add-on development feel much better than a full rebuild loop.

I think that is worth preserving for npm run dev or any future add-on dev mode.

However, HMR should not be over-promised. Adding or removing an add-on, or changing manifest fields like route/sidebar/style metadata, would still require regenerating the add-on registry and restarting/rebuilding the UI unless we add a more dynamic discovery layer later.

Get feedback before deciding route-less JavaScript

I do not think route-less JavaScript should be rushed into the first merge unless there is clear agreement on the shape...but I would really like to see it.

The main options seem to be:

• root-mounted React components returned from register(api)
• compiled IIFE/runtime scripts loaded by the running app
• explicit shell slots first, then add-ons can target those slots
• no route-less JavaScript yet

This choice affects the install story, security model, cleanup lifecycle, typing, testing, HMR, and how much of Agent Canvas internals add-ons can touch.

I would like feedback on that direction before turning it into implementation. See below

Feedback Requested

I’d especially like feedback on:

1. Does the general local UI addon direction make sense for Agent Canvas?
2. Are routed add-ons under /addons/:addonId/* the right first scope?
3. Should user add-ons live under ~/.openhands/agent-canvas/addons?
4. Should source-based add-ons be dev/source-build only?
5. What should the installed/npm package story be?
6. For route-less JavaScript, should we prefer: root-mounted React components, compiled IIFE/runtime scripts, both eventually, or neither until we define shell slots?
7. What security/support concerns would block this from becoming a real PR?

[jpelletier1]

I like the concept a lot. I wonder if there are specific customer use cases we could build initial extensions for. For example, is there a Code Review dashboard or Vulnerability Remediation extension that would make sense here? Like something that monitors repos for CVEs and then lets' you kickoff remediation jobs, something similar to: https://openhands-vulnerability-fixer.vercel.app/

[smolpaws]

Proposal: UI Slots + Conversation-Aware AddonApi

Building on Devin's addon foundation, two additions would make addons far more useful — without changing the overall architecture.

1. UI Slots

Addons that can only render full pages under /addons/:id miss the conversation view where users actually spend time. Named slots would let addons inject UI inline:

// In addon's register function:
export default function register(api: AddonApi): AddonRegistration {
  return {
    Component: AgentStatusPage,
    slots: {
      // Widget in conversation sidebar
      "conversation.sidebar": () => <ConnectionStatus api={api} />,
      
      // Action button in conversation header
      "conversation.header.actions": () => <ExportButton api={api} />,
      
      // Custom renderer for specific event types
      "conversation.event": ({ event }) => {
        if (isHookExecutionEvent(event) && event.hook_event_type === "PreToolUse") {
          return <ToolApprovalCard event={event} api={api} />;
        }
        return null; // fall through to default rendering
      },
      
      // Status indicator in conversation footer
      "conversation.footer": () => <TokenCounter api={api} />,
    },
  };
}

Agent-canvas would mount these via a simple <SlotRenderer> component at each insertion point:

// Inside conversation view layout:
<SlotRenderer slot="conversation.sidebar" context={{ conversationId }} />

This mirrors pi's setWidget / setStatus / registerMessageRenderer pattern but maps naturally to React component slots.

2. Conversation Context in AddonApi

The current AddonApi gives addons fetchJSON, navigate, and queryClient. For interactive addons, add:

interface AddonApi {
  // ... existing methods ...
  
  // Subscribe to conversation events from the WebSocket stream
  onEvent(handler: (event: OpenHandsEvent) => void): () => void;
  
  // Get current conversation state
  getActiveConversation(): { id: string; status: string } | null;
  
  // Send a user message to the active conversation
  sendMessage(content: string): Promise<void>;
  
  // Access conversation history
  getEvents(options?: { limit?: number }): Promise<OpenHandsEvent[]>;
}

This is lightweight to implement — the WebSocket connection and conversation state already exist in agent-canvas contexts. The addon SDK just needs to bridge them:

// In sdk.ts, extend createAddonApi:
onEvent(handler) {
  // Subscribe to the existing WebSocket event stream
  const unsubscribe = conversationEventBus.subscribe(handler);
  // Track for cleanup when addon unmounts
  cleanupFns.push(unsubscribe);
  return unsubscribe;
},

What this enables

With slots + event access, addons can build things like:

• Tool approval UI — custom dialog when PreToolUse hook fires, with approve/reject buttons
• Live token counter — footer widget tracking usage across the conversation
• Custom file previews — event renderer that shows rich diffs instead of raw output
• CI/CD status — sidebar widget showing build status for the current workspace
• Cost dashboard — header action that opens a cost breakdown panel

All without touching agent-canvas core code.

What stays the same

• Devin's manifest, registry, build-time compilation, error boundaries, CSS injection — all good as-is
• Routed pages under /addons/:id remain the primary addon surface for complex UIs
• Slots are opt-in per addon, addons without slots work exactly as before

---

Ref: pi agent extension docs — the slot and event patterns above are adapted from pi's ctx.ui and pi.on() APIs for a React/web context.

[DevinVinson]

I like the concept a lot. I wonder if there are specific customer use cases we could build initial extensions for. For example, is there a Code Review dashboard or Vulnerability Remediation extension that would make sense here? Like something that monitors repos for CVEs and then lets' you kickoff remediation jobs, something similar to: https://openhands-vulnerability-fixer.vercel.app/

I think customer use cases are exactly the right way to frame addons. And something like that vulnerability fixer could probably be turned into an add-on since these are essentially embedded single page apps at this point in my POC.

Expanding to the more in depth support @smolpaws is suggesting could build on that even further.

Once an example or two is in place a user can just ask their agent to generate any other addons they want or need.

[enyst]

• Initial PoC pass: https://github.com/DevinVinson/agent-canvas/tree/dv/addon-support

May I suggest to make a PR or PRs with it? It would be easier to review on source and maybe discuss further. It can be a draft PR, and if you wish, you could name it in some scary… - I mean, clear way 😅

I went through the first in full, and I was mulling on … stuff, while reading, but I’m dense and forget easily 😭

This choice affects [,,,] how much of Agent Canvas internals add-ons can touch.

I think we should give people the tools to touch and shape as much as possible. So I would go to the extreme on this. If there is something truly problematic, I’d love to know what that is and maybe we can look at it.

1. Does the general local UI addon direction make sense for Agent Canvas?

No strong opinion, just thinking, I suppose we could .gitignore a directory in the user repo?

4. Should source-based add-ons be dev/source-build only?

Not sure if I understand correctly, I do think it would be nice if we can offer the maximum of adapt-UI-on-the-fly, it’s fun, and hey, people like fun. 🙏

7. What security/support concerns would block this from becoming a real PR?

Speaking for local use, not sure there’s much… They’re chosen by the user, for their own machine.

[enyst]

One nice property of the prototype is that once an add-on is registered, editing its source files can update through Vite HMR. That makes local add-on development feel much better than a full rebuild loop.

I think that is worth preserving for npm run dev or any future add-on dev mode.

💯

Worth preserving as much as possible, IMHO, for people customizing or asking the agent to customize.

Speaking of which, sorry I read some on the run: how about the agent, I think in a local dev install it should know about this. Are we planning an Agentskill for it, including with info on playing live?

[enyst]

FWIW, I re-read smolpaws’ points and it seems reasonable to me - it’s not much. I think part of the addon story could be: OpenHands agent is modifying itself -ish. Or at least its dressing. 😂

OK, that sounded better in my head!

[enyst]

I like the concept a lot. I wonder if there are specific customer use cases we could build initial extensions for. For example, is there a Code Review dashboard or Vulnerability Remediation extension that would make sense here? Like something that monitors repos for CVEs and then lets' you kickoff remediation jobs, something similar to: https://openhands-vulnerability-fixer.vercel.app/

I think customer use cases are exactly the right way to frame addons. And something like that vulnerability fixer could probably be turned into an add-on since these are essentially embedded single page apps at this point in my POC.

Sorry I’m dense 🤔 I’m hearing functionality here, rather than UI… I’d love to see an example, it would be good to know if it crosses the boundary from “UI addon” to “more complex extension that contains prompts or defines automations”; if so, it may be worth thinking a bit… if that should be the case

[DevinVinson]

If anyone wants to keep following along I started on another branch here with just some more detailed docs: https://github.com/DevinVinson/agent-canvas/tree/dv/extensions-poc-v1/docs

RFC has the the big overview of taking this from Addons to Extensions with agent-canvas CLI install commands for potentially all things extensions: https://github.com/DevinVinson/agent-canvas/blob/dv/extensions-poc-v1/docs/ExtensionsSystemRFC.md

[enyst]

Cross-linking the per-tool visualizer work from PR #1246 with this addon proposal:

• PR: feat(chat): per-tool visualizers for tool calls in the conversation UI #1246
• Architecture note / diagrams: https://enyst.github.io/arch/agent-canvas-tool-visualizers.html

I read the proposal and comments here. The visualizer PR creates a useful internal seam: getEventContent() asks a registry for a React body by action/observation kind, and falls back to the existing markdown path when no renderer is registered. Today, users still need to edit/fork Agent Canvas to add or override a visualizer.

A focused addon follow-up could make custom event rendering the first concrete route-less JS use case:

export default function register(api: AddonApi) {
  api.registerToolVisualizer({
    id: "acme.kubernetes.apply",
    actionKinds: ["MCPToolAction"],
    observationKinds: ["MCPToolObservation"],
    priority: 50,
    matches({ action }) {
      return action?.action.kind === "MCPToolAction"
        && action.action.tool_name === "kubectl_apply";
    },
    Body({ action, observation }) {
      return <KubernetesApplyCard action={action} observation={observation} />;
    },
  });
}

Suggested shape:

1. Add an addon API method (or equivalent slot) named registerToolVisualizer() / conversation.event.visualizers.
2. Compose the active registry as addon visualizers first, built-ins second, markdown fallback last.
3. Support priority plus matches() so an addon can override one specific MCP tool or event variant without replacing the whole kind.
4. Keep the first version in the trusted local source-addon model discussed here (~/.openhands/agent-canvas/addons/<id>, generated registry, Vite compile/HMR in dev/source mode).
5. Export stable visualizer types and primitives from a public subpath such as @openhands/agent-canvas/visualizers.
6. Wrap addon renderers in error boundaries and fall back to the next renderer/default markdown on failure.

This seems to align with the slot/API direction from the comments while staying narrower than a full root-mounted route-less component system. It also gives a practical answer to “how does a user define a different way to render an event?” without requiring them to maintain a fork.

This comment was created by an AI agent (OpenHands) on behalf of the user.

[openhands-ai]

I'm on it! enyst can track my progress at all-hands.dev

[openhands-ai]

Since my last summary, I only made one additional change: I ran Prettier on the modified files to fix the lint formatting errors reported in src/visualizers/tool-visualizer-registry.tsx.

Checklist status:

• ✅ Formatting issue from npm run lint was addressed.
• ⚠️ I did not rerun npm run lint after formatting.
• ⚠️ I did not create a branch, commit, push, or open the PR yet.
• ⚠️ The issue implementation appears mostly complete based on the earlier successful focused tests, typecheck, and build, but final verification and PR steps remain unfinished.

Conciseness:

• The additional change was concise and directly relevant: formatting only.
• No extraneous changes were intentionally added since the last summary.