[Feature] Add per-tool visualizers for tool calls in the conversation UI

[VascoSch92]

Problem

The tool calls are not rendered and presented properly to the end-user. (see video)

display_tool_calls.mov

I think presenting tool calls in a comprehensible way to the end-user not only enhances the user experience, but also makes it easier to debug, perform QA, and create good snapshot tests. For these reasons, I did a bit of scoping on how we could enable that.

Background

Tool calls (Bash, FileEditor, Grep, MCP, etc.) currently render through a single generic markdown-based path. The UI shows readable but unattractive output: bold labels, raw JSON dumps for MCP, one-line backtick commands for Bash, and no diff view for file edits.

The rendering path today:

• src/components/conversation-events/chat/event-message-components/generic-event-message-wrapper.tsx:31 calls getEventContent(event) and feeds the result into GenericEventMessage.
• src/components/features/chat/generic-event-message.tsx is the collapsible card. Its details prop already accepts string | React.ReactNode, but in practice nearly everything passes markdown.
• src/components/conversation-events/chat/event-content-helpers/get-action-content.ts and get-observation-content.ts are two large switch (kind) blocks that build markdown strings per tool.
• The fallback (shared.ts:5) is JSON.stringify(event, null, 2) inside a fenced block.

There is precedent for React-component rendering: TaskTrackerObservation already returns <TaskTrackingObservationContent> instead of markdown (get-event-content.tsx:296). This issue generalizes that pattern.

Worst offenders today

Tool	Current rendering
MCPToolAction (get-action-content.ts:103)	Raw JSON.stringify(action.data, null, 2) inside a code block
ExecuteBashAction (get-action-content.ts:89)	Command:\n\${cmd}`` — single line, no syntax highlight, no copy
FileEditorAction (get-action-content.ts:68)	Only create shows file content; edit / str_replace / view show nothing useful
GrepAction / GlobAction (get-action-content.ts:50)	Bold labels for params; matches rendered as plain markdown bullet list
Default fallback (shared.ts:5)	Full event serialized as JSON

Goals

1. Replace the markdown-string output of high-traffic tools with dedicated React visualizers.
2. Make adding a new visualizer cost one file and one barrel-import line.
3. Preserve the existing card chrome, collapse behavior, success indicator, and ACP rendering path.
4. Keep the markdown path as a fallback so unmigrated tools keep working unchanged.

Non-goals

• No changes to titles, i18n keys, the success indicator, confirmation buttons, or ACP tool-call rendering.
• No new dependencies — hand-rolled with Tailwind, matching current conventions.
• No combined action+observation cards in this PR. Two-card layout (action card, then observation card) stays as-is.
• No migration of low-impact tools (Think, Finish, browser ops, MCP, InvokeSkill, TaskTracker). Those follow in later PRs.

Proposed architecture

Type-safe registry via a defineVisualizer helper

A single helper that narrows Action / Observation by their literal kind so visualizer bodies receive correctly typed props without casts:

// src/components/features/chat/tool-visualizers/define.ts
type ActionKind      = Action["kind"];
type ObservationKind = Observation["kind"];
type ActionByKind<K extends ActionKind>           = Extract<Action, { kind: K }>;
type ObservationByKind<K extends ObservationKind> = Extract<Observation, { kind: K }>;

export interface VisualizerProps<A extends Action, O extends Observation> {
  action: ActionEvent<A>;
  observation?: ObservationEvent<O>;
}

export interface ToolVisualizer<AK extends ActionKind, OK extends ObservationKind> {
  actionKind: AK;
  observationKind?: OK;
  titleKey: string;
  icon?: React.ComponentType<{ className?: string }>;
  Body: React.FC<VisualizerProps<ActionByKind<AK>, ObservationByKind<OK>>>;
}

export const defineVisualizer = <AK extends ActionKind, OK extends ObservationKind>(
  v: ToolVisualizer<AK, OK>,
) => v;

Explicit barrel registration

// src/components/features/chat/tool-visualizers/index.ts
import { bashVisualizer }       from "./bash/bash";
import { fileEditorVisualizer } from "./file-editor/file-editor";
import { searchVisualizer }     from "./search/search";

const ALL = [bashVisualizer, fileEditorVisualizer, searchVisualizer];

export const actionVisualizers      = new Map(ALL.map(v => [v.actionKind, v]));
export const observationVisualizers = new Map(
  ALL.filter(v => v.observationKind).map(v => [v.observationKind!, v]),
);

Auto-discovery via import.meta.glob was considered and rejected: it hurts tree-shaking, breaks test mocking, and hides what is wired.

Dispatch seam

get-event-content.tsx checks the registry before falling through to the existing markdown helpers:

function resolveVisualizer(event, correspondingAction) {
  if (isActionEvent(event)) {
    const v = actionVisualizers.get(event.action.kind);
    if (v) return { title: t(v.titleKey), body: <v.Body action={event} /> };
  }
  if (isObservationEvent(event) && correspondingAction) {
    const v = observationVisualizers.get(event.observation.kind);
    if (v) return { title: t(v.titleKey), body: <v.Body action={correspondingAction} observation={event} /> };
  }
  return null;
}

If resolveVisualizer returns null, the current markdown pipeline runs unchanged.

Shared primitives

Dumb, presentation-only components in tool-visualizers/primitives/. No knowledge of event types. Each is a few dozen lines of Tailwind:

• CodeBlock — monospace, language hint, copy button, expand when long.
• FilePathChip — monospace path with copy icon and optional line range.
• DiffView — unified diff for str_replace-style edits.
• OutputPane — stdout / stderr split, exit-code badge, collapsible.
• KeyValueGrid — for simple key/value displays.

Example visualizer (full file)

// src/components/features/chat/tool-visualizers/bash/bash.tsx
import TerminalIcon from "#/icons/terminal.svg?react";
import { defineVisualizer } from "../define";
import { CodeBlock } from "../primitives/code-block";
import { OutputPane } from "../primitives/output-pane";

export const bashVisualizer = defineVisualizer({
  actionKind: "ExecuteBashAction",
  observationKind: "ExecuteBashObservation",
  titleKey: "ACTION_MESSAGE$RUN",
  icon: TerminalIcon,
  Body: ({ action, observation }) => (
    <div className="flex flex-col gap-2">
      <CodeBlock language="bash" copy>{action.action.command}</CodeBlock>
      {observation && (
        <OutputPane
          stdout={observation.observation.output}
          exitCode={observation.observation.exit_code}
        />
      )}
    </div>
  ),
});

Folder layout

src/components/features/chat/tool-visualizers/
  define.ts
  index.ts
  dispatcher.ts
  test-utils.tsx
  primitives/
    code-block.tsx
    file-path-chip.tsx
    diff-view.tsx
    output-pane.tsx
    key-value-grid.tsx
  bash/
    bash.tsx
    bash.test.tsx
  file-editor/
    file-editor.tsx
    file-editor.test.tsx
  search/
    search.tsx
    search.test.tsx

Scope of this issue (Tier 1)

Migrate the three highest-traffic, ugliest-today tools, each as an action + observation pair:

Tool	Action	Observation
Bash	ExecuteBashAction	ExecuteBashObservation
File editor	FileEditorAction, StrReplaceEditorAction	FileEditorObservation, StrReplaceEditorObservation
Search	GrepAction, GlobAction	GrepObservation, GlobObservation

Visual targets

Tool	Today	After
Bash	One-line backtick command, raw stdout dump	CodeBlock (bash) for command, OutputPane with stdout/stderr split and exit-code badge
FileEditor create	Path + truncated text	FilePathChip + CodeBlock with detected language
FileEditor str_replace	Nothing	DiffView of old vs new
FileEditor view	Nothing	FilePathChip + line-range badge
Grep / Glob	Bold labels + bullet list	Pattern and path chips, match count, file:line list

How to add a new visualizer (post-merge)

Documented in tool-visualizers/index.ts:

1. Create tool-visualizers/<name>/<name>.tsx exporting defineVisualizer({ ... }).
2. Add one import and one entry in the ALL array in index.ts.
3. Add <name>.test.tsx using renderVisualizer with fixtures.

TypeScript enforces the rest: actionKind autocompletes from the Action union, the Body component receives a narrowed event type, and typos in kinds are compile errors.

Testing

• Co-locate *.test.tsx next to each visualizer.
• Reuse fixtures already in __tests__/components/conversation-events/chat/.
• One snapshot per (action, observation) pair, plus error states for Bash (non-zero exit) and FileEditor (failed edit).
• Existing tests in __tests__/components/conversation-events/chat/event-message-*.test.tsx and __tests__/utils/handle-event-for-ui.test.ts must continue to pass for unmigrated tools (fallback path).

Risks and edge cases

• The markdown strings are also consumed by ACP tool-call rendering (get-event-content.tsx:305), which builds details "the same way" as bash output. The dispatcher must only intercept the action/observation details body, not the title pipeline or the ACP path.
• correspondingAction may be undefined for an observation event (e.g., resumed conversation, missing action). The dispatcher falls back to markdown in that case.
• Streaming: action arrives before observation. Visualizers render the action card immediately; the observation card appears later. Two-card layout means no coordination needed.

Acceptance criteria

• tool-visualizers/ directory exists with define.ts, index.ts, dispatcher.ts, test-utils.tsx, and the five primitives.
• Bash, FileEditor (all four commands), and Grep/Glob render via the new visualizers, both action and observation cards.
• All other tools render unchanged (markdown fallback).
• Snapshot tests cover the migrated visualizers, including error states.
• Existing tests in __tests__/ pass without modification.
• README or in-file comment in tool-visualizers/index.ts documents the three-step recipe for adding a new visualizer.

Follow-ups (not in this issue)

• Tier 2: MCP, InvokeSkill, main Browser actions.
• Tier 3: Think, Finish, remaining browser ops, TaskTracker polish.
• Optional: combined action+observation cards behind a per-visualizer opt-in.

[enyst]

💯

Just to note,

stdout={observation.observation.output}

I seem to recall that in the current code, we intently truncate this off; because it could be way too large and then the user would scramble to scroll back. Moreover, I’m not sure if we truncate client side, or we do somewhere in app-server; this varied over time and there are/were actually multiple possible truncations along the execution path to Web FE.

To note, in the sdk we truncate the obs, but

• within very generous limits, and
• the truncated obs is what is saved on disk.

So it’s potentially large.

I think maybe an enhanced UX comes from summarizing this on the fly or capturing it in a summary or some other options; than from giving the user the full terminal-like output of large obs…

———

That said, I love the proposal, actually. I think maybe we can use @DevinVinson ’s help to figure out if we could maybe offer people an ability to

• have a default visualizer, ok
• have a clear way to register a custom visualizer for default tools
• that hopefully is achieved via an addon? not modifying code, not adding files out of nowhere

I love your proposal because I read in it the ability to register one, just the way the user would do this is not fully clear to me.

[DevinVinson]

I can see this from both perspectives, and I think this touches on a struggle I have with how to present and interact with agentic conversations.

On one hand, having rich, per-tool visualizers (like a proper DiffView or a syntax-highlighted CodeBlock) is incredibly valuable. When trying to figure out where an agent is hitting a wall, being able to audit the tool calls with high fidelity is a great for understanding what an agent is doing or failing to do. On the other hand, enyst makes a great point about data bloat. The longer a conversation goes, the harder it becomes to load, read through, and scroll past massive terminal dumps or file renders.

At the same time, we should definitely avoid the other extreme. A lot of agentic tools out there compress everything into a single, updating line like Thinking... Done. While it keeps the feed clean, it completely blinds the user to what’s happening in real time, making it useless for debugging.

What if we explore a hybrid approach that solves both the performance issue and the visibility problem?

Instead of streaming every detailed tool call directly into the primary conversation feed, we could wrap the active agent execution in a single, stateful line. Something like "Thinking/Working/Worked... (5m)". Clicking that line could trigger one of two behaviors:

1. Inline Expansion: It unfolds into the detailed activity stream utilizing the excellent React visualizers proposed by Vasco. These only render if expanded and are collapsed on conversation load.
2. A Dedicated Timeline Panel: It opens a separate canvas or sidebar panel dedicated entirely to a visually rich, heavy-duty log stream of the agent's thinking process and tool outputs.

This could keep the main conversation stream snappy and scannable but still provide a way to deep dive and watch what is happening under the hood.

[enyst]

Instead of streaming every detailed tool call directly into the primary conversation feed, we could wrap the active agent execution in a single, stateful line. Something like "Thinking/Working/Worked... (5m)". Clicking that line could trigger one of two behaviors:

1. Inline Expansion: It unfolds into the detailed activity stream utilizing the excellent React visualizers proposed by Vasco. These only render if expanded and are collapsed on conversation load.
2. A Dedicated Timeline Panel: It opens a separate canvas or sidebar panel dedicated entirely to a visually rich, heavy-duty log stream of the agent's thinking process and tool outputs.

This could keep the main conversation stream snappy and scannable but still provide a way to deep dive and watch what is happening under the hood.

👍 Option 2) sounds great to me!

I think maybe it’s something like the old trajectory visualizer, but better integrated.

It seems to me that while both options are good, we could also add that:

• inline option, 1), will still not work too well when a single agent turn is long-running (e.g. 5k events)
• the dedicated timeline, 2), is by far better adapted for today’s frontier.

Here is an example presentation of where AI agents are today:
Long running agents workflow

It’s by Notion co-founder, I happened to bump on it on X. In reality, though, while he says six months, six months is almost exactly the moment when I experienced this kind of workflow. I mean, I actually saw firsthand that it… works. And it didn’t before!

Maybe it’s just me, the way I see it, this is where agents are, and I would argue that the closer we get to the ability to work that way (or offer people the ability to experience it with OpenHands!), the better off we are. Dedicated timeline is moving us towards there.

[VascoSch92]

The issue was more about how to visualize tool call in the chat UI, but the option to add a feature visualizer can also be interesting and an added value.

I believe @rajshah4 would love that.