Migrate chat UI from Lit to React

[cpsievert]

Summary

Rewrites the front-end from Lit custom elements to React. User-facing behavior is unchanged — chat streaming, tool cards, fullscreen mode, external link dialogs all work the same. The R and Python packages require only minor changes (the two JS/CSS bundles consolidate into one).

Under the hood, this replaces ~2,100 lines of imperative Lit code with ~3,000 lines of React components plus ~4,900 lines of new JS tests (the old codebase had zero).

What's good about this migration

Centralized state. In the old code, "what is the current state of the chat?" required inspecting Lit @property() decorators across multiple class instances, direct DOM reads, and a global window.shinychat.hiddenToolRequests Set. Now it's one useReducer with a typed ChatState and discriminated union of actions. Every state transition is visible in one place and testable in isolation.

No more DOMPurify gymnastics. The old _utils.ts had a complex DOMPurify setup with lifecycle hooks, a WeakMap to preserve custom element attributes, and special allowlists for htmlwidget scripts. The new markdown pipeline produces React elements directly from a HAST (HTML Abstract Syntax Tree), so script tags and event handlers are inert by construction. Embedded Shiny UI (htmlwidgets, inputs) still uses innerHTML via RawHTML, but the markdown rendering path itself never touches innerHTML.

Incremental DOM updates during streaming. In the old code, every streaming chunk replaced the entire message DOM via innerHTML — destroying and rebuilding all code blocks, syntax highlighting, and embedded widgets on every ~50ms chunk arrival. The new code re-parses the markdown into a HAST on each chunk, but React's reconciler diffs the resulting element tree against the previous one and only commits the changed DOM nodes. Existing content within the streaming message (highlighted code, widgets, etc.) is left in place rather than torn down and recreated.

Testable. The transport abstraction, the pure reducer, and composable rehype plugins are all independently testable. 24 test files cover the state machine, all hooks, all plugins, bridge components, and integration flows. Regression tests are anchored to specific bugs.

Build simplification. Two separate entry points with separate CSS bundles consolidate into one shinychat.js + shinychat.css.

Before/After Screenshots

Captured from the tool-basic R test app with Playwright.

Initial state

Before (main)	After (React)

Streaming with tool requests

Before (main)	After (React)

Completed response

Before (main)	After (React)

Mental model for the reviewer

The boundary: where Shiny ends and React begins

The Shiny server still renders <shiny-chat-container> and <shiny-markdown-stream> custom elements in the initial HTML, just like before. What's different is that these custom elements are now thin shells. chat-entry.ts reads attributes from the server-rendered DOM, calls createRoot(this), and hands off to <ChatApp>. From that point inward, everything is React. The same pattern applies to markdown-stream-entry.ts.

A critical detail at this boundary: chat-entry.ts calls transport.unbindAll(this) before mounting the React root. Without this, Shiny's internal binding registry retains stale references from the server-rendered HTML and refuses to re-bind the new React-rendered elements (Shiny thinks the inputs are already bound by ID). This is the kind of Shiny-specific integration concern that's now isolated in the entry files rather than scattered through the component tree.

If you're reviewing R/Python package changes, you mostly only need to care about how attributes are set on the custom elements. If you're reviewing JS, you're in React-land.

State lives in the reducer, imperative stuff lives outside it

state.ts is the single source of truth for message data, input disabled state, and hidden tool requests. Every server action (message, chunk_start, chunk, chunk_end, clear, update_input, remove_loading, hide_tool_request) and one UI action (INPUT_SENT) flows through chatReducer. The reducer is pure — given a state and an action, it returns the next state. No side effects.

However, not everything goes through the reducer. The textarea is intentionally uncontrolled — its value and focus are managed imperatively via a useImperativeHandle ref on ChatInput. This avoids the controlled-input cursor-jump problem and the cost of dispatching on every keystroke during streaming. When the server sends an update_input action with value or focus, ChatApp forwards it to the imperative handle rather than the reducer. The reducer only tracks inputPlaceholder and inputDisabled.

The transport layer bridges Shiny and React

transport/types.ts defines two interfaces:

• ChatTransport — message passing: sendInput(id, value) and onMessage(id, callback). React components use this to send user input and subscribe to server actions.
• ShinyLifecycle — Shiny DOM plumbing: renderDependencies, bindAll, unbindAll, showClientMessage. These are the Shiny-specific operations that React components need but shouldn't call directly.

ShinyTransport implements both. It's a window-level singleton so that the chat entry and markdown-stream entry share one instance and one message handler registration.

The Python/R backends send { id, action, html_deps? } envelopes where action is a discriminated ChatAction union (e.g., { type: "chunk_start", message: { role: "assistant", content: "", content_type: "markdown" } }). The transport renders any html_deps before dispatching, then passes the action directly to the reducer — no translation layer needed.

ShinyTransport also has a pending-message queue: if a shinyChatMessage arrives before any listener has registered for that ID, the action is buffered and flushed on the first onMessage() call. This covers race conditions during page load.

The markdown pipeline: text -> AST -> React elements

The old pipeline was marked -> DOMPurify -> innerHTML. The new pipeline is:

markdown text -> remark (parse) -> rehype (transform) -> HAST -> toJsxRuntime -> React elements

Three frozen processors are defined in processors.ts:

• markdownProcessor: full pipeline with rehype plugins for external link annotation, code highlighting, block-level CE unwrapping, uncontrolled inputs, and accessible suggestions. No rehypeSanitize — because toJsxRuntime produces React elements (not HTML strings), script tags and event handlers are inert by construction.
• htmlProcessor: for raw HTML content — preserves HTML fragment structure while normalizing uncontrolled form inputs, external link attributes, and accessible suggestions.
• semiMarkdownProcessor: includes remarkEscapeHtml (HTML tags become literal text) and rehypeSanitize for defense-in-depth.

Each rehype plugin is independent and tested in isolation. The key plugins to be aware of:

• rehypeUnwrapBlockCEs — promotes block-level custom elements (like <shinychat-raw-html>) out of <p> tags where the markdown parser incorrectly nests them.
• rehypeUncontrolledInputs — rewrites value to defaultValue on <input>/<textarea> elements so React doesn't treat them as controlled inputs (which would conflict with Shiny's input bindings).
• rehypeAccessibleSuggestions — wraps suggestion elements with appropriate ARIA attributes for accessibility.
• rehypeExternalLinks — annotates external links for the external link confirmation dialog.

Two-stage rendering in markdownToReact.ts:

1. Parse (expensive): parseMarkdown() runs the full unified pipeline to produce a HAST, or parseHtml() parses a raw HTML fragment. URL sanitization is applied in both paths. Cached via useMemo keyed on content + processor.
2. Render (cheap): hastToReact() calls toJsxRuntime on the HAST to produce React elements. The streaming dot is injected here via immutable path-copy (O(depth), not O(tree size)) — the cached HAST is never mutated. This stage only re-runs when streaming toggles.

Custom elements with dashes in their tag names get special treatment: fixCustomElementProps in markdownToReact.ts converts React-ified property names (e.g., className, htmlFor) back to their HTML attribute equivalents, because React 19 sets properties (not attributes) on custom elements and properties like htmlFor don't map to anything on a generic HTMLElement.

RawHTML: server-side splitting for Shiny UI content

LLM responses can mix markdown text with embedded Shiny UI (htmlwidgets, inputs, tool cards). React's DOM model is incompatible with Shiny UI that relies on inline scripts, jQuery initialization, and Shiny.bindAll(). The solution is server-side splitting: the server separates React-managed elements from traditional Shiny HTML before sending content to the client.

How it works:

1. On the server (R and Python), non-string content is processed through split_html_islands(). This function walks top-level tag children and checks for the data-shinychat-react attribute.
2. Elements with the attribute (e.g., tool cards) are emitted bare — they flow through the normal HAST → React pipeline via tagToComponentMap.
3. Consecutive elements without the attribute are grouped into <shinychat-raw-html> wrapper tags — these become innerHTML islands on the client.

The client-side RawHTML component sets innerHTML on a wrapper div and manages Shiny binding lifecycle. When ShinyLifecycleContext is available, it automatically calls bindAll after setting innerHTML and unbindAll on cleanup. The display: contents CSS property is used to avoid layout-breaking wrapper divs.

In the HAST → React pipeline, <shinychat-raw-html> elements are handled by MarkdownContent.tsx, which serializes HAST children back to an HTML string via toHtml() and passes the result to <RawHTML html={...} displayContents />.

Adding a new React custom element only requires the server to add data-shinychat-react to the tag. The server-side splitting ensures it bypasses the innerHTML path and flows through React's normal rendering. Internally, shinychat maps tool tags to specific React components via chatTagToComponentMap (e.g., shiny-tool-request → ToolRequestBridge), but this is package-internal — there is not yet a public API for end users to register custom component mappings.

Bridge components: HTML attributes -> typed React props

When toJsxRuntime encounters a <shiny-tool-request request-id="req-1" tool-name="search"> in the HAST, it maps it to the ToolRequestBridge React component (via the components option). But it passes the attributes as raw strings with their original kebab-case names.

Bridge components (ToolRequestBridge, ToolResultBridge) translate these stringly-typed HTML attributes into properly-typed React props:

// ToolResultBridge.tsx
export function ToolResultBridge({
  "request-id": requestId,
  "tool-name": toolName,
  expanded,
  ...
}: ToolResultBridgeProps) {
  return <ToolResult requestId={requestId} toolName={toolName} expanded={isTruthy(expanded)} />
}

This keeps the real components (ToolRequest, ToolResult) clean and testable with typed props. The bridge layer is the only place that knows about the HTML-to-React attribute translation.

What changed in R/Python

• Both packages switch from two HTML dependencies (chat + markdown-stream) to one (shinychat.js + shinychat.css)
• Non-string content (Shiny UI, tool cards) is processed through split_html_islands() before serialization:
  • Elements with data-shinychat-react (tool cards) are emitted bare for React rendering
  • Everything else is wrapped in <shinychat-raw-html> tags for innerHTML rendering
• The {=html} fence convention for raw HTML blocks is replaced by <shinychat-raw-html> tags
• Tool content (e.g., <shiny-tool-request>) gets a data-shinychat-react attribute so the server knows to emit it outside of <shinychat-raw-html> wrappers

Test plan

• cd js && npm run lint
• cd js && npx vitest run (24 test files, ~208 tests)
• uv run python -m pytest pkg-py/tests/ (~56 tests)
• cd pkg-r && Rscript -e "devtools::check(document = FALSE)"
• Manual smoke test: chat streaming, tool cards, fullscreen, external links

🤖 Generated with Claude Code

[gadenbuie]

I think we should do this todo item, either in this PR or in a follow-up (in short, we should widen the input pipe from simple text to more rich data).

shinychat/pkg-py/src/shinychat/_chat_types.py

Line 93 in 1f587d5

# TODO: content should probably be [{"type": "text", "content": "..."}, {"type": "image", ...}]

[gadenbuie]

I think we'll definitely end up wanting to revisit the design of server → client messages too, but that's certainly a follow-up PR. But I do want to call out that the design here — with types and ChatState and chatReducer — consolidates and codifies a lot of the message design structure in a way that will make these kinds of future improvements easier. I think this is a huge win in the React and rewrite column!

[cpsievert]

I think that bookmarking is slightly broken

Nice catch, fixed in 389b65d. Interestingly, this was only broken for R, since R goes through the "streaming" message path when restoring.

I think we should do this todo item

When I wrote that comment a while back I was thinking that it would make the most sense to do this when we add image support.

[gadenbuie]

I think we should do this todo item

When I wrote that comment a while back I was thinking that it would make the most sense to do this when we add image support.

Yeah I know, I'm saying that now that we're rewriting this in React we should do this (as in, commit to doing it as a follow up in the very near future).