diff --git a/Articles/openui-react-renderer-explained.md b/Articles/openui-react-renderer-explained.md new file mode 100644 index 0000000..a90cd4b --- /dev/null +++ b/Articles/openui-react-renderer-explained.md @@ -0,0 +1,611 @@ +# OpenUI's React Renderer Explained: How Progressive Hydration Works with Streamed Model Output + +This deep-dive dissects the React renderer inside `@openuidev/react-lang` — the engine that turns a live token stream from an LLM into progressively hydrating, interactive UI components. We'll walk through the streaming parser, the incremental merge strategy, prop evaluation, error boundaries, and the full pipeline that makes streaming Generative UI possible. + +--- + +## The Problem: The Blank-Screen Tax + +In a traditional AI-powered application, the user sends a prompt and waits for the model to produce its complete response before any UI appears. The flow looks like this: + +``` +User prompt → LLM thinking... (2-8s) → Full response received → Parse → Render → User sees UI +``` + +Even with Server-Sent Events (SSE) streaming, most frameworks only stream _text_. The UI layer still waits for the full response before it knows what to render. The user stares at a blank screen or a generic loading spinner. For complex dashboards, forms, or charts generated by a model, that wait can be 5+ seconds. + +This **blank-screen tax** is the single largest UX friction point in generative AI applications. It kills engagement, hides errors until too late, and makes AI apps feel slow even when the underlying model is fast. + +### What if the UI appeared token by token? + +OpenUI's approach: instead of waiting for the full response, the renderer intercepts the stream at the token level, parses complete statements as they arrive, and progressively hydrates React components — all without waiting for the model to finish. + +``` +User prompt → LLM streams: "root = Card(title=..." → Card shell renders + ... "items=[ContactCard(... → Items fill in + ... ")" → Final shape locks in +``` + +The user sees meaningful UI within milliseconds, not seconds. + +--- + +## The Architecture: Four-Layer Pipeline + +OpenUI's streaming renderer is built as a pipeline of four stages, each operating incrementally: + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ Layer 1: Stream Parser (lang-core/parser/parser.ts) │ +│ Tokenizes, splits statements, auto-closes incomplete code │ +└─────────────────────┬───────────────────────────────────────────┘ + │ Δ only (new completed statements + pending) +┌─────────────────────▼───────────────────────────────────────────┐ +│ Layer 2: Incremental Merge (completed cache + pending parse) │ +│ Completed statements are cached; only the pending tail │ +│ is re-parsed each frame. New IDs merge without clobbering. │ +└─────────────────────┬───────────────────────────────────────────┘ + │ ParseResult (AST tree + metadata) +┌─────────────────────▼───────────────────────────────────────────┐ +│ Layer 3: Prop Evaluation (lang-core runtime) │ +│ Materializes AST → concrete props. Resolves $refs, condition- │ +│ als, ternaries, and dynamic expressions. Caches completed │ +│ statements (no re-eval of finished code). │ +└─────────────────────┬───────────────────────────────────────────┘ + │ Evaluated ElementNode tree +┌─────────────────────▼───────────────────────────────────────────┐ +│ Layer 4: React Renderer (react-lang/src/Renderer.tsx) │ +│ Maps ElementNodes to registered library components. │ +│ ErrorBoundary wraps every node. Renders progressively. │ +└─────────────────────────────────────────────────────────────────┐ +``` + +--- + +## Layer 1: The Streaming Parser + +The streaming parser is the heart of the system. Unlike a traditional parser that requires complete input, OpenUI's `createStreamingParser()` is designed for incremental consumption. + +### The StreamParser Interface + +```typescript +export interface StreamParser { + /** Feed the next SSE/stream chunk and get the latest ParseResult. */ + push(chunk: string): ParseResult; + + /** Set the full text — diffs against internal buffer, pushes only the delta. + * Resets automatically if the text was replaced (not appended). */ + set(fullText: string): ParseResult; + + /** Get the latest ParseResult without consuming new data. */ + getResult(): ParseResult; +} +``` + +The parser is created once per library and lives across the entire streaming session: + +```typescript +// useOpenUIState.ts — parser created via useMemo, lives for the session +const sp = useMemo( + () => createStreamingParser(library.toJSONSchema(), library.root), + [library], +); +``` + +### How It Works Internally + +The streaming parser maintains three internal pieces of state: + +1. **`buf`** — The accumulated raw text buffer (all streamed tokens concatenated). +2. **`completedEnd`** — A watermark marking how far the parser has scanned for completed statements. +3. **`completedStmtMap`** — A `Map` of fully parsed, classified statements that will never change. + +The critical insight: **completed statements are never re-parsed**. Only the "pending" tail — the incomplete portion after `completedEnd` — is re-tokenized and parsed on each update. This gives O(1) amortized parsing cost per statement. + +### Statement Detection: The Watermark Scan + +The `scanNewCompleted()` function walks the buffer byte-by-byte from `completedEnd`, tracking: + +- **Brace/bracket depth** — `( [ {` and `) ] }` +- **String context** — Tracks `"..."` and `'...'` including escape sequences +- **Ternary depth** — `?` / `:` at depth-0 for ternary expressions spanning lines +- **Ternary lookahead** — Before splitting on a newline, peeks ahead past whitespace to check if the next meaningful character is `?` or `:`, preventing premature splits in multi-line ternaries + +A statement is considered complete when the parser encounters a newline at depth-0 with ternary-depth-0 (after ternary lookahead). At that point: + +```typescript +const t = buf.slice(stmtStart, i).trim(); +if (t) addStmt(t); +stmtStart = i + 1; +completedEnd = i + 1; +``` + +### The Auto-Close Trick + +For the pending (incomplete) tail, the parser applies `autoClose()` to syntactically balance the partial code: + +``` +Input: root = Card(title="Hello", items=[CardItem(label="A" +AutoClose → root = Card(title="Hello", items=[CardItem(label="A")] +``` + +This allows the incomplete tail to be parsed as valid syntax, producing a partial but renderable AST. + +### Markdown Fence Stripping + +LLMs often wrap output in ```openui fences. The parser's `stripFences()` function is string-context-aware: it tracks whether it's inside a `"..."` string and won't treat ``` characters mid-string as fence boundaries. During streaming (no closing fence found), it extracts everything after the opening fence as the body. + +--- + +## Layer 2: Incremental Merge + +Each time new data arrives — via `push(chunk)` or `set(fullText)` — the parser produces a `ParseResult` by merging: + +1. **Completed statements** from `completedStmtMap` (cached, never changes) +2. **The pending tail** (re-parsed fresh from the incomplete buffer portion) + +The merge is smart about identity: + +```typescript +// Pending statements can only add NEW IDs — they cannot overwrite completed ones. +// This prevents mid-stream partial text (e.g., `root = Card`) from corrupting +// existing completed statements during edit streaming. +const allStmtMap = new Map(completedStmtMap); +for (const s of stmts) { + if (completedStmtMap.has(s.id)) continue; + const expr = parseExpression(s.tokens); + const stmt = classifyStatement(s, expr); + allStmtMap.set(s.id, stmt); +} +``` + +This means if the model starts generating `root = Card(...)` but later produces `root = Card(...)` again during edit streaming, the completed version on disk takes priority. Only new statement IDs from the pending tail are merged in. + +### Statement Classification + +Each parsed statement is classified into one of four kinds: + +| Kind | Example | Purpose | +|---|---|---| +| `state` | `$count = 0` | Reactive variable declarations | +| `value` | `myCard = Card(...)` | Component value bindings | +| `query` | `data = Query("getUsers")` | Backend data fetching | +| `mutation` | `save = Mutation("saveUser")` | Backend write operations | + +Query and mutation statements are extracted separately and wired to the `QueryManager` for data fetching, while value and state statements feed the reactive rendering pipeline. + +--- + +## Layer 3: Prop Evaluation + +Once the parser produces a `ParseResult` containing `ElementNode` objects with AST-based props, the runtime evaluation layer materializes them into concrete values. + +### The Evaluation Context + +```typescript +const evaluationContext = useMemo( + () => ({ + getState: (name: string) => unwrapFieldValue(store.get(name)), + resolveRef: (name: string) => { + const mutResult = queryManager.getMutationResult(name); + if (mutResult) return mutResult; + return queryManager.getResult(name); + }, + }), + [store, queryManager], +); +``` + +This context provides two capabilities: +1. **`getState(name)`** — Reads reactive state (`$count`, `$filter`, etc.) from the store +2. **`resolveRef(name)`** — Resolves Query/Mutation results by their reference name + +### Eval During Streaming vs. After + +A subtle detail in `useOpenUIState`: query/mutation evaluation is **deferred** while streaming: + +```typescript +useEffect(() => { + if (isStreaming) return; // Skip during streaming + // ... evaluate queries, register mutations +}, [isStreaming, result?.queryStatements, ...]); +``` + +This prevents the runtime from firing backend calls with partial, mid-stream data. Queries and mutations are evaluated only after the model finishes, using the final complete result. + +### Prop Evaluation with Error Collection + +The `evaluateElementProps()` call in the renderer traverses the element tree and resolves every AST-based prop to a concrete value, collecting structured errors: + +```typescript +const evaluatedResult = useMemo(() => { + if (!result?.root) return result; + const errors: OpenUIError[] = []; + const evalCtx: EvalContext = { ctx: evaluationContext, library, store, errors }; + try { + const evaluatedRoot = evaluateElementProps(result.root, evalCtx); + runtimeErrorsRef.current = errors; + return { ...result, root: evaluatedRoot }; + } catch (e) { + // Safety net for per-prop errors not caught individually + const msg = e instanceof Error ? e.message : String(e); + errors.push({ source: "runtime", code: "runtime-error", message: msg }); + runtimeErrorsRef.current = errors; + return result; + } +}, [result, evaluationContext, library, store, storeSnapshot, querySnapshot]); +``` + +Each prop that fails evaluation contributes a structured `OpenUIError` — not a thrown exception — so rendering continues for the rest of the tree. + +--- + +## Layer 4: The React Renderer + +The final layer takes the evaluated `ParseResult` and renders it as React components. + +### Component Resolution + +The `Renderer` component receives a `Library` — a registry of component definitions mapped to React components: + +```typescript +function RenderNode({ node }: { node: ElementNode }) { + const { library, reportError } = useOpenUI(); + const Comp = library.components[node.typeName]?.component; + + if (!Comp) return null; + + return ( + + + + ); +} +``` + +Each node's `typeName` (e.g., `"Card"`, `"Table"`, `"Form"`) is looked up in the library. If registered, the corresponding React component is rendered. If not found, the node silently renders `null`. + +### The `renderDeep` Recursive Renderer + +For props containing nested structures (arrays, nested component calls), `renderDeep` recursively transforms parsed values into React nodes: + +```typescript +function renderDeep(value: unknown): React.ReactNode { + if (value == null) return null; + if (typeof value === "string") return value; + if (typeof value === "number") return String(value); + if (typeof value === "boolean") return String(value); + + if (Array.isArray(value)) { + return value.map((v, i) => {renderDeep(v)}); + } + + if (typeof value === "object" && value !== null) { + const obj = value as Record; + if (obj.type === "element") { + return ; + } + } + + return null; +} +``` + +This handles the full tree: primitives render as text, arrays render as fragments, and nested `ElementNode` objects recursively resolve to registered components. A component's `children` prop, for example, might contain an array of nested `ElementNode` objects, each resolved through `renderDeep`. + +### Error Boundary: "Show Last Good State" + +Every rendered node is wrapped in an `ElementErrorBoundary` with a critical UX design: on render failure, it shows the **last successfully rendered children** instead of going blank: + +```typescript +class ElementErrorBoundary extends Component { + private lastValidChildren: React.ReactNode = null; + + componentDidUpdate(prevProps: ErrorBoundaryProps): void { + if (!this.state.hasError) { + this.lastValidChildren = this.props.children; + } + // Auto-recover when new valid children arrive + if (this.state.hasError && prevProps.children !== this.props.children) { + this.setState({ hasError: false }); + } + } + + render() { + if (this.state.hasError) { + return this.lastValidChildren; // Last good state + } + return this.props.children; + } +} +``` + +During streaming, partial prop data might cause transient render errors. The error boundary ensures the UI never goes blank — it shows the last valid state and auto-recovers when the next streaming chunk provides valid data. + +--- + +## The Full Streaming Pipeline in Action + +Here is what happens when the user submits a prompt: + +``` +1. User sends: "Create a dashboard with a revenue chart and a contact form" + +2. LLM begins streaming (SSE): + Chunk 1: "root = Root(" + Chunk 2: 'children=[' + Chunk 3: 'Card(title="Revenue", charts=[' + Chunk 4: 'LineChart(data=[{"label":"Jan","value":100},' + ... continues token by token + +3. On each chunk: + a. StreamParser.set(fullText) is called + b. Parser scans for completed statements (watermark scan) + c. Completed statements cached, never re-parsed + d. Pending tail auto-closed and parsed + e. Completed + pending merged into ParseResult + f. Prop evaluation materializes concrete values + g. React re-renders with the updated tree + +4. User sees progressive rendering: + t=0ms: Card shell appears + t=200ms: Card title "Revenue" renders + t=500ms: LineChart component mounts + t=800ms: Chart data populates + t=2000ms: Additional Card with Form appears + t=5000ms: Full dashboard complete + +5. isStreaming flips to false: + a. Query/Mutation evaluation runs + b. onError fires with any accumulated errors + c. Form interactions become active +``` + +### State Management and Reactivity + +OpenUI uses React's `useSyncExternalStore` for both the state store and the query manager: + +```typescript +const storeSnapshot = useSyncExternalStore( + store.subscribe, + store.getSnapshot, + store.getSnapshot, +); + +const querySnapshot = useSyncExternalStore( + queryManager.subscribe, + queryManager.getSnapshot, + queryManager.getSnapshot, +) as QuerySnapshot; +``` + +This means any change to reactive state (`$count`, form fields, query results) triggers an immediate re-render with the latest snapshot, keeping the UI in sync with the data layer. The store implements a fine-grained subscription model — only subscribers of the changed key are notified — minimizing unnecessary re-renders. + +### Tool Provider Integration + +The `toolProvider` system allows queries and mutations to connect to any backend: + +```typescript +// Function map — simplest option +toolProvider: { + getUsers: async (args) => fetch("/api/users").then(r => r.json()), + saveUser: async (args) => fetch("/api/users", { method: "POST", body: JSON.stringify(args) }), +} + +// MCP client — Model Context Protocol +toolProvider: mcpClient // any object with callTool({ name, arguments }) +``` + +The `ToolProvider` is wrapped in a stable ref-based proxy so that provider updates (swapping backends, function map changes) are always observed without recreating the `QueryManager`: + +```typescript +const stableToolProvider = useRef({ + async callTool(toolName, args) { + const current = toolProviderInputRef.current ?? null; + // Reads latest provider from ref on every call + // Supports both MCP client and function-map patterns + }, +}); +``` + +--- + +## Code Example: A Streamed Component Rendering Progressively + +Here is a realistic example of how an LLM would stream UI code and what the user sees at each stage: + +### Stage 1: Model begins (after receiving `root = Root(`) + +``` +Streamed so far: root = Root( +Auto-closed: root = Root() +Rendered: Empty Root container (may render nothing yet) +``` + +### Stage 2: Children container starts + +``` +Streamed so far: root = Root( + children=[ + Card( +Auto-closed: root = Root( + children=[ + Card() + ] + ) +Rendered: Root with empty Card shell visible +``` + +### Stage 3: Card props arriving + +``` +Streamed so far: root = Root( + children=[ + Card( + title="Monthly Revenue", + variant="elevated", +Auto-closed: root = Root( + children=[ + Card( + title="Monthly Revenue", + variant="elevated" + ) + ] + ) +Rendered: Card with title "Monthly Revenue" and elevated styling +``` + +### Stage 4: Chart component nests inside + +``` +Streamed so far: ... Card( + title="Monthly Revenue", + variant="elevated", + charts=[ + LineChart( + data=[ + {"label": "Jan", "value": 10000}, + {"label": "Feb", "value": 15000}, + {"label": "Mar", "value": 12000} +Auto-closed: ... LineChart( + data=[ + {"label": "Jan", "value": 10000}, + ... + ] + ) +Rendered: Card shows title + LineChart with 3 data points +``` + +### Stage 5: Stream completes + +``` +Final: root = Root( + children=[ + Card( + title="Monthly Revenue", + variant="elevated", + charts=[LineChart(data=[...])] + ) + ] + ) +Rendered: Complete dashboard with fully-rendered card and chart +``` + +At each stage, only the _pending tail_ is re-parsed. The `Root(children=[Card(...)])` at the top level becomes a completed statement as soon as its closing brackets arrive. All subsequent streaming only appends to the Card's props and children. + +--- + +## Performance Characteristics + +### Token Efficiency + +OpenUI Lang achieves **up to 67% fewer tokens** than equivalent JSON-based streaming formats. Benchmarks against Vercel v0 JSON-Render and Thesys C1 JSON show consistent savings across seven UI scenarios: + +| Scenario | Vercel JSON | C1 JSON | OpenUI Lang | vs Vercel | vs C1 | +|---|---|---|---|---|---| +| Simple table | 340 | 357 | 148 | -56.5% | -58.5% | +| Chart with data | 520 | 516 | 231 | -55.6% | -55.2% | +| Contact form | 893 | 849 | 294 | -67.1% | -65.4% | +| Dashboard | 2247 | 2261 | 1226 | -45.4% | -45.8% | +| Pricing page | 2487 | 2379 | 1195 | -52.0% | -49.8% | +| Settings panel | 1244 | 1205 | 540 | -56.6% | -55.2% | +| E-commerce product | 2449 | 2381 | 1166 | -52.4% | -51.0% | +| **Total** | **10180** | **9948** | **4800** | **-52.8%** | **-51.7%** | + +Fewer tokens means: faster model output, lower API costs, and less data to stream over the network. + +### Streaming Latency + +Because the parser uses a watermark scan with completed-statement caching: +- **First render**: Triggers as soon as enough tokens have arrived to form a complete statement (typically within the first 100-500ms of streaming). +- **Incremental updates**: Each new completed statement is parsed exactly once and cached. +- **Pending tail**: Re-parsed on every chunk, but the pending tail is typically 1-3 statements, keeping re-parse cost negligible. + +### React Re-render Cost + +The renderer uses `useMemo` for both parsing and evaluation, keyed on the raw `response` text. React's reconciliation handles the tree diff efficiently. The `useSyncExternalStore` hook ensures state and query changes trigger re-renders only when data actually changes, not on every stream chunk. + +--- + +## Error Handling: The Self-Correcting Loop + +OpenUI's error system is designed for two audiences: + +### Runtime Errors (User-Facing) + +When a component render fails during streaming, the `ElementErrorBoundary` shows the last valid state and calls `reportError()`. During streaming, these errors are suppressed (transient partial data): + +```typescript +const reportError = useCallback((error: OpenUIError) => { + if (isStreamingRef.current) return; // Skip during streaming + renderErrorsRef.current.push(error); +}, []); +``` + +### Structured Errors (LLM-Facing) + +After streaming completes, all errors are aggregated and fired via `onError`: + +```typescript +// Collectors: parser exceptions, parse failures, validation errors, +// runtime eval errors, render errors, query/mutation tool errors +const errors: OpenUIError[] = []; +// ... collected from all sources ... + +if (propsRef.current.onError) { + propsRef.current.onError(errors); +} +``` + +Each error carries structured metadata: + +```typescript +export type OpenUIError = { + source: "parser" | "runtime"; + code: "parse-exception" | "parse-failed" | "render-error" | "tool-not-found" | ...; + message: string; + component?: string; + hint?: string; // LLM-friendly suggestion +}; +``` + +This structured error output feeds the **self-correcting loop**: the app sends errors back to the LLM, which regenerates the output. The `enrichErrors()` function adds hints like "The component `Chart` doesn't exist. Available options: `LineChart`, `BarChart`, `PieChart`." — enabling the model to autonomously fix its own output. + +--- + +## Key Design Decisions and Trade-offs + +### Why Not JSON? + +JSON requires every key to be quoted, making it verbose. OpenUI Lang uses a compact, expression-based syntax that drops quotes, braces, and boilerplate. The language is specifically designed for streaming: depth-0 newlines delimit statements, auto-close handles incomplete output, and the compact syntax means each token contributes more semantic value. + +### The Statement-Centric Architecture + +Unlike parser combinators or recursive-descent parsers that need the full input, OpenUI's parser is statement-centric. Each statement is independently parseable. This naturally maps to streaming: as soon as a statement is complete, it can be parsed and cached. The pending tail is the only part that needs incremental re-parsing. + +### Caching Completed Statements + +The completed statement cache is the key performance optimization. Without it, every incoming chunk would trigger a full re-parse of all accumulated text. With it, each statement is parsed exactly once, and the incremental cost per chunk is proportional only to the pending tail. + +### Error Boundary Per-Node + +Wrapping every node in its own error boundary is a deliberate choice. In a traditional React app, one error boundary at the top is sufficient. In a streaming Generative UI context, partial data might cause errors at different nodes at different times. Per-node boundaries isolate failures, show last-good-state for the failing node, and let the rest of the tree render normally. + +--- + +## Summary + +OpenUI's React renderer solves the blank-screen problem by treating the LLM's token stream as an incremental source of renderable UI. The streaming parser's watermark-scan approach with completed-statement caching provides O(1) amortized parse cost. The incremental merge prevents partial data from corrupting finished statements. The per-node error boundary keeps the UI resilient during streaming. And the structured error system enables a self-correcting loop where the model iteratively fixes its own output. + +The result: users see meaningful UI within the first second of a model response, watch it incrementally fill in as tokens arrive, and interact with the final complete output — all without the blank-screen wait that plagues traditional AI applications. + +--- + +## References + +- [OpenUI GitHub Repository](https://github.com/thesysdev/openui) +- [`@openuidev/react-lang` — Streaming Renderer Source](https://github.com/thesysdev/openui/tree/main/packages/react-lang/src) +- [`@openuidev/lang-core` — Parser and Runtime Source](https://github.com/thesysdev/openui/tree/main/packages/lang-core/src) +- [OpenUI Documentation](https://openui.com) +- [OpenUI Playground](https://www.openui.com/playground) +- [Token Efficiency Benchmarks](https://github.com/thesysdev/openui/tree/main/benchmarks)