diff --git a/pages/blog/2025-01-rendering-large-json-payloads.mdx b/pages/blog/2025-01-rendering-large-json-payloads.mdx new file mode 100644 index 0000000000..46389dc1b8 --- /dev/null +++ b/pages/blog/2025-01-rendering-large-json-payloads.mdx @@ -0,0 +1,245 @@ +--- +title: "Rendering Large JSON Payloads: Beyond Virtualization" +date: 2025/01/22 +description: "Building a JSON viewer that handles 100K+ line payloads with sub-10ms expand/collapse interactions through hierarchical trees and binary search navigation." +tag: engineering, performance, front-end, react +author: Michael +--- + +import { BlogHeader } from "@/components/blog/BlogHeader"; + + + +When developers use LLM applications, they see requests going in and responses coming out - but not what happens in between. Langfuse makes this visible by capturing and displaying the full input/output payloads of every LLM call. This visibility is essential because developers need to debug failures, optimize token usage, and understand how their applications behave in production. + +For typical use cases - a chatbot message, a function call, a simple completion - these payloads are a few hundred lines of JSON. Browsing them is straightforward: render a tree view, let users expand and collapse nodes, search for specific values. + +With agentic workflows, we see a small but increasing number of traces with significantly larger payloads. Long-running agent loops accumulate tens of thousands of lines of JSON. Tool calls sometimes return entire database query results. Some responses contain 100,000+ lines of structured data. Our initial JSON viewer rendered all nodes to the DOM, which caused browser lags or even crashes on these large payloads. + +## Initial Approach: Virtualization with Eager Flattening + +After implementing virtualization for our [trace viewer](/blog/2025-01-rendering-long-running-traces), we wanted to apply the same approach to JSON rendering. The challenge: we couldn't find proven open-source components for virtualized JSON viewing. One notable exception is [react-obj-view](https://github.com/vothanhdat/react-obj-view) which came close, but lacked features like search and line-wrapping we needed. To have control over the user experience and future extensions, we built our own. + +Our first virtualized implementation followed a straightforward pipeline: + +**Step 1: Deep Parse JSON** - Convert JSON string into JavaScript objects +**Step 2: Build Tree Structure** - Create TreeNode objects with parent-child relationships +**Step 3: Flatten Rows** - Convert tree into flat array based on expansion state (i.e., which rows to show in the viewer) +**Step 4: Virtualize** - Render the subset of rows visible within the viewport from flattened array + +```typescript +const [expandedIds, setExpandedIds] = useState>(new Set()); + +// Build tree and flatten based on expansion state +const flatRows = useMemo(() => { + const tree = buildTreeFromJSON(data); + return flattenTree(tree, expandedIds); +}, [data, expandedIds]); + +function toggleExpand(id: string) { + setExpandedIds(new Set([...expandedIds, id])); + // Triggers rebuild of flatRows: 100K new objects allocated + // Main thread blocked for 300ms +} + +// Virtualizer renders only visible rows +const VirtualRow = ({ index }) => { + const row = flatRows[index]; // O(1) array access + return ; +}; +``` + +This worked for typical payloads. For large datasets, we hit two bottlenecks: + +**Problem 1: Initial Tree Building Blocks UI** + +Deep parsing and tree building for 100K+ nodes took 500ms+ on the main thread, freezing the browser. Solution: conditionally offload to Web Workers for datasets over 10,000 nodes. + +**Problem 2: Expand/Collapse Still Sluggish** + +Even with Web Workers handling initial build, every expand/collapse action rebuilt the entire flat array. Creating 100,000 new row objects on every interaction took 200-300ms. Users experienced noticeable lag when expanding large nodes. + +This is a fundamental performance issue: on every expand or collapse we fully re-flatten the JSON tree. That means walking 100K+ nodes and rebuilding the entire flat row list each time. While individual object creation in Javascript is fast, doing this work tens of thousands of times per interaction adds up to ~200ms of CPU time, along with extra garbage collection from all the short-lived rows, arrays, and strings. In contrast, reusing existing rows and updating only the affected subtree avoids most of this work and completes much faster because it skips the full traversal and rebuild. + +The issue: **virtualization solves rendering performance, not interaction performance.** We only render 50 visible rows, but we're still allocating 100,000 objects on every expansion change. + +## Revised Approach: Hierarchical Tree with JIT Navigation + +How can we resolve this? Like most problems in computer science, the solution is to trade space for time. The revised approach eliminates the flatten step entirely. Instead of pre-computing a flat array, we navigate the hierarchical tree just-in-time (JIT) when the virtualizer requests a specific row. This way we only need to traverse the tree once and can reuse the same data structure for every interaction. + +### Key Decision: Breaking React's Immutability Principle + +This required to break with a core react principle: we mutate the tree directly instead of creating immutable copies. When a user expands a node, we update that node's `isExpanded` property in place and recalculate only the affected `childOffsets` along the path to the root. + +Why? Immutable updates for a 100K-node tree require copying thousands of nodes even with structural sharing. Direct mutation affects only O(log n) nodes - the path from the clicked node to the root, typically 10-20 nodes. We use a version counter to trigger React re-renders explicitly. + +### Data Structure: childOffsets for Binary Search + +To enable JIT lookup, we need a data structure that can answer "what's at row 50,000?" without iterating through all 49,999 preceding rows. A naive hierarchical tree would require walking from the root, counting visible descendants at each level - an O(n) operation. Our solution: augment each node with `childOffsets`, an array of cumulative visible descendant counts that enables binary search navigation in O(log n) time. + +Each TreeNode maintains its hierarchical structure plus navigation metadata: + +```typescript +interface TreeNode { + id: string; + key: string | number; + value: unknown; + type: "object" | "array" | "string" | "number" | "boolean" | "null"; + + // Structure + depth: number; + parentNode: TreeNode | null; + children: TreeNode[]; + + // Expansion state (node owns its state) + isExpandable: boolean; + isExpanded: boolean; + + // Navigation via binary search + childOffsets: number[]; // Cumulative visible descendant counts + visibleDescendantCount: number; +} +``` + +The `childOffsets` array enables O(log n) navigation to any row without pre-computing a flat array: + +```typescript +// Example: A node with 3 children +// Child 0 has 10 visible descendants +// Child 1 has 5 visible descendants +// Child 2 has 8 visible descendants + +childOffsets = [11, 17, 26]; + +// To find row 15: +// Binary search: 15 > 11 && 15 < 17 → descend into Child 1 +// Continue recursively until reaching target row +``` + +This enables O(log n) navigation instead of O(1) array access - but we avoid the O(n) rebuild cost: + +```typescript +function getNodeByIndex(root: TreeNode, index: number): TreeNode | null { + if (index === 0) return root; + if (!root.isExpanded || root.children.length === 0) return null; + + let currentIndex = index - 1; // Account for root node + + // Binary search through childOffsets + for (let i = 0; i < root.children.length; i++) { + const child = root.children[i]; + const offsetEnd = root.childOffsets[i]; + + if (currentIndex < offsetEnd) { + // Target is in this subtree + const childIndex = currentIndex - (i > 0 ? root.childOffsets[i - 1] : 0); + return getNodeByIndex(child, childIndex); + } + } + + return null; +} +``` + +_([View full implementation](https://github.com/langfuse/langfuse/blob/main/web/src/components/ui/AdvancedJsonViewer/utils/treeNavigation.ts))_ + +### In-Place Mutation for Expand/Collapse + +When a user clicks to expand or collapse a node: + +```typescript +export function toggleNodeExpansion(tree: TreeState, nodeId: string): void { + const node = tree.nodeMap.get(nodeId); + if (!node || !node.isExpandable) return; + + // Direct mutation (breaks React patterns) + node.isExpanded = !node.isExpanded; + node.userExpand = node.isExpanded; + + // Update ancestors' offset counts (O(log n) - path to root only) + updateAncestorOffsets(node); +} + +// In React component: +const [expansionVersion, setExpansionVersion] = useState(0); + +function handleToggleExpansion(nodeId: string) { + toggleNodeExpansion(tree, nodeId); + + // Trigger re-render via version counter + setExpansionVersion((v) => v + 1); +} +``` + +Instead of rebuilding the entire tree or flat array, we: + +1. Mutate the `isExpanded` flag on one node +2. Update `childOffsets` along the path to root (O(log n) nodes) +3. Increment version counter to trigger React re-render + +No object allocation. No array rebuilds. Just update a few numbers in place. + +### Progressive Enhancement with Web Workers + +Initial tree construction still requires traversing all nodes. We decided to do a full upfront pass before rendering the JSON to enable additional features like text search, which can quickly find content even in a virtualized view by searching the pre-built tree structure. For datasets over 10,000 nodes, we offload this to a Web Worker: + +```typescript +export function buildTreeFromJSON(data: unknown, config: BuildConfig) { + // Pass 1: Structure - Create TreeNodes with parent-child relationships + const { rootNode, nodeMap } = buildTreeStructureIterative( + data, + config.rootKey, + ); + + // Pass 2: Expansion - Apply initial expansion state + applyExpansionStateIterative(rootNode, config.initialExpansion); + + // Pass 3: Offsets - Compute childOffsets for navigation + computeOffsetsIterative(rootNode); + + // Pass 4: Dimensions - Calculate tree metrics + const { maxDepth, maxContentWidth } = calculateTreeDimensions(rootNode); + + return { rootNode, nodeMap, maxDepth, maxContentWidth }; +} +``` + +All passes use iterative algorithms with explicit stacks to avoid stack overflow on deeply nested JSON. As covered in our [trace viewer post](/blog/2025-01-rendering-long-running-traces), avoiding recursion is critical when dealing with deeply nested structures - JavaScript's call stack limit of ~10,000 frames means recursive algorithms fail on structures with deep nesting. + +**Alternative Approach:** If search functionality isn't required, it's possible to offload tree building to JIT as well. [react-obj-view](https://github.com/vothanhdat/react-obj-view) (no Langfuse affiliation) takes this approach with pixel-based virtualization and JIT node lookups, avoiding upfront tree construction entirely. For our use case, we accepted the few hundred milliseconds of upfront build time, as it is negligible compared to the fetch times for large JSON payloads, in exchange for instant search capabilities. + +## Comparing Approaches + +| Aspect | Initial (Eager Flattening) | Revised (JIT Navigation) | +| --------------- | ---------------------------- | -------------------------- | +| Initial build | O(n) parse + build + flatten | O(n) build only | +| Expand/collapse | O(n) flatten rebuild | O(log n) in-place mutation | +| Row lookup | O(1) array access | O(log n) binary search | +| Memory | 2n (tree + flat array) | n (tree only) | +| React patterns | Immutable ✓ | Mutable (controlled) ✗ | +| Interaction | 200-300ms at 100K nodes | <10ms at any size | + +**Trade-offs of In-Place Mutation:** + +Breaking React's immutability principle requires discipline: + +- Clear ownership: only tree module mutates nodes +- Debug mode validation checks offset correctness +- Version counter makes re-renders explicit and predictable + +The payoff: For large json payloads the expand/collapse completes in <10ms regardless of dataset size, versus 200-300ms+ with immutable rebuilds. + +_The complete implementation is available in the Langfuse repository:_ + +- _[AdvancedJsonViewer](https://github.com/langfuse/langfuse/tree/main/web/src/components/ui/AdvancedJsonViewer) - Full component_ +- _[treeStructure.ts](https://github.com/langfuse/langfuse/blob/main/web/src/components/ui/AdvancedJsonViewer/utils/treeStructure.ts) - Tree building_ +- _[treeNavigation.ts](https://github.com/langfuse/langfuse/blob/main/web/src/components/ui/AdvancedJsonViewer/utils/treeNavigation.ts) - Binary search navigation_ + +--- + +**Building Langfuse?** We're growing our engineering team. If you enjoy solving performance problems with data structures and algorithms, [check out our open positions](https://langfuse.com/careers). diff --git a/pages/blog/2025-01-rendering-long-running-traces.mdx b/pages/blog/2025-01-rendering-long-running-traces.mdx new file mode 100644 index 0000000000..ffa9977a79 --- /dev/null +++ b/pages/blog/2025-01-rendering-long-running-traces.mdx @@ -0,0 +1,322 @@ +--- +title: "Rendering Long-Running Traces: Handling 200K Observations In React Without Stack Overflow" +date: 2025/01/15 +description: "How we built Langfuse's trace viewer to handle extreme cases - from typical 100-observation traces to 10-hour agents with 200K observations - without degrading performance." +tag: engineering, performance, react +author: Michael +--- + +import { BlogHeader } from "@/components/blog/BlogHeader"; + + + +Langfuse is agnostic to what users trace. We don't impose structure or limits on how teams instrument their LLM applications. This flexibility is powerful, but it means we need to handle everything from simple chatbot calls to complex autonomous agents. + +When we built the trace viewer, we designed for the typical case: traces with 100-200 observations, maybe 10-20 levels of nesting. This worked well for most users - until it didn't. As AI capabilities improved, we started seeing production traces from multi-hour autonomous agents. The edge case that broke our assumptions: we saw single traces with over 200,000 observations and 10,000+ levels of nesting depth. + +Research like [METR's work on long-horizon task completion](https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/) suggests this trend will accelerate. As models become more capable, agents will run longer and generate more observations. What's an edge case today may become common tomorrow. + +## The Technical Challenge + +Each Trace can have a number of observations. Our API returns these observations as a flat list. Each observation has an ID and optionally references a parent observation ID. LLM applications involve operations that can run in parallel: tool calls executing concurrently, sub-agents spawning in parallel, multiple API requests happening simultaneously. To make execution flow comprehensible, we need to reconstruct this hierarchy and display it as a tree structure. Users need to see which operations spawned which sub-operations, understand the nesting depth, and navigate the execution graph. + +Building this tree becomes the first step in rendering any trace. Our original implementation had three critical bottlenecks at scale: + +**1. Stack Overflow from Recursive Tree Building** + +JavaScript has a call stack limit of roughly 10,000 frames. Our recursive tree-building algorithm would crash when traces exceeded this depth. + +```typescript +// This works for typical traces... +function buildTree(node) { + return { + ...node, + children: node.children.map((child) => buildTree(child)), + }; +} + +// ...but crashes on deep agent traces with "Maximum call stack size exceeded" +``` + +**2. Quadratic Complexity in Parent-Child Lookups** + +Observations reference their parent by ID. Our initial implementation used `array.find()` to locate parents: + +```typescript +observations.forEach((obs) => { + const parent = observations.find((o) => o.id === obs.parentObservationId); + if (parent) parent.children.push(obs); +}); +``` + +This is O(n²) complexity. For 200 observations, that's 40,000 operations - barely noticeable. For 200,000 observations, it's 40 billion operations. The difference between "instant" and "never finishes." + +**3. Just-in-Time Aggregation on Every Selection** + +Displaying aggregate metrics (total cost, token counts) for an observation requires summing values across all its descendants. Our initial implementation calculated these on-demand when a user selected an observation: + +```typescript +// Recalculated on every selection +function calculateTotalCost(observation: Observation): Decimal { + let total = observation.cost ?? new Decimal(0); + + // Recursively traverse all children + for (const child of observation.children) { + total = total.plus(calculateTotalCost(child)); + } + + return total; +} +``` + +This meant rebuilding aggregates and traversing the entire subtree on every click. For an observation with thousands of descendants, users experienced 200-300ms delays when selecting observations - the UI felt sluggish and unresponsive. + +## Implementation + +### Iterative Tree Building + +To handle arbitrary depth, we replaced recursion with an explicit stack-based approach. + +```typescript +function buildDependencyGraph(observations: Observation[]): { + nodeRegistry: Map; + leafIds: string[]; +} { + const nodeRegistry = new Map(); + + // Pass 1: Create nodes + for (const obs of observations) { + nodeRegistry.set(obs.id, { + observation: obs, + childrenIds: [], + inDegree: 0, + depth: 0, + }); + } + + // Pass 2: Build parent-child relationships + for (const obs of observations) { + if (obs.parentObservationId) { + const parent = nodeRegistry.get(obs.parentObservationId); + if (parent) { + parent.childrenIds.push(obs.id); + } + } + } + + // Pass 3: Calculate depth via breadth-first traversal + const rootIds: string[] = []; + for (const [id, node] of nodeRegistry) { + if (!node.observation.parentObservationId) { + rootIds.push(id); + node.depth = 0; + } + } + + // BFS using index-based queue traversal + const queue = [...rootIds]; + let queueIndex = 0; + while (queueIndex < queue.length) { + const currentId = queue[queueIndex++]; + const currentNode = nodeRegistry.get(currentId)!; + + for (const childId of currentNode.childrenIds) { + const childNode = nodeRegistry.get(childId)!; + childNode.depth = currentNode.depth + 1; + queue.push(childId); + } + } + + // Pass 4: Identify leaf nodes for topological sort + const leafIds: string[] = []; + for (const [id, node] of nodeRegistry) { + node.inDegree = node.childrenIds.length; + if (node.childrenIds.length === 0) { + leafIds.push(id); + } + } + + return { nodeRegistry, leafIds }; +} +``` + +_([View full implementation](https://github.com/langfuse/langfuse/blob/main/web/src/components/trace2/lib/tree-building.ts))_ + +**Key decisions:** + +- **Explicit queue instead of recursion**: Handles unlimited depth +- **Index-based traversal** (`queue[index++]`): Avoids O(n) cost of `array.shift()` +- **Multiple passes**: Clarity over cleverness - each pass has a single responsibility + +### Map-Based Lookups + +We replaced linear search with hash-based lookup for parent-child relationships: + +```typescript +// Before: O(n) for each lookup, O(n²) total +observations.forEach((obs) => { + const parent = observations.find((o) => o.id === obs.parentObservationId); +}); + +// After: O(1) for each lookup, O(n) total +const nodeMap = new Map(observations.map((o) => [o.id, o])); +observations.forEach((obs) => { + const parent = nodeMap.get(obs.parentObservationId); +}); +``` + +This trades additional memory (2n instead of n) for faster lookups: 200ms becomes <1ms on large datasets. + +### Topological Sort for Bottom-Up Aggregation + +Instead of calculating aggregates on-demand, we compute them once during tree building and store them in each node. This transforms an O(n) traversal on every click into an O(1) lookup. + +The key is processing nodes in the right order - children before parents - so each node's aggregate values are ready when we need them. Topological sort gives us this ordering in a single pass: + +```typescript +function buildTreeNodesBottomUp( + nodeRegistry: Map, + leafIds: string[], + traceStartTime: Date, +): string[] { + // Start with leaf nodes (no dependencies) + const queue = [...leafIds]; + let queueIndex = 0; + const rootIds: string[] = []; + + while (queueIndex < queue.length) { + const currentId = queue[queueIndex++]; + const currentNode = nodeRegistry.get(currentId)!; + + // All children have been processed - their costs are available + const childrenTotalCost = currentNode.childrenIds + .map((id) => nodeRegistry.get(id)!.treeNode!.totalCost) + .reduce((acc, cost) => (acc ? acc.plus(cost) : cost), undefined); + + const totalCost = currentNode.nodeCost + ? currentNode.nodeCost.plus(childrenTotalCost ?? 0) + : childrenTotalCost; + + // Create tree node with aggregated data + currentNode.treeNode = { + id: currentNode.observation.id, + totalCost, + // ... other fields + }; + + // Decrement parent's in-degree; queue if ready + if (currentNode.observation.parentObservationId) { + const parent = nodeRegistry.get( + currentNode.observation.parentObservationId, + )!; + parent.inDegree--; + if (parent.inDegree === 0) { + queue.push(currentNode.observation.parentObservationId); + } + } else { + rootIds.push(currentId); + } + } + + return rootIds; +} +``` + +When we process a node, all its children have already been processed. Their costs and token counts are already computed and stored. + +Selecting any observation - even one with 50,000 descendants - now takes <1ms. The cost is pre-calculated and stored in the tree node: + +```typescript +// In the UI component +function ObservationDetailView({ observationId }) { + const { nodeMap } = useTraceData(); + const node = nodeMap.get(observationId); // O(1) lookup + + // totalCost already computed during tree building + return
Total Cost: {node.totalCost}
; +} +``` + +### Context Memoization to Avoid Rebuilds + +With aggregates pre-calculated, we need to ensure the tree is only built once. React Context with memoization provides this guarantee: + +```typescript +// contexts/TraceDataContext.tsx +export function TraceDataProvider({ trace, observations, scores, children }) { + const { minObservationLevel } = useViewPreferences(); + + const uiData = useMemo(() => { + return buildTraceUiData(trace, observations, minObservationLevel); + }, [trace, observations, minObservationLevel]); + + return ( + + {children} + + ); +} +``` + +Memoization ensures we only rebuild the tree when data actually changes, not on every render. Combined with bottom-up aggregation during build, this means we pay the O(n) cost exactly once - when the trace loads. Every subsequent interaction is O(1). + +### Bonus: React Virtualization + +With 200K+ nodes in a tree, rendering all DOM elements would cause a browser crash. We use virtualization to render only the visible rows in the viewport. As users scroll, we dynamically render new rows and unmount off-screen ones. This keeps the DOM size constant regardless of tree size - typically 50-100 elements plus a generous overscroll margin even for massive traces. + +The virtualization library handles the viewport calculations while our tree structure provides the data. When the virtualizer requests row 50,000, we use the tree's navigation to locate and render that specific node without iterating through all preceding nodes or requiring a recalculation of the expensive tree building algorithm. + +## Trade-offs + +**Advantages:** + +- Handles unlimited nesting depth without stack overflow +- O(n) complexity for tree building instead of O(n²) +- O(1) lookups via Map instead of O(n) array search +- O(1) aggregate reads instead of O(n) subtree traversal on every selection + +**Disadvantages:** + +- More complex than naive recursive approach +- Requires understanding of graph algorithms (topological sort, BFS) +- Extra memory overhead (2n for Map + array storage) + +**Remaining Limitation:** + +The bottleneck in loading traces in Langfuse is now the data fetching operation. For long-running traces with 200K observations, fetching data from the database and transferring it to the client takes a few seconds. Once the data arrives, building the tree and rendering the UI completes in under 100ms. After that, all interactions remain responsive with <1ms selection and navigation times. + +## Results + +On a trace with 200,000 observations: + +- Tree building and initial render: <100ms (one-time cost after data arrives) +- Observation selection: <1ms via Map lookup with pre-calculated costs +- Aggregate display: <1ms (direct read, no traversal) +- Expand/collapse: <1ms (no tree rebuild) +- Memory: ~120MB (acceptable for browser context) +- No stack overflow regardless of depth + +These optimizations add complexity but enable the trace viewer to handle production workloads at scale. + +_The full implementation is available in the Langfuse repository:_ + +- _[tree-building.ts](https://github.com/langfuse/langfuse/blob/main/web/src/components/trace2/lib/tree-building.ts) - Core algorithms_ +- _[TraceDataContext.tsx](https://github.com/langfuse/langfuse/blob/main/web/src/components/trace2/contexts/TraceDataContext.tsx) - React integration_ + +--- + +**Building Langfuse?** We're growing our engineering team. If you care about building developer tools that handle edge cases gracefully, [check out our open positions](https://langfuse.com/careers). diff --git a/pages/blog/2025-02-react-architecture-part-1.mdx b/pages/blog/2025-02-react-architecture-part-1.mdx new file mode 100644 index 0000000000..343899b8ed --- /dev/null +++ b/pages/blog/2025-02-react-architecture-part-1.mdx @@ -0,0 +1,394 @@ +--- +title: "Production-Grade React Components Part 1: Separating Data, State, and Presentation" +date: 2025/02/05 +description: "Organizing React components into four distinct layers - data fetching, pure transformation, context orchestration, and presentation - for maintainability and testability." +tag: engineering, react, architecture +author: Michael +--- + +At Langfuse we have a culture governed by [high ownership and proactive engineering](https://langfuse.com/handbook/how-we-work/principles). Engineers work directly with users, plan features across the full stack, and independently ship features and improvements without the need for complicated approval processes. + +This autonomy works well when many developers touch the same components, especially those that are feature-packed and heavily used. However, as components evolve iteratively, responsibilities drift and mix. Data fetching, business logic, state management, and presentation become intertwined. To maintain velocity and performance, periodic refactoring becomes necessary. + +This is the first in a three-part series on how we structure complex React components at Langfuse to improve their performance and maintainability. This post covers layer separation - the architectural foundation that makes everything else possible. + +## The Technical Challenge + +The trace view is one of the most-used components in Langfuse. Users see it every time they open a trace. It combines multiple concerns: progressive data fetching from several endpoints, transforming it into hierarchical structures, managing user interactions, handling responsive layouts, and adapting to different display modes. + +Over the past years, the trace view evolved significantly as multiple engineers added features and improvements. We took the opportunity to refactor it, introducing a component architecture that enables us to maintain and extend it more effectively going forward. + +## Rethinking the Architecture + +The original implementation mixed concerns throughout the component tree. Raw data was passed deep into components and transformed multiple times at different levels. This led to unnecessary re-renders, code duplication, and drift as different parts of the tree handled similar transformations differently. + +We rethought the architecture from data fetching through to presentation. The approach: separate into distinct layers based on their responsibilities and how frequently they change. This aligns with React best practices - keeping data fetching separate from presentation, extracting pure transformations, and controlling re-render boundaries through memoization. + +For the trace view, this resulted in four layers: data fetching, pure transformation, context orchestration, and presentation. + +### Layer 1: Data Fetching + +Data fetching is separated to isolate network concerns and make it easy to swap data sources. This layer knows nothing about tree structures or UI concerns - it only handles API calls, retry logic, and error states. + +```typescript +// api/useTraceData.ts +export function useTraceData({ + traceId, + projectId, + timestamp, +}: UseTraceDataParams) { + const query = api.traces.byIdWithObservationsAndScores.useQuery( + { traceId, projectId, timestamp }, + { + retry(failureCount, error) { + if ( + error.data?.code === "UNAUTHORIZED" || + error.data?.code === "NOT_FOUND" + ) { + return false; + } + return failureCount < 3; + }, + }, + ); + + return { + trace: query.data, + observations: query.data?.observations ?? [], + scores: query.data?.scores ?? [], + isLoading: query.isLoading, + error: query.error, + }; +} +``` + +_([View implementation](https://github.com/langfuse/langfuse/blob/main/web/src/components/trace2/api/useTraceData.ts))_ + +### Layer 2: Pure Transformation + +Business logic is extracted into pure functions with no React dependencies. This enables testing without mocking, reuse across different contexts (React components, Node.js, Web Workers), and ensures transformations happen only once through memoization rather than on every render. + +```typescript +// lib/tree-building.ts +export function buildTraceUiData( + trace: Trace, + observations: Observation[], + minLevel?: ObservationLevel, +): { + tree: TreeNode; + nodeMap: Map; + searchItems: TraceSearchListItem[]; + hiddenObservationsCount: number; +} { + // 1. Filter observations by level + const { sortedObservations, hiddenObservationsCount } = + filterAndPrepareObservations(observations, minLevel); + + // 2. Build dependency graph + const { nodeRegistry, leafIds } = buildDependencyGraph(sortedObservations); + + // 3. Process bottom-up with topological sort + const nodeMap = new Map(); + const rootIds = buildTreeNodesBottomUp( + nodeRegistry, + leafIds, + nodeMap, + trace.timestamp, + ); + + // 4. Create trace root + const tree = createTraceRoot(trace, rootIds, nodeMap); + + // 5. Flatten for search + const searchItems = flattenTreeForSearch(tree); + + return { tree, nodeMap, searchItems, hiddenObservationsCount }; +} +``` + +_([View implementation](https://github.com/langfuse/langfuse/blob/main/web/src/components/trace2/lib/tree-building.ts))_ + +Test example: + +```typescript +// lib/tree-building.clienttest.ts +import { buildTraceUiData } from "./tree-building"; + +test("builds tree with correct parent-child relationships", () => { + const trace = { id: "trace-1", timestamp: new Date() }; + const observations = [ + { id: "obs-1", parentObservationId: null }, + { id: "obs-2", parentObservationId: "obs-1" }, + ]; + + const { tree, nodeMap } = buildTraceUiData(trace, observations); + + expect(tree.children).toHaveLength(1); + expect(tree.children[0].id).toBe("obs-1"); + expect(nodeMap.get("obs-1")!.children[0].id).toBe("obs-2"); +}); +``` + +### Layer 3: Context Orchestration + +Context providers combine data fetching and transformations while controlling re-render boundaries through memoization. This ensures expensive operations (like tree building) run only when inputs change, not on every render, and provides both raw and derived data through a clean API. This approach reduces prop drilling - data flows through context rather than being passed through multiple component layers. + +The trade-off: components depending on context are coupled to it being present in the tree above them. To avoid unnecessary re-renders, contexts should isolate responsibility (avoid fat contexts where unrelated data changes trigger re-renders of all consumers). + +```typescript +// contexts/TraceDataContext.tsx +interface TraceDataContextValue { + trace: TraceType; + observations: Observation[]; + scores: Score[]; + tree: TreeNode; + nodeMap: Map; + searchItems: TraceSearchListItem[]; + hiddenObservationsCount: number; + comments: Map; +} + +export function TraceDataProvider({ + trace, + observations, + scores, + comments, + children, +}: TraceDataProviderProps) { + const { minObservationLevel } = useViewPreferences(); + + const uiData = useMemo(() => { + return buildTraceUiData(trace, observations, minObservationLevel); + }, [trace, observations, minObservationLevel]); + + const value = useMemo( + () => ({ + trace, + observations, + scores, + tree: uiData.tree, + nodeMap: uiData.nodeMap, + searchItems: uiData.searchItems, + hiddenObservationsCount: uiData.hiddenObservationsCount, + comments, + }), + [trace, observations, scores, uiData, comments], + ); + + return ( + + {children} + + ); +} +``` + +_([View implementation](https://github.com/langfuse/langfuse/blob/main/web/src/components/trace2/contexts/TraceDataContext.tsx))_ + +### Layer 4: Presentation + +The presentation layer is responsible for rendering UI and handling user interactions. It receives data via context hooks and props, contains no business logic, and can be further distinguished into three distinct types of components, each with different responsibilities and reusability characteristics: orchestration components, domain components, and pure display components. + +#### Orchestration Components + +These components compose providers and route to platform-specific layouts. They contain no business logic or styling - only composition and platform detection. + +```typescript +// Trace.tsx +export function Trace({ + trace, + observations, + scores, + projectId, + context, +}: TraceProps) { + const commentsMap = useMemo(/* ... */); + + return ( + + + + + + + + + + + + + + ); +} +``` + +#### Domain Components + +Domain components connect domain concepts to display components. They pull data from contexts, apply domain-specific logic, and compose generic components with domain content. + +```typescript +// components/TraceTree.tsx +export function TraceTree() { + const { tree, comments } = useTraceData(); + const { selectedNodeId, setSelectedNodeId, collapsedNodes, toggleCollapsed } = + useSelection(); + + // Domain logic: calculate root totals for heatmap scaling + const rootTotalCost = tree.totalCost; + const rootTotalDuration = + tree.latency != null ? tree.latency * 1000 : undefined; + + return ( + ( + 0} + isCollapsed={isCollapsed} + onToggleCollapse={onToggleCollapse} + isSelected={isSelected} + onSelect={onSelect} + > + + + )} + /> + ); +} +``` + +#### Pure Display Components + +Pure display components are generic UI components with no domain knowledge (think of shadcn/ui components). They are pure functions, work through props, are fully type-parameterized, and can be reused across any domain. + +```typescript +// components/_shared/VirtualizedTree.tsx +interface VirtualizedTreeProps { + tree: T; + collapsedNodes: Set; + selectedNodeId: string | null; + renderNode: (params: { + node: T; + treeMetadata: TreeNodeMetadata; + isSelected: boolean; + isCollapsed: boolean; + onToggleCollapse: () => void; + onSelect: () => void; + }) => ReactNode; + onToggleCollapse: (nodeId: string) => void; + onSelectNode: (nodeId: string | null) => void; +} + +export function VirtualizedTree({ + tree, + collapsedNodes, + selectedNodeId, + renderNode, + onToggleCollapse, + onSelectNode, +}: VirtualizedTreeProps) { + // Generic virtualization logic - works with any tree structure + const flattenedItems = useMemo( + () => flattenTree(tree, collapsedNodes, 0, [], true), + [tree, collapsedNodes], + ); + const rowVirtualizer = useVirtualizer({ + /* ... */ + }); + + return ( +
+ {rowVirtualizer.getVirtualItems().map((virtualRow) => { + const item = flattenedItems[virtualRow.index]; + return renderNode({ + node: item.node, + treeMetadata: { + depth: item.depth, + treeLines: item.treeLines, + isLastSibling: item.isLastSibling, + }, + isSelected: item.node.id === selectedNodeId, + isCollapsed: collapsedNodes.has(item.node.id), + onToggleCollapse: () => onToggleCollapse(item.node.id), + onSelect: () => onSelectNode(item.node.id), + }); + })} +
+ ); +} +``` + +This three-tier structure within the presentation layer provides clear separation of concerns: orchestration components handle composition, domain components apply business rules, and pure display components offer maximum reusability. + +## Sidebar: Context Design + +Instead of a single monolithic context, we create focused contexts with clear boundaries. This allows us to isolate responsibility and avoid unnecessary re-renders. This way each component only subscribes to the contexts it needs - clicking a node doesn't re-render the preference panel, and toggling "show duration" doesn't re-render the tree. + +```typescript + + {" "} + {/* User preferences: show duration, costs, etc. */} + + {" "} + {/* Read-only data: trace, tree, nodeMap */} + + {" "} + {/* UI state: selected node, collapsed nodes */} + + {" "} + {/* Search query and results */} + + + + + +``` + +**Why separate contexts?** + +Contexts are separated by change frequency and responsibility. Data changes rarely (only on refetch), selection changes constantly (every click), and preferences change occasionally (user toggles). Each context has clear ownership: TraceDataProvider owns data and derived structures, ViewPreferencesProvider owns display settings, and SelectionProvider owns interaction state. + +## Key Takeaways + +At Langfuse, our culture of high ownership and proactive engineering requires components that support rapid, confident changes by multiple developers. The layer separation approach we applied to the trace view addresses this need: separating concerns by responsibility and change frequency creates clear boundaries that enable engineers to navigate the codebase confidently and make changes without ripple effects. + +The new architecture has costs - more files and initial setup overhead. However, this increase in files can be managed through intentional directory organization, which we'll explore in [Part 2](/blog/2025-02-react-architecture-part-2) along with other code organization principles that build on these layer separations. The foundation this provides for maintainability and performance aligns with our engineering culture's requirements. + +_The complete trace view implementation:_ + +- _[trace2/](https://github.com/langfuse/langfuse/tree/main/web/src/components/trace2) - Full feature_ +- _[api/](https://github.com/langfuse/langfuse/tree/main/web/src/components/trace2/api) - Data fetching layer_ +- _[lib/](https://github.com/langfuse/langfuse/tree/main/web/src/components/trace2/lib) - Pure transformation layer_ +- _[contexts/](https://github.com/langfuse/langfuse/tree/main/web/src/components/trace2/contexts) - Orchestration layer_ + +--- + +**Building Langfuse?** We're growing our engineering team. If you care about software architecture and maintainable code, [check out our open positions](https://langfuse.com/careers). diff --git a/pages/blog/2025-02-react-architecture-part-2.mdx b/pages/blog/2025-02-react-architecture-part-2.mdx new file mode 100644 index 0000000000..fa4513da61 --- /dev/null +++ b/pages/blog/2025-02-react-architecture-part-2.mdx @@ -0,0 +1,203 @@ +--- +title: "Production-Grade React Components Part 2: Co-Location and Pure Functions" +date: 2025/02/12 +description: "Organizing code by feature instead of file type, and extracting business logic into testable pure functions." +tag: engineering, react, architecture, testing +author: Michael +--- + +At Langfuse, we believe in the power of engineers shipping with [extreme ownership](https://langfuse.com/handbook/how-we-work/principles). As we ship features and improvements iteratively, React components evolve. Occasionally, it makes sense to take a step back and refactor them to maintain velocity and code quality. + +In [Part 1](/blog/2025-02-react-architecture-part-1), we covered layer separation - organizing code into data fetching, pure transformation, context orchestration, and presentation layers. This separation improved maintainability but introduced more files: separate files for API hooks, pure transformation functions, context providers, and presentation components. + +With more files comes a new challenge: _where_ should these files live in your directory structure? + +## The Co-Location Principle + +The co-location principle can be summarized as: place code as close to where it's relevant as possible. Related code should live together, making it easier to find, understand, modify, and delete as a unit. + +The challenge is knowing how close is too close. Move things too far apart and you create friction. Move them too close and you lose clarity. Finding the right distance for each situation is key. + +### Feature-Level Organization + +Traditional folder structures organize by file type - separate folders for components, hooks, utils, and types. Understanding a feature requires opening files across multiple directories. Refactoring means modifying files scattered across folders. Deleting a feature often leaves orphaned code. + +We organize by feature instead. The trace view lives in a single `trace/` folder containing everything related to traces: + +``` +trace/ +├── api/ # Data fetching layer +├── lib/ # Pure transformation layer +├── contexts/ # Context orchestration layer +├── components/ # Presentation layer +└── config/ # Feature configuration +``` + +_([View actual structure](https://github.com/langfuse/langfuse/tree/main/web/src/components/trace2))_ + +Deleting a feature means deleting one folder. Dependencies become clear through import paths - if `TraceLogView` imports from `TraceTimeline`, you see it in the path. Circular dependencies become obvious. Everything related lives together. + +### Component-Level Organization + +Within a feature, complex components get their own folders. The `TraceTimeline` component visualizes observation timing - it needs multiple sub-components, pure calculation functions, and tests. Files share a common prefix pattern that serves as a discovery mechanism - in your IDE, you can type the prefix to see all related files, or use grep/file search to find them programmatically: + +``` +TraceTimeline/ +├── TimelineIndex.tsx # Main component +├── TimelineBar.tsx # Sub-components +├── TimelineRow.tsx +├── TimelineScale.tsx +├── timeline-calculations.ts # Pure functions +├── timeline-calculations.clienttest.ts # Tests co-located +├── timeline-flattening.ts # More utilities +└── types.ts # Type definitions +``` + +This naming convention provides practical benefits for both human developers and automated tools. In your IDE, typing "timeline" shows all related files instantly. For programmatic access, `grep -r "timeline-" .` or similar file search patterns find everything at once - useful when your agent needs to understand or modify a component. Tests live next to the code they verify, marked with `.clienttest.ts`, making coverage visible at a glance. + +### When NOT to Co-Locate + +Co-location is a means to an end, not an end in itself. The goal is making code easier to work with, not rigidly following a principle. Don't over-abstract by creating folder structures and file separations that add no practical value. + +Simple prop interfaces that live in one place stay in the component file - defining them separately adds navigation overhead without benefit. Start with things together. If they become too tightly coupled or create confusion, separate them. The structure should serve the work, not constrain it. + +## Co-Locating Pure Functions and Tests + +Co-location becomes particularly valuable when extracting business logic from React components. Pure functions can be tested without React setup, reused in different contexts, and co-located with both the components that use them and the tests that verify them. + +### Example: Timeline Calculations + +The timeline component needs to position bars, calculate widths, and select appropriate time intervals. Extracting this logic into pure functions enables testing without React, reuse in other contexts, and separation of calculation from rendering: + +```typescript +// timeline-calculations.ts +export const SCALE_WIDTH = 900; +export const STEP_SIZE = 100; + +export const PREDEFINED_STEP_SIZES = [ + 0.25, 0.5, 0.75, 1, 1.25, 1.5, 2, 2.5, 3, 4, 5, 6, 7, 8, 9, 10, 15, 20, 25, + 35, 40, 45, 50, 100, 150, 200, 250, 300, 350, 400, 450, 500, +]; + +/** + * Calculate horizontal offset from trace start time + */ +export function calculateTimelineOffset( + nodeStartTime: Date, + traceStartTime: Date, + totalScaleSpan: number, + scaleWidth: number = SCALE_WIDTH, +): number { + const timeFromStart = + (nodeStartTime.getTime() - traceStartTime.getTime()) / 1000; + return (timeFromStart / totalScaleSpan) * scaleWidth; +} + +/** + * Calculate width of timeline bar from duration + */ +export function calculateTimelineWidth( + duration: number, + totalScaleSpan: number, + scaleWidth: number = SCALE_WIDTH, +): number { + return (duration / totalScaleSpan) * scaleWidth; +} + +/** + * Calculate appropriate step size for time axis + */ +export function calculateStepSize( + traceDuration: number, + scaleWidth: number = SCALE_WIDTH, +): number { + const calculatedStepSize = traceDuration / (scaleWidth / STEP_SIZE); + return ( + PREDEFINED_STEP_SIZES.find((step) => step >= calculatedStepSize) || + PREDEFINED_STEP_SIZES[PREDEFINED_STEP_SIZES.length - 1] + ); +} +``` + +The pure functions live in `timeline-calculations.ts`, co-located with the component that uses them. _([View implementation](https://github.com/langfuse/langfuse/blob/main/web/src/components/trace2/components/TraceTimeline/timeline-calculations.ts))_ + +The component becomes cleaner: + +```typescript +// index.tsx +import { + calculateTimelineOffset, + calculateTimelineWidth, + calculateStepSize, +} from "./timeline-calculations"; + +function TraceTimeline() { + const { tree } = useTraceData(); + const traceDuration = tree.latency ?? 0; + + return ( +
+ {tree.children.map((node) => { + const offset = calculateTimelineOffset( + node.startTime, + tree.startTime, + traceDuration, + ); + const width = calculateTimelineWidth(node.duration, traceDuration); + + return ; + })} +
+ ); +} +``` + +Tests live next to the functions they verify: + +```typescript +// timeline-calculations.clienttest.ts +import { + calculateTimelineOffset, + calculateStepSize, +} from "./timeline-calculations"; + +describe("calculateTimelineOffset", () => { + it("calculates offset for node starting 5 seconds into 10-second trace", () => { + const result = calculateTimelineOffset( + new Date("2024-01-01T00:00:05Z"), + new Date("2024-01-01T00:00:00Z"), + 10, // total span + 900, // scale width + ); + expect(result).toBe(450); // 50% * 900px + }); +}); + +describe("calculateStepSize", () => { + it("selects appropriate step size for 100-second trace", () => { + expect(calculateStepSize(100, 900)).toBe(15); + }); +}); +``` + +Co-locating tests with code makes coverage visible. Looking at the `TraceTimeline/` folder, you immediately see which utilities have tests. _([View tests](https://github.com/langfuse/langfuse/blob/main/web/src/components/trace2/components/TraceTimeline/timeline-calculations.clienttest.ts))_ + +The decision to extract depends on complexity, reusability, and testability. Complex logic, code used across components, and deterministic functions benefit from extraction. Simple logic, lifecycle-coupled code, and hook-heavy operations stay in components. + +## Key Takeaways + +At Langfuse, we prioritize [shipping above little else](https://langfuse.com/handbook/how-we-work/principles). Engineers work on end-to-end ownership - planning, implementing, and supporting features without handoffs. This requires finding and modifying code quickly, with confidence that related pieces are discovered together. The layer separation from [Part 1](/blog/2025-02-react-architecture-part-1) created clear boundaries but introduced more files - addressing where those files live became the next challenge. + +The co-location principle provides a pragmatic framework: place related code together. Feature-level organization means one folder per feature. Component-level organization uses name prefixes for discovery. Tests live next to code. The structure requires discipline - more nesting, consistent naming - but enables developers and coding agents to find code quickly, refactor with confidence, and navigate without tribal knowledge. + +Layer separation created boundaries between concerns. Co-location organized those boundaries into navigable structures. [Part 3](/blog/2025-02-react-architecture-part-3) addresses performance: handling datasets that vary by orders of magnitude within these well-organized components. + +_Browse the actual implementation:_ + +- _[TraceLogView/](https://github.com/langfuse/langfuse/tree/main/web/src/components/trace2/components/TraceLogView) - Feature folder example_ +- _[timeline-calculations.ts](https://github.com/langfuse/langfuse/blob/main/web/src/components/trace2/components/TraceTimeline/timeline-calculations.ts) - Pure functions_ +- _[tree-building.ts](https://github.com/langfuse/langfuse/blob/main/web/src/components/trace2/lib/tree-building.ts) - Complex pure logic_ + +--- + +**Building Langfuse?** We're growing our engineering team. If you value well-organized, maintainable code, [check out our open positions](https://langfuse.com/careers). diff --git a/pages/blog/2025-02-react-architecture-part-3.mdx b/pages/blog/2025-02-react-architecture-part-3.mdx new file mode 100644 index 0000000000..609a4853df --- /dev/null +++ b/pages/blog/2025-02-react-architecture-part-3.mdx @@ -0,0 +1,202 @@ +--- +title: "Production-Grade React Components Part 3: Adaptive Optimization" +date: 2025/02/19 +description: "Every optimization has tradeoffs. Learn how to make the decision at runtime based on data characteristics, preserving the best experience for both small and large datasets." +tag: engineering, react, performance, architecture +author: Michael +--- + +At Langfuse, we support diverse LLM observability use cases - from simple chatbot interactions with a handful of observations to multi-hour autonomous agents generating tens of thousands of observations. We've seen production traces with over 200,000 observations, while most remain small and straightforward to render. + +This creates a design challenge: components need to handle both typical cases and edge cases without degrading either experience. The typical approach optimizes for one end of the spectrum - either build for small data and break at scale, or optimize for scale and add unnecessary complexity for everyone. + +In [Part 1](/blog/2025-02-react-architecture-part-1), we covered layer separation. In [Part 2](/blog/2025-02-react-architecture-part-2), we explored co-location and pure functions. These architectural patterns provide the structure for maintainable components. Within that architecture, we still need to make performance decisions: which optimizations to apply, and when? This post shows how to accommodate performance optimizations by making the decision at runtime based on the data you're actually processing. + +## Adaptive Optimization + +Performance optimizations solve specific problems, but each comes with costs that disproportionately affect different dataset sizes. This post examines how adaptive patterns can balance these tradeoffs across three optimizations: virtualization, lazy data loading, and Web Worker offloading. Rather than applying optimizations universally, each optimization activates only when the data characteristics justify its cost. + +## Pattern 1: Conditional Virtualization + +### Context + +The trace log view displays observations in a table. Each row shows observation metadata, and users can expand rows to see full input/output data. + +### The Tradeoff + +Rendering thousands of DOM elements causes performance degradation and eventually browser crashes. Virtualization solves this by rendering only the visible viewport - typically 50-100 rows regardless of total dataset size. + +However, virtualization removes native browser features. Cmd+F can't find text that isn't in the DOM. Accessibility tools lose context. Print preview only shows the visible portion. For traces with dozens of observations, the browser handles all DOM elements without issue, making these tradeoffs unnecessary. + +### The Adaptive Solution + +The component checks observation count before rendering: + +```typescript +// Determine virtualization based on observation count +const isVirtualized = observations.length >= LOG_VIEW_VIRTUALIZATION_THRESHOLD; +``` + +Below the threshold (350 observations), all rows render to the DOM. Users get full browser search, accessibility support, and native browser features. Above the threshold, virtualization activates, trading those features for the ability to handle thousands of observations without performance issues. + +## Pattern 2: Lazy Loading with Adaptive Download + +### Context + +When users expand an observation row, the component displays the full input/output payloads. For traces with thousands of observations, each with large payloads, the component needs a data fetching strategy. + +### The Tradeoff + +Fetching all observation data upfront creates thousands of network requests and can freeze the browser. Lazy loading solves this - data fetches only when a user expands a specific row. Each expansion takes 50-100ms to load. + +This works well for browsing but complicates download operations. Users expect "Download trace" to include all data, but most observations haven't loaded yet. Fetching everything on demand for large traces takes too long and creates a poor user experience. + +### The Adaptive Solution + +All traces use lazy loading for browsing. The download strategy adapts based on trace size: + +```typescript +// Determine download strategy based on observation count +const isDownloadCacheOnly = observations.length >= LOG_VIEW_DOWNLOAD_THRESHOLD; +``` + +For small traces (under 350 observations), download fetches all data before exporting. Users get complete trace exports with all input/output payloads included. + +For large traces (over 350 observations), download uses cache-only mode. The export includes full data for expanded observations and metadata-only for unexpanded ones. Users see a clear indicator: "Downloaded trace data (cache only)". + +## Pattern 3: Conditional Web Worker Offloading + +### Context + +The JSON viewer needs to build a tree structure from JSON data before rendering. This involves parsing the JSON, creating tree nodes with parent-child relationships, and computing navigation offsets for efficient lookup. + +### The Tradeoff + +Building tree structures from large JSON datasets (100,000+ nodes) can take hundreds of milliseconds on the main thread, blocking all user interaction. Moving this work to a Web Worker keeps the UI responsive. + +However, worker creation, data serialization, and message passing add 10-20ms of overhead. For small JSON payloads that process in a few milliseconds, the worker adds latency and displays loading spinners for operations that should feel instant. + +### The Adaptive Solution + +The component estimates tree size, then chooses between synchronous and asynchronous execution. This fits naturally into the orchestration layer from [Part 1](/blog/2025-02-react-architecture-part-1) - the layer that combines data fetching and transformations while controlling re-render boundaries. + +First, estimate the size without deep traversal: + +```typescript +export function estimateNodeCount(data: unknown): number { + if (data === null || data === undefined) return 1; + if (Array.isArray(data)) return data.length; + if (typeof data === "object") return Object.keys(data).length; + return 1; +} +``` + +Then, provide both synchronous and asynchronous paths: + +```typescript +export function useTreeState(data, config) { + // Estimate size once + const dataSize = useMemo(() => estimateNodeCount(data), [data]); + + // Path 1: Synchronous build for small datasets + const syncTree = useMemo(() => { + if (dataSize > TREE_BUILD_THRESHOLD) return null; + return buildTreeFromJSON(data, config); + }, [data, dataSize, config]); + + // Path 2: Web Worker build for large datasets + const asyncTreeQuery = useQuery({ + queryKey: ["tree-build", data, config], + queryFn: () => buildTreeInWorker(data, config), + enabled: dataSize > TREE_BUILD_THRESHOLD, + staleTime: Infinity, + }); + + // Return whichever path was used + const tree = syncTree || asyncTreeQuery.data; + const isBuilding = + dataSize > TREE_BUILD_THRESHOLD && asyncTreeQuery.isLoading; + + return { tree, isBuilding }; +} +``` + +_([View implementation](https://github.com/langfuse/langfuse/blob/main/web/src/components/ui/AdvancedJsonViewer/hooks/useTreeState.ts))_ + +The same algorithm runs in both paths. The only difference is execution context. Small datasets process synchronously and return instantly. Large datasets process in a Web Worker, keeping the UI responsive. Components receive an identical API regardless of which path was taken. + +The threshold of 10,000 nodes reflects the tradeoff between worker overhead (10-20ms) and UI blocking (hundreds of milliseconds for large datasets). Below that size, the overhead outweighs the benefit. Above that size, keeping the UI responsive justifies the cost. + +## Configuration Centralization + +All thresholds and settings live in a single configuration file: + +```typescript +export const TRACE_VIEW_CONFIG = { + logView: { + virtualizationThreshold: 350, + downloadThreshold: 350, + rowHeight: { + collapsed: 28, + expanded: 150, + }, + maxIndentDepth: 5, + batchFetch: { + concurrency: 10, + }, + }, +} as const; +``` + +_([View implementation](https://github.com/langfuse/langfuse/blob/main/web/src/components/trace2/config/trace-view-config.ts))_ + +## When to Use Adaptive Patterns + +Adaptive optimization makes sense when: + +**Data size varies significantly** - If 95% of your users have datasets under 100 items but 5% have datasets over 10,000 items, you have variance worth addressing. If everyone's datasets are similar in size, optimize for that size. + +**Optimization has meaningful tradeoffs** - If the optimization only improves performance without removing features, apply it universally. Adaptive patterns are for cases where the optimization helps large datasets but hurts small ones. + +**Threshold is measurable and stable** - The decision point (number of rows, data size, nesting depth) should be something you can calculate quickly and reliably. If the threshold depends on complex heuristics or changes frequently, the pattern adds more complexity than value. + +**Maintenance cost is justified** - Running two code paths means testing two scenarios. If 99% of users hit the "small data" path, the benefit of handling the 1% edge case needs to outweigh the maintenance burden. + +## Tradeoffs in Practice + +For traces under the threshold, users get full browser search, expand all functionality, and complete downloads. The UI behaves like a standard web page. + +For traces above the threshold, virtualization activates. Users lose browser search and expand all, but rendering and scrolling remain smooth. Downloads use the cache-only approach. + +For traces with thousands of observations, all optimizations activate. These traces would be unusable without virtualization and lazy loading. The feature limitations are acceptable tradeoffs for making the traces viewable. + +The alternative approaches create problems: + +Optimizing only for large data means most users lose features and see loading spinners for operations that could be instant. + +Optimizing only for small data means traces with thousands of observations cause browser crashes or long freezes. + +## Conclusion + +This concludes our three-part series on production-grade React components at Langfuse: + +- [Part 1](/blog/2025-02-react-architecture-part-1) established layer separation - organizing code into data fetching, pure transformation, context orchestration, and presentation layers +- [Part 2](/blog/2025-02-react-architecture-part-2) covered co-location and pure functions - keeping related code together and extracting testable business logic +- Part 3 introduced adaptive optimization - making performance decisions at runtime based on actual data characteristics + +These patterns work together. Layer separation provides clear boundaries for where optimizations apply. Co-location keeps threshold logic near the code it affects. Pure functions make performance testing straightforward. + +At Langfuse, our engineering culture emphasizes high ownership and shipping with confidence. Engineers work across the full stack, from database queries to React components. The patterns in this series support that culture by making components predictable, maintainable, and capable of handling the extreme variance we see in production LLM applications. + +When developers use Langfuse to debug their applications, they don't know whether they'll open a trace with 10 observations or 10,000 observations. Our job is to make both cases work well. Adaptive optimization lets us preserve the best possible experience for the majority while gracefully handling the edge cases that would otherwise make the product unusable. + +_View the complete implementations:_ + +- _[useTreeState.ts](https://github.com/langfuse/langfuse/blob/main/web/src/components/ui/AdvancedJsonViewer/hooks/useTreeState.ts) - Adaptive tree building_ +- _[TraceLogView.tsx](https://github.com/langfuse/langfuse/blob/main/web/src/components/trace2/components/TraceLogView/TraceLogView.tsx) - Threshold-based virtualization_ +- _[trace-view-config.ts](https://github.com/langfuse/langfuse/blob/main/web/src/components/trace2/config/trace-view-config.ts) - Centralized thresholds_ +- _[trace2/](https://github.com/langfuse/langfuse/tree/main/web/src/components/trace2) - Full application of all three parts_ + +--- + +**Building Langfuse?** We're growing our engineering team. If you care about performance, user experience, and pragmatic engineering, [check out our open positions](https://langfuse.com/careers).