Skip to content

feat(web): single-socket — multiplex terminals + sessions over one WebSocket#887

Open
fastestdevalive wants to merge 22 commits intoComposioHQ:mainfrom
fastestdevalive:feat/single-socket
Open

feat(web): single-socket — multiplex terminals + sessions over one WebSocket#887
fastestdevalive wants to merge 22 commits intoComposioHQ:mainfrom
fastestdevalive:feat/single-socket

Conversation

@fastestdevalive
Copy link
Copy Markdown

Summary

Replaces three separate real-time channels with a single persistent multiplexed WebSocket at /mux.

Before:

  • One WebSocket per terminal (:14801/ws?session=<id>, spawned per tab)
  • SSE stream for session status (/api/events)
  • Legacy ttyd server on port 14800 (dead code, never used by UI)

After:

  • One persistent /mux WebSocket per browser tab (owned by MuxProvider)
  • Terminal I/O, resize, open/close all flow over mux protocol channels
  • Session patches delivered via a shared SSE relay: mux server subscribes once to Next.js /api/events and broadcasts to all connected clients — no per-client polling

Architecture

Browser  ←── WS /mux ──→  Mux Server (:14801)  ←── SSE /api/events ──  Next.js (:3000)
          (bidirectional)     TerminalManager         (event-driven push,
          terminal + sessions  node-pty per session    1 shared connection)

Key changes

  • MuxProvider — React context at app root owning the single WebSocket; DirectTerminal uses hooks instead of managing its own WS
  • SessionBroadcaster — server-side class maintaining one SSE connection to Next.js, lazily connecting on first subscriber and disconnecting when the last one leaves
  • Manual WS upgrade routing — works around a ws library limitation where two WebSocketServer instances with different path options on the same HTTP server don't work correctly
  • useSessionEvents — extended to accept mux session patches, skipping SSE when mux is available
  • Removed terminal-websocket.ts — entire legacy ttyd-based server deleted (port 14800, ~450 lines)

Test plan

  • Open dashboard — terminals connect and show output
  • Resize browser window — terminal resizes correctly
  • Multiple terminals open simultaneously — all receive independent I/O
  • Session status cards update in real time as agent state changes
  • Disconnect/reconnect network — MuxProvider reconnects with backoff, terminals reattach automatically
  • pnpm --filter @composio/ao-web test — 453 tests passing
  • pnpm --filter @composio/ao-web typecheck — clean

🤖 Generated with Claude Code

fastestdevalive and others added 4 commits April 2, 2026 19:01
Implements backend multiplexed WebSocket server on /mux endpoint:
- Add mux-protocol.ts with ClientMessage and ServerMessage types
- Create TerminalManager class for managing PTY processes independently
- Implement mux-websocket.ts with attachMuxWebSocket() function
- Wire mux server into existing direct-terminal-ws server
- Support multiple terminals over single persistent WebSocket connection
- Implement 50KB ring buffer for background terminal output
- Add heartbeat and reconnection logic

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implements client-side multiplexed WebSocket context:
- Create MuxProvider component with persistent connection management
- Implement exponential backoff reconnection (1s to 30s)
- Add subscribeTerminal/writeTerminal/resizeTerminal/openTerminal methods
- Manage per-terminal ring buffers (50KB max) for background output
- Track session patches from server
- Create useMux() hook for easy access in components
- Mount MuxProvider in app root layout via Providers wrapper
- Support dynamic runtime config from /api/runtime/terminal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace direct WebSocket connections with mux channel subscriptions:
- Add useMux() hook to DirectTerminal component
- Replace WebSocket creation with openTerminal() call
- Use subscribeTerminal() for receiving terminal data
- Use writeTerminal() for sending user input
- Use resizeTerminal() for handling window resize
- Remove WebSocket reconnection logic (handled by MuxProvider)
- Remove runtime config fetching (handled by MuxProvider)
- Preserve all existing features:
  - XDA (Extended Device Attributes) handler for clipboard support
  - OSC 52 clipboard handler
  - Keyboard copy handler (Cmd+C/Ctrl+Shift+C)
  - Selection buffering during output
  - Theme switching
  - Fullscreen mode
  - Resize handling

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…bSocket

Replace three separate real-time channels (per-terminal WS, SSE, HTTP poll)
with a single persistent multiplexed WebSocket at /mux.

Architecture:
- Browser ↔ MuxProvider owns one WS connection per tab (/mux)
- Terminal I/O, resize, open/close all flow over mux channels
- Session status patches delivered via a shared SSE relay:
  mux server subscribes once to Next.js /api/events (SSE) and
  broadcasts to all connected browser clients — no per-client polling
- Manual WS upgrade routing fixes ws library limitation with multiple
  WebSocketServer instances on the same HTTP server

Remove:
- terminal-websocket.ts (legacy ttyd-based per-session server, port 14800)
- Per-terminal WebSocket connections from DirectTerminal
- Per-client 5 s HTTP polling for session patches

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…, cleanup

- Use native WS ping frames for heartbeat so idle browser clients are not
  incorrectly disconnected (browser auto-responds to native ping with pong)
- Pass runtime config to buildMuxWsUrl() so dynamic port/proxy path set via
  TERMINAL_WS_PATH env var is actually used (was fetched but never consumed)
- Delay initial WS connect until runtime config fetch resolves, preventing
  race condition on first load
- shutdown() now terminates all mux clients and closes the WSS, preventing
  orphaned PTY processes on restart
- ws 'error' handler now unsubscribes terminal callbacks alongside session
  subscription to prevent leaks in edge cases where 'close' does not follow

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace Node.js Buffer.byteLength() with TextEncoder in MuxProvider
  (browser-safe UTF-8 byte counting for the ring buffer)
- Remove write-only sessionSubscribedRef from MuxProvider
- Remove dead DirectTerminalLocation, DirectTerminalWsUrlOptions,
  buildDirectTerminalWsUrl and their tests (superseded by mux)
- Call closeTerminal(sessionId) on DirectTerminal unmount so PTY
  processes are released and openedTerminalsRef stays accurate
- Add cleanup return to mux effect in useSessionEvents so pending
  refresh timers and abort controllers are cleared on unmount

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- MuxProvider: assign wsRef.current immediately after construction so
  cleanup can close a WebSocket that is still in CONNECTING state;
  add isDestroyedRef to prevent the close handler from scheduling
  reconnects after the component unmounts
- mux-websocket: client "close" message now only unsubscribes that
  client — removed terminalManager.close() which killed the shared PTY
  for every other connected client
- useSessionEvents: mux effect cleanup no longer clears the debounce
  timer; only aborts in-flight requests, preventing rapid mux snapshots
  from starving the membership-change refresh

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- DirectTerminal: remove stale muxStatus read from xterm setup effect
  (muxStatus was captured at mount, leaving reload button permanently
  disabled); reload button now checks muxStatus directly on each render
- TerminalManager: kill PTY and delete map entry when last subscriber
  unsubscribes, preventing orphaned node-pty processes when all mux
  clients disconnect

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- mux-websocket: wire up the exited ServerMessage — add exitCallbacks
  to ManagedTerminal, pass onExit param through subscribe(), and fire
  callbacks when PTY exits and re-attach fails so clients get notified
- DirectTerminal: fix displayStatus to show "error" when there is a
  local error (e.g. xterm.js load failure) regardless of mux state
- useSessionEvents: SSE effect's mux early-return path now returns a
  cleanup function that clears refreshTimerRef and aborts in-flight
  requests on unmount, preventing post-unmount dispatch calls

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- MuxProvider: reset isDestroyedRef to false at the start of the
  effect so React StrictMode's double-invoke doesn't permanently break
  the connection (cleanup sets it true, re-run must reset it)
- DirectTerminal: remove dead status state — it was only set to "error"
  in a catch block but never read; displayStatus already derives the
  error indicator from the error string directly

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Removing the muxSessions.length === 0 guard so an empty snapshot
triggers the membership-key comparison and scheduleRefresh(), which
fetches the full session list and dispatches a reset. Previously,
[] caused an early return that left removed sessions visible indefinitely.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- MuxProvider: handle exited and error terminal messages — exited
  removes the terminal from openedTerminalsRef (preventing re-open on
  reconnect) and writes a red notice into the xterm stream; error is
  logged to console
- MuxProvider: remove dead buffersRef/bufferBytesRef — client-side
  ring buffer was never read; the server already delivers history on
  open
- mux-websocket: use ws.terminate() instead of ws.close() on heartbeat
  timeout — an unresponsive peer won't complete the close handshake,
  so terminate() immediately destroys the socket and frees resources

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The SSE effect had muxSessions (array reference) in its deps, causing
it to re-run on every snapshot and fire cleanup that cleared the
debounce timer the mux effect intentionally preserved.

- Derive muxActive = muxSessions !== undefined and use that in SSE
  effect deps — the effect now only re-runs when mux transitions
  between present/absent, not on every new array reference
- Add reschedule in scheduleRefresh .finally() abort path: when a
  fetch is aborted mid-flight (by a new snapshot) and there is still
  a pending membership key, reschedule so the refresh isn't silently
  dropped

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- useSessionEvents: mux-active SSE cleanup now also resets
  pendingMembershipKeyRef and refreshingRef so the aborted fetch's
  .finally() handler cannot reschedule after unmount
- DirectTerminal: add distinct "Disconnected" label and error-coloured
  dot for muxStatus "disconnected" (WebSocket constructor failure with
  no reconnect), instead of the misleading "Connecting…" shown for
  transient states

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
},
);
subscriptions.set(id, unsub);
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Server open handler sends buffer then subscribes, risking duplicate data

Medium Severity

When handling a terminal open message, the server calls terminalManager.open(id) (line 481) which internally calls subscribe() adding its own subscriber, then the handler separately calls terminalManager.subscribe(id, ...) (line 501) adding a second subscriber. The first open() call alone already subscribes data delivery through the onData handler. Wait — actually, open() only spawns the PTY and sets up internal onData/onExit that push to terminal.subscribers. It doesn't add to terminal.subscribers by itself. The subscribe() call does. So the flow is: open() → buffer sent → opened sent → subscribe() adds the callback. This is correct within a single connection.

However, the real issue: subscribe() calls this.open(id) internally (line 367), which is redundant since the handler already called open(). More importantly, subscribe()'s unsubscribe closure captures the terminal local variable from subscribe(), while open()'s onData/onExit closures capture the terminal local variable from open(). Both point to the same map entry, so this works. No actual bug here after deeper analysis.

Let me re-examine...

Actually — the real bug I see: the opened confirmation is sent (line 496-497) BEFORE the subscription is created (line 500-521). On the client side in MuxProvider, receiving opened triggers openedTerminalsRef.current.add(msg.id). The client considers the terminal "open" and may immediately start writing. This is fine since writeTerminal works independently. Not a bug.

I'll withdraw this and look more carefully.

Additional Locations (1)
Fix in Cursor Fix in Web

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No action needed — the bot concluded its own analysis with 'No actual bug here after deeper analysis' and self-withdrew. The ordering (open → buffer → subscribe) is correct: since everything runs synchronously in a single event-loop tick (no await), node-pty's onData cannot fire between getBuffer() and subscribe().

…le terminals ref

- Move buffer send + subscribe inside !subscriptions.has(id) guard so
  reconnecting clients don't receive the history replay twice (once on
  first open, once after wsRef is reassigned on reconnect)
- Remove `terminals` field from MuxContextValue and useMemo — it was
  populated from a write-only ref and never consumed by any component

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ve dead close()

- SessionBroadcaster.connect(): capture controller in a local variable and
  only clear this.abortController in finally if it still equals the local
  controller; prevents a concurrent connect() call's controller being
  nullified by an older connect()'s finally block
- Heartbeat: send ws.ping() before incrementing missedPongs so all
  MAX_MISSED_PONGS pings are actually transmitted before terminating
  (previously terminated after MAX_MISSED_PONGS-1 sent pings)
- Remove TerminalManager.close(): never called by the mux handler (which
  uses per-subscriber unsubscribe instead) and subtly broken — it killed
  the PTY without clearing subscribers, which would have triggered an
  immediate re-attach via onExit

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…send

- Replace the duplicated ClientMessage/ServerMessage/SessionPatch local
  type definitions in mux-websocket.ts with a single import type from
  src/lib/mux-protocol.ts — eliminates the risk of the two copies drifting
  apart as the protocol evolves
- Add ws.readyState === WebSocket.OPEN guard to the ws.send call inside
  the terminal-operation catch block, consistent with all other sends in
  the file; prevents a throw that would bubble into the outer catch and
  replace the real error with a misleading "Invalid message format"

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
In the ws library, "error" is always followed by "close", so the close
handler's cleanup (clearInterval, unsubscribe sessions, unsubscribe all
terminals) was running twice. Reduced the error handler to just the
console.error log and let close handle cleanup exclusively.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
A throwing callback (e.g. ws.send on a closed socket) would abort the
for-of loop, causing remaining subscribers to miss the data chunk and
skipping the ring buffer update — corrupting replay history for future
reconnections. Wrapping each call in try-catch keeps all subscribers
independent and ensures the buffer update always runs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add reattachAttempts counter to ManagedTerminal and MAX_REATTACH_ATTEMPTS
constant (3). The onExit handler only re-calls open() while the counter is
below the cap, resets it to 0 on a successful attach, and falls through to
notify exit callbacks once the limit is reached — preventing a crash-loop
where a PTY that exits immediately after spawn triggers infinite re-attaches.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
muxStatus was in the deps array solely for the guard check at the top of
the effect. This caused the entire RAF loop + transitionend + backup timers
to re-run on every mux status transition (e.g. reconnecting→connected),
even when no layout change occurred.

Fix: track muxStatus in a ref (updated on every render) and read the ref
inside the effect, so the guard always sees the current value without
muxStatus being a dependency.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

After a reconnect MuxProvider re-opens all terminals, but new PTYs spawn
at 80×24 default. DirectTerminal only sent the initial resize on mount.
Add a dedicated effect that fires whenever muxStatus transitions to
"connected": calls fit.fit() to measure the current container then sends
the live cols/rows to the server, keeping the PTY size in sync.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@illegalcall illegalcall self-requested a review April 3, 2026 20:55
@illegalcall
Copy link
Copy Markdown
Collaborator

@fastestdevalive Have you performed e2e testing for this? Can you attach screenshots/anything how this improves the operation.
A before/after would really help

@fastestdevalive
Copy link
Copy Markdown
Author

@fastestdevalive Have you performed e2e testing for this? Can you attach screenshots/anything how this improves the operation. A before/after would really help

Yes, I have tested it end-to-end on my local machine. TO give context, the problem was the small delay seen when switching between sessions. Its not very evident in the AO's default UI but in my heavily customized fork which is a full blown IDE, the problem manifests as I have multiple session / sub-session selectors in the sidebar. And quickly selecting through them opens/clsoes socket connections which feels slower.
The goal with this is to maintain one websocket and multiplex it for every terminal/session related usage. Similar to how even VSCode does it internally.

I will try to get a before and after video.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants