Skip to content

feat: WebSocket client mode for remote DimOS connection#20

Open
jeff-hykin wants to merge 12 commits intomainfrom
jeff/fix/connect
Open

feat: WebSocket client mode for remote DimOS connection#20
jeff-hykin wants to merge 12 commits intomainfrom
jeff/fix/connect

Conversation

@jeff-hykin
Copy link
Copy Markdown
Member

Problem

dimos-viewer in --connect mode can't send click or teleop events back to DimOS when LCM multicast is unavailable across machines.

Solution

Switched the WebSocket module from server (axum) to client (tokio-tungstenite) that connects to the Python RerunWebSocketServer. Replaced --ws-port with --ws-url for explicit URL control. Added automatic reconnection with buffer drain on reconnect to prevent stale velocity commands from being sent. Moved WS_EVENTS.md protocol doc to repo root.

Breaking Changes

--ws-port CLI flag replaced by --ws-url. If previously using --ws-port 3030, now use --ws-url ws://127.0.0.1:3030/ws.

How to Test

Requires the jeff/fix/rconnect branch of dimos for the server side.

# Terminal 1: start the Python websocket server
cd <path-to-dimos>
git checkout jeff/fix/rconnect
python -c "
from dimos.visualization.rerun.websocket_server import RerunWebSocketServer
import threading
server = RerunWebSocketServer(port=3030)
server.clicked_point.subscribe(lambda pt: print(f'[CLICK] {pt.x:.3f},{pt.y:.3f},{pt.z:.3f}'))
server.tele_cmd_vel.subscribe(lambda tw: print(f'[TWIST] {tw}'))
server.start()
threading.Event().wait()
"

# Terminal 2: build and run the viewer
pixi run build && pixi run viewer -- --ws-url ws://127.0.0.1:3030/ws

Click in the 3D viewport and use WASD keys — should see [CLICK] and [TWIST] in terminal 1.

Contributor License Agreement

  • I have read and approved the CLA.

ruthwikdasyam and others added 10 commits March 14, 2026 01:01
…PC proxy

When --connect is provided (with optional URL), the viewer connects to an
existing gRPC proxy server instead of spawning its own. This mirrors the
stock rerun binary's --connect behavior. Defaults to rerun+http://127.0.0.1:<port>/proxy.
- Remove unused `parking_lot` dependency (cargo-shear)
- Add `[lints] workspace = true` to dimos/Cargo.toml (rerun lint)
- Fix TOML formatting in dimos/pyproject.toml (taplo)
- Rename error variables from `e` to `err` (rerun lint convention)
- Fix trailing whitespace in keyboard.rs
- Add blank lines before doc comments on struct fields/constants
- Use Display instead of Debug for error formatting in viewer.rs
- Commit updated Cargo.lock

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…econnect

- WsPublisher now connects as a client to RerunWebSocketServer (Python)
  instead of running its own axum server
- Replaced axum + broadcast with tokio-tungstenite + mpsc
- Drain queued commands on reconnect to avoid sending stale velocities
- Changed --ws-port to --ws-url for full URL control
- Moved WS_EVENTS.md to repo root with other docs
The Python server (websockets library) sends keepalive pings every 30s
and expects a pong within 30s. The old code only wrote to the stream
and never read, so tungstenite never got a chance to auto-reply with
pong frames. This caused the server to close the connection every
~2 minutes.

Split the stream into read/write halves and spawn a reader task that
consumes incoming frames (triggering tungstenite's automatic pong
replies). The write loop uses tokio::select! to also detect when the
reader exits (server closed), triggering immediate reconnection instead
of waiting for the next send to fail.
- Delete interaction/lcm.rs (LCM publisher no longer needed)
- Simplify KeyboardHandler to take WsPublisher directly
- Remove dual LCM/WS code paths in viewer.rs
- Always connect via WebSocket (default ws://127.0.0.1:3030/ws)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants