Refactor viser into panel-based ViserApp architecture#5
Draft
TToTMooN wants to merge 6 commits into
Draft
Conversation
Inspired by comparing with Raiden's data collection toolkit: - Unified `limb` CLI entry point with subcommands (teleop, record, devices, replay, convert, visualize) — replaces remembering individual script paths - Device discovery (`limb devices`) enumerates RealSense, ZED, CAN interfaces, Dynamixel serial, foot pedals, and SpaceMouse - Motion replay (`limb replay`) streams recorded joint commands back to hardware for verifying recording quality before conversion - Incomplete episode detection: RECORDING_IN_PROGRESS marker tracks in-flight recordings; auto-cleaned on next startup - Depth recording: EpisodeRecorder now captures depth video when cameras provide depth data - Converter skips incomplete/corrupted episodes automatically https://claude.ai/code/session_01BNWyGBKJRszYAY7db6eYMB
- Fix P1: episode cleanup race condition — marker now stores PID, only removes episodes whose owner process is dead - Fix P1: move convert/visualize logic into limb.data package so CLI imports work in wheel installs (scripts become thin wrappers) - Fix P2: validate empty timestamps in replay before accessing indices - Add limb.data.convert_webdataset — WebDataset .tar shard export with Welford online stats, configurable image resize/quality - Add limb.data.upload — S3/GCS/HuggingFace upload with URI scheme (s3://, gs://, hf://) and ~/.config/limb/storage.yaml config - Add shared limb.data.episode_utils for episode discovery/loading - Update CLI: convert-lerobot, convert-webdataset, upload subcommands https://claude.ai/code/session_01BNWyGBKJRszYAY7db6eYMB
Replaces the Rich terminal TUI (StatusDisplay) when a ViserMonitor is active. Renders session status and interactive buttons in the viser sidebar instead of fighting with terminal output. - ViserSessionPanel implements both display interface (update_loop, update_session) and TriggerSource protocol (get_signal) so GUI buttons feed signals into the session alongside keyboard/pedal - Buttons: Start/Stop, Mark Success, Discard, Quit Session - Composed with existing trigger via CompositeTrigger (first wins) - launch.py auto-selects viser panel vs Rich TUI based on monitor - For YamViserAgent + collection, creates lightweight standalone monitor (no URDF) so session panel has a viser server to attach to https://claude.ai/code/session_01BNWyGBKJRszYAY7db6eYMB
This reverts commit 23cfe44.
Replace monolithic ViserMonitor with a composable panel system:
- ViserApp: central coordinator that owns a ViserServer and provides
add_panel()/update()/close() API for declarative panel registration
- ViserPanel protocol: attach(server)/detach() interface for panels
- ObservablePanel protocol: adds update(obs) for control loop data
Split ViserMonitor into focused panels:
- CameraPanel: camera feed thumbnails in sidebar (~40 lines)
- URDFPanel: 3D robot visualization in viewport (~80 lines)
- RecordingPanel: Start/Stop recording button (~90 lines)
- SessionPanel: data collection controls + TriggerSource (~150 lines)
- LoopStatusPanel: Hz/step count display (~30 lines)
Update launch.py to declarative panel registration:
app = ViserApp()
app.add_panel("cameras", CameraPanel())
app.add_panel("urdf", URDFPanel(bimanual=True))
if session: app.add_panel("session", SessionPanel())
Keep ViserMonitor as backward-compat wrapper for external code.
Update YamViserAgent to use CameraPanel directly.
Adding new panels (3D points, policy viz, intervention) is now a
single-file addition + one line in launch.py.
https://claude.ai/code/session_01BNWyGBKJRszYAY7db6eYMB
- Document threading model in ViserApp docstring: main thread calls update(), viser server thread fires on_click callbacks - RecordingPanel now implements ObservablePanel (update(obs)) instead of requiring a get_latest_rgb callback — extracts RGB from obs directly, fixing the bug where frames were never written in the direct-panel path - RecordingPanel.update() uses double-checked locking: fast path skips when not recording, extracts RGB outside lock, then writes inside lock - Panel registration order matters: CameraPanel before RecordingPanel ensures RGB is extracted before frames are written - Remove unused Callable import and stale get_latest_rgb coupling - Simplify ViserMonitor backward-compat wrapper https://claude.ai/code/session_01BNWyGBKJRszYAY7db6eYMB
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ViserMonitorwith composableViserApp+ panel systemapp.add_panel()Architecture
Key changes
limb/visualization/app.py— ViserApp coordinator + ViserPanel/ObservablePanel protocolslimb/visualization/panels/— 5 focused panel classes split from ViserMonitorlimb/envs/launch.py— declarative panel registration replaces conditional ViserMonitor wiringlimb/agents/teleoperation/yam_viser_agent.py— uses CameraPanel directlylimb/visualization/viser_monitor.py— kept as backward-compat wrapperTest plan
uv run limb/envs/launch.py --config_path configs/yam_viser_bimanual.yamlNote: This PR depends on PR #4 — rebase after PR4 merges.
https://claude.ai/code/session_01BNWyGBKJRszYAY7db6eYMB