Skip to content

feat: add a unified WebUI, desktop client, and agent automation interfaces - #9

Closed
Eleanor1018 wants to merge 28 commits into
Roboparty:mainfrom
Eleanor1018:main
Closed

feat: add a unified WebUI, desktop client, and agent automation interfaces#9
Eleanor1018 wants to merge 28 commits into
Roboparty:mainfrom
Eleanor1018:main

Conversation

@Eleanor1018

@Eleanor1018 Eleanor1018 commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR adds a Vue-based WebUI, an Electron desktop client, and a set of agent-facing interfaces around the existing hhtools workflows.

The main changes include:

  • Vue and TypeScript views for human-to-robot, robot-to-robot, batch, video-to-motion, and dataset-analysis workflows.
  • Optional GVHMR-based video-to-motion conversion for single videos and batch jobs, with an isolated Docker runtime.
  • An Electron client that starts and monitors a local Python sidecar.
  • REST, MCP, and JSON CLI interfaces for agent-driven workflows.
  • Durable jobs, asset registration, preflight checks, job recovery, and artifact export.
  • Motion-library, robot-library, task-settings, localization, and navigation updates.
  • Additional schemas, documentation, and regression tests.

These changes share the same contracts, job services, and Web runtime, so they are submitted together for an initial architecture review. I am happy to split the PR if the maintainers would prefer to review the parts separately.

Changes

WebUI

  • Add a Vue 3 and TypeScript frontend.
  • Add views for H2R, R2R, batch processing, video conversion, and dataset analysis.
  • Add categorized motion import and library filtering for plain motion, object interaction, and terrain scenes.
  • Add robot-library entries, preview images, attribution, and license records.
  • Add task and library settings to the workspace settings dialog.
  • Add English and Simplified Chinese translations with system-locale detection.
  • Add the command palette, About dialog, first-run guide, and updated workspace navigation.
  • Commit the generated frontend assets used by the Python package.

Desktop

  • Add an Electron application around the local WebUI and Python runtime.
  • Use Human-Humanoid Tools as the packaged application and window title.
  • Add typed preload and IPC interfaces.
  • Validate IPC senders and restrict application navigation.
  • Add Python runtime discovery, sidecar supervision, logging, and shutdown handling.
  • Persist window state and expose optional-component diagnostics.
  • Add Windows build and NSIS installer support.

Agent interfaces

  • Add versioned contracts shared by REST, MCP, and the JSON CLI.
  • Add capability discovery, asset inspection, asset registration, and retarget preflight.
  • Add durable and idempotent job creation, lookup, retry, and recovery.
  • Add job-scoped artifact listing, verification, and controlled export.
  • Add structured errors and executable next_action responses.
  • Keep MCP stdout limited to JSON-RPC messages.
  • Reject undeclared MCP arguments and sanitize runtime-lease errors.

Services and integrations

  • Add job history, job admission settings, runtime leases, and legacy-job migration.
  • Add upload and asset-boundary validation.
  • Add configurable task concurrency.
  • Add optional GVHMR video-to-motion support and its Docker runtime.
  • Add Agent Skill documentation and exported JSON Schema snapshots.
image image image image

Eleanor1018 and others added 14 commits August 24, 2026 18:25
@Eleanor1018
Merge pull request #1 from Eleanor1018/bugfix/webui
9f5db21 
fix(web): fully hide inactive toast
@Eleanor1018
Merge pull request #2 from Eleanor1018/bugfix/webui
f903e16 
feat: add Electron desktop app and Vue TypeScript WebUI
@Eleanor1018
Merge pull request #3 from Eleanor1018/bugfix/webui
9c62d19 
fix(web):Fix component occlusion issues
@Eleanor1018
Merge pull request #4 from Eleanor1018/bugfix/webui
67e448e 
Bugfix/webui
@Eleanor1018
Merge pull request #5 from Eleanor1018/bugfix/webui
353dcfc 
Bugfix/webui
@Eleanor1018
Merge pull request #6 from Eleanor1018/feature/task-settings
d89dbeb 
Feature/task settings
@Eleanor1018
Merge pull request #7 from Eleanor1018/style/square-controls
39da65f 
Style/square controls
@Eleanor1018
Merge pull request #8 from Eleanor1018/style/square-controls
a0ac3fd 
Style/square controls
@Eleanor1018
Merge pull request #9 from Eleanor1018/style/square-controls
8ccebc1 
chore: standardize code and documentation conventions
@Eleanor1018
Merge pull request #10 from Eleanor1018/style/square-controls
cdf35b6 
fix(agent): align portable URI contracts
@Eleanor1018
test(desktop): refresh workspace smoke assertions
b0e4eb2 
Align the packaged-app E2E checks with the current navigation, unified data analysis entry, and expanded settings dialog.
@Eleanor1018
Merge pull request #11 from Eleanor1018/style/square-controls
2e0cb9e 
Style/square controls
@Eleanor1018
feat: add WebUI, desktop, and agent workflow support
1dd1e80 
Introduce the Vue workspace and Electron client for retargeting, batch, video, and analysis flows, with the packaged application named Human-Humanoid Tools.

Add versioned REST, MCP, and JSON CLI contracts, durable job and artifact services, safer asset boundaries, optional GVHMR integration, documentation, and regression coverage.
@Eleanor1018
Merge pull request #12 from Eleanor1018/feature/consolidated-workbench
433dc08 
Feature/consolidated workbench
@Eleanor1018
Eleanor1018 requested a review from jaggerShen August 31, 2026 07:43
Eleanor1018 and others added 14 commits August 31, 2026 16:42
@Eleanor1018
fix(viewer): stabilize legacy Viser markdown rendering
ea499b8 
Route initial and dynamic Markdown through a single compatibility boundary that removes unsupported string style attributes and normalizes MDX line breaks.

Render clip metadata as an escaped read-only summary, add regression guards, and update the GUI issue records.
@Eleanor1018
fix(i18n): localize retarget workflow state
1fcb7a6 
Propagate the workspace locale through H2R and R2R pipeline, calibration, and result components.

Translate dynamic runtime status messages, refresh state during live locale changes, add regression coverage, and rebuild the packaged WebUI assets.
@Eleanor1018
Merge pull request #13 from Eleanor1018/feature/consolidated-workbench
a047c6b 
Feature/consolidated workbench
@Eleanor1018
feat(desktop): add standalone Linux package support
adf9759 
Bundle the Linux desktop application with its Python runtime and expose separate hhtools and hhtools-desktop entry points.

Add Debian installation guidance, writable CLI defaults, graphics and Wayland fallbacks, runtime staging safeguards, documentation, and regression coverage.
@Eleanor1018
fix(calibration): persist packaged robot overrides in user storage
e62ef0d 
Store H2R and R2R calibration files in the per-user robot directory when bundled presets are read-only, while retaining source-tree sibling behavior.

Prefer validated user overrides across Web, CLI, viewer, retarget, and agent preflight flows. Add content-bound plan storage, strict identity and path validation, and regression coverage.
@Eleanor1018
Merge pull request #14 from Eleanor1018/feature/consolidated-workbench
5f49309 
Feature/consolidated workbench
@Eleanor1018
feat(gvhmr): import external motion results
c7b197a 
Add a direct hmr4d_results.pt path to the Video to Motion workflow and reuse motion ingestion for preview, library registration, and H2R.

Harden tensor loading and the existing isolated Docker bridge, expose best-effort custom checkpoints, document external runtime boundaries, and refresh tests and packaged WebUI assets.
@Eleanor1018
Merge pull request #15 from Eleanor1018/feature/gvhmr-extension
a951595 
feat(gvhmr): import external motion results
@Eleanor1018
fix(startup): validate WebUI dependencies
0fb1861 
Fail cleanly when the browser or desktop sidecar runtime is missing required WebUI packages, and include actionable uv recovery commands. Document the separate terminal, WebUI, and standalone Debian GUI installation paths in both READMEs. Add regression coverage for dependency detection and CLI error handling.
@Eleanor1018
refactor(webui): migrate workbench from Vue to React
63ad363 
- share one React and TypeScript renderer between browser and Electron hosts
- add VS Code-style base, platform, workbench, and service boundaries
- introduce Tailwind CSS and project-owned shadcn/ui primitives
- preserve IK runtime DOM contracts and rebuild production assets
@Eleanor1018
docs(webui): clarify React architecture boundaries
83461d3 
Document workbench ownership, host abstraction, compatibility ports, layout constraints, and long-running batch state flow.
@Eleanor1018
Merge pull request #16 from Eleanor1018/feature/reformation
f139ddb 
Feature/reformation
@Eleanor1018
docs(runtime): clarify frontend orchestration boundaries
2cc97da
@Eleanor1018
Merge pull request #17 from Eleanor1018/feature/reformation
f1541b0 
docs(runtime): clarify frontend orchestration boundaries
@Eleanor1018 Eleanor1018 closed this Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jaggerShen jaggerShen Awaiting requested review from jaggerShen

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@Eleanor1018