feat: add React WebUI, Electron desktop app, and agent interfaces - #11
Open
Eleanor1018 wants to merge 407 commits into
Open
Eleanor1018 wants to merge 407 commits into
Eleanor1018 wants to merge 407 commits into
Conversation
Move the compatibility runtime out of src so the new React frontend cannot treat it as production architecture. Keep the files under old/runtime only as a temporary implementation reference while feature slices are rebuilt against the existing backend API.
Move the complete React/Vite workspace from frontend to frontend-old without changing its contents. This leaves hhtools/web/frontend available for a fresh implementation after the FastAPI surface has been simplified.
Keep the replacement architecture note local and preserve the node_modules ignore rule after archiving the previous frontend.
Move shared robot pose and ground-contact helpers out of the Web serializer and update Retarget, Web, and boundary-policy callers to use the robot layer directly. Preserve the existing numerical and DOF fallback behavior.
Place the Agent REST adapter, transport boundary, artifact response, and H2R executor under hhtools.agent. Update the Web composition root, tests, and dependency policy without changing runtime behavior.
Move Web helpers into jobs, library, analysis, and output packages and update all production, script, and test imports. Keep only the application entry point and dependency check at the Web package root.
Delete production-unused settings overrides, unreachable upload branches, legacy preview diagnostics, unused scheduler state, and mismatched-frame pairing. Preserve format compatibility and recovery paths that still have active callers.
Enumerate clips directly from an explicit local source path and persist a compact source summary instead of embedding every batch entry in job history.
Treat an H2R batch source directory as the persistent replay input and report its recorded clip count without reconstructing the full entry manifest.
Record direct-directory failures by source path without copying clips into the export tree. Preserve failure bundles for uploaded sources and cover the local behavior with a smoke test.
Add content-bound H2R calibration status, constrained proposals, deterministic validation, and front/side PNG previews for vision-capable MCP clients. Allow validated silent saves to a user overlay with stale-baseline protection, previous-version archival, idempotent replay, REST/JSON CLI adapters, GUI assistance, schemas, skill guidance, and end-to-end coverage.
Bind calibration candidates to exact source and target robot bundles, expose status/proposal/validation/preview/save tools, and route missing pair calibration back to the agent. Preserve user-overlay writes, audit notes, archival, stale-baseline checks, schemas, adapters, and end-to-end coverage.
Render a responsive no-prompt homepage for human CLI users, cap ultrawide output, preserve strict help and Agent behavior, and update About credits for Jagger Shen, Nora Sun, and hhtools contributors.
Refactor/frontend webui gui
Run push CI only for main so feature branches with open pull requests do not execute the full workflow twice. Apply Ruff formatting to the maintained Python modules that were blocking the existing formatting gate.
Refactor/frontend webui gui
docs(install): recommend uv source setup
Preserve the configured virtual-environment launcher instead of resolving it to a base interpreter, and persist the selected licensed body-model directory across desktop restarts. Normalize video inputs to the 30 FPS timeline expected by the official runner, publish only the final hmr4d result, refresh Motion Library state immediately, and keep technical tracebacks in Tasks while showing concise workflow errors. Document the optional-runtime boundary and add regression coverage for symlinked Python environments, frame-rate normalization, isolated publication, and desktop configuration. Verified with 1179 Python, 120 frontend, and 65 desktop tests plus real GVHMR inference and DEB smoke testing.
fix(gvhmr): stabilize optional Linux workflow
Managed JSON artifacts were returned as StreamingResponse objects, but the Agent route's JSON portability guard requires a buffered body. Valid job specs, previews, evaluation reports, and manifests therefore failed with HTTP 500 while CSV downloads succeeded. Retain the exact JSON bytes read during SHA-256 verification and return a buffered response. Keep one file open throughout verification, preserve download integrity headers, and continue streaming non-JSON artifacts. Normalize JSON MIME types case-insensitively, including +json subtypes. Preserve strict JSON and portability checks. Add regressions for all four artifact kinds, path replacement, content tampering, duplicate keys, non-finite numbers, malformed JSON, and unsafe payloads. Validation: REST download regressions passed; all 58 retained JSON artifacts downloaded through the HTTP route with matching SHA-256 values.
Saving calibration for a robot installed in the user library could modify its registered bundle, invalidating the asset ID before save retries or status queries. Preflight could also report ready for a calibration that the geometry validator had already rejected. Use a separate .calibration-overlays directory when the historical user calibration directory overlaps a registered bundle. Retain legacy reads, previous-calibration archives, and subsequent Web/CLI writes to adopted overlays. H2R retries reuse existing saved bytes without rewriting them. Persist H2R geometry assessments outside robot bundles. Bind each record to the robot asset, reference family, calibration digest, and validator version, plus the exact motion asset for GLB. Recheck saved bytes against the candidate before publishing evidence. Require current valid evidence for manual-calibration H2R preflight. Missing, corrupt, stale, or failed evidence returns an executable status action instead of a ready plan. Keep preflight free of solver construction. Bind the validator version and controlled overlay paths into plans while preserving execution-time digest checks and stale-plan rejection. Document the status/save/preflight lifecycle and extend service, storage, plan, and MCP coverage. Exercise new and replacement calibrations, restart replays, changed inputs, corrupt evidence, and saves replaced concurrently. Validation: calibration and MCP regressions passed, including the original invalid-G1/preflight reproduction. Seven G1 reference families and the RPO-to-G1 pair passed isolated save, retry, status, and runtime-restart checks without changing either original or copied robot bundle contents.
The previous cache fingerprint used only the input count and maximum mtime rounded to seconds. Renaming a clip could reuse results containing its nonexistent old path; same-second edits and configuration changes could also leave stale analysis results. Fingerprint sorted input and sidecar paths, nanosecond file metadata, streamed content hashes, analysis entries, effective configuration, and the metric schema version. Exclude generated analysis caches, reload the default configuration on demand, and reject missing or changing inputs. Capture input identity before analysis and publish a cache only if that identity is unchanged afterward. Replace manifests atomically to avoid exposing partially written results. Add coverage for renames, deletion, sidecars, same-second edits, preserved metadata, configuration and schema changes, ordering, and concurrent edits. Validation: cache regressions and the original rename reproduction passed. A real LAFAN BVH copy reused its unchanged cache, then reanalyzed after a rename and returned the existing new path. Content hashing adds input I/O to cache validation; the small fixture is not a large-library benchmark.
The Electron CI job failed during prepare:bootstrap because uv had not been installed. The desktop E2E suite also stopped at an obsolete welcome title, leaving later interactions untested. Set up uv 0.12.9 and Python 3.12 before packaging validation and sidecar dependency installation. Keep the real bootstrap build and installer syntax checks, then run Electron E2E under Xvfb with software rendering. Install the required system dependencies and retain failure artifacts. Split the long desktop smoke test into five independent cases covering onboarding, menus/settings, imports, workflow navigation, and the stage. Use stable tutorial step selectors, distinguish top-level menu triggers from submenu items, and update assertions to the current layout controls. Give each case isolated settings and verify runtime readiness, page errors, and Python sidecar shutdown. Validation: removing uv from PATH reproduced the reported CI error. With the pinned uv available, all four packaging-validation commands passed, producing the wheel, dependency export, and valid install script. Desktop type checking, 65 unit tests, and all five E2E cases passed. The completed repair set also passed 1,243 Python tests, 120 frontend tests, all five original issue reproductions, maintained Python lint and format checks, critical Python checks, and git diff --check.
Refactor/frontend webui gui
Depend directly on pkexec, which is the executable used by the system-scope runtime installer, instead of the policykit-1 transitional package removed in Ubuntu 26.04. Keep the Debian version at 0.1.0 and update the package contract test. The rebuilt artifact installs successfully on clean Ubuntu 22.04, 24.04, and 26.04 containers.
fix(package): support Ubuntu 26.04 polkit
Add arm64 DMG and ZIP targets with a native-host check and document the published preview downloads in the English, Chinese, and desktop READMEs. Reuse the verified runtime bootstrap for per-user macOS installation with Python 3.12, native Library paths, and platform-specific checksum tools. Preserve the bundled uv signature and include the installer template in the runtime fingerprint. Prevent App Nap during setup and startup, prepare native imports before marking installation complete, and keep packaged apps from silently using the build checkout. Add regression coverage and a packaged macOS E2E test. Validation: TypeScript checks, 72 desktop unit tests, 16 targeted Python tests, five desktop UI E2E tests, and a clean-home macOS package install, launch, and shutdown test passed. Verified DMG integrity, app signatures, published download links, and the released DMG checksum.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds a React-based WebUI, Windows and Linux(.deb) desktop clients, and agent-facing interfaces for the existing hhtools workflows. It also improves the CLI experience and introduces optional GVHMR-based video-to-motion conversion.
The main additions are:
These features share the same contracts, services, and Web runtime, so they are included together for an initial architecture review. I would also be happy to split the PR if that would make review easier.
Changes
WebUI
Desktop
Agent interfaces
next_actionresponses.Services and integrations