Skip to content

feat(computer-use): integrate native Windows maka.cu/2 path - #4595

Open
sunheyi6 wants to merge 13 commits into
apache:mainfrom
sunheyi6:codex/maka-cu-rust-comparison-pr
Open

feat(computer-use): integrate native Windows maka.cu/2 path#4595
sunheyi6 wants to merge 13 commits into
apache:mainfrom
sunheyi6:codex/maka-cu-rust-comparison-pr

Conversation

@sunheyi6

@sunheyi6 sunheyi6 commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR integrates the native Windows maka.cu/2 Computer Use path into Maka and records the current Windows x64 validation evidence.

  • Keeps the shared Computer Use surface semantic-only; model actions do not expose raw coordinate mutation.
  • Uses the shared supervised lifecycle and a thin Windows native-helper adapter instead of retaining the former private Windows protocol.
  • Enforces fresh target identity using HWND, PID, title, process start time, and windowGeneration; stale observations are invalidated and unknown outcomes are not replayed or upgraded.
  • Adds the helper selection, packaging/provenance seam, Windows host tests, protocol drivers, and bilingual cross-machine evidence.
  • The companion native executor implementation is in maka-agent/maka-cu#8.

Validation

The following evidence is recorded in experiments/maka-cu-windows/CROSS_MACHINE_RUST_NATIVE_SUMMARY.md:

  • Formal Rust native executor: 8 cargo tests passed; shared-protocol lifecycle: 29/29 passed.
  • WPF direct run: 17 pass, 0 fail, 1 honest unknown; six-run WPF matrix: 102 pass, 0 fail, 6 unknown.
  • Chromium direct run: 9 pass, 0 fail, 3 blocked, 1 honest unknown; six-run matrix: 54 pass, 0 fail, 18 blocked, 6 unknown.
  • Computer Use host integration: 115 pass, 0 fail, 2 skipped (the two skipped cases require explicit published-helper environment variables).
  • Desktop host targeted checks: 17 pass plus one Windows symlink EPERM environment block; native capability contract: 13/13 pass.
  • Windows x64 packaging completed with exit code 0; the earlier Spectre/MSB8040 failure did not recur. The installer SHA256 is AD79C2362FF05B5392D79655B85E2B3C20E341AE8ECC2C5E5BDAA387CA37DA12; the ZIP SHA256 is 825213FE198A3140C60B75FD83D80B365C0CD67776DE840D26AD6364DF81B7A2.
  • Real Electron Maka smoke: 7/7 programmatic checks passed.
  • LibreOffice real-app probe: 9 pass, 1 typed dispatch_refused; the mutation was not retried.
  • Manual real Maka Computer Use smoke: user-confirmed pass; an isolated Maka session opened a new Chrome test instance and navigated to https://www.google.com/ without using an existing Chrome profile.
  • The Chromium Enter case remains unknown; a page oracle is never rewritten as helper verification.

Known limits

  • The repository-wide npm run build on the clean PR branch is currently blocked by pre-existing @maka/ui type errors involving settledText, autoScroll, and trailingAction, outside this PR. The CU-scoped build, @maka/computer-use tests, and Desktop main-process build pass.
  • The legacy comparison harness still speaks the former private protocol and is not valid shared-protocol conformance evidence; the new maka-cu2 drivers are authoritative.
  • The dedicated automated real-model runner was not run because its provider/API-key/local-bridge environment was not configured.
  • Full WinUI/UWP mutation coverage was not accepted after Notepad restored an existing user session.
  • Clean-machine validation was intentionally skipped by the current scope. This PR therefore provides local-machine evidence, not clean-machine or production qualification; distributionReady remains false.

Refs #4318

中文翻译

摘要

本 PR 将 native Windows maka.cu/2 Computer Use 路径接入 Maka,并记录当前 Windows x64 验证证据。

  • 保持共享 Computer Use action surface 为 semantic-only;模型 action 不暴露原始坐标 mutation。
  • 使用共享 supervised lifecycle 和薄的 Windows native-helper adapter,不再保留原来的私有 Windows protocol。
  • 使用 HWND、PID、标题、进程启动时间和 windowGeneration 强制执行新鲜目标身份;过期 observation 会失效,unknown 结果不会被重放或升级。
  • 增加 helper 选择、打包/来源追踪 seam、Windows host 测试、protocol driver 和中英双语跨机器证据。
  • 配套的 native executor 实现位于 maka-agent/maka-cu#8

验证情况

以下证据记录在 experiments/maka-cu-windows/CROSS_MACHINE_RUST_NATIVE_SUMMARY.md 中:

  • Formal Rust native executor:8 个 cargo 测试通过;shared-protocol lifecycle:29/29 通过。
  • WPF 直接运行:17 pass、0 fail、1 个诚实 unknown;六次 WPF 矩阵:102 pass、0 fail、6 个 unknown
  • Chromium 直接运行:9 pass、0 fail、3 个 blocked、1 个诚实 unknown;六次矩阵:54 pass、0 fail、18 个 blocked、6 个 unknown
  • Computer Use host integration:115 pass、0 fail、2 个 skipped(两个 skipped 用例需要显式的已发布 helper 环境变量)。
  • Desktop host 定向检查:17 pass,另有 1 个 Windows symlink EPERM 环境阻塞;native capability contract:13/13 通过。
  • Windows x64 打包以退出码 0 完成;之前的 Spectre/MSB8040 失败没有重现。安装包 SHA256 为 AD79C2362FF05B5392D79655B85E2B3C20E341AE8ECC2C5E5BDAA387CA37DA12;ZIP SHA256 为 825213FE198A3140C60B75FD83D80B365C0CD67776DE840D26AD6364DF81B7A2
  • 真实 Electron Maka smoke:7/7 程序化检查通过。
  • LibreOffice 真实应用探测:9 pass、1 个类型化 dispatch_refused;没有重试该 mutation。
  • 手工真实 Maka Computer Use smoke:用户确认通过;隔离的 Maka 会话打开新的 Chrome 测试实例并访问 https://www.google.com/,没有使用已有 Chrome profile。
  • Chromium Enter 用例保持为 unknown;页面 oracle 永远不会被改写为 helper verified。

已知限制

  • 干净 PR 分支上的仓库全量 npm run build 当前被本 PR 范围之外既有的 @maka/ui 类型错误阻塞,涉及 settledTextautoScrolltrailingAction。CU 范围构建、@maka/computer-use 测试和 Desktop main-process 构建通过。
  • 旧 comparison harness 仍使用原来的私有 protocol,因此不是有效的 shared-protocol conformance 证据;新的 maka-cu2 driver 才是权威结果。
  • 专用自动真实模型 runner 未运行,因为没有配置其 provider/API key/本地 bridge 环境。
  • Notepad 恢复了已有用户会话后,没有接受完整的 WinUI/UWP mutation 覆盖结果。
  • 当前范围明确跳过 clean-machine 验证。因此本 PR 提供的是本机证据,不是 clean-machine 或生产资格证明;distributionReady 仍为 false

Refs #4318

@github-actions github-actions Bot added the effort/XL Under 2500 readable lines label Sep 2, 2026
@sunheyi6

sunheyi6 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

@M4n5ter @hqhq1025 cc

@hqhq1025 hqhq1025 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of f24c195182dedafdda6d0ab75755c23f29848ab7

The desired product change is valid: select the Windows native helper on Windows, keep the shared MakaCuService supervisor/backend, verify the helper artifact, and package it only through an explicit readiness gate. The current revision is not merge-ready.

Blocking findings:

  1. This PR is based on the pre-#4497 action-space revision and is now conflicting with main. It contains approximately 693k added lines across 159 files, including the already-landed action-space work, two executor experiments, historical comparison harnesses, and generated result dumps. Please rebuild the PR from current main and retain only the Windows product integration, focused tests, concise documentation, and preparation/provenance tooling.
  2. The checked-in artifact authority points at sunheyi6/maka-cu and records only a local binary digest. The preparation script can set distributionReady: true from a command-line flag without proving an upstream repository commit, reproducible CI artifact, signature, clean-machine acceptance, or packaged conversation E2E. A release gate must be derived from verifiable provenance and qualification evidence, not a caller-provided boolean. Keep readiness false until that pipeline exists.
  3. The companion executor PR currently violates its advertised maka.cu/2 snapshot/image lifecycle and response limit, and its documented clippy command fails. Product integration should not be approved before maka-agent/maka-cu#8 is corrected and the exact corrected artifact is retested through this host path.

Repository hygiene:

  • Remove the two generated browser JSON files (about 663k lines combined) and the repeated raw run outputs from Git history. Publish raw evidence as CI artifacts or a linked release attachment; keep a small machine-readable summary and reproduction commands in the PR.
  • Remove the duplicate Rust executor copy under experiments/. The executor source must have one authority in maka-cu; Maka should consume a pinned artifact, not mirror the implementation.
  • Remove windows-cu-backend.ts and windows-cu-service.ts unless a real platform-specific behavior appears. They only rename/re-export the shared backend and supervisor, which makes the public API look like two implementations even though the architecture requires one. Platform selection can instantiate createMakaCuBackend directly.

The current CI being green proves the branch builds and packages in its current environment, but it does not resolve these ownership, provenance, or executor-contract issues.

Recommended sequence:

  1. Fix and approve maka-agent/maka-cu#8.
  2. Rebase/recreate this PR from current main after #4497.
  3. Keep a small Maka-side adapter/artifact patch only.
  4. Run a real Windows interactive-desktop executor E2E, then packaged Maka conversation E2E, then clean-machine/install/upgrade validation against the exact pinned artifact.
  5. Keep distributionReady: false until signing, provenance, servicing, and clean-machine gates are automated.

Verdict: not ready to merge. The product-side final structure should remain one model action space, one shared protocol/backend/supervisor, and one thin Windows artifact/platform selection seam.


export async function prepareWindowsCuHelper({
source = process.env.MAKA_CU_WINDOWS_SOURCE,
releaseReady = process.argv.includes('--distribution-ready'),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A caller-controlled --distribution-ready flag can promote any locally supplied artifact after only filename/size checks. Readiness must be computed from verified provenance and qualification evidence (upstream commit/artifact identity, signing, clean-machine/package E2E, and servicing), not asserted by the invocation. Please remove this promotion path and keep local preparation development-only.

"distributionReady": false
},
"windowsCu": {
"repo": "sunheyi6/maka-cu",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The product manifest must not establish a contributor fork as the release authority. Pin the canonical maka-agent/maka-cu repository plus an exact reviewed commit or CI artifact provenance once the executor PR is approved. A local digest is useful for integrity, but it is not source provenance.


export type WindowsCuBackendOptions = MakaCuBackendOptions;

export function createWindowsCuBackend(options: WindowsCuBackendOptions): MakaCuBackend {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is only an alias for createMakaCuBackend; the adjacent Windows service file similarly re-exports the shared supervisor. These aliases create a second named public API without a second implementation and weaken the one-backend/one-supervisor boundary. Select the shared backend directly and keep Windows-specific behavior in artifact selection and the native executor.

@github-actions github-actions Bot added effort/XXL Over 2500 readable lines and removed effort/XL Under 2500 readable lines labels Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/XXL Over 2500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants