diff --git a/.trellis/spec/backend/quality-guidelines.md b/.trellis/spec/backend/quality-guidelines.md index 80ccd8f..c426602 100644 --- a/.trellis/spec/backend/quality-guidelines.md +++ b/.trellis/spec/backend/quality-guidelines.md @@ -27,6 +27,7 @@ - idle 收尾要**回溯**到 max(最后输入时刻, 最后媒体活跃时刻),不是收在当前时刻。 - 多命令一次 exec 时用显式分隔符(`---ECHOLOG---`)分段解析,不要靠正则猜哪行是谁的输出(`pmset` 输出里也有裸数字)。 - `lsappinfo info -only name -only bundleid "$(lsappinfo front)"` 无需 TCC 权限;输出形如 `"LSDisplayName"="微信"`。 +- 常驻自动任务访问 macOS Keychain 时必须使用禁止授权 UI 的 helper 模式；`KEYCHAIN_AUTH_REQUIRED` 应静默跳过并暂停该凭据的自动重试。只有显式用户操作可使用至少 60 秒的交互式读取；成功后把凭据缓存于进程内并恢复调度，避免按采样周期重复访问 Keychain 或弹系统对话框。列表/页面刷新不得为了展示状态查询 Keychain，进程停止时必须清空缓存。 ### 常驻采样器模式 diff --git a/README.md b/README.md index c894ccc..f5c15ee 100644 --- a/README.md +++ b/README.md @@ -141,7 +141,9 @@ launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.echolog.daemon.plist ``` screen-understanding helper 不属于 portable build。本机 smoke 组装使用 -`ECHOLOG_MACOS_ADHOC_SMOKE=1 pnpm build:macos-capture`；正式签名使用 +`ECHOLOG_MACOS_ADHOC_SMOKE=1 pnpm build:macos-capture && pnpm smoke:macos-helper`； +launchd 安装后使用 `pnpm smoke:launchd-helper` 检查实际 `WorkingDirectory`； +正式签名使用 `ECHOLOG_MACOS_SIGNING_IDENTITY=... pnpm build:macos-release`。启用识图后，截图只在 单次请求内存中存在；数据库只保存结构化理解结果，不保存图片或 API key。 diff --git a/README_AGENT.md b/README_AGENT.md index 20362f8..6b4bf9a 100644 --- a/README_AGENT.md +++ b/README_AGENT.md @@ -44,6 +44,7 @@ portable `pnpm build` 不编译 Swift helper。macOS 本机测试可显式执行 ```bash ECHOLOG_MACOS_ADHOC_SMOKE=1 pnpm build:macos-capture +pnpm smoke:macos-helper ``` ## 启动后验证 @@ -55,6 +56,19 @@ node dist/cli/index.js plugins list --json node dist/cli/index.js plugins doctor --json ``` +如果使用 launchd，另跑一次真实运行目录检查： + +```bash +pnpm smoke:launchd-helper +``` + +自动识图只允许 helper 以非交互模式读取 Keychain；需要授权时会静默跳过并暂停 +该 Provider 的自动 Keychain 重试，不会周期性弹系统对话框。请在 Web 点击 +“立即识别”（或从本机执行 `el screen understanding run --json`），并在 60 秒内 +完成一次 Keychain 授权。读取成功后 API key 只进入 daemon 的进程内缓存，自动 +调度随即恢复且不再访问 Keychain；daemon 重启会清空缓存，必要时需再次执行一次 +显式识别完成授权。 + 若已把 wrapper 放进 `PATH`，后续使用 `el` 代替 `node dist/cli/index.js`。先读 `el --help` 和目标子命令的 `--help`；它们是参数、枚举、时间格式和示例的权威工具说明。 ## CLI 契约 @@ -98,7 +112,7 @@ screen-understanding 默认关闭。最安全的路径是让人类用户打开 ` 2. 在本机 Keychain 保存 API key； 3. 测试截图并由用户授予屏幕录制权限； 4. 选择 Provider、设置周期/预算后启用； -5. 执行一次“立即识别”并检查结构化中文结果。 +5. 执行一次“立即识别”并检查结构化中文结果；该显式操作可在周期识别尚未启用时完成 Keychain 授权。 Agent 可以读取状态和结果： @@ -113,6 +127,7 @@ Provider、Keychain、settings 和 capture test 的完整 HTTP 契约见 [`docs/ - Key 写入/删除、截图测试、立即识别和历史删除只允许 loopback 请求； - daemon 永远不能调用 `request-permission`，权限必须由交互式用户动作触发； +- Provider 列表和普通页面刷新只能显示内存中已知的密钥状态，不得主动查询或弹出 Keychain； - settings 更新是带 `expectedVersion` 的全量替换；409 时读取 `currentVersion` 后重新决策，不能盲目覆盖； - 启用识别前必须选择存在且已有 Keychain 密钥的 Provider； - 原始截图不得落盘、记录、转发或附在 Agent 回复中，除非用户明确要求处理当前预览且符合其隐私意图。 diff --git a/docs/API.md b/docs/API.md index 55556c4..fc0cdc8 100644 --- a/docs/API.md +++ b/docs/API.md @@ -352,9 +352,9 @@ curl -X PUT http://localhost:19827/api/plugins/screen-time/understanding/setting #### screen-understanding providers 与 Keychain -Provider metadata 存在 PostgreSQL；API Key 只存在当前 macOS 用户的 Keychain， -API 永不返回原文或掩码片段。支持的 provider kind 当前固定为 -`openai-compatible`。 +Provider metadata 存在 PostgreSQL；API Key 只持久化在当前 macOS 用户的 Keychain， +成功读取后可短暂保存在 daemon 进程内存中。API 永不返回原文或掩码片段；daemon +停止或重启会清空缓存。支持的 provider kind 当前固定为 `openai-compatible`。 ```bash curl http://localhost:19827/api/plugins/screen-time/understanding/providers @@ -405,13 +405,15 @@ curl -X DELETE http://localhost:19827/api/plugins/screen-time/understanding/prov - `expectedVersion`：1–2147483647 的整数；PUT/DELETE 均使用乐观并发。 - `apiKey`：UTF-8 1–4096 bytes，不得有首尾空白、换行或 NUL。 -Keychain 状态正常时 `hasApiKey` 为 boolean；helper/Keychain 暂时不可用时为 -`null`，metadata 仍可读取和编辑，Web 显示“密钥状态不可用”。 +`hasApiKey` 只反映 daemon 内存中已知的状态；daemon 刚启动或状态未知时为 `null`。 +`GET /providers` 和普通页面刷新不会为展示状态调用 helper，也不会触发 Keychain UI。 +metadata 始终可读取和编辑，Web 对 `null` 显示“密钥状态不可用”。 新建成功为 `201`；更新成功为 `200` 且 version 加一；删除 metadata 成功为 `204`。非法字段/URL/key 返回 `400`；不存在返回 `404`；版本过期、profile 仍被 settings 选择、启用时删除 key，或启用设置却没有可用 key，返回 `409`。 -Keychain helper 不可用/失败/超时分别返回脱敏的 `503`/`502`/`504`。 +Keychain helper 不可用/失败/超时分别返回脱敏的 `503`/`502`/`504`；需要用户授权 +返回 `409 KEYCHAIN_AUTH_REQUIRED`。 Key 的 PUT/DELETE 仅接受 loopback 请求，远端请求返回 `403 PLUGIN_LOCAL_ONLY`。 Key PUT/DELETE 成功响应分别为： @@ -435,8 +437,8 @@ Key PUT/DELETE 成功响应分别为： ``` 其他稳定冲突 code 包括 `PROVIDER_PROFILE_IN_USE`、`PROVIDER_PROFILE_LIMIT`、 -`PROVIDER_KEY_REQUIRED`；平台/helper code 包括 `KEYCHAIN_UNAVAILABLE`、 -`KEYCHAIN_OPERATION_FAILED` 和 `PLUGIN_TIMEOUT`。 +`PROVIDER_KEY_REQUIRED`；平台/helper code 包括 `KEYCHAIN_AUTH_REQUIRED`、 +`KEYCHAIN_UNAVAILABLE`、`KEYCHAIN_OPERATION_FAILED` 和 `PLUGIN_TIMEOUT`。 #### 显式测试截图 @@ -480,11 +482,18 @@ curl -X POST http://localhost:19827/api/plugins/screen-time/understanding/captur #### AI 屏幕识别 `POST /api/plugins/screen-time/understanding/run` 是 loopback-only 的显式识别入口， -只接受空 JSON 对象。它要求 settings 中 `enabled=true`、选中了有 Keychain 密钥的 -Provider；服务会采集活动显示器、调用 `${baseUrl}/chat/completions`，并只接受包含 +只接受空 JSON 对象。它要求选中了有 Keychain 密钥的 Provider；服务会采集活动 +显示器、调用 `${baseUrl}/chat/completions`，并只接受包含 `summary`、`activity`、`confidence`、`sensitive`、`apps` 的 JSON 结果。失败不会返回 远端响应正文或 API key。 +`enabled` 只控制周期调度；该显式入口在周期识别关闭时仍可执行。它允许 macOS +显示 Keychain 授权 UI，并为用户保留至少 60 秒完成授权。 +成功读取后，凭据进入 daemon 的进程内缓存并解除该 Provider 的自动调度阻断。 +周期任务只执行禁止 UI 的 Keychain 查询；遇到 `KEYCHAIN_AUTH_REQUIRED` 会静默跳过 +本轮并停止重复查询，不会弹窗或把插件降级。后续周期识别直接使用内存缓存，不再 +逐轮访问 helper。daemon 重启后如 Keychain 不能无 UI 读取，请再次调用本显式入口。 + ```bash curl -X POST http://localhost:19827/api/plugins/screen-time/understanding/run \ -H 'Content-Type: application/json' -d '{}' @@ -519,8 +528,8 @@ observation 返回 `404`。它不会删除 request budget ledger。 `dailyRequestBudget` 和可选的 `dailyCostBudgetMicros` 约束执行。一次进程内同时只 允许一个识别任务；临时网络错误按 `maxAttempts` 有界重试。 -识别关闭返回 `409 UNDERSTANDING_DISABLED`；未配置 Provider 返回 -`409 UNDERSTANDING_PROVIDER_REQUIRED`；请求/成本预算耗尽返回 `429`；模型认证、 +未配置 Provider 返回 `409 UNDERSTANDING_PROVIDER_REQUIRED`；请求/成本预算耗尽 +返回 `429`；模型认证、 超时、限流、不可达和非法响应分别返回脱敏的 `PROVIDER_AUTH`、 `PROVIDER_TIMEOUT`、`PROVIDER_RATE_LIMITED`、`PROVIDER_UNAVAILABLE` 或 `UNDERSTANDING_RESPONSE_INVALID`。原始截图测试和识别入口均不提供 diff --git a/docs/PLUGIN_API.md b/docs/PLUGIN_API.md index 31d438a..c0519c1 100644 --- a/docs/PLUGIN_API.md +++ b/docs/PLUGIN_API.md @@ -158,6 +158,19 @@ screen-time packages `native/macos-capture/build/EchoLogScreenCapture.app/Contents/MacOS/echolog-screen-capture`. Permission inspection never prompts, and a missing Screen Recording permission diagnostic does not degrade passive foreground tracking. +`pnpm smoke:macos-helper` verifies the source or release tree runtime path; +`pnpm smoke:launchd-helper` reads the daemon plist `WorkingDirectory` and checks +the helper that launchd will actually load. + +Scheduled Keychain status/get operations MUST use the helper's non-interactive +mode, which applies `kSecUseAuthenticationUIFail`. Authorization-required +results MUST be structured and MUST NOT be treated as timeouts or degrade the +plugin. Explicit local user actions MAY use interactive Keychain access with a +timeout of at least 60 seconds. A successful read SHOULD be cached only in the +daemon process; set/delete MUST synchronize that cache and plugin stop MUST +clear it. Provider listing and ordinary page loads MUST NOT query Keychain. +Secrets MUST NOT enter database rows, configuration, argv, logs, errors, smoke +output, or test output. `GET /api/health` reports Core health. Plugin failures appear in `GET /api/plugins` and `GET /api/plugins/doctor`. A failed doctor request uses diff --git a/package.json b/package.json index 0658904..fe12f8b 100644 --- a/package.json +++ b/package.json @@ -10,7 +10,9 @@ "dev": "tsx src/server/app.ts", "build": "pnpm --filter @echolog/plugin-sdk build && pnpm --filter @echolog/plugin-screen-time build && pnpm --filter @echolog/plugin-tmux-status build && tsup", "build:macos-capture": "bash scripts/build-macos-capture.sh", - "build:macos-release": "pnpm build && pnpm build:macos-capture", + "build:macos-release": "pnpm build && pnpm build:macos-capture && pnpm smoke:macos-helper", + "smoke:macos-helper": "bash scripts/smoke-macos-helper.sh", + "smoke:launchd-helper": "bash scripts/smoke-macos-helper.sh --launchd", "package:macos": "bash scripts/package-release.sh --version 0.2.0 --adhoc", "cli": "tsx src/cli/index.ts", "migrate": "tsx src/migrate.ts", diff --git a/plugins/screen-time/README.md b/plugins/screen-time/README.md index 2606195..bff9204 100644 --- a/plugins/screen-time/README.md +++ b/plugins/screen-time/README.md @@ -33,22 +33,38 @@ explicitly: ```bash ECHOLOG_MACOS_ADHOC_SMOKE=1 pnpm build:macos-capture +pnpm smoke:macos-helper ECHOLOG_MACOS_SIGNING_IDENTITY='Developer ID Application: ...' pnpm build:macos-release ``` +Automatic understanding uses `keychain get --no-auth-ui`. If macOS requires +authorization, the run is skipped without showing UI and further scheduled +Keychain reads for that Provider remain blocked. “立即识别” is an explicit, +interactive operation with a 60-second helper timeout. A successful manual read +caches the credential in daemon memory, clears the block, and lets later +scheduled runs use the cache without invoking the helper again. Set and delete +operations update the same cache; plugin stop or daemon restart clears it. + The artifact is `native/macos-capture/build/EchoLogScreenCapture.app`. The Web “测试截图” and “立即识别” actions invoke that app identity through macOS LaunchServices (`/usr/bin/open -W -n ... --args`) and localhost-only routes. The daemon exclusively pre-creates mode `0600` stdout/stderr files inside one mode `0700` private temporary directory, validates the PNG, sends it to the selected OpenAI-compatible vision endpoint, and deletes the directory in `finally`. -Keychain operations execute the inner helper directly; API keys never enter the -database, argv, logs, or API responses. Successful runs persist only structured +Keychain operations execute the inner helper directly. Provider listing and +ordinary page loads report only the daemon's cached key state and never probe +Keychain. API keys never enter the database, argv, logs, or API responses. +Successful runs persist only structured summary/activity/apps/confidence metadata, with displayed field values required to be in Simplified Chinese. The one-shot capture process has its own 12-second hard watchdog, shorter than the daemon's 15-second request timeout. Never invoke `request-permission` from the daemon. +The `enabled` setting controls periodic scheduling. The loopback-only +“立即识别” action remains available as an explicit one-off run while scheduling +is disabled, so a user can authorize and warm the credential cache before +enabling automatic understanding. + The periodic understanding job checks the database-backed interval every five seconds. This keeps a configured 120-second interval close to two minutes even when the setting changes while the daemon is running, instead of rounding it up diff --git a/plugins/screen-time/native/macos-capture/Sources/EchoLogScreenCaptureCore/Contract.swift b/plugins/screen-time/native/macos-capture/Sources/EchoLogScreenCaptureCore/Contract.swift index ea074cf..9ff045b 100644 --- a/plugins/screen-time/native/macos-capture/Sources/EchoLogScreenCaptureCore/Contract.swift +++ b/plugins/screen-time/native/macos-capture/Sources/EchoLogScreenCaptureCore/Contract.swift @@ -1,5 +1,6 @@ import Foundation import Darwin +import Security public let helperVersion = "0.1.0" public let helperBundleIdentifier = "com.cubeplus1.echolog.screen-capture" @@ -9,8 +10,8 @@ public enum HelperCommand: Equatable, Sendable { case status case requestPermission case capture(output: String, maxPixelEdge: Int) - case keychainStatus(service: String, account: String) - case keychainGet(service: String, account: String) + case keychainStatus(service: String, account: String, noAuthUI: Bool) + case keychainGet(service: String, account: String, noAuthUI: Bool) case keychainSet(service: String, account: String) case keychainDelete(service: String, account: String) case version @@ -106,7 +107,19 @@ public enum CommandParser { guard let operation = args.first, ["status", "get", "set", "delete"].contains(operation) else { throw HelperFailure.invalid("keychain requires status, get, set, or delete") } - let options = try parseOptions(Array(args.dropFirst()), allowed: ["--service", "--account"]) + let rawOptions = Array(args.dropFirst()) + let noAuthUICount = rawOptions.filter { $0 == "--no-auth-ui" }.count + guard noAuthUICount <= 1 else { + throw HelperFailure.keychainInvalid("Duplicate option: --no-auth-ui") + } + let noAuthUI = noAuthUICount == 1 + if noAuthUI && operation != "status" && operation != "get" { + throw HelperFailure.keychainInvalid("--no-auth-ui is only valid for keychain status or get") + } + let options = try parseOptions( + rawOptions.filter { $0 != "--no-auth-ui" }, + allowed: ["--service", "--account"] + ) guard let service = options["--service"], service == screenUnderstandingKeychainService else { throw HelperFailure.keychainInvalid("--service must use the EchoLog screen-understanding namespace") } @@ -114,8 +127,8 @@ public enum CommandParser { throw HelperFailure.keychainInvalid("--account must be 1-255 printable characters") } switch operation { - case "status": return .keychainStatus(service: service, account: account) - case "get": return .keychainGet(service: service, account: account) + case "status": return .keychainStatus(service: service, account: account, noAuthUI: noAuthUI) + case "get": return .keychainGet(service: service, account: account, noAuthUI: noAuthUI) case "set": return .keychainSet(service: service, account: account) default: return .keychainDelete(service: service, account: account) } @@ -208,10 +221,13 @@ public struct SecretInput: Decodable, Sendable { } public enum KeychainStatusMapping { - public enum Result: Equatable { case present, missing, failure } + public enum Result: Equatable { case present, missing, authRequired, failure } public static func map(_ status: Int32) -> Result { - if status == 0 { return .present } - if status == -25300 { return .missing } + if status == errSecSuccess { return .present } + if status == errSecItemNotFound { return .missing } + if [errSecInteractionNotAllowed, errSecAuthFailed, errSecUserCanceled].contains(status) { + return .authRequired + } return .failure } } diff --git a/plugins/screen-time/native/macos-capture/Sources/EchoLogScreenCaptureCore/Runner.swift b/plugins/screen-time/native/macos-capture/Sources/EchoLogScreenCaptureCore/Runner.swift index c3fb15c..e007498 100644 --- a/plugins/screen-time/native/macos-capture/Sources/EchoLogScreenCaptureCore/Runner.swift +++ b/plugins/screen-time/native/macos-capture/Sources/EchoLogScreenCaptureCore/Runner.swift @@ -36,10 +36,10 @@ public struct HelperRunner: Sendable { "bytes": result.bytes, "capturedAt": result.capturedAt, ]) - case let .keychainStatus(service, account): - return try JSONOutput.success(["hasSecret": try keychain.hasSecret(service: service, account: account)]) - case let .keychainGet(service, account): - if let secret = try keychain.getSecret(service: service, account: account) { + case let .keychainStatus(service, account, noAuthUI): + return try JSONOutput.success(["hasSecret": try keychain.hasSecret(service: service, account: account, noAuthUI: noAuthUI)]) + case let .keychainGet(service, account, noAuthUI): + if let secret = try keychain.getSecret(service: service, account: account, noAuthUI: noAuthUI) { return try JSONOutput.success(["hasSecret": true, "secret": secret]) } return try JSONOutput.success(["hasSecret": false]) diff --git a/plugins/screen-time/native/macos-capture/Sources/EchoLogScreenCaptureCore/SystemAdapters.swift b/plugins/screen-time/native/macos-capture/Sources/EchoLogScreenCaptureCore/SystemAdapters.swift index dbd3cc8..c073316 100644 --- a/plugins/screen-time/native/macos-capture/Sources/EchoLogScreenCaptureCore/SystemAdapters.swift +++ b/plugins/screen-time/native/macos-capture/Sources/EchoLogScreenCaptureCore/SystemAdapters.swift @@ -16,34 +16,38 @@ public struct PermissionAdapter: Sendable { public struct KeychainAdapter: Sendable { public init() {} - public func hasSecret(service: String, account: String) throws -> Bool { - let query: [CFString: Any] = [ + public func hasSecret(service: String, account: String, noAuthUI: Bool = false) throws -> Bool { + var query: [CFString: Any] = [ kSecClass: kSecClassGenericPassword, kSecAttrService: service, kSecAttrAccount: account, kSecReturnData: false, kSecMatchLimit: kSecMatchLimitOne, ] + if noAuthUI { query[kSecUseAuthenticationUI] = kSecUseAuthenticationUIFail } let status = SecItemCopyMatching(query as CFDictionary, nil) switch KeychainStatusMapping.map(status) { case .present: return true case .missing: return false + case .authRequired: throw keychainAuthRequired(status: status) case .failure: throw keychainFailure("Unable to query Keychain", status: status) } } - public func getSecret(service: String, account: String) throws -> String? { - let query: [CFString: Any] = [ + public func getSecret(service: String, account: String, noAuthUI: Bool = false) throws -> String? { + var query: [CFString: Any] = [ kSecClass: kSecClassGenericPassword, kSecAttrService: service, kSecAttrAccount: account, kSecReturnData: true, kSecMatchLimit: kSecMatchLimitOne, ] + if noAuthUI { query[kSecUseAuthenticationUI] = kSecUseAuthenticationUIFail } var item: CFTypeRef? let status = SecItemCopyMatching(query as CFDictionary, &item) switch KeychainStatusMapping.map(status) { case .missing: return nil + case .authRequired: throw keychainAuthRequired(status: status) case .failure: throw keychainFailure("Unable to read Keychain item", status: status) case .present: guard let data = item as? Data, let value = String(data: data, encoding: .utf8) else { @@ -117,6 +121,16 @@ public struct KeychainAdapter: Sendable { systemCode: Int(status) ) } + + private func keychainAuthRequired(status: OSStatus) -> HelperFailure { + HelperFailure( + "Keychain authorization is required", + code: "KEYCHAIN_AUTH_REQUIRED", + exitCode: 10, + systemDomain: NSOSStatusErrorDomain, + systemCode: Int(status) + ) + } } public struct CaptureResult: Sendable { diff --git a/plugins/screen-time/native/macos-capture/Tests/EchoLogScreenCaptureCoreTests/ContractTests.swift b/plugins/screen-time/native/macos-capture/Tests/EchoLogScreenCaptureCoreTests/ContractTests.swift index aaa3aaa..f30b4e4 100644 --- a/plugins/screen-time/native/macos-capture/Tests/EchoLogScreenCaptureCoreTests/ContractTests.swift +++ b/plugins/screen-time/native/macos-capture/Tests/EchoLogScreenCaptureCoreTests/ContractTests.swift @@ -1,5 +1,6 @@ import Foundation import Darwin +import Security import XCTest @testable import EchoLogScreenCaptureCore @@ -34,7 +35,8 @@ final class ContractTests: XCTestCase { ]), .keychainStatus( service: "com.cubeplus1.echolog.screen-understanding", - account: "vision-primary" + account: "vision-primary", + noAuthUI: false ) ) XCTAssertEqual( @@ -44,7 +46,30 @@ final class ContractTests: XCTestCase { ]), .keychainGet( service: "com.cubeplus1.echolog.screen-understanding", - account: "vision-primary" + account: "vision-primary", + noAuthUI: false + ) + ) + XCTAssertEqual( + try CommandParser.parse([ + "keychain", "status", "--service", "com.cubeplus1.echolog.screen-understanding", + "--account", "vision-primary", "--no-auth-ui", "--json", + ]), + .keychainStatus( + service: "com.cubeplus1.echolog.screen-understanding", + account: "vision-primary", + noAuthUI: true + ) + ) + XCTAssertEqual( + try CommandParser.parse([ + "keychain", "get", "--service", "com.cubeplus1.echolog.screen-understanding", + "--account", "vision-primary", "--no-auth-ui", "--json", + ]), + .keychainGet( + service: "com.cubeplus1.echolog.screen-understanding", + account: "vision-primary", + noAuthUI: true ) ) XCTAssertThrowsError(try CommandParser.parse([ @@ -53,6 +78,12 @@ final class ContractTests: XCTestCase { ])) { error in XCTAssertEqual((error as? HelperFailure)?.code, "KEYCHAIN_INVALID_ARGUMENT") } + XCTAssertThrowsError(try CommandParser.parse([ + "keychain", "set", "--service", "com.cubeplus1.echolog.screen-understanding", + "--account", "vision-primary", "--no-auth-ui", "--json", + ])) { error in + XCTAssertEqual((error as? HelperFailure)?.code, "KEYCHAIN_INVALID_ARGUMENT") + } } func testSecretInputIsStrictAndBounded() throws { @@ -67,6 +98,9 @@ final class ContractTests: XCTestCase { func testKeychainStatusMapping() { XCTAssertEqual(KeychainStatusMapping.map(0), .present) XCTAssertEqual(KeychainStatusMapping.map(-25300), .missing) + XCTAssertEqual(KeychainStatusMapping.map(errSecInteractionNotAllowed), .authRequired) + XCTAssertEqual(KeychainStatusMapping.map(errSecAuthFailed), .authRequired) + XCTAssertEqual(KeychainStatusMapping.map(errSecUserCanceled), .authRequired) XCTAssertEqual(KeychainStatusMapping.map(-25291), .failure) } diff --git a/plugins/screen-time/src/index.ts b/plugins/screen-time/src/index.ts index 5a85f19..c963e20 100644 --- a/plugins/screen-time/src/index.ts +++ b/plugins/screen-time/src/index.ts @@ -7,7 +7,10 @@ import { MacKeychainClient } from "./macos-keychain-client.js"; import { MacScreenCaptureService, } from "./macos-screen-capture.js"; -import { validateMacosHelperExecutableOverride } from "./macos-helper.js"; +import { + checkMacosHelperInstall, + validateMacosHelperExecutableOverride, +} from "./macos-helper.js"; import { ProviderProfileService } from "./provider-profiles.js"; import { createScreenRoutes } from "./routes.js"; import { ScreenService } from "./screen.js"; @@ -273,6 +276,19 @@ export const screenTimePlugin: PluginDefinition = { }); }, start(context) { + if (process.platform === "darwin") { + const helperPath = typeof context.config.macos_helper_path === "string" + ? context.config.macos_helper_path + : undefined; + const install = checkMacosHelperInstall(helperPath); + if (!install.ok) { + context.logger.warn({ + appBundle: install.appBundle, + executable: install.executable, + buildCommand: install.buildCommand, + }, install.message); + } + } context.logger.info( { sampleSeconds: tracker?.config.sampleSeconds }, "Screen Time plugin started" @@ -280,6 +296,7 @@ export const screenTimePlugin: PluginDefinition = { }, async stop() { await tracker?.stop(); + providerProfiles?.clearCredentialCache(); await store?.close(); tracker = null; service = null; diff --git a/plugins/screen-time/src/macos-helper.ts b/plugins/screen-time/src/macos-helper.ts index 268483c..9803b44 100644 --- a/plugins/screen-time/src/macos-helper.ts +++ b/plugins/screen-time/src/macos-helper.ts @@ -1,13 +1,24 @@ +import { constants, accessSync, statSync } from "node:fs"; import { dirname, isAbsolute } from "node:path"; import { fileURLToPath } from "node:url"; const INNER_EXECUTABLE = "../native/macos-capture/build/EchoLogScreenCapture.app/Contents/MacOS/echolog-screen-capture"; +export const MACOS_HELPER_BUILD_COMMAND = + "ECHOLOG_MACOS_ADHOC_SMOKE=1 pnpm build:macos-capture"; export const DEFAULT_MACOS_HELPER_EXECUTABLE = fileURLToPath( new URL(INNER_EXECUTABLE, import.meta.url) ); +export interface MacosHelperInstallCheck { + ok: boolean; + appBundle: string; + executable: string; + buildCommand: string; + message: string; +} + export function validateMacosHelperExecutableOverride( value: unknown ): string | null { @@ -39,3 +50,53 @@ export function resolveMacosHelperApp(override?: unknown): string { const executable = resolveMacosHelperExecutable(override); return dirname(dirname(dirname(executable))); } + +export function checkMacosHelperInstall(override?: unknown): MacosHelperInstallCheck { + const error = validateMacosHelperExecutableOverride(override); + const executable = error + ? typeof override === "string" + ? override + : DEFAULT_MACOS_HELPER_EXECUTABLE + : resolveMacosHelperExecutable(override); + const appBundle = dirname(dirname(dirname(executable))); + const base = { appBundle, executable, buildCommand: MACOS_HELPER_BUILD_COMMAND }; + if (error) { + return { ...base, ok: false, message: error }; + } + try { + if (!statSync(appBundle).isDirectory()) { + return { + ...base, + ok: false, + message: `EchoLogScreenCapture.app is not a directory at ${appBundle}; run ${MACOS_HELPER_BUILD_COMMAND}`, + }; + } + } catch { + return { + ...base, + ok: false, + message: `EchoLogScreenCapture.app is missing at ${appBundle}; run ${MACOS_HELPER_BUILD_COMMAND}`, + }; + } + try { + if (!statSync(executable).isFile()) { + return { + ...base, + ok: false, + message: `screen capture helper executable is not a file at ${executable}; run ${MACOS_HELPER_BUILD_COMMAND}`, + }; + } + accessSync(executable, constants.X_OK); + } catch { + return { + ...base, + ok: false, + message: `screen capture helper executable is not runnable at ${executable}; run ${MACOS_HELPER_BUILD_COMMAND}`, + }; + } + return { + ...base, + ok: true, + message: `EchoLogScreenCapture.app is installed at ${appBundle}`, + }; +} diff --git a/plugins/screen-time/src/macos-keychain-client.ts b/plugins/screen-time/src/macos-keychain-client.ts index 3983aa2..1dd2d8e 100644 --- a/plugins/screen-time/src/macos-keychain-client.ts +++ b/plugins/screen-time/src/macos-keychain-client.ts @@ -4,19 +4,25 @@ import type { } from "@echolog/plugin-sdk"; import { ProviderError, + type ProviderSecretAccess, type ProviderSecretStore, } from "./provider-profiles.js"; -import { resolveMacosHelperExecutable } from "./macos-helper.js"; +import { + checkMacosHelperInstall, + resolveMacosHelperExecutable, +} from "./macos-helper.js"; export const SCREEN_UNDERSTANDING_KEYCHAIN_SERVICE = "com.cubeplus1.echolog.screen-understanding"; +export const INTERACTIVE_KEYCHAIN_TIMEOUT_MS = 60_000; +export const NON_INTERACTIVE_KEYCHAIN_TIMEOUT_MS = 5_000; export type KeychainCommandAdapter = ( request: PluginCommandRequest, signal?: AbortSignal ) => Promise; -function helperFailure(error: unknown): ProviderError { +function helperFailure(error: unknown, executable: string): ProviderError { if (error instanceof ProviderError) return error; const candidate = error as { code?: unknown; @@ -33,7 +39,39 @@ function helperFailure(error: unknown): ProviderError { return new ProviderError("PLUGIN_TIMEOUT", "Keychain helper timed out", 504); } if (candidate?.code === "ENOENT") { - return new ProviderError("KEYCHAIN_UNAVAILABLE", "Keychain helper is unavailable", 503); + const check = checkMacosHelperInstall(executable); + return new ProviderError( + "KEYCHAIN_UNAVAILABLE", + check.ok ? `Keychain helper could not be launched at ${executable}` : check.message, + 503 + ); + } + return new ProviderError("KEYCHAIN_OPERATION_FAILED", "Keychain operation failed", 502); +} + +function helperResultFailure(result: PluginCommandResult): ProviderError { + let value: unknown; + try { + value = JSON.parse(result.stdout); + } catch { + return new ProviderError("KEYCHAIN_OPERATION_FAILED", "Keychain operation failed", 502); + } + if (!value || typeof value !== "object" || Array.isArray(value)) { + return new ProviderError("KEYCHAIN_OPERATION_FAILED", "Keychain operation failed", 502); + } + const object = value as Record; + if (object.ok !== false || typeof object.code !== "string") { + return new ProviderError("KEYCHAIN_OPERATION_FAILED", "Keychain operation failed", 502); + } + if (object.code === "KEYCHAIN_AUTH_REQUIRED") { + return new ProviderError( + "KEYCHAIN_AUTH_REQUIRED", + "Keychain authorization is required", + 409 + ); + } + if (object.code === "KEYCHAIN_UNAVAILABLE") { + return new ProviderError("KEYCHAIN_UNAVAILABLE", "Keychain is unavailable", 503); } return new ProviderError("KEYCHAIN_OPERATION_FAILED", "Keychain operation failed", 502); } @@ -43,7 +81,7 @@ function parseResponse( expectedSecretState: boolean | null ): boolean { if (result.exitCode !== 0) { - throw new ProviderError("KEYCHAIN_OPERATION_FAILED", "Keychain operation failed", 502); + throw helperResultFailure(result); } let value: unknown; try { @@ -68,7 +106,7 @@ function parseResponse( function parseSecret(result: PluginCommandResult): string | null { if (result.exitCode !== 0) { - throw new ProviderError("KEYCHAIN_OPERATION_FAILED", "Keychain operation failed", 502); + throw helperResultFailure(result); } let value: unknown; try { @@ -104,6 +142,8 @@ function parseSecret(result: PluginCommandResult): string | null { export class MacKeychainClient implements ProviderSecretStore { private readonly executable: string; + private readonly cache = new Map(); + private readonly knownSecretState = new Map(); constructor( private readonly exec: KeychainCommandAdapter, @@ -112,12 +152,34 @@ export class MacKeychainClient implements ProviderSecretStore { this.executable = resolveMacosHelperExecutable(executableOverride); } - async has(id: string, signal?: AbortSignal): Promise { + cachedState(id: string): boolean | null { + if (this.cache.has(id)) return true; + return this.knownSecretState.get(id) ?? null; + } + + hasCachedValue(id: string): boolean { + return this.cache.has(id); + } + + clearCache(): void { + this.cache.clear(); + this.knownSecretState.clear(); + } + + async has( + id: string, + signal?: AbortSignal, + access: ProviderSecretAccess = "non-interactive" + ): Promise { + const cached = this.cachedState(id); + if (cached !== null) return cached; try { - const result = await this.run("status", id, undefined, signal); - return parseResponse(result, null); + const result = await this.run("status", id, undefined, access, signal); + const hasSecret = parseResponse(result, null); + this.knownSecretState.set(id, hasSecret); + return hasSecret; } catch (error) { - throw helperFailure(error); + throw helperFailure(error, this.executable); } } @@ -127,28 +189,49 @@ export class MacKeychainClient implements ProviderSecretStore { "set", id, JSON.stringify({ secret: value }), + "interactive", signal ); parseResponse(result, true); + this.cache.set(id, value); + this.knownSecretState.set(id, true); } catch (error) { - throw helperFailure(error); + throw helperFailure(error, this.executable); } } - async get(id: string, signal?: AbortSignal): Promise { + async get( + id: string, + signal?: AbortSignal, + access: ProviderSecretAccess = "interactive" + ): Promise { + const cached = this.cache.get(id); + if (cached !== undefined) return cached; + if (access === "non-interactive" && this.knownSecretState.get(id) === false) { + return null; + } try { - return parseSecret(await this.run("get", id, undefined, signal)); + const secret = parseSecret(await this.run("get", id, undefined, access, signal)); + if (secret === null) { + this.knownSecretState.set(id, false); + } else { + this.cache.set(id, secret); + this.knownSecretState.set(id, true); + } + return secret; } catch (error) { - throw helperFailure(error); + throw helperFailure(error, this.executable); } } async delete(id: string, signal?: AbortSignal): Promise { try { - const result = await this.run("delete", id, undefined, signal); + const result = await this.run("delete", id, undefined, "interactive", signal); parseResponse(result, false); + this.cache.delete(id); + this.knownSecretState.set(id, false); } catch (error) { - throw helperFailure(error); + throw helperFailure(error, this.executable); } } @@ -156,8 +239,17 @@ export class MacKeychainClient implements ProviderSecretStore { operation: "status" | "get" | "set" | "delete", id: string, stdin: string | undefined, + access: ProviderSecretAccess, signal?: AbortSignal ): Promise { + const check = checkMacosHelperInstall(this.executable); + if (!check.ok) { + return Promise.reject(new ProviderError( + "KEYCHAIN_UNAVAILABLE", + check.message, + 503 + )); + } return this.exec({ executable: this.executable, args: [ @@ -167,9 +259,14 @@ export class MacKeychainClient implements ProviderSecretStore { SCREEN_UNDERSTANDING_KEYCHAIN_SERVICE, "--account", id, + ...(access === "non-interactive" && (operation === "status" || operation === "get") + ? ["--no-auth-ui"] + : []), "--json", ], - timeoutMs: 5_000, + timeoutMs: access === "interactive" + ? INTERACTIVE_KEYCHAIN_TIMEOUT_MS + : NON_INTERACTIVE_KEYCHAIN_TIMEOUT_MS, maxBufferBytes: 16 * 1024, ...(stdin === undefined ? {} : { stdin }), }, signal); diff --git a/plugins/screen-time/src/macos-screen-capture.ts b/plugins/screen-time/src/macos-screen-capture.ts index 471752e..8e1bbc1 100644 --- a/plugins/screen-time/src/macos-screen-capture.ts +++ b/plugins/screen-time/src/macos-screen-capture.ts @@ -17,6 +17,8 @@ import type { PluginDoctorCheck, } from "@echolog/plugin-sdk"; import { + checkMacosHelperInstall, + resolveMacosHelperExecutable, resolveMacosHelperApp, } from "./macos-helper.js"; @@ -303,6 +305,7 @@ async function validatePng( } export class MacScreenCaptureService { + private readonly executable: string; private readonly appBundle: string; private captureInFlight = false; @@ -310,7 +313,8 @@ export class MacScreenCaptureService { private readonly exec: CaptureCommandAdapter, executableOverride?: string ) { - this.appBundle = resolveMacosHelperApp(executableOverride); + this.executable = resolveMacosHelperExecutable(executableOverride); + this.appBundle = resolveMacosHelperApp(this.executable); } private async invokeViaLaunchServices( @@ -319,6 +323,10 @@ export class MacScreenCaptureService { timeoutMs: number, signal?: AbortSignal ): Promise { + const check = checkMacosHelperInstall(this.executable); + if (!check.ok) { + throw new CaptureError("PLUGIN_EXEC_FAILED", check.message, 502); + } const resultPath = resolve(directory, "helper-result.json"); const stderrPath = resolve(directory, "helper-stderr.log"); const deadlineMs = Date.now() + timeoutMs; @@ -430,6 +438,20 @@ export class MacScreenCaptureService { async doctor(signal?: AbortSignal): Promise { let createdDirectory: string | null = null; try { + const install = checkMacosHelperInstall(this.executable); + if (!install.ok) { + return [{ + id: "screen-understanding:helper", + ok: false, + message: install.message, + details: { + appBundle: install.appBundle, + executable: install.executable, + buildCommand: install.buildCommand, + launchMethod: "LaunchServices", + }, + }]; + } createdDirectory = await mkdtemp(join(tmpdir(), "echolog-screen-understanding-")); const temporaryDirectory = await realpath(createdDirectory); await chmod(temporaryDirectory, 0o700); diff --git a/plugins/screen-time/src/provider-profiles.ts b/plugins/screen-time/src/provider-profiles.ts index 448637b..be32a6d 100644 --- a/plugins/screen-time/src/provider-profiles.ts +++ b/plugins/screen-time/src/provider-profiles.ts @@ -18,6 +18,7 @@ export type ProviderErrorCode = | "PROVIDER_PROFILE_IN_USE" | "PROVIDER_PROFILE_LIMIT" | "PROVIDER_KEY_REQUIRED" + | "KEYCHAIN_AUTH_REQUIRED" | "KEYCHAIN_UNAVAILABLE" | "KEYCHAIN_OPERATION_FAILED" | "PLUGIN_TIMEOUT"; @@ -54,12 +55,25 @@ export interface ProviderProfileStore { } export interface ProviderSecretStore { - has(id: string, signal?: AbortSignal): Promise; - get?(id: string, signal?: AbortSignal): Promise; + has( + id: string, + signal?: AbortSignal, + access?: ProviderSecretAccess + ): Promise; + get?( + id: string, + signal?: AbortSignal, + access?: ProviderSecretAccess + ): Promise; set(id: string, value: string, signal?: AbortSignal): Promise; delete(id: string, signal?: AbortSignal): Promise; + cachedState?(id: string): boolean | null; + hasCachedValue?(id: string): boolean; + clearCache?(): void; } +export type ProviderSecretAccess = "interactive" | "non-interactive"; + export interface ProviderProfile { id: string; version: number; @@ -226,23 +240,6 @@ function serializeProfile( }; } -async function mapWithConcurrency( - values: T[], - concurrency: number, - mapper: (value: T) => Promise -): Promise { - const results = new Array(values.length); - let next = 0; - const workers = Array.from({ length: Math.min(concurrency, values.length) }, async () => { - while (next < values.length) { - const index = next++; - results[index] = await mapper(values[index]); - } - }); - await Promise.all(workers); - return results; -} - export class ProviderProfileService { private readonly locks = new Map>(); @@ -268,13 +265,7 @@ export class ProviderProfileService { async list(signal?: AbortSignal): Promise { const rows = await this.store.listProviderProfiles(); - return mapWithConcurrency(rows, 4, async (row) => { - try { - return serializeProfile(row, await this.secrets.has(row.id, signal)); - } catch { - return serializeProfile(row, null); - } - }); + return rows.map((row) => serializeProfile(row, this.cachedSecretState(row.id))); } async create(input: ProviderProfileMetadataInput): Promise { @@ -306,12 +297,7 @@ export class ProviderProfileService { existing.version ); } - let hasApiKey: boolean | null = null; - try { - hasApiKey = await this.secrets.has(id, signal); - } catch { - // Metadata remains editable when the native Keychain helper is down. - } + const hasApiKey = this.cachedSecretState(id); const updated = await this.store.updateProviderProfile(id, expectedVersion, input); if (!updated) await this.throwMissingOrConflict(id); return serializeProfile(updated!, hasApiKey); @@ -375,7 +361,11 @@ export class ProviderProfileService { ): Promise { return this.exclusive(id, async () => { await this.requireProfile(id); - if (requireKey && !(await this.secrets.has(id, signal))) { + if ( + requireKey && + this.cachedSecretState(id) !== true && + !(await this.secrets.has(id, signal, "non-interactive")) + ) { throw new ProviderError("PROVIDER_KEY_REQUIRED", `provider profile ${id} requires an API key`, 409); } return operation(); @@ -384,7 +374,8 @@ export class ProviderProfileService { async getForInference( id: string, - signal?: AbortSignal + signal?: AbortSignal, + access: ProviderSecretAccess = "interactive" ): Promise<{ profile: ProviderProfile; apiKey: string }> { return this.exclusive(id, async () => { const row = await this.requireProfile(id); @@ -395,7 +386,7 @@ export class ProviderProfileService { 503 ); } - const apiKey = await this.secrets.get(id, signal); + const apiKey = await this.secrets.get(id, signal, access); if (!apiKey) { throw new ProviderError( "PROVIDER_KEY_REQUIRED", @@ -410,6 +401,18 @@ export class ProviderProfileService { }); } + hasCachedCredential(id: string): boolean { + return this.secrets.hasCachedValue?.(id) ?? false; + } + + clearCredentialCache(): void { + this.secrets.clearCache?.(); + } + + private cachedSecretState(id: string): boolean | null { + return this.secrets.cachedState?.(id) ?? null; + } + private async requireProfile(id: string): Promise { const row = await this.store.getProviderProfile(id); if (!row) { diff --git a/plugins/screen-time/src/understanding.ts b/plugins/screen-time/src/understanding.ts index 66135b3..39e3dba 100644 --- a/plugins/screen-time/src/understanding.ts +++ b/plugins/screen-time/src/understanding.ts @@ -2,7 +2,11 @@ import { nanoid } from "nanoid"; import { setTimeout as delay } from "node:timers/promises"; import type { ScreenUnderstandingObservation } from "./schema.js"; import type { UnderstandingSettingsService } from "./understanding-settings.js"; -import type { ProviderProfile } from "./provider-profiles.js"; +import { + ProviderError, + type ProviderProfile, + type ProviderSecretAccess, +} from "./provider-profiles.js"; import type { CapturedPng, MacScreenCaptureService } from "./macos-screen-capture.js"; import type { VisionCompletion, VisionProviderClient } from "./vision-provider.js"; import { VisionProviderError } from "./vision-provider.js"; @@ -10,6 +14,13 @@ import { VisionProviderError } from "./vision-provider.js"; const MAX_HISTORY_LIMIT = 100; const RETRY_BASE_DELAY_MS = 250; +function isSilentScheduledCredentialFailure(error: unknown): boolean { + return error instanceof ProviderError && [ + "KEYCHAIN_AUTH_REQUIRED", + "PROVIDER_KEY_REQUIRED", + ].includes(error.code); +} + export interface UnderstandingResult { summary: string; activity: string; @@ -88,8 +99,10 @@ export interface UnderstandingStore { export interface UnderstandingProviderResolver { getForInference( id: string, - signal?: AbortSignal + signal?: AbortSignal, + access?: ProviderSecretAccess ): Promise<{ profile: ProviderProfile; apiKey: string }>; + hasCachedCredential?(id: string): boolean; } export interface UnderstandingCapture { @@ -238,6 +251,7 @@ function cancellationError(): Error { export class ScreenUnderstandingService { private inFlight = false; private lastScheduledAt = 0; + private scheduledKeychainBlockedFor: string | null = null; constructor( private readonly store: UnderstandingStore, @@ -263,7 +277,7 @@ export class ScreenUnderstandingService { this.inFlight = true; try { const configuration = await this.settings.get(); - if (!configuration.enabled) { + if (!configuration.enabled && options.scheduled) { throw new UnderstandingError( "UNDERSTANDING_DISABLED", "Screen understanding is disabled", @@ -288,8 +302,10 @@ export class ScreenUnderstandingService { const resolved = await this.providers.getForInference( configuration.providerProfileId, - signal + signal, + options.scheduled ? "non-interactive" : "interactive" ); + if (!options.scheduled) this.scheduledKeychainBlockedFor = null; const captured = await this.capture.captureForInference(signal); const capturedAt = new Date(captured.capturedAt); if (!Number.isFinite(capturedAt.getTime())) { @@ -406,6 +422,17 @@ export class ScreenUnderstandingService { if (this.inFlight) return null; const configuration = await this.settings.get(); if (!configuration.enabled) return null; + if ( + this.scheduledKeychainBlockedFor !== null && + this.scheduledKeychainBlockedFor === configuration.providerProfileId + ) { + if (!this.providers.hasCachedCredential?.(this.scheduledKeychainBlockedFor)) { + return null; + } + this.scheduledKeychainBlockedFor = null; + } else if (this.scheduledKeychainBlockedFor !== null) { + this.scheduledKeychainBlockedFor = null; + } const now = this.now(); if ( this.lastScheduledAt > 0 && @@ -415,6 +442,13 @@ export class ScreenUnderstandingService { try { return await this.run(signal, { scheduled: true }); } catch (error) { + if ( + configuration.providerProfileId && + isSilentScheduledCredentialFailure(error) + ) { + this.scheduledKeychainBlockedFor = configuration.providerProfileId; + } + if (isSilentScheduledCredentialFailure(error)) return null; if ( error instanceof UnderstandingError && error.code === "UNDERSTANDING_DISABLED" diff --git a/plugins/screen-time/web/index.js b/plugins/screen-time/web/index.js index d08fc74..bf5ad3f 100644 --- a/plugins/screen-time/web/index.js +++ b/plugins/screen-time/web/index.js @@ -161,14 +161,14 @@ export async function activate({ api }) { - +
${esc(profile.id)} · v${esc(profile.version)} - - ${profile.hasApiKey ? `` : ""} + + ${profile.hasApiKey !== false ? `` : ""}
` diff --git a/scripts/package-release.sh b/scripts/package-release.sh index bf63849..0db93a4 100755 --- a/scripts/package-release.sh +++ b/scripts/package-release.sh @@ -123,6 +123,7 @@ minimum_os="$(/usr/libexec/PlistBuddy -c 'Print :LSMinimumSystemVersion' "$app_p status_json="$("$app_executable" status --json)" STATUS_JSON="$status_json" node -e 'const value = JSON.parse(process.env.STATUS_JSON); if (value.ok !== true || value.command !== "status" || value.bundleIdentifier !== "com.cubeplus1.echolog.screen-capture" || !["granted", "request-needed"].includes(value.permission)) process.exit(1);' +bash scripts/smoke-macos-helper.sh --root "$bundle_root" # Keep the ready-to-run bundle portable: production runtime dependencies stay, # while pnpm's machine-local install metadata and dev-only toolchain are removed. diff --git a/scripts/smoke-macos-helper.sh b/scripts/smoke-macos-helper.sh new file mode 100755 index 0000000..27f5840 --- /dev/null +++ b/scripts/smoke-macos-helper.sh @@ -0,0 +1,76 @@ +#!/bin/bash +set -euo pipefail + +repo_root="$(cd "$(dirname "$0")/.." && pwd)" +target_root="$repo_root" +launchd_plist="" + +usage() { + cat >&2 <<'EOF' +usage: scripts/smoke-macos-helper.sh [--root DIR] [--launchd [PLIST]] + +Verifies the packaged EchoLogScreenCapture.app at the runtime path used by +screen-time. With --launchd, reads WorkingDirectory from the daemon plist and +checks that installed runtime tree instead of this source checkout. +EOF + exit 2 +} + +while [[ $# -gt 0 ]]; do + case "$1" in + --root) + [[ $# -ge 2 ]] || usage + target_root="$2" + shift 2 + ;; + --launchd) + launchd_plist="${2:-$HOME/Library/LaunchAgents/com.echolog.daemon.plist}" + if [[ $# -ge 2 && "$2" != --* ]]; then + shift 2 + else + shift + fi + ;; + *) usage ;; + esac +done + +[[ "$(uname -s)" == "Darwin" ]] || { echo "macOS is required" >&2; exit 2; } + +if [[ -n "$launchd_plist" ]]; then + [[ -f "$launchd_plist" ]] || { echo "launchd plist not found: $launchd_plist" >&2; exit 1; } + target_root="$(/usr/libexec/PlistBuddy -c 'Print :WorkingDirectory' "$launchd_plist")" +fi + +target_root="$(cd "$target_root" && pwd)" +app="$target_root/plugins/screen-time/native/macos-capture/build/EchoLogScreenCapture.app" +executable="$app/Contents/MacOS/echolog-screen-capture" +build_command="ECHOLOG_MACOS_ADHOC_SMOKE=1 pnpm build:macos-capture" + +if [[ ! -d "$app" ]]; then + echo "EchoLogScreenCapture.app is missing at $app" >&2 + echo "build it with: (cd $target_root && $build_command)" >&2 + exit 1 +fi +if [[ ! -x "$executable" ]]; then + echo "screen capture helper executable is missing or not runnable at $executable" >&2 + echo "build it with: (cd $target_root && $build_command)" >&2 + exit 1 +fi + +/usr/bin/codesign --verify --deep --strict --verbose=2 "$app" >/dev/null +status_json="$("$executable" status --json)" +STATUS_JSON="$status_json" /usr/bin/python3 - <<'PY' +import json +import os + +value = json.loads(os.environ["STATUS_JSON"]) +assert value["ok"] is True +assert value["command"] == "status" +assert value["bundleIdentifier"] == "com.cubeplus1.echolog.screen-capture" +assert value["permission"] in ("granted", "request-needed") +PY + +echo "helper_root=$target_root" +echo "helper_app=$app" +echo "helper_status=$status_json" diff --git a/tests/screen-capture.test.ts b/tests/screen-capture.test.ts index 0cff984..7621d4d 100644 --- a/tests/screen-capture.test.ts +++ b/tests/screen-capture.test.ts @@ -1,6 +1,8 @@ import assert from "node:assert/strict"; +import { execFile } from "node:child_process"; import { chmod, + mkdir, mkdtemp, rm, stat, @@ -10,17 +12,42 @@ import { import { tmpdir } from "node:os"; import { dirname, join } from "node:path"; import test from "node:test"; +import { fileURLToPath } from "node:url"; import { CaptureError, MacScreenCaptureService, } from "../plugins/screen-time/src/macos-screen-capture.js"; import { + MACOS_HELPER_BUILD_COMMAND, DEFAULT_MACOS_HELPER_EXECUTABLE, + checkMacosHelperInstall, resolveMacosHelperExecutable, validateMacosHelperExecutableOverride, } from "../plugins/screen-time/src/macos-helper.js"; import { createScreenRoutes } from "../plugins/screen-time/src/routes.js"; +const repoRoot = dirname(dirname(fileURLToPath(import.meta.url))); +let fakeHelperRoot = ""; +let fakeHelperExecutable = ""; + +test.before(async () => { + fakeHelperRoot = await mkdtemp(join(tmpdir(), "echolog-capture-helper-")); + fakeHelperExecutable = join( + fakeHelperRoot, + "EchoLogScreenCapture.app", + "Contents", + "MacOS", + "echolog-screen-capture" + ); + await mkdir(dirname(fakeHelperExecutable), { recursive: true }); + await writeFile(fakeHelperExecutable, "#!/bin/sh\nexit 0\n"); + await chmod(fakeHelperExecutable, 0o755); +}); + +test.after(async () => { + await rm(fakeHelperRoot, { recursive: true, force: true }); +}); + const PNG_1X1 = Buffer.from( "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=", "base64" @@ -95,6 +122,70 @@ test("macOS helper resolution uses the packaged app inner executable and validat assert.throws(() => resolveMacosHelperExecutable("/tmp/wrong-helper")); }); +test("macOS helper install check reports a missing app bundle with the build command", async () => { + const root = await mkdtemp(join(tmpdir(), "echolog-missing-helper-")); + const executable = join( + root, + "EchoLogScreenCapture.app", + "Contents", + "MacOS", + "echolog-screen-capture" + ); + try { + const check = checkMacosHelperInstall(executable); + assert.equal(check.ok, false); + assert.equal(check.executable, executable); + assert.equal(check.buildCommand, MACOS_HELPER_BUILD_COMMAND); + assert.match(check.message, /EchoLogScreenCapture\.app is missing/); + assert.match(check.message, /pnpm build:macos-capture/); + + let invoked = false; + const service = new MacScreenCaptureService(async () => { + invoked = true; + throw new Error("must not invoke LaunchServices"); + }, executable); + const checks = await service.doctor(); + assert.equal(invoked, false); + assert.equal(checks[0]?.ok, false); + assert.equal(checks[0]?.details?.buildCommand, MACOS_HELPER_BUILD_COMMAND); + assert.match(checks[0]?.message ?? "", /EchoLogScreenCapture\.app is missing/); + } finally { + await rm(root, { recursive: true, force: true }); + } +}); + +test("macOS helper smoke script fails clearly when the runtime app bundle is missing", { + skip: process.platform !== "darwin", +}, async () => { + const root = await mkdtemp(join(tmpdir(), "echolog-helper-smoke-")); + try { + const result = await new Promise<{ + exitCode: number; + stdout: string; + stderr: string; + }>((resolve) => { + execFile( + join(repoRoot, "scripts/smoke-macos-helper.sh"), + ["--root", root], + (error, stdout, stderr) => { + resolve({ + exitCode: typeof error?.code === "number" ? error.code : 0, + stdout, + stderr, + }); + } + ); + }); + assert.equal(result.exitCode, 1); + assert.equal(result.stdout, ""); + assert.match(result.stderr, /EchoLogScreenCapture\.app is missing/); + assert.match(result.stderr, /pnpm build:macos-capture/); + assert.match(result.stderr, new RegExp(root.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"))); + } finally { + await rm(root, { recursive: true, force: true }); + } +}); + test("capture service returns a bounded in-memory PNG preview and removes the private directory", async () => { let capturedPath = ""; const service = new MacScreenCaptureService(async (request) => { @@ -105,14 +196,14 @@ test("capture service returns a bounded in-memory PNG preview and removes the pr assert.deepEqual(request.args.slice(0, 9), [ "-W", "-n", "-o", argumentValue(request.args, "-o"), "--stderr", argumentValue(request.args, "--stderr"), - DEFAULT_MACOS_HELPER_EXECUTABLE.replace("/Contents/MacOS/echolog-screen-capture", ""), + fakeHelperExecutable.replace("/Contents/MacOS/echolog-screen-capture", ""), "--args", "capture", ]); assert.equal(request.args.indexOf("-o") < request.args.indexOf("--args"), true); assert.equal(request.args.indexOf("--stderr") < request.args.indexOf("--args"), true); await writePrivatePng(capturedPath); return completeLaunch(request.args, success(capturedPath)); - }); + }, fakeHelperExecutable); const result = await service.captureTest(); assert.equal(result.preview.base64, PNG_1X1.toString("base64")); assert.equal(result.preview.mediaType, "image/png"); @@ -132,7 +223,7 @@ test("capture service rejects concurrent tests with a bounded busy response", as await gate; await writePrivatePng(outputPath); return completeLaunch(request.args, success(outputPath)); - }); + }, fakeHelperExecutable); const first = service.captureTest(); await started; await assert.rejects( @@ -217,7 +308,7 @@ test("capture service rejects malformed output, unsafe paths, symlinks, oversize try { for (const item of cases) { await t.test(item.name, async () => { - const service = new MacScreenCaptureService(item.run); + const service = new MacScreenCaptureService(item.run, fakeHelperExecutable); await assert.rejects( service.captureTest(), (error) => error instanceof CaptureError && error.code === item.code && @@ -242,7 +333,7 @@ test("capture service waits for an asynchronous LaunchServices result", async () void writeLaunchFiles(request.args, success(outputPath).stdout); }, 40); return { stdout: "ignored", stderr: "Unable to block", exitCode: 0 }; - }); + }, fakeHelperExecutable); const result = await service.captureTest(); assert.equal(result.preview.base64, PNG_1X1.toString("base64")); }); @@ -257,7 +348,7 @@ test("capture service bounds a missing LaunchServices result and cleans up", asy await chmod(stderrPath, 0o600); setTimeout(() => controller.abort(), 40); return { stdout: "ignored", stderr: "Unable to block", exitCode: 0 }; - }); + }, fakeHelperExecutable); await assert.rejects( service.captureTest(controller.signal), (error) => error instanceof CaptureError && error.code === "PLUGIN_TIMEOUT" @@ -272,7 +363,7 @@ test("capture service rejects an insecure LaunchServices result and cleans up", await writeLaunchFiles(request.args, JSON.stringify({ ok: true })); await chmod(argumentValue(request.args, "-o"), 0o644); return { stdout: "ignored", stderr: "Unable to block", exitCode: 0 }; - }); + }, fakeHelperExecutable); await assert.rejects( service.captureTest(), (error) => error instanceof CaptureError && error.code === "PLUGIN_OUTPUT_INVALID" @@ -308,7 +399,7 @@ test("capture helper errors are mapped safely and permission diagnostics stay no "private diagnostic" ); return { stdout: "ignored", stderr: "benign open diagnostic", exitCode: 0 }; - }); + }, fakeHelperExecutable); await assert.rejects( permission.captureTest(), (error) => error instanceof CaptureError && diff --git a/tests/screen-time.test.ts b/tests/screen-time.test.ts index e0f6b1c..3ea36df 100644 --- a/tests/screen-time.test.ts +++ b/tests/screen-time.test.ts @@ -1,13 +1,23 @@ import assert from "node:assert/strict"; import { existsSync } from "node:fs"; +import { chmod, mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; import test from "node:test"; import Fastify from "fastify"; import { screenTimePlugin } from "@echolog/plugin-screen-time"; import type { AppRule } from "../plugins/screen-time/src/schema.js"; import { classifySegment } from "../plugins/screen-time/src/screen.js"; import { createScreenRoutes } from "../plugins/screen-time/src/routes.js"; -import { MacKeychainClient } from "../plugins/screen-time/src/macos-keychain-client.js"; -import { DEFAULT_MACOS_HELPER_EXECUTABLE } from "../plugins/screen-time/src/macos-helper.js"; +import { + INTERACTIVE_KEYCHAIN_TIMEOUT_MS, + MacKeychainClient, + NON_INTERACTIVE_KEYCHAIN_TIMEOUT_MS, +} from "../plugins/screen-time/src/macos-keychain-client.js"; +import { + DEFAULT_MACOS_HELPER_EXECUTABLE, + MACOS_HELPER_BUILD_COMMAND, +} from "../plugins/screen-time/src/macos-helper.js"; import { ProviderError, ProviderProfileService, @@ -481,64 +491,158 @@ test("provider validators are strict and normalize safe base URLs", () => { assert.equal(validateProviderKey({ apiKey: "test-value" }).ok, true); }); -test("macOS Keychain client follows the write-only helper contract", async () => { - const requests: Array<{ args: string[]; stdin?: string }> = []; - const client = new MacKeychainClient(async (request) => { - requests.push({ args: request.args, stdin: request.stdin }); - const operation = request.args[1]; - return { - stdout: JSON.stringify({ - ok: true, - hasSecret: operation === "delete" ? false : true, - }), - stderr: "", - exitCode: 0, - }; - }, "/test/EchoLogScreenCapture.app/Contents/MacOS/echolog-screen-capture"); - assert.equal(await client.has("vision-primary"), true); - await client.set("vision-primary", "test-credential-value"); - await client.delete("vision-primary"); - assert.deepEqual(requests[0]?.args, [ - "keychain", - "status", - "--service", - "com.cubeplus1.echolog.screen-understanding", - "--account", - "vision-primary", - "--json", - ]); - assert.equal(requests[0]?.stdin, undefined); - assert.deepEqual(JSON.parse(requests[1]?.stdin ?? ""), { - secret: "test-credential-value", - }); - assert.equal(requests[1]?.args.join(" ").includes("test-credential-value"), false); - - const failed = new MacKeychainClient(async () => { - const error = new Error("spawn failed") as Error & { code: string }; - error.code = "ENOENT"; - throw error; - }); - await assert.rejects( - failed.has("vision-primary"), - (error) => error instanceof ProviderError && - error.code === "KEYCHAIN_UNAVAILABLE" && - !error.message.includes("test-credential-value") +test("macOS Keychain client follows the bounded helper and cache contract", async () => { + const fakeRoot = await mkdtemp(join(tmpdir(), "echolog-keychain-fake-")); + const fakeExecutable = join( + fakeRoot, + "EchoLogScreenCapture.app", + "Contents", + "MacOS", + "echolog-screen-capture" ); + await mkdir(join(fakeRoot, "EchoLogScreenCapture.app", "Contents", "MacOS"), { + recursive: true, + }); + await writeFile(fakeExecutable, "#!/bin/sh\nexit 0\n"); + await chmod(fakeExecutable, 0o755); + const requests: Array<{ args: string[]; stdin?: string; timeoutMs?: number }> = []; + try { + const client = new MacKeychainClient(async (request) => { + requests.push({ + args: request.args, + stdin: request.stdin, + timeoutMs: request.timeoutMs, + }); + const operation = request.args[1]; + return { + stdout: JSON.stringify(operation === "get" + ? { ok: true, hasSecret: true, secret: "cached-credential-value" } + : { ok: true, hasSecret: operation !== "delete" }), + stderr: "", + exitCode: 0, + }; + }, fakeExecutable); + assert.equal(await client.has("vision-primary"), true); + assert.deepEqual(requests[0]?.args, [ + "keychain", + "status", + "--service", + "com.cubeplus1.echolog.screen-understanding", + "--account", + "vision-primary", + "--no-auth-ui", + "--json", + ]); + assert.equal(requests[0]?.timeoutMs, NON_INTERACTIVE_KEYCHAIN_TIMEOUT_MS); + assert.equal(requests[0]?.stdin, undefined); + + assert.equal( + await client.get("manual-profile", undefined, "interactive"), + "cached-credential-value" + ); + assert.equal(requests[1]?.args.includes("--no-auth-ui"), false); + assert.equal(requests[1]?.timeoutMs, INTERACTIVE_KEYCHAIN_TIMEOUT_MS); + assert.equal( + await client.get("manual-profile", undefined, "non-interactive"), + "cached-credential-value" + ); + assert.equal(requests.length, 2); - const timedOut = new MacKeychainClient(async () => { - throw Object.assign(new Error("command timed out"), { - code: null, - killed: true, - signal: "SIGTERM", + await client.set("vision-primary", "test-credential-value"); + assert.deepEqual(JSON.parse(requests[2]?.stdin ?? ""), { + secret: "test-credential-value", }); - }); - await assert.rejects( - timedOut.has("vision-primary"), - (error) => error instanceof ProviderError && - error.code === "PLUGIN_TIMEOUT" && - error.statusCode === 504 && - error.message === "Keychain helper timed out" - ); + assert.equal(requests[2]?.args.join(" ").includes("test-credential-value"), false); + assert.equal(requests[2]?.timeoutMs, INTERACTIVE_KEYCHAIN_TIMEOUT_MS); + assert.equal(await client.get("vision-primary", undefined, "non-interactive"), "test-credential-value"); + assert.equal(requests.length, 3); + assert.equal(client.cachedState("vision-primary"), true); + assert.equal(client.hasCachedValue("vision-primary"), true); + + await client.delete("vision-primary"); + assert.equal(requests[3]?.timeoutMs, INTERACTIVE_KEYCHAIN_TIMEOUT_MS); + assert.equal(client.cachedState("vision-primary"), false); + assert.equal(client.hasCachedValue("vision-primary"), false); + assert.equal(await client.get("vision-primary", undefined, "non-interactive"), null); + assert.equal(requests.length, 4); + + client.clearCache(); + assert.equal(client.cachedState("manual-profile"), null); + assert.equal(client.hasCachedValue("manual-profile"), false); + + const failed = new MacKeychainClient(async () => { + const error = new Error("spawn failed") as Error & { code: string }; + error.code = "ENOENT"; + throw error; + }, fakeExecutable); + await assert.rejects( + failed.has("vision-primary"), + (error) => error instanceof ProviderError && + error.code === "KEYCHAIN_UNAVAILABLE" && + error.message.includes("could not be launched") && + !error.message.includes("test-credential-value") + ); + + const timedOut = new MacKeychainClient(async () => { + throw Object.assign(new Error("command timed out"), { + code: null, + killed: true, + signal: "SIGTERM", + }); + }, fakeExecutable); + await assert.rejects( + timedOut.has("vision-primary"), + (error) => error instanceof ProviderError && + error.code === "PLUGIN_TIMEOUT" && + error.statusCode === 504 && + error.message === "Keychain helper timed out" + ); + + const authRequired = new MacKeychainClient(async () => ({ + stdout: JSON.stringify({ + ok: false, + error: "private helper detail", + code: "KEYCHAIN_AUTH_REQUIRED", + retryable: false, + }), + stderr: "test-credential-value must not escape", + exitCode: 10, + }), fakeExecutable); + await assert.rejects( + authRequired.get("vision-primary", undefined, "non-interactive"), + (error) => error instanceof ProviderError && + error.code === "KEYCHAIN_AUTH_REQUIRED" && + error.statusCode === 409 && + !error.message.includes("test-credential-value") && + !error.message.includes("private helper detail") + ); + } finally { + await rm(fakeRoot, { recursive: true, force: true }); + } + + const missingRoot = await mkdtemp(join(tmpdir(), "echolog-keychain-helper-")); + try { + const missingExecutable = join( + missingRoot, + "EchoLogScreenCapture.app", + "Contents", + "MacOS", + "echolog-screen-capture" + ); + const missing = new MacKeychainClient(async () => { + throw new Error("must not spawn a missing helper"); + }, missingExecutable); + await assert.rejects( + missing.has("vision-primary"), + (error) => error instanceof ProviderError && + error.code === "KEYCHAIN_UNAVAILABLE" && + error.statusCode === 503 && + error.message.includes("EchoLogScreenCapture.app is missing") && + error.message.includes(MACOS_HELPER_BUILD_COMMAND) + ); + } finally { + await rm(missingRoot, { recursive: true, force: true }); + } }); test("macOS Keychain client matches the real Swift helper contract", { @@ -575,13 +679,16 @@ test("provider metadata remains readable and editable when Keychain status is un return { id: "default", version: 1, ...DEFAULT_UNDERSTANDING_SETTINGS, updatedAt: now }; }, }; + let helperQueries = 0; const unavailable = async () => { + helperQueries++; throw new ProviderError("KEYCHAIN_UNAVAILABLE", "Keychain helper is unavailable", 503); }; const service = new ProviderProfileService(store, { has: unavailable, set: unavailable, delete: unavailable, + cachedState() { return null; }, }); assert.equal((await service.list())[0]?.hasApiKey, null); const updated = await service.update("vision-primary", 1, { @@ -592,6 +699,7 @@ test("provider metadata remains readable and editable when Keychain status is un }); assert.equal(updated.displayName, "Updated"); assert.equal(updated.hasApiKey, null); + assert.equal(helperQueries, 0); }); test("provider service enforces CAS, references, and write-only key state", async () => { @@ -643,6 +751,8 @@ test("provider service enforces CAS, references, and write-only key state", asyn async has(id: string) { return keys.has(id); }, async set(id: string) { keys.add(id); }, async delete(id: string) { keys.delete(id); }, + cachedState(id: string) { return keys.has(id); }, + hasCachedValue(id: string) { return keys.has(id); }, }; const service = new ProviderProfileService(store, secrets); const created = await service.create({ @@ -891,6 +1001,31 @@ test("screen-time Web clears submitted keys and never renders them", async () => assert.equal(html.includes("test-credential-value"), false); assert.equal(JSON.stringify(data).includes("test-credential-value"), false); + provider.hasApiKey = null; + const unknownKeyHtml = contribution.renderFace( + { type: "understanding-providers" }, + { + data, + esc: (value: unknown) => String(value), + escA: (value: unknown) => String(value), + fmtDur: String, + } + ); + assert.match(unknownKeyHtml, /密钥状态不可用/); + assert.match(unknownKeyHtml, /保存或替换密钥/); + assert.match(unknownKeyHtml, /删除密钥/); + + const standalone = await readFile( + join(process.cwd(), "web/screen-understanding.js"), + "utf8" + ); + assert.match(standalone, /hasApiKey === null \? "密钥状态不可用"/); + assert.match(standalone, /\$\("deleteKey"\)\.hidden = provider\.hasApiKey === false/); + assert.match( + await readFile(join(process.cwd(), "web/screen-understanding.html"), "utf8"), + /id="deleteKey"/ + ); + provider.displayName = ''; provider.model = ""; const escapeText = (value: unknown) => String(value) diff --git a/tests/screen-understanding.test.ts b/tests/screen-understanding.test.ts index e44ce3b..104f9f8 100644 --- a/tests/screen-understanding.test.ts +++ b/tests/screen-understanding.test.ts @@ -1,6 +1,13 @@ import assert from "node:assert/strict"; +import { chmod, mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; import test from "node:test"; -import type { ProviderProfile } from "../plugins/screen-time/src/provider-profiles.js"; +import { + ProviderError, + ProviderProfileService, + type ProviderProfile, +} from "../plugins/screen-time/src/provider-profiles.js"; import { ScreenUnderstandingService, UnderstandingError, @@ -47,22 +54,39 @@ test("vision payload requires Simplified Chinese display values", () => { }); test("Keychain client reads a secret only through the helper response", async () => { + const fakeRoot = await mkdtemp(join(tmpdir(), "echolog-understanding-keychain-")); + const fakeExecutable = join( + fakeRoot, + "EchoLogScreenCapture.app", + "Contents", + "MacOS", + "echolog-screen-capture" + ); + await mkdir(join(fakeRoot, "EchoLogScreenCapture.app", "Contents", "MacOS"), { + recursive: true, + }); + await writeFile(fakeExecutable, "#!/bin/sh\nexit 0\n"); + await chmod(fakeExecutable, 0o755); const requests: string[][] = []; - const client = new MacKeychainClient(async (request) => { - requests.push(request.args); - return { - stdout: JSON.stringify( - request.args[1] === "get" - ? { ok: true, hasSecret: true, secret: "secret-value" } - : { ok: true, hasSecret: request.args[1] !== "delete" } - ), - stderr: "", - exitCode: 0, - }; - }, "/test/EchoLogScreenCapture.app/Contents/MacOS/echolog-screen-capture"); - assert.equal(await client.get("vision-primary"), "secret-value"); - assert.equal(requests[0]?.includes("secret-value"), false); - assert.equal(requests[0]?.[1], "get"); + try { + const client = new MacKeychainClient(async (request) => { + requests.push(request.args); + return { + stdout: JSON.stringify( + request.args[1] === "get" + ? { ok: true, hasSecret: true, secret: "secret-value" } + : { ok: true, hasSecret: request.args[1] !== "delete" } + ), + stderr: "", + exitCode: 0, + }; + }, fakeExecutable); + assert.equal(await client.get("vision-primary"), "secret-value"); + assert.equal(requests[0]?.includes("secret-value"), false); + assert.equal(requests[0]?.[1], "get"); + } finally { + await rm(fakeRoot, { recursive: true, force: true }); + } }); test("vision client accepts OpenAI-compatible JSON and maps provider failures safely", async () => { @@ -180,22 +204,287 @@ test("understanding service captures, retries transient provider failures, and p assert.equal(JSON.stringify(result).includes("secret-key"), false); }); -test("disabled understanding does not capture or call the provider", async () => { +test("disabled understanding skips scheduled work but permits an explicit run", async () => { let captures = 0; let calls = 0; + const current = new Date("2026-08-26T09:00:00+08:00"); const service = new ScreenUnderstandingService( makeStore(), - { async get() { return { id: "default", version: 1, ...DEFAULT_UNDERSTANDING_SETTINGS, enabled: false, updatedAt: new Date() }; } } as any, - { async getForInference() { throw new Error("must not resolve provider"); } }, - { async captureForInference() { captures++; throw new Error("must not capture"); } }, - { async complete() { calls++; throw new Error("must not call"); } }, - ); - await assert.rejects( - service.run(), - (error) => error instanceof UnderstandingError && error.code === "UNDERSTANDING_DISABLED" + { async get() { return { id: "default", version: 1, ...DEFAULT_UNDERSTANDING_SETTINGS, enabled: false, providerProfileId: profile.id, updatedAt: current }; } } as any, + { async getForInference() { return { profile, apiKey: "secret-key" }; } }, + { async captureForInference() { + captures++; + return { format: "png" as const, displayId: 1, widthPixels: 1, heightPixels: 1, bytes: 3, capturedAt: current.toISOString(), png: Buffer.from("png") }; + } }, + { async complete() { + calls++; + return { content: '{"summary":"显式识别","activity":"授权测试","confidence":0.95,"sensitive":false,"apps":["代码编辑器"]}', latencyMs: 12, costMicros: null }; + } }, + { isIdle() { return false; } }, + () => current ); + assert.equal(await service.runScheduled(), null); assert.equal(captures, 0); assert.equal(calls, 0); + assert.equal((await service.run()).summary, "显式识别"); + assert.equal(captures, 1); + assert.equal(calls, 1); +}); + +test("scheduled understanding uses non-interactive credentials and resumes after a manual read", async () => { + let current = new Date("2026-08-26T10:00:00+08:00"); + let providerResolutions = 0; + let credentialCached = false; + const accessModes: string[] = []; + const settings = { + async get() { + return { + id: "default", + version: 1, + ...DEFAULT_UNDERSTANDING_SETTINGS, + enabled: true, + providerProfileId: profile.id, + captureIntervalSeconds: 60, + updatedAt: current, + }; + }, + }; + const service = new ScreenUnderstandingService( + makeStore(), + settings as any, + { + async getForInference(_id, _signal, access) { + providerResolutions++; + accessModes.push(access ?? "interactive"); + if (access === "non-interactive" && !credentialCached) { + throw new ProviderError( + "KEYCHAIN_AUTH_REQUIRED", + "Keychain authorization is required", + 409 + ); + } + credentialCached = true; + return { profile, apiKey: "secret-key" }; + }, + hasCachedCredential() { return credentialCached; }, + }, + { + async captureForInference() { + return { + format: "png" as const, + displayId: 1, + widthPixels: 1, + heightPixels: 1, + bytes: 3, + capturedAt: current.toISOString(), + png: Buffer.from("png"), + }; + }, + }, + { + async complete() { + return { + content: '{"summary":"编辑代码","activity":"编写功能","confidence":0.95,"sensitive":false,"apps":["代码编辑器"]}', + latencyMs: 12, + costMicros: null, + }; + }, + }, + { isIdle() { return false; } }, + () => current + ); + + assert.equal(await service.runScheduled(), null); + current = new Date(current.getTime() + 60_000); + assert.equal(await service.runScheduled(), null); + assert.equal(providerResolutions, 1); + assert.deepEqual(accessModes, ["non-interactive"]); + + await service.run(); + current = new Date(current.getTime() + 60_000); + assert.ok(await service.runScheduled()); + assert.equal(providerResolutions, 3); + assert.deepEqual(accessModes, ["non-interactive", "interactive", "non-interactive"]); +}); + +test("scheduled understanding retries transient Keychain failures", async () => { + const failures = [ + ["KEYCHAIN_UNAVAILABLE", 503], + ["KEYCHAIN_OPERATION_FAILED", 502], + ["PLUGIN_TIMEOUT", 504], + ] as const; + + for (const [code, statusCode] of failures) { + let current = new Date("2026-08-26T10:30:00+08:00"); + let providerResolutions = 0; + const settings = { + async get() { + return { + id: "default", + version: 1, + ...DEFAULT_UNDERSTANDING_SETTINGS, + enabled: true, + providerProfileId: profile.id, + captureIntervalSeconds: 60, + updatedAt: current, + }; + }, + }; + const service = new ScreenUnderstandingService( + makeStore(), + settings as any, + { + async getForInference() { + providerResolutions++; + throw new ProviderError(code, "Safe transient Keychain failure", statusCode); + }, + }, + { + async captureForInference() { + throw new Error("must not capture without credentials"); + }, + }, + { + async complete() { + throw new Error("must not call provider without credentials"); + }, + }, + { isIdle() { return false; } }, + () => current + ); + + await assert.rejects( + service.runScheduled(), + (error) => error instanceof ProviderError && error.code === code + ); + current = new Date(current.getTime() + 60_000); + await assert.rejects( + service.runScheduled(), + (error) => error instanceof ProviderError && error.code === code + ); + assert.equal(providerResolutions, 2, `${code} must be retried`); + } +}); + +test("explicit authorization while disabled unlocks enablement and scheduled cache use", async () => { + const fakeRoot = await mkdtemp(join(tmpdir(), "echolog-understanding-cache-")); + const fakeExecutable = join( + fakeRoot, + "EchoLogScreenCapture.app", + "Contents", + "MacOS", + "echolog-screen-capture" + ); + await mkdir(join(fakeRoot, "EchoLogScreenCapture.app", "Contents", "MacOS"), { + recursive: true, + }); + await writeFile(fakeExecutable, "#!/bin/sh\nexit 0\n"); + await chmod(fakeExecutable, 0o755); + const helperRequests: Array<{ args: string[]; timeoutMs?: number }> = []; + let current = new Date("2026-08-26T11:00:00+08:00"); + let enabled = false; + try { + const keychain = new MacKeychainClient(async (request) => { + helperRequests.push({ args: request.args, timeoutMs: request.timeoutMs }); + if (request.args.includes("--no-auth-ui")) { + return { + stdout: JSON.stringify({ + ok: false, + error: "Keychain authorization is required", + code: "KEYCHAIN_AUTH_REQUIRED", + retryable: false, + }), + stderr: "", + exitCode: 10, + }; + } + return { + stdout: JSON.stringify({ + ok: true, + hasSecret: true, + secret: "integration-credential-value", + }), + stderr: "", + exitCode: 0, + }; + }, fakeExecutable); + const providers = new ProviderProfileService({ + async getProviderProfile() { + return { + id: profile.id, + version: profile.version, + displayName: profile.displayName, + providerKind: profile.providerKind, + baseUrl: profile.baseUrl, + model: profile.model, + createdAt: new Date(profile.createdAt), + updatedAt: new Date(profile.updatedAt), + }; + }, + } as any, keychain); + const settings = { + async get() { + return { + id: "default", + version: 1, + ...DEFAULT_UNDERSTANDING_SETTINGS, + enabled, + providerProfileId: profile.id, + captureIntervalSeconds: 60, + updatedAt: current, + }; + }, + }; + const service = new ScreenUnderstandingService( + makeStore(), + settings as any, + providers, + { + async captureForInference() { + return { + format: "png" as const, + displayId: 1, + widthPixels: 1, + heightPixels: 1, + bytes: 3, + capturedAt: current.toISOString(), + png: Buffer.from("png"), + }; + }, + }, + { + async complete() { + return { + content: '{"summary":"编辑代码","activity":"编写功能","confidence":0.95,"sensitive":false,"apps":["代码编辑器"]}', + latencyMs: 12, + costMicros: null, + }; + }, + }, + { isIdle() { return false; } }, + () => current + ); + + await assert.rejects( + providers.withSelectable(profile.id, true, async () => undefined), + (error) => error instanceof ProviderError && error.code === "KEYCHAIN_AUTH_REQUIRED" + ); + assert.equal(helperRequests.length, 1); + assert.equal(helperRequests[0]?.args.includes("--no-auth-ui"), true); + + await service.run(); + assert.equal(helperRequests.length, 2); + assert.equal(helperRequests[1]?.args.includes("--no-auth-ui"), false); + assert.equal(keychain.hasCachedValue(profile.id), true); + + await providers.withSelectable(profile.id, true, async () => { enabled = true; }); + assert.equal(helperRequests.length, 2); + current = new Date(current.getTime() + 60_000); + assert.ok(await service.runScheduled()); + assert.equal(helperRequests.length, 2); + } finally { + await rm(fakeRoot, { recursive: true, force: true }); + } }); test("understanding routes expose local run and safe history endpoints", async () => { diff --git a/web/screen-understanding.css b/web/screen-understanding.css index f900260..5672461 100644 --- a/web/screen-understanding.css +++ b/web/screen-understanding.css @@ -70,6 +70,7 @@ button { cursor: pointer; } .mini-status { padding: 6px 9px; } .mini-status.is-saved { border-color: rgba(133, 181, 138, 0.36); color: var(--green); } .mini-status.is-missing { border-color: rgba(201, 164, 75, 0.32); color: var(--gold); } +.mini-status.is-unknown { color: var(--ink-muted); } .field-label { display: grid; gap: 7px; margin-top: 14px; color: var(--ink-muted); font-size: 12px; letter-spacing: 0.03em; } .field-note { color: var(--ink-dim); font-family: ui-monospace, SFMono-Regular, Menlo, monospace; font-size: 10px; } diff --git a/web/screen-understanding.html b/web/screen-understanding.html index b7471c7..8555b30 100644 --- a/web/screen-understanding.html +++ b/web/screen-understanding.html @@ -64,6 +64,7 @@

模型连接

+
diff --git a/web/screen-understanding.js b/web/screen-understanding.js index 0e66d4e..e8face6 100644 --- a/web/screen-understanding.js +++ b/web/screen-understanding.js @@ -50,12 +50,16 @@ function selectedProvider() { return state.providers.find((provider) => provider.id === state.selectedProviderId) || null; } +function keyStateText(hasApiKey) { + return hasApiKey === null ? "密钥状态不可用" : hasApiKey ? "密钥已保存" : "尚无密钥"; +} + function renderProvider() { const select = $("providerSelect"); const empty = $("providerEmpty"); const details = $("providerDetails"); select.innerHTML = state.providers.map((provider) => - `` + `` ).join(""); if (state.providers.length === 0) { select.hidden = true; @@ -63,6 +67,7 @@ function renderProvider() { details.hidden = true; $("keyBadge").textContent = "未选择"; $("keyBadge").className = "mini-status"; + $("deleteKey").hidden = true; return; } select.hidden = false; @@ -79,8 +84,13 @@ function renderProvider() { $("providerBaseUrl").value = provider.baseUrl; $("providerVersion").textContent = `v${provider.version}`; const badge = $("keyBadge"); - badge.textContent = provider.hasApiKey ? "密钥已保存" : "尚无密钥"; - badge.className = `mini-status ${provider.hasApiKey ? "is-saved" : "is-missing"}`; + badge.textContent = keyStateText(provider.hasApiKey); + badge.className = `mini-status ${provider.hasApiKey === null ? "is-unknown" : provider.hasApiKey ? "is-saved" : "is-missing"}`; + $("apiKeyInput").placeholder = provider.hasApiKey === true + ? "输入新密钥以替换" + : provider.hasApiKey === null ? "输入密钥以保存或替换" : "输入 API Key"; + $("saveKey").textContent = provider.hasApiKey === true ? "替换密钥" : "保存密钥"; + $("deleteKey").hidden = provider.hasApiKey === false; } function renderSettings() { @@ -178,6 +188,21 @@ async function saveKey() { } } +async function deleteKey() { + const provider = selectedProvider(); + if (!provider || !window.confirm("删除此配置的密钥？")) return; + showMessage("keyMessage", "正在从本机 Keychain 删除…"); + try { + await api(`${API_ROOT}/providers/${encodeURIComponent(provider.id)}/key`, { method: "DELETE" }); + showMessage("keyMessage", "密钥已从 macOS Keychain 删除。", false); + toast("API Key 已删除"); + await load(); + } catch (error) { + showMessage("keyMessage", error.message, true); + toast(error.message, true); + } +} + async function saveProvider() { const provider = selectedProvider(); if (!provider) return; @@ -267,6 +292,7 @@ async function deleteHistory(id) { $("providerSelect").addEventListener("change", (event) => { state.selectedProviderId = event.target.value; renderProvider(); }); $("saveKey").addEventListener("click", saveKey); +$("deleteKey").addEventListener("click", deleteKey); $("saveProvider").addEventListener("click", saveProvider); $("createProvider").addEventListener("click", createProvider); $("saveSettings").addEventListener("click", saveSettings);