diff --git a/README.md b/README.md index 68380da..5b986a7 100644 --- a/README.md +++ b/README.md @@ -231,6 +231,47 @@ serves an Apple Silicon `llama.cpp` server; Kev (`node skills/jev-browser/bin/je tier table with every measured number — scores, latency, disk, memory, each backend's `goal_done` bar and the known holes — lives in [SKILL.md](skills/jev-browser/SKILL.md#judging-tiers). +A local tier is one command: `setup` fetches whatever is missing through the launcher itself, starts +the server in the background, writes `baseUrl` and the placeholder key into your config, then proves +the endpoint works by asking it one real question. + +```bash +jev-browser setup local-readout # llama.cpp + the registry GGUF; prints the log path and the pid +jev-browser setup status # installed / running / configured, per local tier +jev-browser setup stop local-readout +``` + +`jev-browser setup kev` is the same for the accuracy tier and needs `uv` as well (it clones the Kev +checkout and runs `uv sync --extra serve`). Both write under `~/.jev-browser/`, never into this repo. +The manual route still exists — start `bin/jev-local.mjs` yourself and export the two variables it +prints — and `jev-browser tier use --persist` still stores that line's values for you. + +## WebUI + +```bash +node skills/jev-browser/bin/jev-webui.mjs # prints http://127.0.0.1:8765/ +node skills/jev-browser/bin/jev-webui.mjs --port 9000 --open +``` + +One page over the same skill, not a second implementation: the panels call the same `lib/tiers.mjs`, +`lib/doctor.mjs` and `lib/config.mjs` the CLI does — so they cannot disagree with `tier status`, +`doctor` or `config show` — and edits go to the same `~/.config/jev-browser/config.json`. Every +command it starts is one of this repo's own `bin/*.mjs` scripts, spawned with an argv array (never a +shell), so no field on the page can become a command. + +The server binds **`127.0.0.1` only** — never `0.0.0.0`, so nothing on it is reachable from your +network — and no route returns your API key: the config panel shows only whether one is set, and +secrets typed into the Run panel are never echoed back into the page or the log pane. + +| panel | what it does | +| --- | --- | +| Tiers | the three judging tiers — what each is, what it needs, its port, its 20-item score and its `goal_done` bar, hosted marked as the default — plus what a run would use right now, start/stop for the two local servers with their output streaming into the page, and `tier use`'s text (saving it to the user config is its own labelled button) | +| Config | the effective configuration with its sources, editable, with the diff a save produced and an unset for every key the user file sets | +| Doctor | `doctor()` live or offline: every check with its status, detail and hint | +| Models | the local registry: which GGUF is downloaded, which is the default, which file is serving, and a one-click start with `--model-name` or an existing on-disk `--model ` | +| Judge | a state and a question set against the configured endpoint, with each answer's choice, confidence and probability table | +| Run | a real run (`jev-browser run --json`) with live progress, the final status / steps / cost and the per-step journal | + ## MCP server `jev-browser mcp` speaks MCP over stdio with zero dependencies. Tools: `jev_browse`, `jev_observe`, diff --git a/README.zh-CN.md b/README.zh-CN.md index be62ea6..a730924 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -210,6 +210,29 @@ jev-browser tier use kev # 打印某个 tier 的 export 行和启动命令( 表格与全部实测数字(得分、延迟、磁盘、内存、各后端 `goal_done` 阈值、已知短板)见 [`SKILL.md`](skills/jev-browser/SKILL.md#judging-tiers);`references/config.md` 只保留阈值 profile 细节。 +本地 tier 现在是一条命令:`setup` 通过 launcher 自己把缺的文件拉齐、后台起服务、把 `baseUrl` 和占位 key 写进配置,再用一个真实问题验证端点确实会答。 + +```bash +jev-browser setup local-readout # llama.cpp + 注册表里的 GGUF;打印日志路径和 pid +jev-browser setup status # 每个本地 tier:已安装 / 运行中 / 已配置 +jev-browser setup stop local-readout +``` + +`jev-browser setup kev` 是精度档的同款命令,额外需要 `uv`(它会 clone Kev 检出并跑 `uv sync --extra serve`)。两者只写 `~/.jev-browser/` 之下,不会写进本仓库。手动方式照旧 —— 自己起 `bin/jev-local.mjs` 并 export 它打印的两行 —— `jev-browser tier use --persist` 也能把那两行存下来。 + +## WebUI + +```bash +node skills/jev-browser/bin/jev-webui.mjs # 打印 http://127.0.0.1:8765/ +node skills/jev-browser/bin/jev-webui.mjs --port 9000 --open +``` + +同一个 skill 之上的一层页面,不是第二套实现:各面板直接调用 CLI 用的 `lib/tiers.mjs`、`lib/doctor.mjs`、`lib/config.mjs`,因此不会和 `tier status` / `doctor` / `config show` 给出不一致的结论,改动也写进同一个 `~/.config/jev-browser/config.json`。它启动的每条命令都是本仓库 `bin/*.mjs` 里的脚本,以 argv 数组方式拉起,绝不经过 shell。 + +**只绑定回环地址**:服务只监听 `127.0.0.1`(绝不 `0.0.0.0`),同网络的其他机器访问不到;任何路由都不会返回你的 API key —— 配置面板只显示"是否已设置",Run 面板输入的 secret 也不会被回显到页面或日志里。 + +六个面板(tier / 配置 / doctor / 模型注册表 / judge 试跑 / run)见英文 README 的 [WebUI](README.md#webui) 一节。 + ## MCP server `jev-browser mcp` 通过 stdio 提供 MCP,零依赖。工具:`jev_browse`、`jev_observe`、`jev_judge`、`jev_pick`、`jev_doctor`、`jev_config`。结果同时以 JSON 文本和 `structuredContent` 返回。见 [`references/mcp.md`](skills/jev-browser/references/mcp.md)。 diff --git a/skills/jev-browser/SKILL.md b/skills/jev-browser/SKILL.md index 9d86d38..5f2600b 100644 --- a/skills/jev-browser/SKILL.md +++ b/skills/jev-browser/SKILL.md @@ -166,7 +166,24 @@ somewhere local. A local endpoint never costs anything, so a loopback `baseUrl` One command serves the same `/v1/systemone` contract from a local llama.cpp server — no API key, no network, no cost. The answer is read out of the model's first generated token (options -are labelled A, B, C …), so nothing is generated and nothing is parsed: +are labelled A, B, C …), so nothing is generated and nothing is parsed. + +`jev-browser setup local-readout` does the whole first run: it finds llama.cpp, fetches the registry +GGUF through the launcher's own `--download-only`, serves it detached (log and pid under +`~/.jev-browser/run/`), writes `baseUrl` + the placeholder key into your user config, and then asks +the endpoint one real question — so "it works" is printed, not assumed: + +```bash +jev-browser setup local-readout +# server 127.0.0.1:8092 (pid …) +# config ~/.config/jev-browser/config.json — baseUrl + apiKey=local +# log ~/.jev-browser/run/jev-local-8092.log +# verified noul: P(yes)=0.95 — 569 ms through http://127.0.0.1:8092 +# `jev-browser setup status` reports installed / running / configured per tier; +# `jev-browser setup stop local-readout` stops the server it started. +``` + +Or by hand — the export line the launcher prints is all a run needs: ```bash node /bin/jev-local.mjs @@ -178,6 +195,11 @@ export TYPESAFE_BASE_URL=http://127.0.0.1:8092 TYPESAFE_API_KEY=local jev-browser judge --state-file state.json --questions-file questions.json --json # $0 ``` +A local tier's key is the literal placeholder `local`: the local server ignores the `Authorization` +header, but the client requires a non-empty key. `jev-browser config set apiKey local` stores it by +hand, and both `jev-browser setup ` and `jev-browser tier use --persist` store it for you +(the export still works and is what the launchers print). + Which model is served is data, not code: `lib/local-models.json` lists each entry with its GGUF filename, download URL, exact byte size and label, and its `"default"` picks the one the launcher uses. Switching models is an edit to that JSON — or one flag for a single run: @@ -203,6 +225,11 @@ and exits 0, and `doctor` reports the active registry id in its `local model` li ### Kev 4B — the accuracy tier (`jev-kev`, port 8008) +`jev-browser setup kev` is the one-command path for this tier: it checks `uv` (and says +`brew install uv` when it is missing), clones the Kev checkout and runs `uv sync --extra serve` +— skip both with `--skip-deps` — then fetches the pinned files, serves detached, writes +`baseUrl` + the placeholder key and verifies, exactly like `setup local-readout`. By hand: + ```bash node /bin/jev-kev.mjs # verifies the checkpoint and its 9.34 GB base in the HF cache (fetching what is missing, resuming @@ -289,6 +316,30 @@ instead of ~0.99). When the option labels cannot be read out at all the server a `experiments/gguf-provider/RESULTS.md` and `experiments/gguf-provider/results/local-models-4b.md`. +## WebUI + +`node skills/jev-browser/bin/jev-webui.mjs` serves one page on `127.0.0.1:8765` and prints its URL +(`--port` moves it, `--open` opens your browser, `-q` prints the URL alone). It is a view over this +skill, not a second implementation: the panels call the same `lib/tiers.mjs`, `lib/doctor.mjs` and +`lib/config.mjs` the CLI does — so they cannot disagree with `tier status`, `doctor` or +`config show` — and every edit is written to the same `~/.config/jev-browser/config.json`. Each +command it starts is one of this skill's own `bin/*.mjs` scripts, spawned with an argv array and +never a shell, so nothing typed into the page can become a command. + +The server binds **127.0.0.1 only** (never `0.0.0.0`): nothing on it is reachable from your network. +No route returns your API key — the config panel shows only whether one is set — and secrets typed +into the Run panel are never echoed back into the page or the log pane. Reach for it when you want +to see what a run would use, start a local tier, or drive one goal by hand without a terminal. + +| panel | what it does | +| --- | --- | +| Tiers | the three judging tiers (what each needs, its port, its 20-item score, its `goal_done` bar; hosted is marked as the default) and what a run would use right now | +| Config | the effective configuration with its sources, editable, with the diff a save produced | +| Doctor | `doctor()` live or offline: every check with its status, detail and hint | +| Models | the local registry: downloaded? default? serving? — and a start with `--model-name` or an on-disk `--model ` | +| Judge | a state and a question set against the configured endpoint, with each answer's choice, confidence and probabilities | +| Run | a real `run` with live progress, the final status / steps / cost and the per-step journal | + ## MCP (Claude Desktop, Cursor, Codex) `jev-browser mcp` is a stdio MCP server with tools `jev_browse`, `jev_observe`, diff --git a/skills/jev-browser/bin/jev-browser.mjs b/skills/jev-browser/bin/jev-browser.mjs index e967ecf..de0e292 100755 --- a/skills/jev-browser/bin/jev-browser.mjs +++ b/skills/jev-browser/bin/jev-browser.mjs @@ -8,6 +8,7 @@ import { loadConfig, describeConfig, saveUserConfig, unsetUserConfig, patchFromK import { executeJob } from "../lib/runner.mjs"; import { TypeSafeClient } from "../lib/typesafe.mjs"; import { doctor, formatDoctor } from "../lib/doctor.mjs"; +import { setupCommand } from "../lib/setup.mjs"; import { DEFAULT_TIER, TIERS, describeTier, fetchModels, formatTierList, formatTierStatus, formatTierUse, launcherCommand, probeEndpoint, tierByName, tierEnv, tierRows } from "../lib/tiers.mjs"; import { installTargets, formatInstall, DEFAULT_TARGETS, DEFAULT_SKILL_DIR } from "../lib/install.mjs"; import { parseKeyValue, rankProbabilities } from "../lib/util.mjs"; @@ -23,6 +24,7 @@ Usage: jev-browser judge --state --questions (or --state-file / --questions-file) jev-browser pick --question "" --candidate id=description... [--context ] jev-browser doctor [--json] [--offline] + jev-browser setup [local-readout | kev | status | stop ] [--model-name ] [--skip-deps] jev-browser tier [list | status | use ] [--json] [--persist] jev-browser config show | path | set | unset | set-key [ | --from-env] jev-browser install [--targets a,b,c] [--dry-run] [--copy] [--uninstall] @@ -49,11 +51,27 @@ tier: thresholds profile and value, and whether a local endpoint answers. use print that tier's export line (and the command that starts it) — nothing is written to your configuration unless you add --persist. - --persist with "use": store that tier's baseUrl in the user config (never automatic) + --persist with "use": store that tier's baseUrl (and, for a local tier, its placeholder + apiKey) in the user config — never automatic --json machine-readable list / status / use --offline with "status": skip the loopback probe instead of asking the endpoint exit codes: 0 ok, 2 unknown tier or bad usage +setup — one command per local tier, from nothing installed to a verified, configured server: + setup what is installed, what is running, what a run uses, and what to type next + setup local-readout fetch the registry model unattended, serve it in the background, write + baseUrl + apiKey into the user config, then ask it one real question + (llama.cpp is required: brew install llama.cpp) + setup kev clone the Kev checkout and uv sync it (unless --skip-deps), fetch the pinned + checkpoint, serve it in the background, write the config, ask it a question + (uv is required: brew install uv) + --model-name with "setup local-readout": serve that registry entry instead of the default + --skip-deps with "setup kev": skip git clone + uv sync, require them to exist already + --json machine-readable result (each setup prints its log file and pid) + setup status per local tier: installed / running / configured, and the log path + setup stop stop the launcher that setup started (its pid file) — never a foreign process + exit codes: 0 ok, 2 usage or a missing prerequisite, 1 a runtime failure (the log path is named) + Config precedence: defaults < ~/.config/jev-browser/config.json < ./jev-browser.config.json (or $JEV_BROWSER_CONFIG) < env < flags Env: TYPESAFE_API_KEY TYPESAFE_BASE_URL TYPESAFE_DEFAULT_MODEL JEV_BROWSER_BACKEND JEV_BROWSER_MAX_STEPS JEV_BROWSER_BUDGET_USD JEV_BROWSER_JOURNAL_DIR JEV_BROWSER_CHROME_CDP_URL JEV_BROWSER_HEADLESS JEV_BROWSER_EGO_SERVER_NAME `; @@ -90,6 +108,8 @@ const OPTIONS = { offline: { type: "boolean" }, targets: { type: "string" }, home: { type: "string" }, + "model-name": { type: "string" }, + "skip-deps": { type: "boolean" }, copy: { type: "boolean" }, uninstall: { type: "boolean" }, "from-env": { type: "boolean" }, @@ -180,7 +200,7 @@ async function main(argv) { switch (command) { case "run": { - const { config } = await loadConfig({ flags: flagsFromValues(values) }); + const { config } = await loadConfig({ flags: flagsFromValues(values), ...(values.home ? { home: values.home } : {}) }); if (!values.goal && !values["dry-run"]) throw new Error("--goal is required"); if (!values.url && !values["space-id"]) throw new Error("--url is required (or --space-id to resume an ego task space)"); if (!config.apiKey && !values["dry-run"]) throw new Error("no TypeSafe API key: export TYPESAFE_API_KEY or run `jev-browser config set-key --from-env`"); @@ -208,14 +228,14 @@ async function main(argv) { return result.status === "success" ? 0 : result.status === "needs_user" ? 3 : 2; } case "observe": { - const { config } = await loadConfig({ flags: flagsFromValues(values) }); + const { config } = await loadConfig({ flags: flagsFromValues(values), ...(values.home ? { home: values.home } : {}) }); if (!values.url) throw new Error("--url is required"); const out = await executeJob({ config, job: { mode: "observe", startUrl: values.url, headless: values.headless || undefined, cdpUrl: values["cdp-url"], screenshotPath: values.screenshot ? path.resolve(values.screenshot) : undefined, keep: values.keep ?? false }, log }); print(values.json ? out.page : { backend: out.backend, page: out.page }, values, `${out.page.title} <${out.page.url}> — ${out.page.elements.length} interactive elements`); return 0; } case "judge": { - const { config } = await loadConfig({ flags: flagsFromValues(values) }); + const { config } = await loadConfig({ flags: flagsFromValues(values), ...(values.home ? { home: values.home } : {}) }); const state = values["state-file"] ? parseMaybeJson(await fs.readFile(values["state-file"], "utf8")) : parseMaybeJson(values.state); const questions = values["questions-file"] ? JSON.parse(await fs.readFile(values["questions-file"], "utf8")) : values.questions ? JSON.parse(values.questions) : undefined; if (state === undefined || !questions) throw new Error("--state/--state-file and --questions/--questions-file are required"); @@ -225,7 +245,7 @@ async function main(argv) { return 0; } case "pick": { - const { config } = await loadConfig({ flags: flagsFromValues(values) }); + const { config } = await loadConfig({ flags: flagsFromValues(values), ...(values.home ? { home: values.home } : {}) }); if (!values.question || !values.candidate?.length) throw new Error("--question and at least two --candidate id=description are required"); const criteria = toMap(values.candidate); if (!values["no-none"] && !("none" in criteria)) criteria.none = "No candidate fits."; @@ -244,6 +264,19 @@ async function main(argv) { else process.stdout.write(`${formatDoctor(report)}\n`); return report.ok ? 0 : 1; } + case "setup": { + // The one-click path to a local tier: `setupCommand` composes the launchers and throws a + // `config: true` error (exit 2) for anything the user has to change. + const { config } = await loadConfig({ flags: flagsFromValues(values), ...(values.home ? { home: values.home } : {}) }); + const out = await setupCommand({ + sub: positionals[1], + rest: positionals.slice(2), + options: { home: values.home, modelName: values["model-name"], skipDeps: values["skip-deps"], json: values.json, config, skillDir: SKILL_DIR, env: process.env, log }, + }); + if (values.json) print(out.json, { json: true }); + else process.stdout.write(`${out.text}\n`); + return out.code; + } case "config": { const sub = positionals[1] ?? "show"; const home = values.home; @@ -304,7 +337,9 @@ async function main(argv) { const tier = tierByName(name); if (!tier) throw configError(`unknown tier "${name}" (expected ${TIERS.map((t) => t.name).join(", ")})`); const home = values.home; - const persisted = values.persist ? await saveUserConfig({ baseUrl: tier.baseUrl }, home ? { home } : {}) : null; + // A local tier is only half configured by baseUrl: its key is the literal placeholder the + // client requires and the server ignores, so --persist stores both. Hosted has no key. + const persisted = values.persist ? await saveUserConfig({ baseUrl: tier.baseUrl, ...(tier.apiKey ? { apiKey: tier.apiKey } : {}) }, home ? { home } : {}) : null; if (values.json) print({ tier: tier.name, baseUrl: tier.baseUrl, apiKey: tier.apiKey, port: tier.port, command: launcherCommand(tier, SKILL_DIR), env: tierEnv(tier), persisted }, { json: true }); else process.stdout.write(`${formatTierUse(tier, { skillDir: SKILL_DIR, persisted, configPath: userConfigPath(home) })}\n`); return 0; diff --git a/skills/jev-browser/bin/jev-kev.mjs b/skills/jev-browser/bin/jev-kev.mjs index c0ffd32..f26d9c9 100755 --- a/skills/jev-browser/bin/jev-kev.mjs +++ b/skills/jev-browser/bin/jev-kev.mjs @@ -24,6 +24,8 @@ import { createHash } from "node:crypto"; import { spawn } from "node:child_process"; import { Readable } from "node:stream"; import { pipeline } from "node:stream/promises"; +import { fileURLToPath } from "node:url"; +import { KEV_DEFAULT_CLONE, KEV_UV_HINT, kevSetupCommands, probeKevRuntime } from "../lib/kev.mjs"; // ---------------------------------------------------------------------------- assets @@ -81,7 +83,7 @@ const KEV_ASSETS = { // different .gitattributes, so its small files are expected to fail and fall through to HF). const SOURCES = ["modelscope", "hf", "hf-mirror"]; -const DEFAULT_CLONE = path.join(os.homedir(), ".local", "share", "jev-browser", "kev"); +const DEFAULT_CLONE = KEV_DEFAULT_CLONE; const CACHE = path.join(os.homedir(), ".cache", "huggingface", "hub"); const RUN_ID = KEV_ASSETS.run.repo; const BASE_ID = KEV_ASSETS.base.repo; @@ -98,6 +100,7 @@ Usage: jev-kev --list-files print the pinned manifest and exit jev-kev --patch-row-limit apply the optional row-limit patch to the local clone jev-kev --unpatch-row-limit revert it + jev-kev --detach run in the background; print the env line once serving What it does, in order: 1. fetches ${RUN_ID} (16 files, 152 MiB) and its base ${BASE_ID} @@ -124,6 +127,9 @@ Tuning: --run checkpoint to serve (default ${RUN_ID}) --clone Kev checkout to use (default ${DEFAULT_CLONE.replace(os.homedir(), "~")}) --timeout wait for the server to become ready (default ${READY_MS / 1000}) + --detach background: logs to ~/.jev-browser/run/jev-kev-8008.log, writes the pid + file, prints the env line once the server answers and returns (the server + keeps running; opt in, never the default) -h, --help this text Env (passed through to the server): KEV_TEMPERATURE, KEV_BACKEND, KEV_DTYPE, KEV_PREFIX_CACHE, ... @@ -388,45 +394,19 @@ const patchApplied = async (clone) => (await fsp.readFile(path.join(clone, "kev" /** The venv python, with the exact command that creates it when it is missing. */ async function findPython(clone) { - const python = path.join(clone, ".venv", "bin", "python"); - if (!fs.existsSync(clone)) { - throw configError( - `Kev checkout not found at ${clone}\n\nClone it with:\n\n git clone --depth 1 https://github.com/jaredpalmer/kev.git ${clone}\n uv sync --extra serve --project ${clone}\n`, - ); - } - if (!fs.existsSync(python)) { - throw configError( - `no Python venv at ${path.join(clone, ".venv")}\n\nCreate it with:\n\n uv sync --extra serve --project ${clone}\n\n` + - `(uv follows the repo's .python-version, 3.13 — not 3.14, which has no torch wheel)`, - ); + const status = await probeKevRuntime({ clone }); + if (status.ok) return status.python; + const setup = kevSetupCommands(clone); + if (status.kind === "missing-clone") { + throw configError(`Kev checkout not found at ${clone}\n\nClone it with:\n\n ${setup[0]}\n ${setup[1]}\n\n(${KEV_UV_HINT})`); } - const probe = await run(python, ["-c", "import torch, mlx_lm; print('ok')"], { cwd: clone, quiet: true }).catch((error) => error); - if (probe instanceof Error || probe.code !== 0) { - const detail = (probe instanceof Error ? probe.message : `${probe.stdout}${probe.stderr}`).trim().split("\n").slice(-3).join("\n"); + if (status.kind === "missing-venv") { throw configError( - `the venv at ${python} cannot import torch and mlx_lm, so the MLX backend is unavailable\n ${detail}\n\n` + - `Install the serving extras with:\n\n uv sync --extra serve --project ${clone}\n`, + `no Python venv at ${path.join(clone, ".venv")}\n\nCreate it with:\n\n ${setup[1]}\n\n` + + `(${KEV_UV_HINT}; uv follows the repo's .python-version, 3.13 — not 3.14, which has no torch wheel)`, ); } - return python; -} - -function run(application, args, { cwd, env, quiet } = {}) { - return new Promise((resolve, reject) => { - const child = spawn(application, args, { cwd, env: env ?? process.env, stdio: ["ignore", "pipe", "pipe"] }); - let stdout = ""; - let stderr = ""; - child.stdout.on("data", (chunk) => { - stdout += chunk; - if (!quiet) process.stderr.write(chunk); - }); - child.stderr.on("data", (chunk) => { - stderr += chunk; - if (!quiet) process.stderr.write(chunk); - }); - child.once("error", reject); - child.once("close", (code) => resolve({ code, stdout, stderr })); - }); + throw configError(`${status.detail}\n\nInstall the serving extras with:\n\n ${setup[1]}\n\n(${KEV_UV_HINT})`); } async function getJson(url, timeoutMs = 1500) { @@ -514,6 +494,30 @@ async function main() { const verifyOnly = flag("--verify-only"); const downloadOnly = flag("--download-only"); + + // --detach: run this launcher again in the background (args minus the flag), wait for the card it + // serves, and print the one line callers read. Opt-in; the foreground path below is untouched. + if (flag("--detach")) { + if (verifyOnly || downloadOnly) throw configError(`--detach cannot be combined with ${verifyOnly ? "--verify-only" : "--download-only"}: one backgrounds a server, the other only inspects or fetches and exits`); + const { spawnSelfDetached, waitUntilReady } = await import("../lib/detach.mjs"); + const runDir = path.join(os.homedir(), ".jev-browser", "run"); + const logFile = path.join(runDir, `jev-kev-${port}.log`); + const detachedUrl = `http://127.0.0.1:${port}`; + const child = spawnSelfDetached({ script: fileURLToPath(import.meta.url), args: argv.filter((argument) => argument !== "--detach"), logFile }); + const ready = await waitUntilReady(async () => Boolean((await getJson(`${detachedUrl}/v1/models`, 1500))?.models?.length), { timeoutMs: READY_MS }); + if (!ready) { + log(`[kev] --detach: nothing answered ${detachedUrl}/v1/models within ${READY_MS / 1000}s; see ${logFile} (pid ${child.pid})`); + return 1; + } + const pidFile = path.join(runDir, `jev-kev-${port}.pid`); + if (child.exitCode === null && !fs.existsSync(pidFile)) { + await fsp.mkdir(runDir, { recursive: true }); + await fsp.writeFile(pidFile, `${child.pid}\n`); + } + process.stdout.write(`TYPESAFE_BASE_URL=${detachedUrl} TYPESAFE_API_KEY=local\n`); + return 0; + } + // Check the runtime before the (expensive) cache verification: a missing venv should fail in a // second, not after re-hashing 9.5 GB. --verify-only and --download-only never need it. const python = verifyOnly || downloadOnly ? null : await findPython(clone); @@ -595,6 +599,8 @@ async function main() { spawned.kill("SIGTERM"); throw new Error(`the server came up serving "${card.run}", not "${run}" — refusing to hand out a URL for it`); } + const runDir = path.join(os.homedir(), ".jev-browser", "run"); + const pidFile = path.join(runDir, `jev-kev-${port}.pid`); const shutdown = (signal) => { log(`[kev] ${signal} — stopping the Kev server${spawned ? ` (pid ${spawned.pid})` : ""}`); try { @@ -602,6 +608,7 @@ async function main() { } catch { // already gone } + fs.rmSync(pidFile, { force: true }); process.exit(0); }; process.on("SIGINT", () => shutdown("SIGINT")); @@ -610,6 +617,11 @@ async function main() { log(`[kev] server exited (code ${code})`); }); + // This launcher's own pid: `setup stop` signals this process, and the handler above stops the + // server it started together with it. + await fsp.mkdir(runDir, { recursive: true }); + await fsp.writeFile(pidFile, `${process.pid}\n`); + log(`[kev] serving /v1/systemone on ${serviceUrl} (${card.run} · ${card.base} · ${card.device}/${card.backend} · ${card.dtype} · T=${Number(card.temperature).toFixed(4)})`); log(`[kev] limits: needs a Python venv + MLX, ~18 GB idle and 36 GB GPU footprint under load at the raised limit;`); log(`[kev] the released cap is 8192 tokens (state + one branch) — see experiments/kev-4b/README.md`); diff --git a/skills/jev-browser/bin/jev-local.mjs b/skills/jev-browser/bin/jev-local.mjs index bac3970..10b0445 100755 --- a/skills/jev-browser/bin/jev-local.mjs +++ b/skills/jev-browser/bin/jev-local.mjs @@ -21,6 +21,7 @@ import path from "node:path"; import { spawn } from "node:child_process"; import { Readable } from "node:stream"; import { pipeline } from "node:stream/promises"; +import { fileURLToPath } from "node:url"; // Static on purpose: --help must work even while the registry is being edited. const HELP = `jev-local — serve the local (experimental) Jev backend for jev-browser @@ -48,6 +49,9 @@ Tuning: --port N port of the Jev-compatible server (default 8092) --llama-port N port of the llama.cpp server (default 8090) --download-only fetch the model file and exit + --detach run in the background: logs to ~/.jev-browser/run/jev-local-8092.log, + writes the pid file, prints the env line once it answers /health and + returns (the server keeps running; opt in, never the default) -h, --help this text Env: @@ -172,7 +176,7 @@ async function main() { log(`[local] ${error.message}`); return 2; } - const { LOCAL_DEFAULTS, LocalProvider, SERVICE, defaultLocalModel, findLlamaServer, getJson, handleLocalRequest, llamaServedModel, localModel, localPaths } = local; + const { LOCAL_DEFAULTS, LocalProvider, SERVICE, defaultLocalModel, findLlamaServer, getJson, handleLocalRequest, llamaMissingHint, llamaServedModel, localModel, localPaths } = local; const port = Number(arg("--port", LOCAL_DEFAULTS.port)); const llamaPort = Number(arg("--llama-port", LOCAL_DEFAULTS.llamaPort)); @@ -193,6 +197,35 @@ async function main() { if (flag("--list-models")) return listModels(local); + // --detach: run this same launcher again in the background (args minus the flag), then wait for + // the endpoint it starts to answer and print the one line callers read. Opt-in; the foreground + // path below is untouched. + if (flag("--detach")) { + if (downloadOnly) throw configError("--detach cannot be combined with --download-only: one backgrounds a server, the other only fetches and exits"); + const { spawnSelfDetached, waitUntilReady } = await import("../lib/detach.mjs"); + const runDir = localPaths().run; + const pidFile = path.join(runDir, `jev-local-${port}.pid`); + const logFile = path.join(runDir, `jev-local-${port}.log`); + const child = spawnSelfDetached({ script: fileURLToPath(import.meta.url), args: argv.filter((argument) => argument !== "--detach"), logFile }); + const serving = async () => { + const health = await getJson(`${serviceUrl}/health`, 1500); + return health?.service === SERVICE && health?.status === "ok"; + }; + const ready = await waitUntilReady(serving, { timeoutMs: 180_000 }); + if (!ready) { + log(`[local] --detach: nothing answered ${serviceUrl}/health within 180s; see ${logFile} (pid ${child.pid})`); + return 1; + } + // The child writes this itself once it listens; a detached server must be stoppable even if it + // was still writing when the health check passed. + if (child.exitCode === null && !fs.existsSync(pidFile)) { + await fsp.mkdir(runDir, { recursive: true }); + await fsp.writeFile(pidFile, `${child.pid}\n`); + } + process.stdout.write(`${envLine}\n`); + return 0; + } + // Resolve which GGUF to serve: a registry entry (default or --model-name), or an explicit path. const entry = requestedId ? localModel(requestedId) : explicitPath ? null : defaultLocalModel(); const file = explicitPath ? path.resolve(explicitPath) : localPaths(undefined, entry).modelFile; @@ -225,7 +258,7 @@ async function main() { const bin = findLlamaServer(); if (!bin) { - log("[local] llama-server not found (looked on PATH and in /opt/homebrew/bin).\n\nInstall it with:\n\n brew install llama.cpp\n"); + log(`[local] ${llamaMissingHint()}`); return 2; } log(`[local] llama-server: ${bin}`); diff --git a/skills/jev-browser/bin/jev-webui.mjs b/skills/jev-browser/bin/jev-webui.mjs new file mode 100755 index 0000000..99c5859 --- /dev/null +++ b/skills/jev-browser/bin/jev-webui.mjs @@ -0,0 +1,135 @@ +#!/usr/bin/env node +// jev-webui — the local WebUI for jev-browser: the three judging tiers, the effective config, +// doctor, the local model registry, a judge playground and a run launcher in one browser page. +// +// It is a view over the existing lib/, not a second implementation: the page is served by Node's +// own http server, bound to 127.0.0.1 only (never 0.0.0.0, so nothing here is on the LAN), and it +// reads and writes the same ~/.config/jev-browser/config.json the CLI uses. +// +// node skills/jev-browser/bin/jev-webui.mjs +// node skills/jev-browser/bin/jev-webui.mjs --port 9000 --open +// +// The URL is the only thing on stdout, so it can be piped; everything else goes to stderr. +import fs from "node:fs/promises"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; +import { parseArgs } from "node:util"; +import { userConfigPath } from "../lib/config.mjs"; +import { DEFAULT_WEBUI_PORT, WebUiError, createWebUiServer, listenWebUi, openBrowser, parsePort } from "../lib/webui.mjs"; + +const SKILL_DIR = path.resolve(path.dirname(fileURLToPath(import.meta.url)), ".."); +const VERSION = JSON.parse(await fs.readFile(new URL("../package.json", import.meta.url), "utf8")).version; + +const HELP = `jev-webui ${VERSION} — local WebUI for jev-browser (tiers, config, doctor, models, judge, run) + +Usage: + jev-webui [--port ${DEFAULT_WEBUI_PORT}] [--open] [-q] + + Serves one page on 127.0.0.1 and does everything the CLI does, from a browser: + Tiers the three judging tiers, what a run would use right now, and start/stop for the two + local servers (their output is streamed into the page, so a download is visible) + Config the effective configuration with its sources; edits go to ~/.config/jev-browser/config.json + Doctor the same checks as \`jev-browser doctor\`, live or offline + Models the local registry (lib/local-models.json): what is downloaded, what is serving + Judge one System One request against the configured endpoint, with the answers rendered + Run a real run (\`jev-browser run --json\`), with live progress and the per-step journal + + Loopback only. The page never receives your API key, and every command is spawned with an argv + array — the WebUI cannot be talked into running something else. + +options: + --port N bind 127.0.0.1:N (default ${DEFAULT_WEBUI_PORT}; 1024..65535) + --open open the page in your default browser (best effort) + -q, --quiet no banner on stderr; the URL is still printed + -h, --help this text + -v, --version print the version + +Exit codes: 0 ok, 2 bad usage or the port is taken. +`; + +const OPTIONS = { + port: { type: "string" }, + open: { type: "boolean" }, + quiet: { type: "boolean", short: "q" }, + help: { type: "boolean", short: "h" }, + version: { type: "boolean", short: "v" }, +}; + +function banner(url, skillDir) { + return [ + `jev-webui ${VERSION} — local WebUI for jev-browser`, + ` url ${url}`, + " bind 127.0.0.1 only — not reachable from your network, no LAN exposure", + ` config ${userConfigPath()} (the same file the CLI reads and writes)`, + ` scripts ${skillDir}/bin/{jev-browser,jev-local,jev-kev}.mjs, spawned with an argv array`, + " Ctrl-C stops the page and any local model server it started.", + ].join("\n"); +} + +async function main(argv) { + let values; + try { + ({ values } = parseArgs({ args: argv, options: OPTIONS, allowPositionals: false, strict: true })); + } catch (error) { + process.stderr.write(`error: ${error.message}\n\n${HELP}`); + return 2; + } + if (values.help) { + process.stdout.write(HELP); + return 0; + } + if (values.version) { + process.stdout.write(`${VERSION}\n`); + return 0; + } + + let port; + try { + port = values.port === undefined ? DEFAULT_WEBUI_PORT : parsePort(values.port, { name: "--port", min: 1024, max: 65535 }); + } catch (error) { + process.stderr.write(`error: ${error.message}\n`); + return 2; + } + + const log = values.quiet ? () => {} : (message) => process.stderr.write(`${message}\n`); + const ui = createWebUiServer({ log }); + let address; + try { + address = await listenWebUi(ui.server, { port }); + } catch (error) { + if (error.code === "EADDRINUSE") { + process.stderr.write(`error: port ${port} is already in use — start it with another one: jev-webui --port \n`); + return 2; + } + if (error.code === "EACCES") { + process.stderr.write(`error: port ${port} needs root — pick a port above 1023\n`); + return 2; + } + throw error; + } + + const url = ui.url(address.port); + process.stdout.write(`${url}\n`); + if (!values.quiet) process.stderr.write(`${banner(url, SKILL_DIR)}\n`); + if (values.open) openBrowser(url); + + let stopping = false; + await new Promise((resolve) => { + const stop = () => { + if (stopping) return; + stopping = true; + if (!values.quiet) process.stderr.write("\nstopping…\n"); + ui.close().then(() => resolve(), () => resolve()); + }; + process.once("SIGINT", stop); + process.once("SIGTERM", stop); + }); + return 0; +} + +try { + process.exitCode = await main(process.argv.slice(2)); +} catch (error) { + process.stderr.write(`error: ${error.message}\n`); + process.exitCode = error instanceof WebUiError ? 2 : 1; +} diff --git a/skills/jev-browser/lib/detach.mjs b/skills/jev-browser/lib/detach.mjs new file mode 100644 index 0000000..2281ef1 --- /dev/null +++ b/skills/jev-browser/lib/detach.mjs @@ -0,0 +1,46 @@ +// Backgrounding for the two local launchers. `--detach` runs the same script again with the same +// arguments minus the flag, sends its output to a log file under ~/.jev-browser/run, and returns +// once the endpoint answers. Opt-in only: the foreground launcher is unchanged, and only a process +// whose pid file this launcher wrote is ever stopped. +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import { spawn } from "node:child_process"; + +/** The run directory, pid file and log file one launcher owns (both launchers use these names). */ +export function runPaths(home = os.homedir(), name, port) { + const run = path.join(home, ".jev-browser", "run"); + return { run, pidFile: path.join(run, `${name}-${port}.pid`), logFile: path.join(run, `${name}-${port}.log`) }; +} + +/** + * Run `script` (this launcher) again in the background with `args`, output into `logFile`. + * A new process group, so it survives the caller's exit and its terminal's Ctrl-C. + */ +export function spawnSelfDetached({ script, args, logFile, cwd = process.cwd(), env = process.env }) { + fs.mkdirSync(path.dirname(logFile), { recursive: true }); + const fd = fs.openSync(logFile, "a"); + try { + const child = spawn(process.execPath, [script, ...args], { cwd, env, detached: true, stdio: ["ignore", fd, fd] }); + child.unref(); + return child; + } finally { + fs.closeSync(fd); + } +} + +/** Poll `ready` until it returns true or the deadline passes. Never throws. */ +export async function waitUntilReady(ready, { timeoutMs, intervalMs = 500 } = {}) { + const deadline = Date.now() + timeoutMs; + for (;;) { + let ok = false; + try { + ok = Boolean(await ready()); + } catch { + ok = false; + } + if (ok) return true; + if (Date.now() >= deadline) return false; + await new Promise((resolve) => setTimeout(resolve, intervalMs)); + } +} diff --git a/skills/jev-browser/lib/doctor.mjs b/skills/jev-browser/lib/doctor.mjs index c788788..ac85f49 100644 --- a/skills/jev-browser/lib/doctor.mjs +++ b/skills/jev-browser/lib/doctor.mjs @@ -7,8 +7,8 @@ import { promisify } from "node:util"; import { findChromeExecutable } from "./backends/chrome.mjs"; import { SAFARI_ENABLE_HINT } from "./backends/safari.mjs"; import { describeConfig, userConfigPath } from "./config.mjs"; -import { describeTier, loopbackPort, probeEndpoint } from "./tiers.mjs"; -import { TypeSafeClient } from "./typesafe.mjs"; +import { describeTier, loopbackPort, probeEndpoint, tierByPort } from "./tiers.mjs"; +import { TypeSafeClient, isLoopbackBaseUrl } from "./typesafe.mjs"; import { installTargets } from "./install.mjs"; const run = promisify(execFile); @@ -38,7 +38,20 @@ export async function doctor({ config, sources, home = os.homedir(), skillDir, l const major = Number(process.versions.node.split(".")[0]); add("node", major >= 22, `node ${process.version}`, major >= 22 ? undefined : "Node 22+ is required (global fetch + WebSocket)"); - add("api key", Boolean(config.apiKey), config.apiKey ? `present (${sources.some((s) => s.kind === "env" && s.keys.includes("apiKey")) ? "env TYPESAFE_API_KEY" : "config file"})` : "missing", config.apiKey ? undefined : "export TYPESAFE_API_KEY=... or run: jev-browser config set-key --from-env"); + // A local tier is a complete configuration without a real key: the server ignores the header, + // but the client refuses to send without one — so only the placeholder is missing, and that is a + // warning, not a failure. + const loopback = isLoopbackBaseUrl(config.baseUrl); + add( + "api key", + config.apiKey ? true : loopback ? null : false, + config.apiKey ? `present (${sources.some((s) => s.kind === "env" && s.keys.includes("apiKey")) ? "env TYPESAFE_API_KEY" : "config file"})` : "missing", + config.apiKey + ? undefined + : loopback + ? "a local tier takes the literal placeholder: export TYPESAFE_API_KEY=local, or run: jev-browser config set-key local (or: jev-browser setup local-readout)" + : "export TYPESAFE_API_KEY=... or run: jev-browser config set-key --from-env", + ); let models = null; if (config.apiKey && live) { try { @@ -50,40 +63,67 @@ export async function doctor({ config, sources, home = os.homedir(), skillDir, l } } - // The fully local backend is optional: report it, never fail on it, never throw. The registry - // is imported dynamically so a half-edited lib/local-models.json shows up as a line here - // instead of breaking doctor. + // The fully local backends are optional: report them, never fail on them, never throw. Both + // modules are imported dynamically so a half-edited registry (or a broken Kev checkout) shows up + // as a line here instead of breaking doctor. const endpoint = await probeEndpoint({ config, live, models }); - try { - const { localStatus } = await import("./local.mjs"); - // Probe the port the *run* uses, so a Kev endpoint on 8008 is reported rather than the GGUF - // default of 8092 sitting idle beside it. - const port = loopbackPort(config.baseUrl) ?? undefined; - const local = await localStatus({ home, ...(port ? { port } : {}) }); - const ready = Boolean(local.llamaServer && local.bytes > 0); - const size = local.bytes > 0 ? `${Math.round(local.bytes / 1024 / 1024)} MiB` : `not downloaded (${Math.round(local.expectedBytes / 1024 / 1024)} MiB expected)`; - const livePort = local.serving - ? `127.0.0.1:${local.port} up${local.servingModel ? ` (serving ${local.servingModel})` : ""}` - : local.endpoint - ? `127.0.0.1:${local.port} up (${local.endpoint.kind}${local.endpoint.run ? ` ${local.endpoint.run}` : ` "${local.endpoint.name}"`})` - : `127.0.0.1:${local.port} not running`; - const detail = [ - local.llamaServer ? `llama-server ${local.llamaServer}` : "llama-server not found", - `${local.id}${local.label ? ` "${local.label}"` : ""} ${size}`, - livePort, - ].join(" · "); - const hint = local.endpoint - ? `the port your baseUrl uses is serving a ${local.endpoint.kind}, not this registry entry — see the goal_done bar line` - : !local.llamaServer - ? "brew install llama.cpp" - : local.bytes === 0 - ? `download it: node /bin/jev-local.mjs --download-only (registry: ${local.registry.path})` - : local.serving - ? undefined - : "start it: node /bin/jev-local.mjs (--list-models shows the registry)"; - add("local model", local.endpoint || ready ? true : null, detail, hint); - } catch (error) { - add("local model", null, `registry problem: ${error.message}`, "fix skills/jev-browser/lib/local-models.json, or run --list-models with a working file"); + const port = loopbackPort(config.baseUrl); + const kevTarget = tierByPort(port)?.name === "kev" || endpoint?.profile === "kev"; + if (kevTarget) { + // Kev needs a checkout, a venv that imports torch + mlx_lm and its cached assets — not the GGUF + // registry. A Kev endpoint never gets a jev-local hint. + try { + const { probeKevRuntime } = await import("./kev.mjs"); + const kev = await probeKevRuntime({ clone: path.join(home, ".local", "share", "jev-browser", "kev") }); + add( + "kev", + kev.ok ? true : null, + `${kev.clone} · ${kev.detail}`, + kev.ok ? undefined : "prepare it with: node /bin/jev-kev.mjs (it prints the clone and uv commands), or: node /bin/jev-browser.mjs setup kev", + ); + } catch (error) { + add("kev", null, `runtime check failed: ${error.message}`, "node /bin/jev-kev.mjs --help"); + } + } else { + try { + const { localStatus } = await import("./local.mjs"); + // Probe the port the *run* uses, so a Kev endpoint on 8008 is reported rather than the GGUF + // default of 8092 sitting idle beside it. + const local = await localStatus({ home, ...(port ? { port } : {}) }); + // The registry's exact byte size is the check: a non-empty file is not a usable model. + const truncated = local.bytes > 0 && local.expectedBytes > 0 && local.bytes !== local.expectedBytes; + const ready = Boolean(local.llamaServer && local.bytes > 0 && !truncated); + const size = + local.bytes === 0 + ? `not downloaded (${Math.round(local.expectedBytes / 1024 / 1024)} MiB expected)` + : truncated + ? `truncated (${local.bytes} of ${local.expectedBytes} bytes)` + : `${Math.round(local.bytes / 1024 / 1024)} MiB`; + const livePort = local.serving + ? `127.0.0.1:${local.port} up${local.servingModel ? ` (serving ${local.servingModel})` : ""}` + : local.endpoint + ? `127.0.0.1:${local.port} up (${local.endpoint.kind}${local.endpoint.run ? ` ${local.endpoint.run}` : ` "${local.endpoint.name}"`})` + : `127.0.0.1:${local.port} not running`; + const detail = [ + local.llamaServer ? `llama-server ${local.llamaServer}` : "llama-server not found", + `${local.id}${local.label ? ` "${local.label}"` : ""} ${size}`, + livePort, + ].join(" · "); + const hint = local.endpoint + ? `the port your baseUrl uses is serving a ${local.endpoint.kind}, not this registry entry — see the goal_done bar line` + : !local.llamaServer + ? "brew install llama.cpp" + : truncated + ? `delete it and download it again: node /bin/jev-local.mjs --download-only (or in one step: jev-browser setup local-readout)` + : local.bytes === 0 + ? `download it: node /bin/jev-local.mjs --download-only (registry: ${local.registry.path}); or in one step: jev-browser setup local-readout` + : local.serving + ? undefined + : "start it: node /bin/jev-local.mjs (--list-models shows the registry)"; + add("local model", local.endpoint || ready ? true : null, detail, hint); + } catch (error) { + add("local model", null, `registry problem: ${error.message}`, "fix skills/jev-browser/lib/local-models.json, or run --list-models with a working file"); + } } // The bar and the tier come from one resolution (lib/tiers.mjs), so the name on this line and the diff --git a/skills/jev-browser/lib/kev.mjs b/skills/jev-browser/lib/kev.mjs new file mode 100644 index 0000000..d539209 --- /dev/null +++ b/skills/jev-browser/lib/kev.mjs @@ -0,0 +1,45 @@ +// The Kev accuracy tier's runtime location and readiness probe, shared by its launcher +// (bin/jev-kev.mjs, which owns the fetch/verify/serve flow) and by `doctor` / `jev-browser setup`, +// so the "checkout + venv + torch/mlx" rules exist once. The checkout is a third-party clone the +// user makes — it lives outside this repo on purpose. +import { execFile } from "node:child_process"; +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import { promisify } from "node:util"; + +const run = promisify(execFile); + +/** Where jev-kev keeps the Kev checkout it serves from (default; its --clone flag overrides it). */ +export const KEV_DEFAULT_CLONE = path.join(os.homedir(), ".local", "share", "jev-browser", "kev"); + +/** The import that says the MLX serving extras are really installed in the venv. */ +export const KEV_IMPORT_PROBE = ["-c", "import torch, mlx_lm; print('ok')"]; + +/** The two commands that create the checkout and its venv — printed by the launcher and by doctor. */ +export const kevSetupCommands = (clone) => [ + `git clone --depth 1 https://github.com/jaredpalmer/kev.git ${clone}`, + `uv sync --extra serve --project ${clone}`, +]; + +/** uv is what builds the venv; it is not a Node dependency, so nothing else can install it. */ +export const KEV_UV_HINT = "install it first: brew install uv"; + +/** + * What Kev needs from this machine, without downloading or starting anything: the checkout, its + * venv, and a venv that can import torch + mlx_lm. + * + * Never throws — a probe that could not run is a state the callers report, not an exception. + * @returns {Promise<{ok: boolean, clone: string, python: string|null, kind: "ok"|"missing-clone"|"missing-venv"|"no-import", detail: string}>} + */ +export async function probeKevRuntime({ clone = KEV_DEFAULT_CLONE, timeoutMs = 60_000 } = {}) { + const python = path.join(clone, ".venv", "bin", "python"); + if (!fs.existsSync(clone)) return { ok: false, clone, python: null, kind: "missing-clone", detail: `no Kev checkout at ${clone}` }; + if (!fs.existsSync(python)) return { ok: false, clone, python, kind: "missing-venv", detail: `no Python venv at ${path.join(clone, ".venv")}` }; + const probe = await run(python, KEV_IMPORT_PROBE, { cwd: clone, timeout: timeoutMs }).catch((error) => error); + if (probe instanceof Error || probe.code !== 0) { + const detail = (probe instanceof Error ? probe.message : `${probe.stdout}${probe.stderr}`).trim().split("\n").slice(-3).join("\n"); + return { ok: false, clone, python, kind: "no-import", detail: `the venv at ${python} cannot import torch and mlx_lm\n ${detail}` }; + } + return { ok: true, clone, python, kind: "ok", detail: `${python} imports torch and mlx_lm` }; +} diff --git a/skills/jev-browser/lib/local.mjs b/skills/jev-browser/lib/local.mjs index ab3f1ee..35b0889 100644 --- a/skills/jev-browser/lib/local.mjs +++ b/skills/jev-browser/lib/local.mjs @@ -121,6 +121,18 @@ export function localPaths(home = os.homedir(), model = { id: LOCAL_MODEL_ID, fi /** `llama-server` on PATH, then the Homebrew prefix (macOS default). Never throws. */ export function findLlamaServer({ env = process.env, extra = ["/opt/homebrew/bin"] } = {}) { + // An explicit path wins outright: JEV_LLAMA_SERVER is how a build outside Homebrew is pointed + // at, and a value that does not resolve is "not found" rather than a silent fall-through. + const override = env.JEV_LLAMA_SERVER?.trim(); + if (override) { + try { + const stat = fs.statSync(override); + if (stat.isFile() && (stat.mode & 0o111) !== 0) return override; + } catch { + // fall through to not-found + } + return null; + } const dirs = [...new Set([...(env.PATH ?? "").split(path.delimiter).filter(Boolean), ...extra])]; for (const dir of dirs) { const candidate = path.join(dir, "llama-server"); @@ -134,6 +146,15 @@ export function findLlamaServer({ env = process.env, extra = ["/opt/homebrew/bin return null; } +/** The one hint for a missing llama.cpp, shared by the launcher and `jev-browser setup`. */ +export function llamaMissingHint({ env = process.env } = {}) { + const override = env.JEV_LLAMA_SERVER?.trim(); + const where = override + ? `llama-server not found at ${override} (JEV_LLAMA_SERVER overrides PATH and /opt/homebrew/bin)` + : "llama-server not found (looked on PATH and in /opt/homebrew/bin)"; + return `${where}.\n\nInstall it with:\n\n brew install llama.cpp\n`; +} + /** GET JSON with a short timeout; null on any failure (unreachable, non-JSON, timeout). */ export async function getJson(url, timeoutMs = 1500) { try { diff --git a/skills/jev-browser/lib/setup.mjs b/skills/jev-browser/lib/setup.mjs new file mode 100644 index 0000000..f1b2037 --- /dev/null +++ b/skills/jev-browser/lib/setup.mjs @@ -0,0 +1,432 @@ +// `jev-browser setup`: the one-click path to a local judging backend. It composes the launchers — +// their `--download-only` to fetch, their `--detach` to serve — and never re-implements them, then +// proves the endpoint answers one real question before it reports success. +// +// Every write lands under the effective home directory (the user config, ~/.jev-browser/run, the +// model cache, and by default the Kev checkout), so `--home ` isolates a scratch setup. +import { execFile, spawn } from "node:child_process"; +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { promisify } from "node:util"; +import { fileURLToPath } from "node:url"; +import { DEFAULT_TIER, describeTier, tierByName } from "./tiers.mjs"; +import { defaultLocalModel, findLlamaServer, llamaMissingHint, localModel, localPaths } from "./local.mjs"; +import { KEV_UV_HINT, kevSetupCommands, probeKevRuntime } from "./kev.mjs"; +import { saveUserConfig, userConfigPath } from "./config.mjs"; +import { TypeSafeClient } from "./typesafe.mjs"; +import { runPaths, waitUntilReady } from "./detach.mjs"; + +const exec = promisify(execFile); +const SKILL_DIR = path.resolve(path.dirname(fileURLToPath(import.meta.url)), ".."); + +/** The tiers `setup` can install. Hosted Jev is the default and needs nothing. */ +export const SETUP_TIERS = ["local-readout", "kev"]; +export const SETUP_USAGE = "usage: jev-browser setup [local-readout | kev | status | stop ] [--model-name ] [--skip-deps] [--json]"; + +const configError = (message) => Object.assign(new Error(message), { config: true }); + +/** The question a fresh setup asks through the endpoint it just started: the proof it prints. */ +export const VERIFY_STATE = { ticket: "My card was charged twice" }; +export const VERIFY_QUESTIONS = { refund: { type: "noul", instructions: "Does `ticket` ask for a refund?" } }; + +/** Ask the endpoint one real question and return its parsed answer. Throws when it cannot answer. */ +export async function verifyEndpoint({ baseUrl, apiKey, model, timeoutMs = 120_000 }) { + const client = new TypeSafeClient({ apiKey, baseUrl, model, timeoutMs, maxRetries: 0 }); + const started = performance.now(); + const result = await client.systemOne({ state: VERIFY_STATE, questions: VERIFY_QUESTIONS }); + const answer = result.answers.refund; + const detail = answer.type === "noul" ? `noul: P(yes)=${answer.noul.toFixed(2)}` : `${answer.type}: ${answer.choice ?? answer.score}`; + return { answer, detail, ms: Math.round(performance.now() - started), model: result.model, costUsd: result.costUsd }; +} + +/** Run a child with the terminal attached, printing the command first (clone / uv sync / fetch). */ +async function runStepStreaming({ command, args, env, cwd, log }) { + log(`$ ${command} ${args.join(" ")}`); + return { + code: await new Promise((resolve) => { + const child = spawn(command, args, { cwd, env, stdio: ["ignore", "inherit", "inherit"] }); + child.once("error", () => resolve(1)); + child.once("exit", (code) => resolve(code ?? 1)); + }), + }; +} + +/** Start a launcher detached: it returns once it printed its env line, or failed with a log. */ +async function startLauncherDetached({ script, args, env, cwd, timeoutMs }) { + const child = spawn(process.execPath, [script, ...args], { cwd, env, stdio: ["ignore", "pipe", "inherit"] }); + let stdout = ""; + child.stdout.on("data", (chunk) => { + stdout += chunk; + }); + const code = await new Promise((resolve) => { + const timer = setTimeout(() => { + try { + child.kill("SIGTERM"); + } catch { + // already gone + } + resolve(124); + }, timeoutMs); + child.once("error", () => { + clearTimeout(timer); + resolve(1); + }); + child.once("exit", (exit) => { + clearTimeout(timer); + resolve(exit ?? 1); + }); + }); + return { code, stdout: stdout.trim() }; +} + +/** `uv` on PATH, or null. uv builds Kev's venv and is not a Node dependency. */ +async function findUv({ env = process.env } = {}) { + try { + const { stdout } = await exec("uv", ["--version"], { env, timeout: 20_000 }); + return { path: "uv", version: String(stdout).trim() }; + } catch { + return null; + } +} + +function defaultDeps(env) { + return { + findLlamaServer: () => findLlamaServer({ env }), + findUv: () => findUv({ env }), + runStep: runStepStreaming, + startDetached: startLauncherDetached, + verify: (options) => verifyEndpoint(options), + kevRuntime: (options = {}) => probeKevRuntime(options), + }; +} + +const exists = (file) => + fs + .stat(file) + .then(() => true) + .catch(() => false); + +const statOrNull = (file) => fs.stat(file).catch(() => null); + +const readJsonOrNull = async (file) => { + try { + return JSON.parse(await fs.readFile(file, "utf8")); + } catch { + return null; + } +}; + +async function readPid(file) { + const text = await fs.readFile(file, "utf8").catch(() => null); + if (text === null) return null; + const pid = Number(text.trim()); + return Number.isInteger(pid) && pid > 0 ? pid : null; +} + +/** Is `pid` alive, and does its command line look like the launcher that wrote the pid file? */ +async function pidState(pid, expect = null) { + if (!pid) return { pid, alive: false, ours: false, command: null }; + try { + const { stdout } = await exec("ps", ["-p", String(pid), "-o", "command="], { timeout: 5000 }); + const command = String(stdout).trim(); + if (!command) return { pid, alive: false, ours: false, command: null }; + return { pid, alive: true, ours: expect ? command.includes(expect) : true, command }; + } catch { + return { pid, alive: false, ours: false, command: null }; + } +} + +async function getJsonOrNull(url, timeoutMs = 1500) { + try { + return await (await fetch(url, { signal: AbortSignal.timeout(timeoutMs) })).json(); + } catch { + return null; + } +} + +const launcherPath = (tier, skillDir) => path.join(skillDir, "bin", tier === "kev" ? "jev-kev.mjs" : "jev-local.mjs"); +const pidName = (tier) => (tier === "kev" ? "jev-kev" : "jev-local"); +const kevClone = (home) => path.join(home, ".local", "share", "jev-browser", "kev"); +const humanBytes = (bytes) => (bytes >= 1024 ** 3 ? `${(bytes / 1024 ** 3).toFixed(2)} GiB` : `${Math.round(bytes / 1024 ** 2)} MiB`); + +/** What either endpoint answers when it is up. */ +async function endpointServing(tier, port) { + if (tier === "kev") { + const card = await getJsonOrNull(`http://127.0.0.1:${port}/v1/models`); + return Boolean(card?.models?.length); + } + const health = await getJsonOrNull(`http://127.0.0.1:${port}/health`); + return health?.service === "jev-local"; +} + +/** Installed (and verified) state for one tier, from the files this machine actually has. */ +async function installedState({ tier, home, deps }) { + if (tier === "kev") { + const clone = kevClone(home); + const runtime = await deps.kevRuntime({ clone }); + const checkpoint = await exists(path.join(home, ".cache", "huggingface", "hub", "models--jaredpalmer--kev-4b")); + const base = await exists(path.join(home, ".cache", "huggingface", "hub", "models--Qwen--Qwen3.5-4B-Base")); + return { installed: runtime.ok && checkpoint && base, detail: `${runtime.detail} · assets ${checkpoint && base ? "cached" : "not cached (the launcher verifies them with --verify-only)"}` }; + } + let entry; + try { + entry = defaultLocalModel(); + } catch (error) { + return { installed: false, detail: `registry problem: ${error.message}` }; + } + const { modelFile } = localPaths(home, entry); + const stat = await statOrNull(modelFile); + const bytes = stat?.size ?? 0; + return { + installed: bytes > 0 && bytes === entry.bytes, + detail: + bytes === 0 + ? `${entry.id} not downloaded (${humanBytes(entry.bytes)} expected)` + : bytes === entry.bytes + ? `${entry.id} verified (${humanBytes(bytes)})` + : `${entry.id} truncated (${bytes} of ${entry.bytes} bytes)`, + }; +} + +/** installed / running / configured / serving for one tier. */ +async function tierState({ tier: name, home, skillDir, deps }) { + const tier = tierByName(name); + const paths = runPaths(home, pidName(name), tier.port); + const pid = await readPid(paths.pidFile); + const alive = await pidState(pid, path.basename(launcherPath(name, skillDir))); + const userConfig = await readJsonOrNull(userConfigPath(home)); + const installed = await installedState({ tier: name, home, deps }); + return { + tier: name, + baseUrl: tier.baseUrl, + port: tier.port, + installed: installed.installed, + installedDetail: installed.detail, + running: alive.alive, + serving: await endpointServing(name, tier.port), + configured: userConfig?.baseUrl === tier.baseUrl && Boolean(userConfig?.apiKey), + pid: alive.alive ? pid : null, + pidFile: paths.pidFile, + logFile: paths.logFile, + setupCommand: `node ${path.join(skillDir, "bin", "jev-browser.mjs")} setup ${name}`, + }; +} + +const stateFlags = (state) => `${state.installed ? "installed" : "not installed"} · ${state.running || state.serving ? "running" : "not running"} · ${state.configured ? "configured" : "not configured"}`; +const stateText = (state) => [`${state.tier.padEnd(15)} ${stateFlags(state)}`, ` ${state.installedDetail}`, ` log ${state.logFile}`]; + +/** `jev-browser setup` — what is installed, what is running, what a run uses, and what to type next. */ +async function overview({ home, config, skillDir, deps }) { + const states = []; + for (const tier of SETUP_TIERS) states.push(await tierState({ tier, home, skillDir, deps })); + const using = describeTier({ config, classification: null, skillDir }); + const lines = ["jev-browser setup — the local judging backends", "", `a run right now uses: ${using.tier} — ${config.baseUrl}`, ""]; + for (const state of states) { + lines.push(...stateText(state)); + if (!(state.installed && state.configured)) lines.push(` set it up: ${state.setupCommand}`); + lines.push(""); + } + lines.push(`details: jev-browser setup status · stop one: jev-browser setup stop <${SETUP_TIERS.join("|")}>`); + lines.push("hosted Jev is the default and needs no setup."); + return { code: 0, text: lines.join("\n"), json: { uses: { tier: using.tier, baseUrl: config.baseUrl }, tiers: states } }; +} + +/** `jev-browser setup status` — per local tier: installed, running, configured, log. */ +async function statusReport({ home, skillDir, deps }) { + const states = []; + for (const tier of SETUP_TIERS) states.push(await tierState({ tier, home, skillDir, deps })); + const lines = ["jev-browser setup status", ""]; + for (const state of states) { + lines.push(...stateText(state)); + lines.push(""); + } + const next = states.filter((state) => !(state.installed && state.configured)); + lines.push(next.length ? `not set up yet: ${next.map((state) => state.setupCommand).join(" · ")}` : "every local tier is installed and configured."); + return { code: 0, text: lines.join("\n"), json: { tiers: states } }; +} + +/** `jev-browser setup stop ` — stop the process this pid file names, and nothing else. */ +async function stopTier({ tier: name, home, skillDir, log }) { + const tier = tierByName(name); + if (!tier || !SETUP_TIERS.includes(name)) throw configError(`unknown tier "${name}" (expected ${SETUP_TIERS.join(", ")})`); + const paths = runPaths(home, pidName(name), tier.port); + const pid = await readPid(paths.pidFile); + if (!pid) { + return { + code: 0, + text: `nothing to stop: no pid file at ${paths.pidFile}\n${name} was not started by setup (or it already exited); a server you started by hand is stopped with Ctrl-C.`, + json: { tier: name, stopped: false, reason: "no pid file", pidFile: paths.pidFile }, + }; + } + const live = await pidState(pid, path.basename(launcherPath(name, skillDir))); + if (!live.alive) { + await fs.rm(paths.pidFile, { force: true }); + return { code: 0, text: `removed a stale pid file: pid ${pid} is gone (${paths.pidFile})`, json: { tier: name, stopped: false, reason: "stale pid file", pid } }; + } + if (!live.ours) { + return { + code: 1, + text: `pid ${pid} is not a ${name} launcher (${live.command}); leaving it alone.\nDelete ${paths.pidFile} if that file is stale.`, + json: { tier: name, stopped: false, reason: "pid is not ours", pid, command: live.command }, + }; + } + log(`stopping ${name} (pid ${pid})…`); + try { + process.kill(pid, "SIGTERM"); + } catch (error) { + return { code: 1, text: `could not signal pid ${pid}: ${error.message}`, json: { tier: name, stopped: false, pid, error: error.message } }; + } + const gone = await waitUntilReady(async () => !(await pidState(pid)).alive, { timeoutMs: 15_000, intervalMs: 250 }); + if (!gone) return { code: 1, text: `sent SIGTERM to pid ${pid} but it is still running after 15s — see ${paths.logFile}`, json: { tier: name, stopped: false, pid, logFile: paths.logFile } }; + await fs.rm(paths.pidFile, { force: true }); + return { code: 0, text: `stopped ${name} (pid ${pid}).\nlog: ${paths.logFile}`, json: { tier: name, stopped: true, pid, logFile: paths.logFile } }; +} + +/** A failed child: exit 2 stays a usage/config problem, anything else is a runtime failure. */ +function stepFailure(what, code, logFile) { + const message = `${what} failed (exit ${code})${logFile ? `; log: ${logFile}` : ""}`; + return code === 2 ? configError(message) : new Error(message); +} + +/** Write baseUrl+apiKey, then prove the endpoint answers. Shared by both tiers. */ +async function finishTier({ tier, home, skillDir, deps, paths, started, model, log }) { + const configFile = await saveUserConfig({ baseUrl: tier.baseUrl, apiKey: tier.apiKey }, { home }); + const verified = await deps.verify({ baseUrl: tier.baseUrl, apiKey: tier.apiKey, model }).catch((error) => { + throw new Error(`the endpoint is up but did not answer the verification question: ${error.message}\nlog: ${paths.logFile}`); + }); + const pid = await readPid(paths.pidFile); + log(`verified ${tier.baseUrl}: ${verified.detail} in ${verified.ms} ms`); + const lines = [ + `jev-browser setup ${tier.name}`, + "", + ` server 127.0.0.1:${tier.port}${pid ? ` (pid ${pid})` : ""}`, + ` config ${configFile} — baseUrl + apiKey=${tier.apiKey}`, + ` log ${paths.logFile}`, + ` verified ${verified.detail} — ${verified.ms} ms through ${tier.baseUrl}`, + "", + `A run right now uses the ${tier.name} tier:`, + "", + ` node ${path.join(skillDir, "bin", "jev-browser.mjs")} run --goal "…" --url https://example.com`, + "", + `stop it with: jev-browser setup stop ${tier.name}`, + ]; + return { + code: 0, + text: lines.join("\n"), + json: { + tier: tier.name, + baseUrl: tier.baseUrl, + apiKey: tier.apiKey, + port: tier.port, + pid, + pidFile: paths.pidFile, + logFile: paths.logFile, + configFile, + verified: { detail: verified.detail, ms: verified.ms, answer: verified.answer, model: verified.model }, + }, + }; +} + +/** `jev-browser setup local-readout` — llama.cpp check, unattended fetch, detached serve, verify. */ +async function setupLocalReadout({ tier, home, skillDir, env, deps, options, log }) { + let entry; + try { + entry = options.modelName ? localModel(options.modelName) : defaultLocalModel(); + } catch (error) { + throw error?.name === "LocalModelRegistryError" ? configError(error.message) : error; + } + const llama = deps.findLlamaServer(); + if (!llama) throw configError(llamaMissingHint({ env })); + + const launcher = launcherPath(tier.name, skillDir); + const paths = runPaths(home, pidName(tier.name), tier.port); + const { modelFile } = localPaths(home, entry); + const stat = await statOrNull(modelFile); + const onDisk = stat?.size ?? 0; + const ready = onDisk > 0 && onDisk === entry.bytes; + log(`[setup] llama.cpp: ${llama}`); + log( + `[setup] model ${entry.id}: ${ + ready ? `already on disk (${humanBytes(onDisk)})` : onDisk === 0 ? `not downloaded (${humanBytes(entry.bytes)})` : `truncated (${onDisk} of ${entry.bytes} bytes)` + }`, + ); + + if (!ready) { + const fetch = await deps.runStep({ command: process.execPath, args: [launcher, "--download-only", "--model-name", entry.id], env: { ...env, HOME: home }, cwd: skillDir, log }); + if (fetch.code !== 0) throw stepFailure(`fetching ${entry.id}`, fetch.code, paths.logFile); + } + + const started = await deps.startDetached({ script: launcher, args: ["--detach", "--model-name", entry.id], env: { ...env, HOME: home }, cwd: skillDir, timeoutMs: options.timeoutMs ?? 300_000 }); + if (started.code !== 0) throw stepFailure("starting jev-local", started.code, paths.logFile); + log(`[setup] serving on 127.0.0.1:${tier.port} (${started.stdout || "env line printed"})`); + + const out = await finishTier({ tier, home, skillDir, deps, paths, started, model: options.config?.model, log }); + out.json.model = { id: entry.id, file: entry.file, bytes: entry.bytes, url: entry.url }; + return out; +} + +/** `jev-browser setup kev` — uv check, clone + uv sync (unless --skip-deps), fetch, serve, verify. */ +async function setupKev({ tier, home, skillDir, env, deps, options, log }) { + const launcher = launcherPath(tier.name, skillDir); + const paths = runPaths(home, pidName(tier.name), tier.port); + const clone = kevClone(home); + + if (!options.skipDeps) { + const uv = await deps.findUv(); + if (!uv) throw configError(`uv not found on PATH — ${KEV_UV_HINT}\n\nKev's venv is built with uv:\n\n ${kevSetupCommands(clone).join("\n ")}`); + log(`[setup] ${uv.version}`); + if (!(await exists(clone))) { + const step = await deps.runStep({ command: "git", args: ["clone", "--depth", "1", "https://github.com/jaredpalmer/kev.git", clone], env, cwd: home, log }); + if (step.code !== 0) throw stepFailure("cloning the Kev checkout", step.code); + } + const sync = await deps.runStep({ command: uv.path, args: ["sync", "--extra", "serve", "--project", clone], env, cwd: home, log }); + if (sync.code !== 0) throw stepFailure("uv sync --extra serve", sync.code); + } + + const runtime = await deps.kevRuntime({ clone }); + if (!runtime.ok) throw configError(`${runtime.detail}\n\nPrepare it with:\n\n ${kevSetupCommands(clone).join("\n ")}\n\n(${KEV_UV_HINT})`); + log(`[setup] ${runtime.detail}`); + + const fetch = await deps.runStep({ command: process.execPath, args: [launcher, "--download-only", "--clone", clone], env: { ...env, HOME: home }, cwd: skillDir, log }); + if (fetch.code !== 0) throw stepFailure("fetching the Kev checkpoint", fetch.code, paths.logFile); + + const started = await deps.startDetached({ script: launcher, args: ["--detach", "--clone", clone], env: { ...env, HOME: home }, cwd: skillDir, timeoutMs: options.timeoutMs ?? 900_000 }); + if (started.code !== 0) throw stepFailure("starting jev-kev", started.code, paths.logFile); + log(`[setup] serving on 127.0.0.1:${tier.port} (${started.stdout || "env line printed"})`); + + const out = await finishTier({ tier, home, skillDir, deps, paths, started, model: options.config?.model, log }); + out.json.clone = clone; + return out; +} + +/** + * The `setup` command. Throws a `config: true` error for anything the user has to change (exit 2), + * a plain error for a runtime failure (exit 1), and returns `{ code, text, json }` otherwise. + */ +export async function setupCommand({ sub, rest = [], options = {}, deps = {} } = {}) { + const env = options.env ?? process.env; + const home = options.home ?? os.homedir(); + const skillDir = options.skillDir ?? SKILL_DIR; + const log = options.log ?? ((message) => process.stderr.write(`${message}\n`)); + const seams = { ...defaultDeps(env), ...deps }; + + if (sub === undefined) { + if (!options.config) throw configError(`${SETUP_USAGE} (setup needs the effective configuration, which only the CLI loads)`); + return overview({ home, config: options.config, skillDir, deps: seams }); + } + if (sub === "status") return statusReport({ home, skillDir, deps: seams }); + if (sub === "stop") return stopTier({ tier: rest[0], home, skillDir, log }); + if (sub === DEFAULT_TIER) { + return { + code: 0, + text: `hosted Jev needs no setup — it is the default.\n\n export TYPESAFE_API_KEY=... # https://console.typesafe.ai\n node ${path.join(skillDir, "bin", "jev-browser.mjs")} doctor\n\nA local tier is optional: jev-browser setup ${SETUP_TIERS.join(" | ")}`, + json: { tier: DEFAULT_TIER, needsSetup: false }, + }; + } + const tier = SETUP_TIERS.includes(sub) ? tierByName(sub) : null; + if (!tier) throw configError(`unknown setup target "${sub}"\n${SETUP_USAGE}`); + if (options.modelName && sub !== "local-readout") throw configError("--model-name applies to `setup local-readout` (Kev serves its pinned checkpoint)"); + if (options.skipDeps && sub !== "kev") throw configError("--skip-deps applies to `setup kev`"); + return sub === "kev" ? setupKev({ tier, home, skillDir, env, deps: seams, options, log }) : setupLocalReadout({ tier, home, skillDir, env, deps: seams, options, log }); +} diff --git a/skills/jev-browser/lib/tiers.mjs b/skills/jev-browser/lib/tiers.mjs index 63b04e2..357782a 100644 --- a/skills/jev-browser/lib/tiers.mjs +++ b/skills/jev-browser/lib/tiers.mjs @@ -252,7 +252,9 @@ export function formatTierStatus(status) { /** * `tier use` — the export line, the way both launchers print it. Nothing is written to disk here; - * only `--persist` (or `config set baseUrl`) stores anything, and the text says so either way. + * only `--persist` (or `config set baseUrl`) stores anything: a local tier's baseUrl and its + * placeholder apiKey, because a key the client can see is what a local run still needs; hosted + * stores its baseUrl alone. The text says which either way. */ export function formatTierUse(tier, { skillDir = null, persisted = null, configPath = userConfigPath() } = {}) { const out = []; @@ -284,6 +286,10 @@ export function formatTierUse(tier, { skillDir = null, persisted = null, configP ); } out.push(""); - out.push(persisted ? `Stored baseUrl=${tier.baseUrl} in ${persisted} — runs use it without the export.` : `Nothing was written: add --persist to store baseUrl in ${configPath} instead of exporting it.`); + out.push( + persisted + ? `Stored ${tier.apiKey ? `apiKey=${tier.apiKey} and baseUrl=${tier.baseUrl}` : `baseUrl=${tier.baseUrl}`} in ${persisted} — runs use it without the export.` + : `Nothing was written: add --persist to store ${tier.apiKey ? "that tier's baseUrl and apiKey" : "baseUrl"} in ${configPath} instead of exporting it.`, + ); return out.join("\n"); } diff --git a/skills/jev-browser/lib/webui-page.mjs b/skills/jev-browser/lib/webui-page.mjs new file mode 100644 index 0000000..4da4077 --- /dev/null +++ b/skills/jev-browser/lib/webui-page.mjs @@ -0,0 +1,800 @@ +// The page the local WebUI serves. One file, inline CSS and JS, no external assets and no CDN — +// the server has no route that could fetch one, and the page carries no configuration: everything +// it shows arrives later from /api/*, so a served page can never contain an API key. +// +// The markup is a template literal, so the embedded script uses string concatenation and +// document.createElement (never its own template literals) and renders every value with +// textContent — which is also what keeps a config value or a log line from becoming markup. +export const WEBUI_PAGE = ` + + + + +jev-browser WebUI + + + +
+

jev-browser WebUI

+ 127.0.0.1 only + loading… +
+ +
+
+
+

What a run would use right now

+

+      
+ + Same resolution as jev-browser tier status — lib/tiers.mjs, not a copy. +
+
+
+
+

Launcher output

+

stdout / stderr of the local server this page started — the model download shows up here.

+
+
+
+ +
+
+

Edit (writes ~/.config/jev-browser/config.json)

+
+ + + + + + + + + +
+ + + +
+
+
+
+

Effective configuration and where each value came from

+
+

+
+
+

Unset a key this WebUI wrote

+
+

Only keys present in the user config file are listed; unsetting restores whatever the defaults and the environment provide.

+
+ +
+ +
+
+

Environment

+
+ + + +
+
+
+
+ +
+
+ +
+
+

Registry (lib/local-models.json)

+
+

+
+
+

Use a file that is already on disk

+
+ + +
+

+
+
+

Local backend status

+
loading…
+
+
+ +
+
+

One request against the configured endpoint

+
+ + + + +
+ + + +
+
+
+
+

Answers

+

Nothing yet.

+
+
+ +
+
+

Run a goal

+
+
+ + + + + +
+
+
+

inputs — sent to the model as candidate values

+ +
+
+
+
+
+

secrets — typed but never sent to the model, never echoed here

+ +
+
+
+
+ + + +
+
+
+
+

Progress (stderr of jev-browser run)

+
no run yet
+
+
+

Result

+

Nothing yet.

+
+
+

Journal (per step)

+

Nothing yet.

+
+
+
+ + + +`; diff --git a/skills/jev-browser/lib/webui.mjs b/skills/jev-browser/lib/webui.mjs new file mode 100644 index 0000000..c567019 --- /dev/null +++ b/skills/jev-browser/lib/webui.mjs @@ -0,0 +1,980 @@ +// The local WebUI: a loopback-only page over the same lib/ every CLI command uses. +// +// Every panel is an adapter, never a second implementation: the tiers and the resolved goal_done +// bar come from lib/tiers.mjs (describeTier + probeEndpoint), health from lib/doctor.mjs, config +// reads and writes from lib/config.mjs, the model registry from lib/local.mjs. That is why the page +// can never disagree with `jev-browser tier status`, `doctor` or `config show`. +// +// A local tool that spawns processes on request is the one genuinely dangerous part of a WebUI, so +// every field is treated as hostile input: the scripts come from this module's own bin/ (never from +// a request body), arguments are validated and enumerated (tier from TIERS, model from the +// registry, paths under one root, ports as integers in range), and children are spawned with an +// argv array — never a shell, so no value can be re-parsed into a command. +import fs from "node:fs/promises"; +import http from "node:http"; +import os from "node:os"; +import path from "node:path"; +import { spawn as nodeSpawn } from "node:child_process"; +import { fileURLToPath } from "node:url"; +import { DEFAULTS, PROFILE_NAMES, THRESHOLD_PROFILES, describeConfig, loadConfig, saveUserConfig, unsetUserConfig } from "./config.mjs"; +import { DEFAULT_TIER, TIERS, describeTier, fetchModels, formatTierStatus, formatTierUse, launcherCommand, probeEndpoint, tierByName, tierEnv, tierRows } from "./tiers.mjs"; +import { doctor, formatDoctor } from "./doctor.mjs"; +import { TypeSafeClient, isLoopbackBaseUrl, validateQuestions } from "./typesafe.mjs"; +import { expandHome, readJson } from "./util.mjs"; +import { WEBUI_PAGE } from "./webui-page.mjs"; + +/** The only address this server ever binds. */ +export const LOOPBACK = "127.0.0.1"; +export const DEFAULT_WEBUI_PORT = 8765; + +/** The skill directory this module belongs to; every script the WebUI spawns is resolved under it. */ +const MODULE_SKILL_DIR = path.resolve(path.dirname(fileURLToPath(import.meta.url)), ".."); + +/** The channels a client may read logs from. Anything else is a 400, not a file read. */ +export const LOG_CHANNELS = Object.freeze(["run", ...TIERS.filter((tier) => tier.launcher).map((tier) => `tier:${tier.name}`)]); + +/** + * The route table, as data: the dispatcher and the 405 answer are both built from it, so the + * documented list and the served list cannot drift apart. + */ +export const ROUTES = Object.freeze([ + { method: "GET", path: "/", purpose: "the page itself (static, no config in it)" }, + { method: "GET", path: "/api/tiers", purpose: "the three tiers + the live tier/endpoint/bar a run would use" }, + { method: "POST", path: "/api/tiers/start", purpose: "start a local tier's launcher (argv array, pinned script)" }, + { method: "POST", path: "/api/tiers/stop", purpose: "stop the launcher this server started (whole process group)" }, + { method: "GET", path: "/api/tiers/use", purpose: "what `tier use` prints, without writing anything" }, + { method: "POST", path: "/api/tiers/use", purpose: "store that tier's baseUrl in the user config (explicit write)" }, + { method: "GET", path: "/api/logs", purpose: "buffered stdout/stderr of a child, from a cursor" }, + { method: "GET", path: "/api/config", purpose: "effective config (key value never sent) + sources + user-set keys" }, + { method: "POST", path: "/api/config", purpose: "save an allow-listed patch through lib/config.mjs" }, + { method: "POST", path: "/api/config/unset", purpose: "remove one key from the user config" }, + { method: "GET", path: "/api/doctor", purpose: "lib/doctor.mjs, live or offline" }, + { method: "GET", path: "/api/models", purpose: "the local registry, what is on disk, what is serving" }, + { method: "POST", path: "/api/models/start", purpose: "start the GGUF launcher for a chosen registry id or file" }, + { method: "POST", path: "/api/judge", purpose: "one TypeSafe System One call against the effective endpoint" }, + { method: "POST", path: "/api/run", purpose: "spawn `jev-browser run --json` (argv array, never a shell)" }, + { method: "GET", path: "/api/run", purpose: "the last run's state, result JSON and journal rows" }, + { method: "POST", path: "/api/run/stop", purpose: "kill the run child" }, +]); + +const MAX_LOG_LINES = 500; +const MAX_BODY_BYTES = 1 << 20; +const MAX_CHILD_OUTPUT = 2 << 20; + +/** A usage/config problem the user has to fix: 4xx, with the message shown as-is. */ +export class WebUiError extends Error { + constructor(message, status = 400) { + super(message); + this.name = "WebUiError"; + this.status = status; + } +} + +const isPlainObject = (value) => value !== null && typeof value === "object" && !Array.isArray(value); + +/** True when `child` is `parent` or lives inside it (after both are resolved). */ +const isWithin = (child, parent) => child === parent || child.startsWith(parent.endsWith(path.sep) ? parent : `${parent}${path.sep}`); + +/** + * ~/.jev-browser/models — the one directory a `--model ` may name. It is the same directory + * lib/local.mjs's localPaths() builds, kept as an expression here so validation needs no registry + * import (a half-edited local-models.json must not be able to widen it). + */ +export const modelRoot = (home = os.homedir()) => path.join(home, ".jev-browser", "models"); + +// ----------------------------------------------------------------------------- validation + +export function parsePort(raw, { name = "port", min = 1024, max = 65535 } = {}) { + const value = typeof raw === "string" && raw.trim() !== "" ? Number(raw) : raw; + if (!Number.isInteger(value)) throw new WebUiError(`${name} must be a whole number (got ${JSON.stringify(raw)})`); + if (value < min || value > max) throw new WebUiError(`${name} must be between ${min} and ${max} (got ${value})`); + return value; +} + +export function requireNumber(raw, name, { min, max, integer = false } = {}) { + const value = typeof raw === "string" && raw.trim() !== "" ? Number(raw) : raw; + if (typeof value !== "number" || !Number.isFinite(value)) throw new WebUiError(`${name} must be a number (got ${JSON.stringify(raw)})`); + if (integer && !Number.isInteger(value)) throw new WebUiError(`${name} must be a whole number (got ${value})`); + if (min !== undefined && value < min) throw new WebUiError(`${name} must be at least ${min} (got ${value})`); + if (max !== undefined && value > max) throw new WebUiError(`${name} must be at most ${max} (got ${value})`); + return value; +} + +export function requireChoice(raw, name, options) { + if (typeof raw !== "string" || !options.includes(raw)) throw new WebUiError(`${name} must be one of ${options.join(", ")} (got ${JSON.stringify(raw)})`); + return raw; +} + +export function requireText(raw, name, maxChars) { + if (typeof raw !== "string" || !raw.trim()) throw new WebUiError(`${name} is required`); + if (raw.length > maxChars) throw new WebUiError(`${name} is longer than ${maxChars} characters`); + return raw.trim(); +} + +export function requireUrl(raw, name = "url") { + const value = requireText(raw, name, 4000); + let parsed; + try { + parsed = new URL(value); + } catch { + throw new WebUiError(`${name} must be an absolute http(s) URL`); + } + if (parsed.protocol !== "http:" && parsed.protocol !== "https:") throw new WebUiError(`${name} must be an http(s) URL`); + return value; +} + +export function requireToken(raw, name) { + const value = requireText(raw, name, 128); + if (!/^[A-Za-z0-9_.:-]+$/.test(value)) throw new WebUiError(`${name} must be letters, digits, ".", ":", "_" or "-" (got ${JSON.stringify(value)})`); + return value; +} + +export function requireApiKey(raw) { + const value = requireText(raw, "apiKey", 512); + if (/[\r\n]/.test(value)) throw new WebUiError("apiKey must be a single line"); + return value; +} + +/** One tier of TIERS, or a 400 naming the ones that exist. */ +export function requireTier(raw) { + const tier = tierByName(raw); + if (!tier) throw new WebUiError(`unknown tier ${JSON.stringify(raw)} (expected ${TIERS.map((tier) => tier.name).join(", ")})`); + return tier; +} + +/** A registry id, checked against the registry itself rather than a hard-coded list. */ +export function requireModelId(raw, registry) { + const id = requireToken(raw, "modelName"); + if (!registry?.models?.[id]) { + throw new WebUiError(`unknown model id ${JSON.stringify(id)} (the registry has ${Object.keys(registry?.models ?? {}).join(", ") || "no models"})`); + } + return id; +} + +/** + * A `--model ` value: a .gguf file inside MODEL_ROOTS. Anything else — a relative path, a + * symlink target outside the root, a shell-ish string — is rejected before a process exists. + */ +export function resolveModelPath(raw, { home = os.homedir() } = {}) { + const value = requireText(raw, "modelPath", 4096); + if (!value.endsWith(".gguf")) throw new WebUiError("modelPath must name a .gguf file"); + const resolved = path.resolve(expandHome(value, home)); + const root = modelRoot(home); + if (!isWithin(resolved, root)) throw new WebUiError(`modelPath must be under ${root}`); + return resolved; +} + +/** Request key=value rows: an array of {key, value}, as `--input` / `--secret` see them. */ +export function requirePairs(raw, name) { + if (raw === undefined || raw === null) return {}; + const list = Array.isArray(raw) ? raw : isPlainObject(raw) ? Object.entries(raw).map(([key, value]) => ({ key, value })) : null; + if (!list) throw new WebUiError(`${name} must be an array of { key, value } rows`); + const out = {}; + for (const row of list) { + if (!isPlainObject(row)) throw new WebUiError(`${name} rows must be objects with key and value`); + const { key, value } = row; + if (key === undefined || key === "" || value === undefined || value === null || value === "") continue; // an empty row is not an error + if (typeof key !== "string" || !/^[A-Za-z0-9_.-]{1,64}$/.test(key)) throw new WebUiError(`${name} key ${JSON.stringify(key)} must match [A-Za-z0-9_.-]{1,64}`); + if (typeof value !== "string") throw new WebUiError(`${name}.${key} must be a string`); + if (value.length > 2000) throw new WebUiError(`${name}.${key} is longer than 2000 characters`); + out[key] = value; + } + return out; +} + +/** A path like "a.b.c" must not be able to reach Object.prototype. */ +function assertSafeKeyPath(key) { + for (const part of key.split(".")) { + if (part === "__proto__" || part === "constructor" || part === "prototype") throw new WebUiError(`config key ${JSON.stringify(key)} is not allowed`); + } +} + +/** + * The config keys the WebUI may write, each with its own validator. Everything else is refused, so + * a request cannot reach a config field the panel does not show (chrome.extraArgs, say). + */ +export const EDITABLE_KEYS = Object.freeze({ + baseUrl: (value) => requireUrl(value, "baseUrl"), + model: (value) => requireToken(value, "model"), + backend: (value) => requireChoice(value, "backend", ["ego", "chrome", "safari"]), + maxSteps: (value) => requireNumber(value, "maxSteps", { min: 1, max: 1000, integer: true }), + budgetUsd: (value) => requireNumber(value, "budgetUsd", { min: 0.0001, max: 1000 }), + "thresholds.profile": (value) => requireChoice(value, "thresholds.profile", PROFILE_NAMES), + "thresholds.goalDone": (value) => requireNumber(value, "thresholds.goalDone", { min: 0, max: 1 }), + "thresholds.goalDoneFinal": (value) => requireNumber(value, "thresholds.goalDoneFinal", { min: 0, max: 1 }), +}); + +/** "a.b" -> {a:{b:value}}, refusing prototype paths first. */ +export function nestPatch(flat) { + const out = {}; + for (const [key, value] of Object.entries(flat)) { + assertSafeKeyPath(key); + const parts = key.split("."); + let node = out; + for (const part of parts.slice(0, -1)) node = node[part] ??= {}; + node[parts.at(-1)] = value; + } + return out; +} + +/** Validate a {dotted.path: value} patch; empty strings mean "leave this field alone". */ +export function normalizePatch(raw) { + if (raw === undefined || raw === null) return {}; + if (!isPlainObject(raw)) throw new WebUiError("patch must be an object keyed by config path"); + const flat = {}; + for (const [key, value] of Object.entries(raw)) { + if (!Object.hasOwn(EDITABLE_KEYS, key)) { + throw new WebUiError(`config key ${JSON.stringify(key)} is not editable from the WebUI (editable: ${Object.keys(EDITABLE_KEYS).join(", ")})`); + } + if (value === undefined || value === null || (typeof value === "string" && value.trim() === "")) continue; + flat[key] = EDITABLE_KEYS[key](value); + } + return nestPatch(flat); +} + +// ----------------------------------------------------------------------------- config views + +/** Flatten a nested config into {dotted.path: leaf}; arrays and nulls are leaves. */ +export function flattenConfig(node, prefix = "", out = {}) { + for (const [key, value] of Object.entries(isPlainObject(node) ? node : {})) { + const p = prefix ? `${prefix}.${key}` : key; + if (isPlainObject(value)) flattenConfig(value, p, out); + else out[p] = value; + } + return out; +} + +/** The config a browser may see: describeConfig's shape, with the key reduced to "is one set". */ +export function viewConfig(config) { + const view = describeConfig(config); + view.apiKey = config.apiKey ? "(set)" : null; + return view; +} + +/** Every leaf that differs between two views — what "the resulting diff" means. */ +export function diffConfig(before, after) { + const a = flattenConfig(before); + const b = flattenConfig(after); + const out = []; + for (const key of [...new Set([...Object.keys(a), ...Object.keys(b)])].sort()) { + if (JSON.stringify(a[key] ?? null) === JSON.stringify(b[key] ?? null)) continue; + out.push({ path: key, from: a[key] ?? null, to: b[key] ?? null }); + } + return out; +} + +/** + * A scrubber for anything that leaves the server. The API key and every secret typed into the Run + * panel are replaced wherever they appear — including inside a child's output or an error message + * that echoed a request — so "no route returns the key" holds for every response, not just the + * config one. + */ +export function makeScrubber(values = []) { + const needles = [...new Set(values.filter((value) => typeof value === "string" && value.length >= 3))]; + if (!needles.length) return (text) => text; + return (text) => { + let out = String(text); + for (const needle of needles) out = out.split(needle).join("‹redacted›"); + return out; + }; +} + +// ----------------------------------------------------------------------------- http plumbing + +export function assertLoopbackHost(host) { + if (host !== LOOPBACK && host !== "localhost" && host !== "::1") { + throw new WebUiError(`refusing to bind ${host}: this WebUI is loopback-only (${LOOPBACK})`, 500); + } + return host; +} + +/** Bind the server, refusing anything but a loopback host. `port: 0` picks a free port. */ +export async function listenWebUi(server, { port = DEFAULT_WEBUI_PORT, host = LOOPBACK } = {}) { + assertLoopbackHost(host); + await new Promise((resolve, reject) => { + const onError = (error) => reject(error); + server.once("error", onError); + server.listen(port, host, () => { + server.off("error", onError); + resolve(); + }); + }); + return server.address(); +} + +/** Best-effort: open the printed URL. Never throws, never blocks. */ +export function openBrowser(url, { spawn = nodeSpawn } = {}) { + const [command, args] = process.platform === "darwin" ? ["open", [url]] : process.platform === "win32" ? ["cmd", ["/c", "start", "", url]] : ["xdg-open", [url]]; + try { + const child = spawn(command, args, { stdio: "ignore", detached: true }); + child.on?.("error", () => {}); + child.unref?.(); + return true; + } catch { + return false; + } +} + +function sendJson(res, status, body, scrub = (text) => text) { + const text = scrub(JSON.stringify(body)); + res.writeHead(status, { "content-type": "application/json; charset=utf-8", "content-length": Buffer.byteLength(text), "cache-control": "no-store" }); + res.end(text); +} + +function sendHtml(res, html) { + res.writeHead(200, { "content-type": "text/html; charset=utf-8", "content-length": Buffer.byteLength(html), "cache-control": "no-store" }); + res.end(html); +} + +function readJsonBody(req, limit = MAX_BODY_BYTES) { + return new Promise((resolve, reject) => { + let size = 0; + let settled = false; + const chunks = []; + const fail = (error) => { + if (settled) return; + settled = true; + reject(error); + req.destroy(); + }; + req.on("data", (chunk) => { + size += chunk.length; + if (size > limit) return fail(new WebUiError(`request body larger than ${limit} bytes`, 413)); + chunks.push(chunk); + }); + req.on("error", (error) => fail(new WebUiError(`request failed: ${error.message}`))); + req.on("end", () => { + if (settled) return; + settled = true; + const text = Buffer.concat(chunks).toString("utf8").trim(); + if (!text) return resolve({}); + let parsed; + try { + parsed = JSON.parse(text); + } catch (error) { + return reject(new WebUiError(`request body must be JSON (${error.message})`)); + } + if (!isPlainObject(parsed)) return reject(new WebUiError("request body must be a JSON object")); + resolve(parsed); + }); + }); +} + +/** The CLI's `parseMaybeJson`: JSON when it parses, the raw string when it does not. */ +function parseMaybeJson(text) { + if (typeof text !== "string") return text; + try { + return JSON.parse(text); + } catch { + return text; + } +} + +// ----------------------------------------------------------------------------- the server + +/** + * Build the server without starting it. + * + * @param {object} [options] + * @param {string} [options.skillDir] the skill root the spawned scripts are resolved under + * @param {string} [options.home] the HOME whose config file is read and written + * @param {object} [options.env] environment for loadConfig and for every child + * @param {string} [options.cwd] working directory for every child + * @param {Function} [options.spawn] child_process.spawn, injectable so tests can assert argv + * @param {boolean} [options.detached] put children in their own process group (see killGroup) + * @param {(message: string) => void} [options.log] + */ +export function createWebUiServer({ skillDir = MODULE_SKILL_DIR, home = os.homedir(), env = process.env, cwd = process.cwd(), spawn = nodeSpawn, detached = spawn === nodeSpawn, log = () => {} } = {}) { + const channels = new Map(); // name -> { cursor, lines, partial } + const children = new Map(); // name -> record + const spawned = []; // { name, script, args } — what this server actually launched + const redactions = new Set(); // values that must never reach the browser + let redactionsAt = 0; + + /** + * The literal key a local tier's launcher prints (`TYPESAFE_API_KEY=local`) is not a secret: it is + * a constant the launchers publish on stdout and it also sits inside every local tier name, so + * putting it in the redaction set would rewrite half the page. Any other configured key goes in, + * whatever it looks like. + */ + const LOCAL_API_KEY = "local"; + function addApiKeyRedaction(apiKey) { + if (typeof apiKey === "string" && apiKey && apiKey !== LOCAL_API_KEY) redactions.add(apiKey); + } + + // ---------------------------------------------------------------- log buffers + + function channelFor(name) { + let channel = channels.get(name); + if (!channel) { + channel = { cursor: 0, lines: [], partial: { stdout: "", stderr: "" } }; + channels.set(name, channel); + } + return channel; + } + + function appendLine(name, stream, text) { + const channel = channelFor(name); + channel.lines.push({ i: ++channel.cursor, stream, text }); + if (channel.lines.length > MAX_LOG_LINES) channel.lines.splice(0, channel.lines.length - MAX_LOG_LINES); + log(text); + } + + function writeChunk(name, stream, chunk) { + const channel = channelFor(name); + const text = channel.partial[stream] + chunk.toString("utf8"); + const parts = text.split(/\r?\n/); + channel.partial[stream] = parts.pop() ?? ""; + for (const part of parts) if (part !== "") appendLine(name, stream, part); + } + + function flushChannel(name) { + const channel = channelFor(name); + for (const stream of ["stdout", "stderr"]) { + const rest = channel.partial[stream]; + channel.partial[stream] = ""; + if (rest) appendLine(name, stream, rest); + } + } + + // ---------------------------------------------------------------- children + + const cap = (text) => (text.length > MAX_CHILD_OUTPUT ? text.slice(-MAX_CHILD_OUTPUT) : text); + + /** + * Kill a child and everything it started. The launchers spawn llama-server / the MLX server + * themselves, so a signal to the launcher alone would orphan the model server: children run + * detached (their own process group) and the whole group is signalled instead. + */ + function killTree(record, signal) { + const child = record?.child; + if (!child || typeof child.kill !== "function") return; + if (record.detached && Number.isInteger(child.pid) && child.pid > 1) { + try { + process.kill(-child.pid, signal); + return; + } catch { + // no such group (already gone, or a platform without them) — fall back to the child + } + } + try { + child.kill(signal); + } catch { + // already gone + } + } + + function isRunning(name) { + return Boolean(children.get(name)?.running); + } + + function startChild(name, script, args, extra = {}) { + if (!isWithin(script, skillDir)) throw new WebUiError(`refusing to run ${script}: outside the skill directory`, 500); + const child = spawn(process.execPath, [script, ...args], { cwd, env, stdio: ["ignore", "pipe", "pipe"], detached }); + const record = { name, script, args, child, pid: child.pid ?? null, startedAt: Date.now(), running: true, ready: false, exitCode: null, signal: null, error: null, detached, stdout: "", stdoutToLog: true, ...extra }; + children.set(name, record); + spawned.push({ name, script, args: [...args] }); + if (spawned.length > 100) spawned.shift(); + child.stdout?.on("data", (chunk) => { + record.stdout = cap(record.stdout + chunk.toString("utf8")); + // The run child's stdout is its JSON result, which the panel renders separately; only a + // launcher's stdout (its download progress and env line) belongs in the log pane. + if (record.stdoutToLog) writeChunk(name, "stdout", chunk); + }); + child.stderr?.on("data", (chunk) => writeChunk(name, "stderr", chunk)); + child.on?.("error", (error) => { + record.error = error.message; + record.running = false; + record.ready = true; + appendLine(name, "stderr", `spawn failed: ${error.message}`); + }); + child.on?.("close", (code, signal) => { + record.exitCode = code; + record.signal = signal ?? null; + record.running = false; + flushChannel(name); + appendLine(name, "stderr", `[exited: code=${code === null ? "?" : code} signal=${signal ?? "none"}]`); + void afterExit(record); + }); + return record; + } + + /** The run channel parses its result and journal once the child is gone. */ + async function afterExit(record) { + if (record.name === "run") { + const text = record.stdout.trim(); + if (text) { + try { + record.result = JSON.parse(text.slice(text.indexOf("{"))); + } catch (error) { + record.resultError = `could not parse the run result: ${error.message}`; + } + } else if (!record.error) { + record.resultError = `the run wrote nothing to stdout (exit code ${record.exitCode})`; + } + record.journal = await readJournal(record.journalDir); + } + record.ready = true; + } + + /** + * The run journal under the temp directory: `//{run.json,steps.jsonl}`. Compacted to + * the row the panel prints (goal_done / action / choice per step), never the whole state. + */ + async function readJournal(baseDir) { + if (!baseDir) return null; + const entries = await fs.readdir(baseDir, { withFileTypes: true }).catch(() => []); + const dirs = entries.filter((entry) => entry.isDirectory()).map((entry) => path.join(baseDir, entry.name)).sort(); + const dir = dirs.at(-1); + if (!dir) return { dir: baseDir, run: null, rows: [], error: `no journal was written under ${baseDir}` }; + const run = await readJson(path.join(dir, "run.json"), null); + const steps = await fs.readFile(path.join(dir, "steps.jsonl"), "utf8").catch(() => ""); + const rows = []; + for (const line of steps.split("\n")) { + if (!line.trim()) continue; + let row; + try { + row = JSON.parse(line); + } catch { + continue; + } + rows.push({ + step: row.step, + url: row.url ?? null, + title: row.title ?? null, + goalDone: row.goalDone ?? row.answers?.goal_done?.noul ?? null, + blocker: row.blocker ?? row.answers?.blocker?.top ?? null, + action: row.chosen?.kind ?? null, + label: row.chosen?.label ?? null, + changed: row.changed ?? null, + finalCheck: Boolean(row.finalCheck), + }); + } + return { dir, run, rows }; + } + + // ---------------------------------------------------------------- config + scrub + + async function currentConfig({ refreshRedactions = true } = {}) { + let loaded; + try { + loaded = await loadConfig({ env, cwd, home }); + } catch (error) { + throw new WebUiError(error.message, 400); + } + if (refreshRedactions) addApiKeyRedaction(loaded.config.apiKey); + return loaded; + } + + /** + * Refresh the redaction set at most once a second: the key can change under us (the Config panel + * writes it) and every response must already know it, but a log poll should not re-read config + * three times a second. + */ + async function refreshRedactions(force = false) { + if (!force && Date.now() - redactionsAt < 1000) return; + redactionsAt = Date.now(); + try { + const { config } = await loadConfig({ env, cwd, home }); + addApiKeyRedaction(config.apiKey); + } catch { + // a broken config is reported by the route that needs it + } + } + + /** The scrubber for the current request: the key plus every secret typed into the Run panel. */ + const scrubber = () => makeScrubber([...redactions]); + + // ---------------------------------------------------------------- api handlers + + async function apiTiers() { + const { config } = await currentConfig(); + const models = await fetchModels({ config }); + const classification = await probeEndpoint({ config, models }); + const status = describeTier({ config, classification, skillDir }); + return { + defaultTier: DEFAULT_TIER, + tiers: tierRows({ skillDir }), + status, + statusText: formatTierStatus(status), + classification, + baseUrl: config.baseUrl, + keySet: Boolean(config.apiKey), + local: TIERS.filter((tier) => tier.launcher).map((tier) => { + const record = children.get(`tier:${tier.name}`); + return { + tier: tier.name, + port: tier.port, + command: launcherCommand(tier, skillDir), + env: tierEnv(tier), + running: Boolean(record?.running), + startedAt: record?.startedAt ?? null, + exitCode: record?.exitCode ?? null, + }; + }), + }; + } + + async function loadRegistry() { + try { + const { loadLocalModels } = await import("./local.mjs"); + return loadLocalModels(); + } catch (error) { + throw new WebUiError(`the local model registry could not be read: ${error.message}`, 500); + } + } + + /** Shared by /api/tiers/start and /api/models/start: start bin/jev-local.mjs with validated args. */ + async function startLocalTier(tier, body) { + if (!tier.launcher) throw new WebUiError(`${tier.name} has no local server to start — it is a hosted service`); + const name = `tier:${tier.name}`; + if (isRunning(name)) throw new WebUiError(`${tier.name} is already running (started by this WebUI) — stop it first`, 409); + const port = body.port === undefined || body.port === "" ? tier.port : parsePort(body.port, { name: "port" }); + const args = ["--port", String(port)]; + if (tier.name !== "local-readout") { + if (body.modelName || body.modelPath) throw new WebUiError(`${tier.name} serves its own checkpoint: modelName/modelPath apply to jev-local only`); + } else { + if (body.llamaPort !== undefined && body.llamaPort !== "") args.push("--llama-port", String(parsePort(body.llamaPort, { name: "llamaPort" }))); + if (body.modelName) args.push("--model-name", requireModelId(body.modelName, await loadRegistry())); + if (body.modelPath) { + const file = resolveModelPath(body.modelPath, { home }); + const stat = await fs.stat(file).catch(() => null); + if (!stat?.isFile()) throw new WebUiError(`no such model file: ${file} (only ${modelRoot(home)} may be named)`, 404); + args.push("--model", file); + } + } + const script = path.join(skillDir, tier.launcher); + const record = startChild(name, script, args); + return { tier: tier.name, channel: name, pid: record.pid, port, command: `node ${script} ${args.join(" ")}` }; + } + + async function apiTiersStart(body) { + return startLocalTier(requireTier(body.tier), body); + } + + async function apiTiersStop(body) { + const tier = requireTier(body.tier); + const name = `tier:${tier.name}`; + const record = children.get(name); + if (!record || !record.running) throw new WebUiError(`${tier.name} is not running (only a launcher this WebUI started can be stopped here)`, 409); + killTree(record, "SIGTERM"); + return { tier: tier.name, stopped: true, pid: record.pid }; + } + + /** `tier use` as data, plus the canonical export line for that tier. */ + function apiTierUse(url) { + const tier = requireTier(url.searchParams.get("tier")); + const rows = tierRows({ skillDir }); + return { + tier: tier.name, + text: formatTierUse(tier, { skillDir, configPath: path.join(home, ".config", "jev-browser", "config.json") }), + env: tierEnv(tier), + command: launcherCommand(tier, skillDir), + baseUrl: tier.baseUrl, + port: tier.port, + apiKey: tier.apiKey, + row: rows.find((row) => row.tier === tier.name) ?? null, + }; + } + + /** The explicit write behind the "save baseUrl" button — the only config write this route does. */ + async function apiTierUsePersist(body) { + const tier = requireTier(body.tier); + const port = body.port === undefined || body.port === "" ? tier.port : parsePort(body.port, { name: "port" }); + const baseUrl = tier.port && port !== tier.port ? `http://${LOOPBACK}:${port}` : tier.baseUrl; + // A local tier is only half configured by baseUrl: its key is the literal placeholder the client + // requires and the server ignores, so — exactly like `tier use --persist` — both are stored. + const file = await saveUserConfig({ baseUrl, ...(tier.apiKey ? { apiKey: tier.apiKey } : {}) }, { home }); + const { config } = await currentConfig({ refreshRedactions: false }); + addApiKeyRedaction(config.apiKey); + return { tier: tier.name, baseUrl, apiKey: tier.apiKey ?? null, persisted: file, text: formatTierUse(tier, { skillDir, persisted: file, configPath: file }), config: viewConfig(config) }; + } + + function apiLogs(url) { + const channel = url.searchParams.get("channel") ?? ""; + if (!LOG_CHANNELS.includes(channel)) throw new WebUiError(`unknown log channel ${JSON.stringify(channel)} (expected ${LOG_CHANNELS.join(", ")})`); + const since = Number(url.searchParams.get("since") ?? 0); + if (!Number.isFinite(since) || since < 0) throw new WebUiError("since must be a non-negative number"); + const buffer = channelFor(channel); + const record = children.get(channel); + return { + channel, + lines: buffer.lines.filter((line) => line.i > since), + next: buffer.cursor, + running: Boolean(record?.running), + ready: record ? record.ready : false, + exitCode: record?.exitCode ?? null, + }; + } + + async function apiConfigGet() { + const { config, sources, paths } = await currentConfig(); + const userFile = await readJson(paths.userFile, null); + return { + config: viewConfig(config), + keySet: Boolean(config.apiKey), + sources, + paths, + userSetKeys: Object.keys(flattenConfig(userFile)), + editable: Object.keys(EDITABLE_KEYS), + profiles: PROFILE_NAMES, + thresholds: THRESHOLD_PROFILES, + backends: ["ego", "chrome", "safari"], + defaults: { + baseUrl: DEFAULTS.baseUrl, + model: DEFAULTS.model, + backend: DEFAULTS.backend, + maxSteps: DEFAULTS.maxSteps, + budgetUsd: DEFAULTS.budgetUsd, + profile: DEFAULTS.thresholds.profile, + }, + }; + } + + async function apiConfigPost(body) { + const before = viewConfig((await currentConfig()).config); + const patch = normalizePatch(body.patch); + if (body.apiKey !== undefined && body.apiKey !== null && String(body.apiKey).trim() !== "") patch.apiKey = requireApiKey(String(body.apiKey)); + const saved = Object.keys(flattenConfig(patch)); + if (!saved.length) throw new WebUiError("nothing to save: send a patch (and/or apiKey) with at least one value"); + const file = await saveUserConfig(patch, { home }); + const after = viewConfig((await currentConfig()).config); + return { saved, file, config: after, keySet: Boolean(after.apiKey), diff: diffConfig(before, after) }; + } + + async function apiConfigUnset(body) { + const key = requireText(body.key, "key", 128); + if (!Object.hasOwn(EDITABLE_KEYS, key) && key !== "apiKey") { + throw new WebUiError(`config key ${JSON.stringify(key)} is not editable from the WebUI (editable: ${[...Object.keys(EDITABLE_KEYS), "apiKey"].join(", ")})`); + } + const before = viewConfig((await currentConfig()).config); + const file = await unsetUserConfig(key, { home }); + const after = viewConfig((await currentConfig()).config); + return { removed: key, file, config: after, keySet: Boolean(after.apiKey), diff: diffConfig(before, after) }; + } + + async function apiDoctor(url) { + const { config, sources } = await currentConfig(); + const live = url.searchParams.get("live") !== "0"; + const report = await doctor({ config, sources, skillDir, home, live }); + return { report, text: formatDoctor(report) }; + } + + async function apiModels() { + const root = modelRoot(home); + let registry = null; + let registryError = null; + let registryModels = {}; + try { + const { loadLocalModels } = await import("./local.mjs"); + const loaded = loadLocalModels(); + registry = { path: loaded.path, default: loaded.default, ids: Object.keys(loaded.models) }; + registryModels = loaded.models; + } catch (error) { + registryError = error.message; + } + const entries = []; + for (const [id, entry] of Object.entries(registryModels)) { + const file = path.join(root, entry.file); + const stat = await fs.stat(file).catch(() => null); + entries.push({ + id, + label: entry.label, + file, + expectedBytes: entry.bytes, + bytesOnDisk: stat?.size ?? 0, + downloaded: Boolean(stat && stat.size === entry.bytes), + partial: Boolean(stat && stat.size !== entry.bytes), + default: registry.default === id, + }); + } + let onDisk = []; + const files = await fs.readdir(root, { withFileTypes: true }).catch(() => []); + for (const entry of files.slice(0, 200)) { + if (!entry.isFile() || !entry.name.endsWith(".gguf")) continue; + const stat = await fs.stat(path.join(root, entry.name)).catch(() => null); + onDisk.push({ name: entry.name, path: path.join(root, entry.name), bytes: stat?.size ?? 0 }); + } + onDisk.sort((a, b) => a.name.localeCompare(b.name)); + let status = null; + let statusError = null; + try { + const { localStatus } = await import("./local.mjs"); + status = await localStatus({ home }); + } catch (error) { + statusError = error.message; + } + return { root, registry, registryError, entries, onDisk, status, statusError }; + } + + async function apiModelsStart(body) { + return startLocalTier(tierByName("local-readout"), body); + } + + async function apiJudge(body) { + const { config } = await currentConfig(); + const apiKey = config.apiKey ?? (isLoopbackBaseUrl(config.baseUrl) ? "local" : null); + if (!apiKey) throw new WebUiError("no TypeSafe API key: store one in the Config panel, or point baseUrl at a local tier", 400); + if (body.state === undefined || body.state === null) throw new WebUiError("state is required"); + const stateText = typeof body.state === "string" ? body.state : JSON.stringify(body.state); + if (stateText.length > 200_000) throw new WebUiError("state is longer than 200000 characters"); + const questionsRaw = typeof body.questions === "string" ? parseMaybeJson(body.questions) : body.questions; + if (!isPlainObject(questionsRaw)) throw new WebUiError("questions must be a JSON object keyed by question id"); + try { + validateQuestions(questionsRaw); + } catch (error) { + throw new WebUiError(error.message, 400); + } + const model = body.model ? requireToken(body.model, "model") : config.model; + const client = new TypeSafeClient({ + apiKey, + baseUrl: config.baseUrl, + model, + timeoutMs: config.timeoutMs, + maxRetries: config.maxRetries, + pricePerMtok: config.pricePerMtok, + }); + const result = await client.systemOne({ state: parseMaybeJson(stateText), questions: questionsRaw }); + return { + model: result.model, + answers: result.answers, + usage: result.usage, + costUsd: result.costUsd, + ms: result.ms, + cacheHit: result.cacheHit, + baseUrl: config.baseUrl, + profile: config.thresholds.profile, + }; + } + + async function apiRun(body) { + if (isRunning("run")) throw new WebUiError("a run is already in progress — stop it first", 409); + const goal = requireText(body.goal, "goal", 2000); + const url = requireUrl(body.url, "url"); + const inputs = requirePairs(body.inputs, "inputs"); + const secrets = requirePairs(body.secrets, "secrets"); + const backend = body.backend === undefined || body.backend === "" ? null : requireChoice(body.backend, "backend", ["ego", "chrome", "safari"]); + const maxSteps = body.maxSteps === undefined || body.maxSteps === "" ? null : requireNumber(body.maxSteps, "maxSteps", { min: 1, max: 1000, integer: true }); + const budgetUsd = body.budgetUsd === undefined || body.budgetUsd === "" ? null : requireNumber(body.budgetUsd, "budgetUsd", { min: 0.0001, max: 1000 }); + // The journal goes under the OS temp directory: the WebUI writes nothing of its own anywhere + // else, and `--journal-dir` is how the CLI is told where to put it. + const runId = `webui-${new Date().toISOString().replace(/[:.]/g, "-")}`; + const journalDir = path.join(os.tmpdir(), "jev-browser-webui", runId); + const args = ["run", "--json", "--goal", goal, "--url", url, "--journal-dir", journalDir]; + if (backend) args.push("--backend", backend); + if (maxSteps !== null) args.push("--max-steps", String(maxSteps)); + if (budgetUsd !== null) args.push("--budget-usd", String(budgetUsd)); + for (const [key, value] of Object.entries(inputs)) args.push("--input", `${key}=${value}`); + for (const [key, value] of Object.entries(secrets)) args.push("--secret", `${key}=${value}`); + for (const value of Object.values(secrets)) redactions.add(value); + const script = path.join(skillDir, "bin", "jev-browser.mjs"); + const record = startChild("run", script, args, { runId, journalDir, goal, url, secretKeys: Object.keys(secrets), stdoutToLog: false }); + return { + runId, + channel: "run", + pid: record.pid, + journalDir, + // The command line the panel shows, with every secret value replaced. + command: [`node ${script}`, "run", "--json", "--goal", JSON.stringify(goal), "--url", JSON.stringify(url), `--journal-dir ${journalDir}`] + .concat(backend ? ["--backend", backend] : []) + .concat(maxSteps !== null ? ["--max-steps", String(maxSteps)] : []) + .concat(budgetUsd !== null ? ["--budget-usd", String(budgetUsd)] : []) + .concat(Object.entries(inputs).map(([key, value]) => `--input ${key}=${JSON.stringify(value)}`)) + .concat(Object.keys(secrets).map((key) => `--secret ${key}=‹secret›`)) + .join(" "), + }; + } + + function apiRunGet() { + const record = children.get("run"); + if (!record) return { running: false, ready: false, result: null, resultError: null, journal: null, exitCode: null }; + return { + runId: record.runId ?? null, + goal: record.goal ?? null, + url: record.url ?? null, + startedAt: record.startedAt, + running: Boolean(record.running), + ready: Boolean(record.ready), + exitCode: record.exitCode ?? null, + signal: record.signal ?? null, + error: record.error ?? null, + result: record.result ?? null, + resultError: record.resultError ?? null, + journal: record.journal ?? null, + }; + } + + function apiRunStop() { + const record = children.get("run"); + if (!record || !record.running) throw new WebUiError("no run is in progress", 409); + killTree(record, "SIGTERM"); + return { stopped: true, pid: record.pid }; + } + + // ---------------------------------------------------------------- dispatch + + const HANDLERS = { + "GET /": (_url, _req, res) => sendHtml(res, WEBUI_PAGE), + "GET /api/tiers": () => apiTiers(), + "POST /api/tiers/start": (_url, req) => readJsonBody(req).then(apiTiersStart), + "POST /api/tiers/stop": (_url, req) => readJsonBody(req).then(apiTiersStop), + "GET /api/tiers/use": (url) => apiTierUse(url), + "POST /api/tiers/use": (_url, req) => readJsonBody(req).then(apiTierUsePersist), + "GET /api/logs": (url) => apiLogs(url), + "GET /api/config": () => apiConfigGet(), + "POST /api/config": (_url, req) => readJsonBody(req).then(apiConfigPost), + "POST /api/config/unset": (_url, req) => readJsonBody(req).then(apiConfigUnset), + "GET /api/doctor": (url) => apiDoctor(url), + "GET /api/models": () => apiModels(), + "POST /api/models/start": (_url, req) => readJsonBody(req).then(apiModelsStart), + "POST /api/judge": (_url, req) => readJsonBody(req).then(apiJudge), + "POST /api/run": (_url, req) => readJsonBody(req).then(apiRun), + "GET /api/run": () => apiRunGet(), + "POST /api/run/stop": () => apiRunStop(), + }; + + async function handle(req, res) { + let scrub = (text) => text; + try { + await refreshRedactions(); + scrub = scrubber(); + const url = new URL(req.url ?? "/", `http://${LOOPBACK}`); + const key = `${req.method} ${url.pathname}`; + const handler = HANDLERS[key]; + if (!handler) { + const allowed = ROUTES.filter((route) => route.path === url.pathname).map((route) => route.method); + if (allowed.length) throw new WebUiError(`method ${req.method} is not allowed on ${url.pathname} (allowed: ${allowed.join(", ")})`, 405); + throw new WebUiError(`unknown route ${key}`, 404); + } + const body = await handler(url, req, res); + if (body !== undefined && !res.headersSent) sendJson(res, 200, body, scrub); + } catch (error) { + const status = error instanceof WebUiError ? error.status : Number.isInteger(error?.status) && error.status >= 400 && error.status < 600 ? error.status : 500; + if (!res.headersSent) sendJson(res, status, { error: error?.message ?? "internal error" }, scrub); + else res.end(); + } + } + + const server = http.createServer((req, res) => { + void handle(req, res); + }); + + /** Stop every child this server started, then close the socket. */ + async function close({ timeoutMs = 2000 } = {}) { + const living = [...children.values()].filter((record) => record.running); + for (const record of living) killTree(record, "SIGTERM"); + if (living.length) { + await Promise.race([ + Promise.all(living.map((record) => new Promise((resolve) => (record.running ? record.child.once?.("close", resolve) : resolve())))), + new Promise((resolve) => setTimeout(resolve, timeoutMs)), + ]); + for (const record of living) if (record.running) killTree(record, "SIGKILL"); + } + await new Promise((resolve) => { + server.close(() => resolve()); + server.closeAllConnections?.(); + }); + } + + return { server, channels, children, spawned, state: { redactions }, close, url: (port) => `http://${LOOPBACK}:${port}/` }; +} diff --git a/skills/jev-browser/package.json b/skills/jev-browser/package.json index fe54ff6..d58e94f 100644 --- a/skills/jev-browser/package.json +++ b/skills/jev-browser/package.json @@ -5,7 +5,9 @@ "type": "module", "license": "MIT", "bin": { - "jev-browser": "./bin/jev-browser.mjs" + "jev-browser": "./bin/jev-browser.mjs", + "jev-local": "./bin/jev-local.mjs", + "jev-kev": "./bin/jev-kev.mjs" }, "engines": { "node": ">=22" diff --git a/skills/jev-browser/references/config.md b/skills/jev-browser/references/config.md index d1d5298..42ce509 100644 --- a/skills/jev-browser/references/config.md +++ b/skills/jev-browser/references/config.md @@ -11,7 +11,7 @@ masked and the list of sources that contributed. | key | default | meaning | | --- | --- | --- | -| `apiKey` | null | TypeSafe API key; prefer the `TYPESAFE_API_KEY` env var. `config set-key --from-env` stores it (0600) for MCP hosts | +| `apiKey` | null | TypeSafe API key; prefer the `TYPESAFE_API_KEY` env var. `config set-key --from-env` stores it (0600) for MCP hosts. A local tier uses the literal placeholder `local` — the local server ignores the `Authorization` header, but the client requires a non-empty key; `jev-browser setup ` and `jev-browser tier use --persist` store a local tier's `baseUrl` + `apiKey` for you (`config set apiKey local` by hand) | | `baseUrl` | `https://api.typesafe.ai` | API base URL (`TYPESAFE_BASE_URL`) | | `model` | `jev-latest` | `jev-latest`, `jev-preview`, or a pinned id such as `jev-1.13.0` (`TYPESAFE_DEFAULT_MODEL`) | | `timeoutMs` | 20000 | per-request timeout | diff --git a/tests/unit/cli-home.test.mjs b/tests/unit/cli-home.test.mjs new file mode 100644 index 0000000..617c8b0 --- /dev/null +++ b/tests/unit/cli-home.test.mjs @@ -0,0 +1,51 @@ +// `--home` means the same thing to every command: the user config (and everything derived from it) +// comes from that directory. doctor/config/tier/install always honoured it; run/observe/judge/pick +// did not, so a scratch-home probe silently read the real user config instead. +import test from "node:test"; +import assert from "node:assert/strict"; +import { execFile } from "node:child_process"; +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { promisify } from "node:util"; +import { BIN } from "../helpers/env.mjs"; + +const run = promisify(execFile); + +async function cli(args, { env = {}, home }) { + try { + const { stdout, stderr } = await run(process.execPath, [BIN, ...args], { env: { PATH: process.env.PATH, HOME: home, ...env }, cwd: home }); + return { code: 0, stdout, stderr }; + } catch (error) { + return { code: typeof error.code === "number" ? error.code : 1, stdout: error.stdout ?? "", stderr: error.stderr ?? "" }; + } +} + +// Each command's own required-flag error: proof it got past config loading without reading any +// other config. The broken `backend` in the other home only loadConfig can produce. +const CASES = [ + { name: "run", args: ["run", "--goal", "open the page"], clean: /--url is required/ }, + { name: "observe", args: ["observe"], clean: /--url is required/ }, + { name: "judge", args: ["judge"], clean: /--state\/--state-file and --questions\/--questions-file are required/ }, + { name: "pick", args: ["pick"], clean: /--question and at least two --candidate id=description are required/ }, +]; + +test("run/observe/judge/pick read --home the way doctor/config/tier/install do", async (t) => { + const clean = await fs.mkdtemp(path.join(os.tmpdir(), "jev-home-clean-")); + const bad = await fs.mkdtemp(path.join(os.tmpdir(), "jev-home-bad-")); + t.after(async () => { + await fs.rm(clean, { recursive: true, force: true }); + await fs.rm(bad, { recursive: true, force: true }); + }); + await fs.mkdir(path.join(bad, ".config", "jev-browser"), { recursive: true }); + await fs.writeFile(path.join(bad, ".config", "jev-browser", "config.json"), JSON.stringify({ backend: "nope" })); + + for (const scenario of CASES) { + const fromBad = await cli([...scenario.args, "--home", bad], { home: clean }); + assert.match(fromBad.stderr, /Unknown backend "nope"/, `${scenario.name} must read the config --home names`); + assert.notEqual(fromBad.code, 0, `${scenario.name} fails on a config it cannot load`); + + const fromClean = await cli([...scenario.args, "--home", clean], { home: clean }); + assert.match(fromClean.stderr, scenario.clean, `${scenario.name} must not read any other config`); + } +}); diff --git a/tests/unit/doctor.test.mjs b/tests/unit/doctor.test.mjs new file mode 100644 index 0000000..6397a52 --- /dev/null +++ b/tests/unit/doctor.test.mjs @@ -0,0 +1,113 @@ +// `doctor`'s local lines, offline: a loopback endpoint is a complete configuration without a real +// key, the GGUF check compares the registry's exact size, and a Kev endpoint gets a Kev line that +// never sends the reader to the jev-local launcher. +import test from "node:test"; +import assert from "node:assert/strict"; +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; +import { doctor } from "../../skills/jev-browser/lib/doctor.mjs"; +import { loadConfig } from "../../skills/jev-browser/lib/config.mjs"; +import { defaultLocalModel } from "../../skills/jev-browser/lib/local.mjs"; + +const SKILL_DIR = fileURLToPath(new URL("../../skills/jev-browser", import.meta.url)); + +/** A scratch home, doctor against it, offline. */ +async function scratchHome() { + return fs.mkdtemp(path.join(os.tmpdir(), "jev-doctor-home-")); +} + +async function runDoctor({ home, baseUrl }) { + const env = { PATH: process.env.PATH, HOME: home, ...(baseUrl ? { TYPESAFE_BASE_URL: baseUrl } : {}) }; + const { config, sources } = await loadConfig({ env, home, cwd: home }); + return doctor({ config, sources, home, skillDir: SKILL_DIR, live: false }); +} + +const check = (report, name) => report.checks.find((c) => c.name === name); + +/** + * Run `body` with JEV_LLAMA_SERVER pinned, then restore it. Whether llama.cpp is installed is an + * input to doctor (it resolves JEV_LLAMA_SERVER, then PATH, then /opt/homebrew/bin), and a test + * that inherits it asserts the machine it happens to run on. + */ +async function withLlamaServer(value, body) { + const previous = process.env.JEV_LLAMA_SERVER; + process.env.JEV_LLAMA_SERVER = value; + try { + return await body(); + } finally { + if (previous === undefined) delete process.env.JEV_LLAMA_SERVER; + else process.env.JEV_LLAMA_SERVER = previous; + } +} + +test("a loopback baseUrl without a key is a warning that names the placeholder, not a failure", async () => { + const home = await scratchHome(); + try { + const report = await runDoctor({ home, baseUrl: "http://127.0.0.1:8092" }); + const key = check(report, "api key"); + assert.equal(key.status, "warn", "the local tiers are complete without a real key"); + assert.equal(key.detail, "missing"); + assert.match(key.hint, /TYPESAFE_API_KEY=local/); + assert.match(key.hint, /config set-key local/); + + // The hosted default keeps the hard failure and the hosted remedy. + const hosted = await runDoctor({ home }); + assert.equal(check(hosted, "api key").status, "fail"); + assert.match(check(hosted, "api key").hint, /config set-key --from-env/); + } finally { + await fs.rm(home, { recursive: true, force: true }); + } +}); + +test("a truncated model file is reported as truncated, not accepted as ready", async () => { + const home = await scratchHome(); + try { + const entry = defaultLocalModel(); + const file = path.join(home, ".jev-browser", "models", entry.file); + await fs.mkdir(path.dirname(file), { recursive: true }); + await fs.writeFile(file, Buffer.alloc(1024)); + + // With llama.cpp present, the next action is to replace the file — while the truncation is + // still the thing the check refuses to call ready. + const stub = path.join(home, "llama-server-stub"); + await fs.writeFile(stub, "#!/bin/sh\n", { mode: 0o755 }); + const withLlama = await withLlamaServer(stub, () => runDoctor({ home, baseUrl: "http://127.0.0.1:8092" })); + const local = check(withLlama, "local model"); + assert.equal(local.status, "warn"); + assert.match(local.detail, new RegExp(`truncated \\(1024 of ${entry.bytes} bytes\\)`)); + assert.match(local.detail, /127\.0\.0\.1:8092 not running/); + assert.match(local.hint, /download it again/); + + // Without llama.cpp (a bare CI runner) the same file is still described as truncated and is + // still not ready; the hint names the prerequisite that is missing first. The two states are + // asserted separately because the machine decides which one a run meets. + const withoutLlama = await withLlamaServer(path.join(home, "no-llama-server-here"), () => runDoctor({ home, baseUrl: "http://127.0.0.1:8092" })); + const bare = check(withoutLlama, "local model"); + assert.equal(bare.status, "warn"); + assert.match(bare.detail, new RegExp(`truncated \\(1024 of ${entry.bytes} bytes\\)`)); + assert.match(bare.detail, /llama-server not found/); + assert.match(bare.hint, /brew install llama\.cpp/); + } finally { + await fs.rm(home, { recursive: true, force: true }); + } +}); + +test("a Kev endpoint gets a Kev line and no jev-local hint anywhere in the report", async () => { + const home = await scratchHome(); + try { + const report = await runDoctor({ home, baseUrl: "http://127.0.0.1:8008" }); + assert.equal(check(report, "local model"), undefined, "the GGUF registry line is not what a Kev endpoint needs"); + const kev = check(report, "kev"); + assert.ok(kev, "a Kev endpoint gets its own check"); + assert.equal(kev.status, "warn", "a scratch home has no Kev checkout"); + assert.match(kev.detail, /\.local\/share\/jev-browser\/kev/); + assert.match(kev.hint, /bin\/jev-kev\.mjs/); + for (const entry of report.checks) { + assert.doesNotMatch(`${entry.name} ${entry.detail} ${entry.hint ?? ""}`, /jev-local/, `no jev-local hint for a Kev endpoint (${entry.name})`); + } + } finally { + await fs.rm(home, { recursive: true, force: true }); + } +}); diff --git a/tests/unit/setup.test.mjs b/tests/unit/setup.test.mjs new file mode 100644 index 0000000..7a95dfe --- /dev/null +++ b/tests/unit/setup.test.mjs @@ -0,0 +1,205 @@ +// `jev-browser setup`: argument validation, status/stop against a scratch home, the clean refusal +// when a prerequisite is missing, and — in-process, with the launcher seams injected — the +// fetch → start → persist → verify path. No test starts a server, downloads a model, or writes +// outside its own temporary home. +import test from "node:test"; +import assert from "node:assert/strict"; +import { execFile } from "node:child_process"; +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { promisify } from "node:util"; +import { fileURLToPath } from "node:url"; +import { BIN } from "../helpers/env.mjs"; +import { setupCommand } from "../../skills/jev-browser/lib/setup.mjs"; +import { defaultLocalModel } from "../../skills/jev-browser/lib/local.mjs"; + +const run = promisify(execFile); +const SKILL_DIR = fileURLToPath(new URL("../../skills/jev-browser", import.meta.url)); + +async function cli(args, { env = {}, home, cwd = home } = {}) { + try { + const { stdout, stderr } = await run(process.execPath, [BIN, ...args], { env: { PATH: process.env.PATH, HOME: home, ...env }, cwd }); + return { code: 0, stdout, stderr }; + } catch (error) { + return { code: typeof error.code === "number" ? error.code : 1, stdout: error.stdout ?? "", stderr: error.stderr ?? "" }; + } +} + +async function scratch(t, prefix = "jev-setup-home-") { + const home = await fs.mkdtemp(path.join(os.tmpdir(), prefix)); + t.after(() => fs.rm(home, { recursive: true, force: true })); + return home; +} + +test("setup with no argument reports each tier and the command that sets it up", async (t) => { + const home = await scratch(t); + const json = await cli(["setup", "--json", "--home", home], { home }); + assert.equal(json.code, 0, json.stderr); + const report = JSON.parse(json.stdout); + assert.equal(report.uses.tier, "hosted", "a clean home still runs on hosted Jev"); + assert.deepEqual( + report.tiers.map((tier) => tier.tier), + ["local-readout", "kev"], + ); + for (const tier of report.tiers) { + assert.equal(tier.installed, false, `${tier.tier} is not installed in a scratch home`); + assert.equal(tier.running, false); + assert.equal(tier.configured, false); + assert.match(tier.setupCommand, new RegExp(`setup ${tier.tier}$`)); + assert.ok(tier.logFile.startsWith(home), "the log lives under the home that was named"); + } + + const text = await cli(["setup", "--home", home], { home }); + assert.equal(text.code, 0, text.stderr); + assert.match(text.stdout, /set it up: .*setup local-readout/); + assert.match(text.stdout, /set it up: .*setup kev/); +}); + +test("setup status reports not installed / not running / not configured and writes nothing", async (t) => { + const home = await scratch(t); + const status = await cli(["setup", "status", "--json", "--home", home], { home }); + assert.equal(status.code, 0, status.stderr); + const report = JSON.parse(status.stdout); + for (const tier of report.tiers) { + assert.deepEqual( + { installed: tier.installed, running: tier.running, configured: tier.configured }, + { installed: false, running: false, configured: false }, + tier.tier, + ); + assert.match(tier.installedDetail, /not downloaded|no Kev checkout/); + } + assert.deepEqual(await fs.readdir(home), [], "status is read-only: it does not even create the run directory"); +}); + +test("setup stop stops nothing without a pid file, and refuses a pid that is not a launcher", async (t) => { + const home = await scratch(t); + const idle = await cli(["setup", "stop", "local-readout", "--json", "--home", home], { home }); + assert.equal(idle.code, 0, idle.stderr); + const idleReport = JSON.parse(idle.stdout); + assert.equal(idleReport.stopped, false); + assert.equal(idleReport.reason, "no pid file"); + + // A pid file naming a live process that is not one of our launchers: never killed. + const runDir = path.join(home, ".jev-browser", "run"); + await fs.mkdir(runDir, { recursive: true }); + await fs.writeFile(path.join(runDir, "jev-local-8092.pid"), `${process.pid}\n`); + const foreign = await cli(["setup", "stop", "local-readout", "--home", home], { home }); + assert.equal(foreign.code, 1); + assert.match(foreign.stdout, /not a local-readout launcher/); + assert.doesNotThrow(() => process.kill(process.pid, 0), "the foreign process is untouched"); + assert.equal(await fs.readFile(path.join(runDir, "jev-local-8092.pid"), "utf8"), `${process.pid}\n`, "an ambiguous pid file is left alone"); +}); + +test("setup rejects unknown targets and misplaced flags", async (t) => { + const home = await scratch(t); + const unknownTarget = await cli(["setup", "bogus", "--home", home], { home }); + assert.equal(unknownTarget.code, 2); + assert.match(unknownTarget.stderr, /unknown setup target "bogus"/); + assert.match(unknownTarget.stderr, /usage: jev-browser setup/); + + const unknownTier = await cli(["setup", "stop", "bogus", "--home", home], { home }); + assert.equal(unknownTier.code, 2); + assert.match(unknownTier.stderr, /unknown tier "bogus"/); + + const unknownModel = await cli(["setup", "local-readout", "--model-name", "nope", "--home", home], { home }); + assert.equal(unknownModel.code, 2); + assert.match(unknownModel.stderr, /unknown local model "nope"/); + + const misplacedSkip = await cli(["setup", "local-readout", "--skip-deps", "--home", home], { home }); + assert.equal(misplacedSkip.code, 2); + assert.match(misplacedSkip.stderr, /--skip-deps applies to `setup kev`/); + + const misplacedModel = await cli(["setup", "kev", "--model-name", "qwen3.5-0.8b-q8", "--home", home], { home }); + assert.equal(misplacedModel.code, 2); + assert.match(misplacedModel.stderr, /--model-name applies to `setup local-readout`/); +}); + +test("setup local-readout refuses cleanly when llama.cpp is absent", async (t) => { + const home = await scratch(t); + // JEV_LLAMA_SERVER pins the lookup, so this is deterministic even where Homebrew has llama.cpp. + const out = await cli(["setup", "local-readout", "--home", home], { home, env: { PATH: "/nonexistent", JEV_LLAMA_SERVER: "/nonexistent/llama-server" } }); + assert.equal(out.code, 2, "a missing prerequisite is a usage/config problem"); + assert.match(out.stderr, /brew install llama\.cpp/); + assert.match(out.stderr, /JEV_LLAMA_SERVER/); + assert.deepEqual(await fs.readdir(home), [], "nothing is fetched or written when the prerequisite is missing"); +}); + +test("setup kev asks for uv before it touches anything", async (t) => { + const home = await scratch(t); + const out = await cli(["setup", "kev", "--home", home], { home, env: { PATH: "/nonexistent" } }); + assert.equal(out.code, 2); + assert.match(out.stderr, /brew install uv/); + assert.match(out.stderr, /uv sync --extra serve/); + assert.deepEqual(await fs.readdir(home), [], "no clone, no cache, no config"); +}); + +test("setup kev --skip-deps refuses without a prepared checkout instead of downloading", async (t) => { + const home = await scratch(t); + const out = await cli(["setup", "kev", "--skip-deps", "--home", home], { home }); + assert.equal(out.code, 2); + assert.match(out.stderr, /Prepare it with/); + assert.match(out.stderr, /uv sync --extra serve --project .*jev-browser\/kev/); + assert.deepEqual(await fs.readdir(home), [], "--skip-deps never clones, syncs or fetches"); +}); + +test("a successful local-readout setup fetches, starts, persists both values and verifies", async (t) => { + const home = await scratch(t); + const entry = defaultLocalModel(); + const calls = { runStep: [], startDetached: [], verify: [] }; + const pidFile = path.join(home, ".jev-browser", "run", "jev-local-8092.pid"); + const deps = { + findLlamaServer: () => "/opt/homebrew/bin/llama-server", + runStep: async (step) => { + calls.runStep.push(step); + return { code: 0 }; + }, + startDetached: async (step) => { + calls.startDetached.push(step); + await fs.mkdir(path.dirname(pidFile), { recursive: true }); + await fs.writeFile(pidFile, "4242\n"); + return { code: 0, stdout: "TYPESAFE_BASE_URL=http://127.0.0.1:8092 TYPESAFE_API_KEY=local" }; + }, + verify: async (options) => { + calls.verify.push(options); + return { answer: { type: "noul", noul: 1 }, detail: "noul: P(yes)=1.00", ms: 12, model: "qwen3.5-4b-q4-k-m" }; + }, + kevRuntime: async () => ({ ok: true, clone: "/none", python: "/none", kind: "ok", detail: "unused" }), + }; + + const out = await setupCommand({ sub: "local-readout", options: { home, env: { HOME: home, PATH: "/nonexistent" }, skillDir: SKILL_DIR, log: () => {} }, deps }); + assert.equal(out.code, 0); + assert.deepEqual(calls.runStep[0].args, [path.join(SKILL_DIR, "bin", "jev-local.mjs"), "--download-only", "--model-name", entry.id], "the launcher's own --download-only fetches"); + assert.deepEqual(calls.startDetached[0].args, ["--detach", "--model-name", entry.id], "the launcher's own --detach serves"); + for (const call of [...calls.runStep, ...calls.startDetached]) assert.equal(call.env.HOME, home, "children run against the home that was named"); + assert.equal(calls.verify[0].baseUrl, "http://127.0.0.1:8092"); + assert.equal(calls.verify[0].apiKey, "local"); + + const config = JSON.parse(await fs.readFile(path.join(home, ".config", "jev-browser", "config.json"), "utf8")); + assert.equal(config.baseUrl, "http://127.0.0.1:8092"); + assert.equal(config.apiKey, "local"); + assert.equal(out.json.pid, 4242); + assert.match(out.text, /verified\s+noul: P\(yes\)=1\.00/); + assert.match(out.text, /log\s+.*jev-local-8092\.log/); + assert.match(out.text, /setup stop local-readout/); +}); + +test("a setup whose endpoint cannot answer still persists the config and names the log", async (t) => { + const home = await scratch(t); + const deps = { + findLlamaServer: () => "/opt/homebrew/bin/llama-server", + runStep: async () => ({ code: 0 }), + startDetached: async () => ({ code: 0, stdout: "" }), + verify: async () => { + throw new Error("TypeSafe connection failed: fetch failed"); + }, + kevRuntime: async () => ({ ok: true, clone: "/none", python: "/none", kind: "ok", detail: "unused" }), + }; + + await assert.rejects( + setupCommand({ sub: "local-readout", options: { home, env: { HOME: home, PATH: "/nonexistent" }, skillDir: SKILL_DIR, log: () => {} }, deps }), + /did not answer the verification question: TypeSafe connection failed[\s\S]*log: .*jev-local-8092\.log/, + ); + const config = JSON.parse(await fs.readFile(path.join(home, ".config", "jev-browser", "config.json"), "utf8")); + assert.deepEqual(config, { version: 1, baseUrl: "http://127.0.0.1:8092", apiKey: "local" }, "the config is written before the endpoint is asked"); +}); diff --git a/tests/unit/tier.test.mjs b/tests/unit/tier.test.mjs index 7cec648..d2e50fa 100644 --- a/tests/unit/tier.test.mjs +++ b/tests/unit/tier.test.mjs @@ -236,12 +236,31 @@ test("tier use prints the launcher's export line and writes nothing without --pe assert.match(hosted.stdout, /already the default/); await assert.rejects(fs.access(configFile)); - // --persist is the explicit opt-in, and it stores only the baseUrl. + // --persist is the explicit opt-in: a local tier stores baseUrl AND its placeholder key, so the + // run that follows needs no export. const persisted = JSON.parse((await cli(["tier", "use", "kev", "--persist", "--json"], { env, home })).stdout); assert.equal(persisted.persisted, configFile); - assert.equal(JSON.parse(await fs.readFile(configFile, "utf8")).baseUrl, "http://127.0.0.1:8008"); + const stored = JSON.parse(await fs.readFile(configFile, "utf8")); + assert.equal(stored.baseUrl, "http://127.0.0.1:8008"); + assert.equal(stored.apiKey, "local", "a local tier persists its placeholder key too"); assert.equal(JSON.parse((await cli(["config", "show", "--json"], { env, home })).stdout).config.baseUrl, "http://127.0.0.1:8008"); + // Hosted has no key to store: baseUrl alone, and the text says which it wrote. + const hostedHome = await fs.mkdtemp(path.join(os.tmpdir(), "jev-tier-hosted-")); + try { + const hostedPersist = await cli(["tier", "use", "hosted", "--persist"], { env, home: hostedHome }); + assert.equal(hostedPersist.code, 0, hostedPersist.stderr); + assert.match(hostedPersist.stdout, /Stored baseUrl=https:\/\/api\.typesafe\.ai/); + const hostedConfig = JSON.parse(await fs.readFile(path.join(hostedHome, ".config", "jev-browser", "config.json"), "utf8")); + assert.equal(hostedConfig.baseUrl, "https://api.typesafe.ai"); + assert.equal("apiKey" in hostedConfig, false, "hosted has no placeholder key to store"); + + const kevPersist = await cli(["tier", "use", "kev", "--persist"], { env, home: hostedHome }); + assert.match(kevPersist.stdout, /Stored apiKey=local and baseUrl=http:\/\/127\.0\.0\.1:8008/); + } finally { + await fs.rm(hostedHome, { recursive: true, force: true }); + } + const unknown = await cli(["tier", "use", "bogus"], { env, home }); assert.equal(unknown.code, 2, "an unknown tier is a usage error"); assert.match(unknown.stderr, /unknown tier "bogus"/); diff --git a/tests/unit/webui.test.mjs b/tests/unit/webui.test.mjs new file mode 100644 index 0000000..1def486 --- /dev/null +++ b/tests/unit/webui.test.mjs @@ -0,0 +1,419 @@ +// The local WebUI: what it refuses, what it never sends to a browser, and that its answers are the +// CLI's answers. Offline throughout — the only endpoint any test needs is the mock TypeSafe +// fixture, and every child process is a stub, so no test can start a real model server. +import test from "node:test"; +import assert from "node:assert/strict"; +import { EventEmitter } from "node:events"; +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { PassThrough } from "node:stream"; +import { loadConfig } from "../../skills/jev-browser/lib/config.mjs"; +import { describeTier, fetchModels, formatTierStatus, probeEndpoint, tierRows } from "../../skills/jev-browser/lib/tiers.mjs"; +import { WebUiError, assertLoopbackHost, createWebUiServer, listenWebUi, modelRoot, parsePort, requireModelId, resolveModelPath } from "../../skills/jev-browser/lib/webui.mjs"; +import { createMockTypeSafe } from "../helpers/mock-typesafe.mjs"; + +const SKILL_DIR = path.resolve(path.dirname(new URL(import.meta.url).pathname), "..", "..", "skills", "jev-browser"); +const KEY = "sk-webui-test-0f3a9c2b7d1e4a5f"; +const DEAD_ENDPOINT = "http://127.0.0.1:9/"; + +async function tempHome(config = null) { + const home = await fs.mkdtemp(path.join(os.tmpdir(), "jev-webui-home-")); + if (config) { + const dir = path.join(home, ".config", "jev-browser"); + await fs.mkdir(dir, { recursive: true }); + await fs.writeFile(path.join(dir, "config.json"), JSON.stringify({ version: 1, ...config }), { mode: 0o600 }); + } + return home; +} + +/** A child_process.spawn stand-in: every test asserts on `calls` instead of starting anything. */ +function stubSpawn(calls) { + return (file, args, options) => { + calls.push({ file, args: [...args], options }); + const child = new EventEmitter(); + child.pid = 4242; + child.stdout = new PassThrough(); + child.stderr = new PassThrough(); + child.kill = (signal) => { + child.emit("close", 0, signal ?? null); + return true; + }; + return child; + }; +} + +async function startUi({ home, env, ...options } = {}) { + const calls = []; + const ui = createWebUiServer({ + skillDir: SKILL_DIR, + home, + env: env ?? { HOME: home, PATH: process.env.PATH }, + cwd: home, + spawn: stubSpawn(calls), + log: () => {}, + ...options, + }); + const address = await listenWebUi(ui.server, { port: 0 }); + return { ui, calls, address, base: `http://127.0.0.1:${address.port}` }; +} + +const get = (base, url) => fetch(base + url, { cache: "no-store" }); +const post = (base, url, body) => + fetch(base + url, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify(body ?? {}) }); + +async function withUi(home, fn, options = {}) { + const context = await startUi({ home, ...options }); + try { + return await fn(context); + } finally { + await context.ui.close(); + } +} + +test("the server binds loopback only, and the served page carries no secret", async () => { + assert.throws(() => assertLoopbackHost("0.0.0.0"), /loopback-only/); + assert.throws(() => assertLoopbackHost("192.168.1.10"), /loopback-only/); + assert.equal(assertLoopbackHost("127.0.0.1"), "127.0.0.1"); + + const home = await tempHome({ apiKey: KEY }); + try { + await withUi(home, async ({ base, address }) => { + assert.equal(address.address, "127.0.0.1", "the listening socket must be loopback"); + const never = createWebUiServer({ home }); + await assert.rejects(listenWebUi(never.server, { port: 0, host: "0.0.0.0" }), /loopback-only/); + assert.equal(never.server.listening, false); + + const response = await get(base, "/"); + assert.equal(response.status, 200); + assert.match(response.headers.get("content-type"), /text\/html/); + const html = await response.text(); + assert.ok(!html.includes(KEY), "the served page must not contain the API key"); + assert.ok(!/src="https?:|href="https?:|@import\s|cdn\./i.test(html), "the page must not reference any external asset"); + assert.match(html, /