feat(jev-browser): one-command local setup and a loopback WebUI - #2
Merged
Merged
Conversation
Bringing up a local tier meant running a launcher, reading an export line, exporting it, and
remembering which of two servers a port belonged to. Every step had a silent failure mode, and
four of them were live in the shipped code:
- `run`, `observe`, `judge` and `pick` never passed --home to loadConfig, so the flag was
accepted and ignored: a scratch HOME still read the real config, and a test that set a
loopback baseUrl there was testing nothing.
- A local tier needs an apiKey the client will send, but the local server ignores it, so
--persist stored baseUrl alone and the next run refused to start with "no TypeSafe API key".
- doctor called a missing key a failure even when baseUrl was loopback, where the only thing
missing is the placeholder; accepted a non-empty model file as ready, so a truncated download
was reported as installed; and offered "brew install llama.cpp" for a Kev endpoint that does
not use the GGUF registry at all.
`jev-browser setup <local-readout|kev>` is now the whole path: it finds the prerequisite (and
says which command installs it), fetches through the launcher's own --download-only, starts it
detached, writes baseUrl + the placeholder key, then asks the endpoint one real question
(noul: "Does `ticket` ask for a refund?") and prints the answer and the round trip — so "it
works" is a measured fact rather than a claim. setup status reports installed / running /
configured per tier; setup stop <tier> signals only a pid whose command line is the launcher
that wrote it, and removes a stale pid file instead of guessing. Exit 2 is everything the user
has to change, exit 1 everything that broke, and a runtime failure always names the log.
The shared seams exist once: lib/detach.mjs owns the backgrounding (new process group so it
survives Ctrl-C, pid + log under ~/.jev-browser/run) and lib/kev.mjs owns "checkout + venv +
torch/mlx", which the launcher, doctor and setup all read instead of each restating the three
commands. Both launchers gained --detach (opt-in; the foreground path is unchanged), and
JEV_LLAMA_SERVER now points at a llama.cpp outside Homebrew, where a value that does not
resolve is "not found" rather than a silent fall-through to PATH.
Tests: tests/unit/{setup,doctor,cli-home,tier}.test.mjs drive the real CLI against a scratch
HOME — the setup flows through injected seams (fetch, start, verify) so no server is needed,
doctor's three new lines are pinned by name, and cli-home covers the four commands that
ignored --home. Full suite at this commit: 108 tests, 103 pass, 0 fail, 5 skipped (Safari).
Choosing a judging tier now means holding three backends and three different goal_done bars in your head, and bringing one up means a launcher, an export line and a server whose output only exists in a log file. This adds a browser view of exactly that: one page on 127.0.0.1 whose panels call the same lib/tiers.mjs, lib/doctor.mjs and lib/config.mjs the CLI does, so the page cannot disagree with `tier status`, `doctor` or `config show` — and a download or a run can be watched instead of tailed. It is a view, not a second implementation: children are always this skill's own bin/*.mjs scripts, tier and model names are checked against TIERS and the registry, --model paths must resolve inside ~/.jev-browser/models, ports are integers in range, config edits go through saveUserConfig's allow-list, and the doctor/judge/run panels call the same functions the CLI calls. A local tool that starts processes on request is the one genuinely dangerous part of a WebUI, so every field is treated as hostile input: bodies are capped at 1 MiB, children are spawned with an argv array and never a shell, logs are buffered from a cursor (500 lines), and the server binds 127.0.0.1 only — never 0.0.0.0, so nothing here is on the LAN. Two things are deliberately withheld. No route returns the key: the config panel reports `keySet`, and every log line a child prints is passed through a redaction set containing the configured key. And stopping is scoped — each child is put in its own process group and the whole group is signalled, and closing the page stops every child it started, so a Ctrl-C on the WebUI does not leave a 4B model server holding 18 GB. tests/unit/webui.test.mjs drives the real server over HTTP on an ephemeral port and pins the promises above by name: loopback-only bind and no secret in the served page, no route returning the key in any shape, hostile fields rejected with 4xx and no process spawned, commands spawned as argv arrays so a value can never reach a shell, log streaming from a cursor, `use this tier` writing only when asked, closing the page killing every child, and the tier panel matching the CLI's own resolution rather than restating it. Full suite at this commit: 108 tests, 103 pass, 0 fail, 5 skipped (Safari).
Both front pages described a local tier as something you assemble: start a launcher, copy the export line, keep the terminal open. That was true when the GGUF readout was the only tier; now the setup is subsumed by one command that also verifies the endpoint, and the placeholder key it needs is the one detail a manual export hid. So README.md and SKILL.md lead with `jev-browser setup <tier>`, keep the by-hand route below it (it still works and the launchers still print it), and README.zh-CN.md carries the same block so the Chinese page does not lag the English one. SKILL.md gains a WebUI section next to the judging-tier table, and README.md one next to the MCP section, each with the panel list and the two properties that decide whether this is safe to run: the server binds 127.0.0.1 only, and no route returns your API key. That claim is made in the docs because it is what the code and its tests enforce — spawn with an argv array, scripts pinned to this skill's bin/, secrets redacted out of streamed logs — not as a promise the reader has to take on faith. references/config.md's apiKey row now says what a local tier actually stores: the literal placeholder `local`, because the client requires a non-empty key and the local server ignores the Authorization header, and which two commands will store it for you.
Test-only change; no product code is touched.
The truncation test asserted doctor's "download it again" hint while letting doctor resolve
llama-server from whatever machine it ran on, so on the ubuntu-latest runner — no llama.cpp and
no Homebrew to hold it — the hint was the missing prerequisite instead:
not ok 50 - a truncated model file is reported as truncated, not accepted as ready
location: '.../tests/unit/doctor.test.mjs:48:1'
error: |-
The input did not match the regular expression /download it again/. Input:
'brew install llama.cpp'
code: 'ERR_ASSERTION'
That precedence is the product's and it stays: the detail carries both facts at once
(`llama-server not found · qwen3.5-4b-q4-k-m "Qwen3.5-4B Q4_K_M" truncated (1024 of 2740937888
bytes) · 127.0.0.1:8092 not running`) and the hint names the prerequisite that is missing first,
so a truncated file is never reported ready in either state — which is what the test is about.
What was wrong was the test's assumption, so it now pins the input instead of inheriting it:
JEV_LLAMA_SERVER (the override added in the setup commit) points at a stub for the present case
and at a path that does not resolve for the absent case, and the two are asserted separately —
truncated in the detail and not ready in both, with the hint each state actually gives.
Verified in both environments: with llama.cpp hidden (PATH without /opt/homebrew/bin plus a
JEV_LLAMA_SERVER that does not resolve — the CI condition) the file passes, and it passes here
where llama.cpp is installed. A throwaway script running the old assertion in that no-llama
environment reproduces CI exactly: status warn, the truncated detail, hint "brew install
llama.cpp", assertion fails. Full suite: 108 tests, 103 pass, 0 fail, 5 skipped.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings the local judging tiers from "assemble it yourself" to one command, and adds a loopback WebUI over the same
lib/. Three commits, no product behaviour changed for a hosted-Jev user.What is in it
feat(jev-browser): one-command local setup—jev-browser setup <local-readout|kev|status|stop <tier>>is the whole path to a working local tier: check the prerequisite (and name the command that installs it), fetch through the launcher's own--download-only, start it detached, writebaseUrl+ the placeholder key, then ask the endpoint one real question and print the answer and the round trip, so "it works" is a measured fact.setup stopsignals only a pid whose command line is the launcher that wrote it, and removes a stale pid file instead of guessing. It also fixes four silent failures that were live in the shipped code:run,observe,judgeandpickaccepted--homeand never passed it toloadConfig, so a scratch HOME still read the real config.tier use --persiststoredbaseUrlalone, and the next run then refused to start with "no TypeSafe API key" — a local tier needs the literal placeholderlocal.doctorcalled a missing key a failure for a loopback baseUrl (where only the placeholder is missing), accepted a truncated model file as installed, and offered "brew install llama.cpp" for a Kev endpoint that never uses the GGUF registry.New shared seams:
lib/detach.mjs(backgrounding with a pid + log under~/.jev-browser/run; a new process group, so the server survives the caller's Ctrl-C) andlib/kev.mjs(the "checkout + venv + torch/mlx" rules, read by the launcher, doctor and setup instead of restated three times). Both launchers gained--detach(opt-in; the foreground path is unchanged),JEV_LLAMA_SERVERnow points at a llama.cpp outside Homebrew, andpackage.jsonexposesjev-local/jev-kevas bins.feat(jev-webui): loopback WebUI over the same lib/— one page on127.0.0.1:8765whose six panels (tiers, config, doctor, models, judge, run) call the samelib/tiers.mjs,lib/doctor.mjs,lib/config.mjsthe CLI does, so the page cannot disagree withtier status/doctor/config show. A local tool that starts processes on request is the dangerous part, so: bodies capped at 1 MiB, children spawned with an argv array and never a shell, scripts pinned to this skill'sbin/, tier and model names checked againstTIERSand the registry,--modelpaths confined to~/.jev-browser/models, config edits through the allow-list, no route returns the key (the panel reportskeySet; streamed logs are redacted), and each child gets its own process group so closing the page stops everything it started.docs: the local setup path and the WebUI— README.md, README.zh-CN.md, SKILL.md andreferences/config.mdnow lead withjev-browser setup <tier>(the by-hand route stays, and stays accurate), document the WebUI's six panels, and state the two properties that decide whether it is safe to run: loopback-only bind, no key in any response.Measured evidence
tests/unit/{setup,doctor,cli-home,webui}.test.mjsplus thetieradditions — all driving the real CLI or the real HTTP server on a scratch HOME, with setup's fetch/start/verify injected as seams so no server is needed.truncated (N of M bytes), and a Kev endpoint gets akevline with nojev-localhint anywhere in the report.--home <dir>isolates it) — nothing is written into the repo, and no model file is in this PR.What is not included
https://api.typesafe.ai.setup kevstill needsuvand a multi-GB download; it is exercised in tests through seams, not by pulling 9.34 GB in CI.0.0.0.0, and--portonly moves the loopback port.