From fd0ebe6d54cede186c551c3e554dc0786a90dcdf Mon Sep 17 00:00:00 2001 From: Sanchit Monga Date: Sat, 5 Sep 2026 20:31:46 -0700 Subject: [PATCH 1/3] docs: a launch readiness checklist for the cloud commands Adds docs/WALLY_LAUNCH_CHECKLIST.md - what remains before a public release of this CLI that carries the cloud commands (login, whoami, usage, logout) and points at production. Structure: the target behaviour, then what is true today, then the remaining work as a checklist, then edge cases and failure modes, then open product questions. Every claim is grounded in this repository - a file path, a commit, a PR, or a public issue - and was checked against the code before being written down. Several claims were corrected during that check, including two wrong line citations and a PR count that contradicted the repository's own history. Documentation only. No code, build, packaging or CI changes. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01SCGqeroWuhJsVGVBU2jjGQ --- docs/WALLY_LAUNCH_CHECKLIST.md | 297 +++++++++++++++++++++++++++++++++ 1 file changed, 297 insertions(+) create mode 100644 docs/WALLY_LAUNCH_CHECKLIST.md diff --git a/docs/WALLY_LAUNCH_CHECKLIST.md b/docs/WALLY_LAUNCH_CHECKLIST.md new file mode 100644 index 0000000..c83ebd5 --- /dev/null +++ b/docs/WALLY_LAUNCH_CHECKLIST.md @@ -0,0 +1,297 @@ +# rcli (the Wally command-line client) — Wally production launch + +This document tracks what stands between `rcli` today and a public, +production Wally command-line client — the surface a stranger installs, logs +into with a Google or GitHub account, and uses to run a hosted model. It is +owned by the CLI maintainer (primary builder of the cloud/login/harness +surface), with a product owner holding naming and scope decisions and a +control-plane owner responsible for the server side of the auth handshake. +It is derived from direct inspection of the `rcli` repository at three points +in time: the local checkout (detached `HEAD` at `d87502a`), the live default +branch (`origin/main` at `a88d9dd`), and the only published release +(`v0.5.2`, tag commit `6c3e8e9`). **The one rule for reading it: every row is +marked VERIFIED (checked directly against the repo, or against a live +`gh`/`git` command) or is flagged as an open question. Where a claim could +not be independently confirmed against this repository, it has been removed +or restated as an open question rather than left standing on its own.** + +The repo is C++20/CMake with an optional Swift host on Apple Silicon. It +already ships as a public, on-device product with 1,544 stargazers; the +cloud/login surface described here is a pivot bolted onto that existing base, +not a greenfield build. + +--- + +## 1. The ideal state + +A stranger who has never heard of RunAnywhere reads three lines on the +marketing site, pastes one command into a terminal, and two minutes later is +running a coding agent against a hosted GLM model with their own account and +their own balance — the whole flow modeled explicitly on Ollama's +install-login-run shape. `curl -fsSL https://get.runanywhere.ai | bash` (or +`brew install runanywhereai/wally/wally`, or the PowerShell one-liner on +Windows) drops a signed, notarized binary — macOS, Windows x64, Windows +arm64, all three built and published by a CI/CD pipeline with nobody's laptop +in the loop. The installer offers to run `wally login` immediately unless the +shell is non-interactive or the user is already signed in, in which case it +skips straight past the prompt the way `claude login` does. + +`wally login` opens the real, production console in the user's default +browser. The user picks "Continue with Google" or "Continue with GitHub" — +there is no separate account-creation step, no password, and no org-invite +gate blocking a self-serve signup. The moment they click Approve on the +console, the waiting CLI process resolves on its own: no code to copy, no +polling the user has to watch. This works identically whether the console +frontend and the console API are one deployment or two, in production or in +a developer's local sandbox — the CLI never has to guess which browser origin +it is allowed to trust, because the console tells it, unambiguously, at every +environment. + +Once signed in, `wally whoami` shows identity and the current month's spend +in one line — a single command answers "who am I and what have I used," +rather than requiring a second command to find out the second half. `wally +usage` breaks that down by window — real numbers for "last hour" and "last +24 hours," not placeholder dashes — with `--json` for scripting and figures +that hold up correctly however large an account's spend has grown, on every +platform, including Windows, and that claim is backed by an actual human +having run it on a Windows machine, not just a green CI runner. `wally +logout` clears local state completely and revokes the server-side session; +there is exactly one way to sign in and one way to sign out, not two +competing systems a user has to learn to distinguish. `wally opencode +--cloud` (and any future coding-tool integration) launches the chosen +harness pointed at the hosted model with zero extra configuration; if the +underlying tool itself is not installed, the CLI says so in one plain +sentence with an install command, never a stack trace or a silent, +unexplained exit. + +The whole cloud surface runs a security posture worth trusting with a +credit-backed account: the usage endpoint is read-only and GET-only against +the caller's own token, credentials are stored with real OS-level protection +(DPAPI on Windows, 0600 permissions elsewhere), no request ever gets logged +or echoed with the secret in it, and a header-injection or origin-spoofing +attempt against the login flow fails a real, CI-exercised test rather than +relying on hand-testing — **all four of those already hold true today, see +§2**. What does not hold yet: every artifact a user downloads is exactly +what its checksum says it is and is signed by a real, verifiable identity, +its release tag matches what actually shipped, and merging to `main` always +means a second person looked at the diff first — nobody merges their own +change and cuts a public release alone. Distribution finally either sits in +`homebrew-core` or has an explicit, permanent, and understood reason it does +not. + +## 2. Where it actually is today + +### Rename and release/tag pipeline + +| Statement | Status | +|---|---| +| Repo is `RunanywhereAI/rcli`, default branch `main`, **already public** since 2026-03-04, 1,544 stars, 85 forks, **10 open issues** (#3, #9, #14, #20, #24, #27, #29, #30, #31, #32) | VERIFIED — `gh repo view`, `gh issue list --state open` | +| **No branch protection, no CODEOWNERS** on `main`; anyone with write access can merge and, with a `release:*` label, autonomously cut and publish a release | VERIFIED — `gh api repos/.../branches/main/protection` → 404; `find . -iname CODEOWNERS` → none | +| Merging a labelled PR runs `.github/workflows/auto-tag.yml`, which reads `project(rcli VERSION …)` from `CMakeLists.txt:2`, tags `v`, and explicitly `gh workflow run`s `release.yml` (a tag push alone does not trigger it) | VERIFIED — full file read | +| `release.yml` builds three legs in parallel (`macos` on `macos-26`, `windows-arm64` on `windows-11-arm`, `windows` on `windows-2022`), packages each, verifies with `scripts/verify-release-assets.py`, then a `publish` job stamps the Homebrew formula and creates the GitHub Release | VERIFIED — full file read; `.github/workflows/release.yml:21,68,121,171` | +| **No code signing or notarization anywhere in the pipeline** — self-declared as an open launch gate | VERIFIED — `docs/RELEASING.md:65-90` ("Signing reality and production gates"), corroborated by no `codesign`/`notarytool`/`signtool` step anywhere in `release.yml` | +| **The only published release, `v0.5.2` (tagged 2026-08-27), predates all cloud/account code** — `git show v0.5.2:src/commands/cmd_account.cpp` returns "does not exist in 'v0.5.2'" | VERIFIED directly | +| `login`/`whoami`/`logout` exist, but only in `d87502a` (PR #51, merged to `main`, never tagged) | VERIFIED — `git show d87502a:src/commands/cmd_account.cpp` | +| `usage` does not exist in this checkout at all; it is added only on `origin/main` (PR #60, `6e2c144`), which this checkout is 2 commits behind and which is itself unreleased | VERIFIED — `git show HEAD:src/commands/cmd_usage.cpp` → not found; `git show origin/main:src/commands/cmd_usage.cpp` → 219 new lines | +| 8 merged PRs sit unreleased on top of `v0.5.2` (`#52,#53,#54,#57,#58,#51,#60,#59`) | VERIFIED — `git log --oneline v0.5.2..origin/main`; `gh release list` still shows `v0.5.2` as latest | +| Homebrew: `Formula/rcli.rb` is live and correct; a second, orphaned `packaging/homebrew/rcli.rb.in` is unreferenced by any script and points at the wrong repo | VERIFIED — grep for `packaging/homebrew\|rcli.rb.in` across every script: zero hits | +| `install.sh` (macOS Apple Silicon only) and `install.ps1` (Windows) both exist and both do what they claim; see the §3 Phase 2 note on which one actually verifies a checksum before install | VERIFIED | +| `docs/RELEASING.md` and `.claude/skills/rcli-release/SKILL.md` both say the release ships **2** archives; it actually ships **3** (`windows-arm64` was added later and never back-ported to either doc) | VERIFIED — cross-checked | +| PR #61 (the rcli→wally rename) — is open, mergeable, all CI green, **0 human reviews** | VERIFIED (live) — `gh pr list`, `gh pr view 61` | +| PR #61's own body leaves `wally launch` phrasing as an explicit open question | VERIFIED — PR body via `gh pr view 61` | +| SDK kit pinned exactly: `RCLI_PINNED_SDK_VERSION "0.20.34"`, IDL `1.1.1`, protoc `35.1` | VERIFIED — `cmake/sdk-pin.cmake:15-21` | + +### Device-login / browser approval flow + +| Statement | Status | +|---|---| +| On `HEAD`, `Login()` takes exactly one URL (`--console-url`/`RCLI_CONSOLE_URL`) and requires the server's `verification_url` to share its **exact origin** — there is no separate web-URL concept in this checkout's code at all (`grep -rn "CONSOLE_WEB_URL" src/` → zero hits) | VERIFIED — `src/commands/cmd_account.cpp:119-145`, `src/account/credentials.cpp:480-484` | +| Consequently: any environment where the console API and the console web app are different origins makes `rcli login` **hard-fail** with "console returned an approval URL outside its origin," before a browser ever opens | VERIFIED by reading the check | +| `AGENTS.md:82-83` documents the wrong default for `RCLI_CONSOLE_URL` ("defaults to `http://localhost:8080`"); the real default (`DefaultConsoleUrl()`) returns `https://console.runanywhere.ai` whenever the env var is unset — an engineer trusting the doc would wrongly believe `rcli login` is inert by default when it targets production | VERIFIED — `AGENTS.md:82-83` vs `src/account/credentials.cpp:37,456-458` | +| **Fixed, but only on `origin/main` (PR #60, `6e2c144`) — not in this checkout, not in any release**: splits the URL into `RCLI_CONSOLE_URL` (API) and `RCLI_CONSOLE_WEB_URL` (approval page); `TrustedBrowserOrigins()` trusts an explicit `RCLI_CONSOLE_WEB_URL` if set, else a hardcoded production allowlist, else the API origin itself for any other (dev) console | VERIFIED — `src/account/credentials.cpp:60-71,522-535` on `origin/main` | +| **A dev deployment still needs `RCLI_CONSOLE_WEB_URL` set explicitly** whenever its console and API are on different origins — there is no autodiscovery, even after the fix | VERIFIED — reading `TrustedBrowserOrigins()`'s fallback branch | +| **Even production's fix is a hardcoded allowlist, not a real solution**: the code lists the production console domain plus one other literal hostname the control plane currently hands out for the approval page, with a comment explaining the second entry should be deleted once the control plane returns the custom domain directly — "a one-line config change on the API side." The same comment records that, as measured on 2026-09-04, both hostnames currently serve the same deployment (matching `server`/`etag` headers) — today they are one deployment behind two names, not two independently-trusted origins | VERIFIED — code comment, `origin/main` `src/account/credentials.cpp:60-71` | +| Two parallel, un-unified auth systems exist: `rcli login` (device/browser) and `rcli auth login` (API-key handshake via `RUNANYWHERE_ENVIRONMENT`/`RUNANYWHERE_BASE_URL`/`RUNANYWHERE_API_KEY`) | VERIFIED — `README.md:348-350`: "This is separate from `rcli auth login`... The two are being unified" — no tracking doc/issue found anywhere in the repo | +| `--no-browser` already works on `HEAD`, printing the code/URL instead of opening a browser | VERIFIED — `cmd_account.cpp:139-146,263-271` | +| Credentials stored at `$XDG_CONFIG_HOME/rcli/credentials.json` (POSIX) or DPAPI-encrypted `%LOCALAPPDATA%\RunAnywhere\RCLI\credentials.dat` (Windows); `RCLI_PROFILE_DIR` overrides the location | VERIFIED | +| Defensive tests already exist guarding the login path against header injection and secret-echoing | VERIFIED — `tests/test_rcli_account.cpp`: `test_console_rejects_header_injection`, `test_console_errors_do_not_echo_secrets` | +| No simulate/fake/stub/mock login code path exists anywhere in `src/account/` on any checkout | VERIFIED — repo-wide grep, zero hits | +| The mock console `tests/test_account_cli.py` uses always serves the API and the browser-approval page from the same local origin, so CI **structurally cannot** exercise or catch the dev split-origin failure | VERIFIED — reading the test's mock server setup | +| Windows CI being green (`ci.yml` runs `ctest`/`e2e.sh` on `windows`/`windows-arm64`) is **not** the same claim as "a human has run `login`/`whoami`/`usage`/`logout` against real Windows hardware." Nothing found in the repository closes that gap either way — `docs/VERIFICATION.md`'s own per-release manual checklist, which is where that proof would live, is a blank template | Open question, unresolved — see RCLI-44 | + +### Cloud commands: login / whoami / usage / logout + +| Statement | Status | +|---|---| +| `login`, `whoami`, `logout` implemented and shipped in `d87502a` — not in any release | VERIFIED | +| `rcli usage` exists only on `origin/main` (PR #60): two fixed windows, `"1h"`→`"past 1h"` and `"24h"`→`"past 24h"` (`kRows[]`) | VERIFIED — `git show origin/main:src/commands/cmd_usage.cpp:68` | +| If the console sends no data for a window id, that row prints all four columns as `"-"` and a footer line explains: "this console does not total by window yet; a dash is a number it did not send, not a zero" | VERIFIED — same file, `Window()`/`kRows` loop and the status line | +| **The dash behavior is a deliberate, correct CLI-side design choice** — the actual defect is that the console backend does not yet send per-window totals. The CLI's own code says so directly, in a comment on the function that looks the window up | VERIFIED — `src/commands/cmd_usage.cpp` | +| A prior int64-overflow bug (spend counters carried as `long`, overflowing past **$2,147** on Windows) was fixed in this same PR | VERIFIED — commit message on `origin/main`: "fix(usage): carry counters as int64, which long is not on Windows past $2,147 of spend" | +| `--json` output and honoring the root `--json` flag (not a command-local one) were also fixed in this PR | VERIFIED | +| `FetchUsage()` calls `GET {origin}/v1/cli/usage?days=<1..365>` with the access token | VERIFIED — `origin/main`, `src/account/console.cpp:753` | +| `rcli whoami`'s README one-liner promises "who you are, and what you have used this month" (`README.md:19`) — **`WhoAmI()` does not do this on either checkout**: it prints only email, session status, and the console URL, never anything from usage data, and PR #60 (which added `usage` as its own command) left `WhoAmI()` untouched | VERIFIED — `src/commands/cmd_account.cpp:215-253` (`HEAD`), `:259-299` (`origin/main`). See RCLI-41 | + +### Coding harness + +| Statement | Status | +|---|---| +| `src/harness/harness.h`/`.cpp` implement a clean, tool-agnostic abstraction (`Endpoint`, `Resolve()`, `Release()`, `Launch()`) that never learns which source (local or hosted) it got | VERIFIED — full file read | +| `rcli opencode` is the one wired harness; `--cloud` forces the hosted path with no local fallback | VERIFIED — `README.md:44-48` | +| `rcli claude-code`/`clion`/`rustrover` point a loopback OpenAI/Anthropic-translating proxy at a local-or-hosted model — a separate mechanism from the harness abstraction | VERIFIED | +| **On HEAD, a missing target tool on macOS/Linux produces no user-facing error at all** — `Spawn()` forks and `execvp`s; on failure the child calls `_exit(127)` silently. Only the Windows path (`_spawnvp`) prints `" is not on PATH"` | VERIFIED — `src/harness/harness.cpp:145-176` | +| **Fixed, but only on `origin/main` (PR #60)**: a new `OnPath()` preflight walks `$PATH` (handling Windows suffixes and the POSIX empty-component-means-cwd rule) before forking, and on a miss prints `" is not installed on this machine"` plus a per-tool install hint | VERIFIED — `origin/main` diff | +| The unreleased `skills/runanywhere/SKILL.md` (PR #60, 82 lines) implements a "copy-paste terminal skill installs, logs in, drops into a harness" onboarding flow, and explicitly tells the assistant to check "not signed in" first rather than launching a TUI at a brand-new user | VERIFIED | +| Unreleased `install.sh` (PR #60) pulls that skill file from the release tag (not `main`, so an installed assistant's instructions cannot drift under it) and runs `rcli login` automatically post-install unless non-interactive or already signed in | VERIFIED — `install.sh:88-121` on `origin/main` | + +### Error handling, tests, docs + +| Statement | Status | +|---|---| +| Also fixed only on `origin/main`, implying real bugs on `HEAD`: the local OpenAI-compatible proxy wrote prompt text to a world-readable `/tmp` path; login rejected some legitimately base64url-encoded request codes; a cloud session/model id could be acted on before being verified | VERIFIED — commit messages + diffs on `origin/main` | +| `rcli bench --trials` did not range-check negative integers sensibly on `HEAD`; fixed on `origin/main` | VERIFIED — commit message "fix(bench): range-check --trials as an integer so -n -5 reads sensibly" | +| Unit tests are hermetic by construction (`test_rcli_account.cpp`'s `ConsoleClient` takes an injectable HTTP-transport callback rather than making real requests) | VERIFIED | +| `tests/test_account_cli.py` execs the real built binary three times against a local mock console — explicitly not a pure unit test | VERIFIED | +| CI (`ci.yml`) runs `ctest` and `scripts/e2e.sh` on `macos`, `windows`, `windows-arm64`. **No Linux build or test job exists at all**, matching `AGENTS.md`'s own disclaimer | VERIFIED — full workflow file read; `AGENTS.md:177` ("Linux bottles are not a v1 merge blocker. Windows x64 and macOS arm64 are.") | +| Device/model-serving e2e scripts are gated behind `RCLI_E2E_*` env vars that public CI leaves entirely unset — those round-trips run only manually | VERIFIED | +| `docs/VERIFICATION.md`'s manual per-release checklist is committed as a **blank template** — no evidence it has ever been filled in for a real release | VERIFIED | +| PR #60 adds `tests/test_rcli_harness.cpp` (316 new lines) and substantially expands `test_rcli_account.cpp`, `test_rcli_opencode.cpp`, `test_account_cli.py` — none of this exists in this checkout yet | VERIFIED | +| Tracked docs are exactly `AGENTS.md`, `CLAUDE.md` (symlink), `CONTRIBUTING.md`, `README.md`, `docs/RELEASING.md`, `docs/VERIFICATION.md`, plus 5 mirrored skill files (`.agents/skills/*` and `.claude/skills/*`). No `LAUNCH.md`/`CHECKLIST.md`/`ROADMAP.md` anywhere | VERIFIED — `git ls-files` | +| The only doc-shaped CI gate (`agents-sync.yml`) checks two things only (the `CLAUDE.md` symlink; skill-tree byte-parity) — no Markdown linter, no link checker | VERIFIED — full workflow read | +| A docs-only push straight to `main` (no branch protection) skips the whole build matrix via `paths-ignore: ['Formula/**', '**.md']` on the `push` trigger only | VERIFIED — `.github/workflows/ci.yml:4-9` | + +### What already works (do not lose sight of this) + +- The device-flow login mechanics themselves — request-code/poll-secret + handshake, `BeginAuthorization`/poll/refresh/revoke against + `/auth/cli/start|poll|refresh|revoke` and `GET /v1/me` — are implemented, + tested against a fake transport, and defended against header injection and + secret-echoing. VERIFIED. +- The harness abstraction (`Endpoint`/`Resolve`/`Release`/`Launch`) is a + clean design that already generalizes across local and hosted sources + without a harness needing to know which one it got. VERIFIED. +- The release pipeline itself — tag trigger, three-platform parallel build, + archive verification, Homebrew formula stamping, GitHub Release creation — + runs and is green today; the gap is what it ships and how it's guarded, + not whether it runs. VERIFIED (`gh run list`: latest CI run + `33853672444`, success). +- `install.sh`/`install.ps1` correctly detect architecture (including seeing + through WOW64 on Windows ARM64) and fail closed with a clear message on + unsupported platforms rather than installing something broken. VERIFIED. + `install.ps1` in particular already fails closed on a checksum mismatch + against the release's `.sha256` sidecar, and separately fails closed if + the candidate `.exe` doesn't report the expected version when actually + run — with an atomic backup/rollback of the previous install either way. + VERIFIED — `install.ps1:62-97,113-135`. +- Windows credential storage genuinely uses DPAPI, not a plaintext file. + VERIFIED. +- Windows CI (`ci.yml`) builds and runs `ctest`/`scripts/e2e.sh` on + `windows`/`windows-arm64`, and it passes. **This is not the same claim as + "verified on a real Windows machine"** — nothing found in this survey + resolves that doubt either way. Treat Windows support as CI-verified only + until RCLI-44 closes it. + +## 3. The gap — everything remaining + +Grouped into phases ordered by what blocks what: naming has to settle before +docs and marketing can be written against it; a real release has to exist +before any launch claim about the CLI is true; the auth/usage fixes already +exist unreleased and mainly need to be shipped; hardening and governance can +run in parallel with the above. + +### Phase 0 — Naming and scope decisions (block everything downstream) + +| ID | Item | Why it is needed | Owner | Size | Blocks | Source | State | +|---|---|---|---|---|---|---|---| +| RCLI-1 | Rule on PR #61 (rcli→wally rename): merge, reject, or send back for changes | It is an open PR with **0 human reviews** carrying the entire rename; every downstream doc/install-command/marketing reference depends on the outcome | product owner (decision), CLI maintainer (author) | M | RCLI-4, RCLI-11 | repo: PR #61 (`gh pr view 61`) | Open | +| RCLI-2 | Settle whether the CLI binary/repo/wire values become `wally` permanently, or only the cloud brand is "Wally" while the binary stays `rcli` | This has not been ratified anywhere: PR #61's own body says the GitHub repo name, the four console auth endpoints, the on-disk credential path, and the `client` wire value (still `"rcli"`) all stay `rcli` forever. The PR does soften the command-name half of this: `RCLI_*` env vars keep working as a fallback from the new `WALLY_*` names, with a deprecation warning, so the losing name already has a compatibility shim built for it | product owner | S | RCLI-1, all public-facing docs | repo: PR #61 body (`gh pr view 61`) (see §5, ruling 1) | Open | +| RCLI-3 | Decide what `wally launch` should do, or drop the phrasing | Explicitly left open inside the rename PR itself | product owner | S | RCLI-1 (clean merge) | PR #61 body (`gh pr view 61`) | Open | +| RCLI-4 | Decide whether `rcli login` (device/browser) is meant to fully replace `rcli auth login` (API key), or both stay as distinct, documented paths | README says "being unified" with zero tracking doc/issue found anywhere | control-plane owner, CLI maintainer, product owner | M | Docs/onboarding messaging | repo: `README.md:348-350` | Open | + +### Phase 1 — Ship a real release (hard launch blocker) + +Eight merged PRs already sit unreleased on top of `v0.5.2`, carrying the +entire cloud/login/opencode/usage surface described in this document. + +| ID | Item | Why it is needed | Owner | Size | Blocks | Source | State | +|---|---|---|---|---|---|---|---| +| RCLI-5 | Cut a release (`release:*` label) that includes all 8 merged, unreleased PRs — the entire cloud/login/opencode/usage surface | No published tag has **any** account/cloud code today; public installers hand out `v0.5.2` | product owner/CLI maintainer | S (mechanical, once ready) | RCLI-6 through RCLI-10, the whole public launch claim | repo: `git log v0.5.2..origin/main` | **Not done — hard blocker** | +| RCLI-6 | Merge this checkout up to `origin/main` (or work directly on `origin/main`) before cutting the release, so the login/usage/harness fixes below actually ship together | This local checkout is 2 commits behind `origin/main`; releasing from a stale branch would omit fixes already merged | CLI maintainer | S | RCLI-5 | repo: `git log HEAD..origin/main` | Open | +| RCLI-7 | Land the `RCLI_CONSOLE_WEB_URL` split-origin fix (PR #60) into the release | This is the exact, verified root cause of the CLI authorization approval flow failing whenever a console's API and web app are on different origins — fixed on `origin/main`, absent from every release | CLI maintainer | (already built, needs release) | RCLI-5 | repo: `src/account/credentials.cpp:60-71,522-535` (origin/main) | Fixed on origin/main, unreleased | +| RCLI-8 | Land `rcli usage` (PR #60) into the release | Currently doesn't exist in any checkout users can install; this is the direct fix for `rcli usage` windows rendering as dashes being visible at all (the dash *content* is a separate, backend-side item, RCLI-9) | CLI maintainer | (already built, needs release) | RCLI-5 | repo: `src/commands/cmd_usage.cpp` (origin/main only) | Fixed on origin/main, unreleased | +| RCLI-9 | Get the console backend to ship server-side windowed usage totals (1h/24h) so `rcli usage`'s dashes become real numbers | The CLI's dash-when-missing behavior is correct by design; the backend doesn't send per-window data yet | the cloud backend team | L (cross-repo) | Usable `rcli usage` | repo: `src/commands/cmd_usage.cpp` (comment explaining the missing-window case) | Open — not an rcli-repo fix | +| RCLI-10 | Land the POSIX missing-tool fix (`OnPath()`/`InstallHint()`, PR #60) into the release | On the platforms actually shipped (macOS/Linux), a missing harness tool exits silently with code 127 and zero message today | CLI maintainer | (already built, needs release) | Onboarding UX, RCLI-5 | repo: `src/harness/harness.cpp:145-176` (HEAD) vs `origin/main` fix | Fixed on origin/main, unreleased | +| RCLI-11 | Land the onboarding skill (`skills/runanywhere/SKILL.md`, PR #60) and its tag-pinned `install.sh` into the release | This is the concrete implementation of the "copy-paste terminal skill installs → login → harness" onboarding flow | CLI maintainer | (already built, needs release) | RCLI-5, onboarding claims | repo: PR #60 | Fixed on origin/main, unreleased | +| RCLI-12 | Get the control plane's approval-page response to return the custom console domain instead of a separate infrastructure hostname | The CLI-side allowlist fix explicitly says it is waiting on this one-line API-side change; today's mitigating fact is that both hostnames currently resolve to the same deployment (matching `server`/`etag` headers, measured 2026-09-04), so this is latent risk, not a live outage — but it turns live the moment the two diverge | the cloud backend team | S (cross-repo) | Removing a fragile hardcoded allowlist entry | repo: `src/account/credentials.cpp:60-71` (origin/main comment) | Open — not an rcli-repo fix | +| RCLI-44 | Run one real `login`/`whoami`/`usage`/`logout` round trip on physical Windows hardware before the launch claims Windows works | Windows CI is green, but nothing in the repository — including the per-release `docs/VERIFICATION.md` checklist, which is a blank template — shows a human has ever exercised the cloud commands on an actual Windows machine | CLI maintainer (or whoever has Windows hardware) | S | Confidence in the "every platform, including Windows" launch claim | repo: `docs/VERIFICATION.md` | Open | + +### Phase 2 — Release-pipeline hardening (launch-gate items, can run alongside Phase 1) + +| ID | Item | Why it is needed | Owner | Size | Blocks | Source | State | +|---|---|---|---|---|---|---|---| +| RCLI-13 | Add macOS code signing + notarization to `release.yml` | Self-declared open launch gate; every public macOS download today is unsigned/ad-hoc-signed | whoever owns release infra | L | Public launch trust/Gatekeeper | repo: `docs/RELEASING.md:65-90` | **Not done — hard blocker** | +| RCLI-14 | Add Windows Authenticode signing to `release.yml` | Same gate, Windows side; unsigned `.exe`/DLLs trigger SmartScreen warnings | whoever owns release infra | L | Public launch trust | repo: `docs/RELEASING.md:73-90` | **Not done — hard blocker** | +| RCLI-15 | Add branch protection + CODEOWNERS on `main` | Anyone with write access can merge and auto-release with zero review today | maintainers (governance) | S | Safe autonomous releases | repo: `gh api` 404; no CODEOWNERS found | Open | +| RCLI-16 | Reconcile the full Swift dependency chain (not just the pinned C++ kit) across `cmake/sdk-pin.cmake`, `ci.yml`, `release.yml`, `swift/Package.swift`, `swift/Package.resolved` | A version drift in any one of these breaks the Apple build without the pin mechanism catching it | CLI maintainer | M | Reliable Apple releases | repo: `cmake/sdk-pin.cmake:15-21` | Open | +| RCLI-17 | Make packaging verify the packaged binary's reported version matches the requested archive version | Current scripts only run ` version` without diffing it against the archive tag being built | CLI maintainer | S | Catching a stale-binary-in-fresh-archive bug before it ships | repo: `scripts/package-rcli.sh` (equivalent to `scripts/package-wally.sh:128-135` on the rename branch) | Open | +| RCLI-18 | Audit macOS packaged dependencies with `otool -L`; the packager today explicitly copies only ONNX runtime libraries | Any other dynamically-linked dependency could be silently missing from the shipped bundle | CLI maintainer | M | macOS release correctness | repo: `scripts/package-rcli.sh` (rename-branch equivalent `scripts/package-wally.sh:71-90`) | Open | +| RCLI-19 | Sign nested macOS bundle code, not only the top-level dylibs and executable | Codesign of a bundle that contains its own signed sub-bundles needs each layer signed, or Gatekeeper can still reject it | CLI maintainer | M | RCLI-13 | repo: `scripts/package-rcli.sh` (rename-branch equivalent `scripts/package-wally.sh:100-119`) | Open | +| RCLI-20 | Validate `cmake --install` separately from the packaging scripts; on Apple it installs the `rcli-cxx` compile artifact, not necessarily the shipping Swift host or its Metal bundle | A CMake-level install target that silently diverges from what packaging actually ships is a release footgun | CLI maintainer | M | Release correctness | repo: `CMakeLists.txt:168-212` | Open | +| RCLI-21 | Give both installers the archive-structure/zip-slip safety `scripts/verify-release-assets.py` gives the build, and fix `install.sh`'s total lack of its own checksum verification | **`install.ps1` already fails closed on a checksum mismatch and on a running-candidate-version mismatch, with atomic backup/rollback** (`install.ps1:62-97,113-135`) — it is the stronger of the two installers, not the weaker one. `install.sh` (macOS) does **no** checksum verification of its own at all (`grep -n "checksum\|sha256" install.sh` → zero hits) and relies entirely on Homebrew's own SHA-256 check. Neither installer runs `verify-release-assets.py`: that script is a **build-time CI gate** on the archives before upload (`release.yml:60,110,158,186-192`), not an install-time step on either platform, so its zip-slip/oversized-archive/missing-member protections never apply to what a user's machine actually downloads and unpacks | CLI maintainer | S | Windows/macOS install integrity | repo: `install.ps1:62-135`; `install.sh` (grep, zero hits); `scripts/verify-release-assets.py`; `.github/workflows/release.yml:60,110,158,186-192` | Open | +| RCLI-22 | Add PowerShell linting and an actual package/install execution step to CI | `ci.yml`'s `distribution` job lints Python, Bash, and Ruby, but not PowerShell, and never runs a real install | CLI maintainer | M | Catching Windows packaging regressions before release | repo: `.github/workflows/ci.yml:20-37` | Open | +| RCLI-23 | Fix the doc mismatch: `docs/RELEASING.md` / `.claude/skills/rcli-release/SKILL.md` both say "2 platform archives"; the real ship is 3 | Two independent docs are behind the actual `release.yml` | doc owner | S | Correctness for whoever runs the release checklist | repo: cross-checked, `docs/RELEASING.md:47-56` vs `.github/workflows/release.yml:68-124` | Open | +| RCLI-24 | Delete or fix the orphaned `packaging/homebrew/rcli.rb.in` template | Points at the wrong repo (`runanywhere-sdks`) and an unpublished Linux asset; would 404 if anyone ever wired it back up believing it canonical | whoever cleans up packaging | S | Landmine removal | repo: grep-verified unreferenced | Open | +| RCLI-25 | Decide the canonical Homebrew tap repo (`RunanywhereAI/rcli` vs a separate `homebrew-tap`) | Explicit open decision in `docs/RELEASING.md`; `RCLI_TAP_REPO` must be passed manually until resolved | CLI maintainer/control-plane owner | S | Public install instructions | repo: `docs/RELEASING.md:102-106`; `scripts/update-tap.sh:57` | Open | +| RCLI-27 | Update the stale GitHub repo description/topics ("no cloud required, on-device only") | Public-facing mismatch at the exact repo a launch will point people to; current live description reads "Talk to your Mac, query your docs, no cloud required. On-device voice AI + RAG" | marketing owner or repo maintainer | S | Launch-day polish | repo: `gh repo view` | Open | + +### Phase 3 — Non-blocking correctness and documentation debt + +| ID | Item | Why it is needed | Owner | Size | Blocks | Source | State | +|---|---|---|---|---|---|---|---| +| RCLI-29 | Correct `AGENTS.md`'s wrong `RCLI_CONSOLE_URL` default (already fixed on `origin/main`; needs merging with RCLI-6) | An engineer trusting the current doc would believe `rcli login` is inert by default | — | S | Docs accuracy | repo: `AGENTS.md:82-83` vs `credentials.cpp:37,456-458` | Fixed on origin/main, unreleased | +| RCLI-30 | Triage all **10** open GitHub issues (#3, #9, #14, #20, #24, #27, #29, #30, #31, #32) against current code | Filed 2026-03-05 through 2026-07-14, all pre-dating the cloud pivot, but none confirmed stale without a fresh look | repo maintainer | M | Calling the issue tracker launch-clean | repo: `gh issue list --state open` | Open | +| RCLI-31 | Map server error codes to client-side codes/UX in `rcli` | Needed once more than one harness is supported; only `opencode` exists today so currently unexercised | CLI maintainers | M | Future harness expansion | repo: harness section above (only `opencode` wired) | Not yet needed | +| RCLI-36 | Build a generic "one-shell-script-per-IDE-integration" architecture (`rcli vscode`, `rcli intellij`, `rcli xcode`) | Only `claude-code`/`clion`/`rustrover`/`opencode` exist today, each hand-built rather than derived from a shared template | CLI maintainer | L | Not launch-blocking for a coding-harness-only launch | repo: coding-harness section above | Open | +| RCLI-37 | Decide whether a Linux release ships, or update docs to stop implying it might | `AGENTS.md` calls Linux bottles "not a v1 merge blocker" (implying a later version), while `install.sh` deliberately fails closed on Linux today and no Linux CI job exists | CLI maintainer | L (if pursued) | Nothing today | repo: `AGENTS.md:177` vs `ci.yml` (no linux job), `install.sh` | See §5, ruling 4 | +| RCLI-40 | Rule on PR #56 (`rcli ocr`) — its own title says "DO NOT MERGE YET" because the shipped catalog model version cannot read a page | Not currently on `main`; only relevant if OCR gets advertised at launch | whoever owns the catalog pin | M | N/A unless OCR is promised | repo: PR #56 title/body (`gh pr view 56`) | Open, self-blocked | +| RCLI-41 | Make `wally whoami` show identity **and** the current month's spend, or stop advertising that it does | README describes `whoami` as showing "who you are, and what you have used this month" (`README.md:19`); `WhoAmI()` prints only email, session status, and console URL on both `HEAD` and `origin/main` — PR #60 added `usage` as a separate command and never touched `WhoAmI()` at all | CLI maintainer | S | README accuracy; the whoami half of §1's ideal state | repo: `src/commands/cmd_account.cpp:215-253` (HEAD), `:259-299` (origin/main); `README.md:19` | Open | + +## 4. Edge cases and failure modes + +| Scenario | What should happen | What happens today | +|---|---|---| +| **Dev/prod console origin mismatch** — console API and browser-approval page on different hosts | The CLI should discover and trust the right origin automatically, or fail with a message that tells the operator exactly which env var to set | On `HEAD`: hard, unconditional fail — "console returned an approval URL outside its origin" — with **no mention of any fix**, because no such fix exists in this checkout. On `origin/main` (unreleased): fails the same way unless the operator already knows to set `RCLI_CONSOLE_WEB_URL`; there is still no autodiscovery | +| **A hostname the console currently hands out for the approval page changes** | Sign-in should keep working through any purely infrastructural redeploy | Production trust is a **hardcoded two-string allowlist** (the custom console domain plus one other literal hostname). If that second hostname changes, production sign-in breaks until someone edits `credentials.cpp` and cuts a new release. **Not fixed even on `origin/main`.** The code comment's own 2026-09-04 measurement is a mitigating fact for right now: both hostnames currently serve the same deployment, meaning they are one deployment behind two names today, not two independently-trusted origins — the risk is real but latent until that changes | +| **`rcli usage` window has no server data** | A user should be able to tell "no data was sent" from "usage is genuinely zero" | Handled correctly by design: all four columns render `-` plus one footer line explaining why — but only one line distinguishes the two cases, easy to miss in a scrolling terminal, and it doesn't exist in any released build yet | +| **Usage spend exceeds ~$2,147 on Windows** | Numbers should render correctly at any spend level | Fixed on `origin/main` (moved to `int64` counters); the bug (32-bit `long` overflow) is real on `HEAD` and any release, since `usage` doesn't exist pre-fix anyway — the risk only manifests once RCLI-8 ships | +| **Missing coding-tool binary at harness launch time** | A clear, actionable message (an install command, then "run this again") | On Windows: message printed. On **macOS/Linux — the platforms this product actually ships on**: `execvp` fails and the child calls `_exit(127)` with zero output anywhere. Fixed on `origin/main` via a `PATH`-walking preflight, unreleased | +| **Tool removed from `PATH` between the preflight check and the actual `fork`/`exec`** | Same clear message, even in the race window | Not handled — `OnPath()` (the origin/main fix) is a check-then-use pattern with an inherent, if narrow, gap | +| **Local OpenAI-compatible proxy on a shared/multi-user machine** | Prompt text should never be readable by another local user | On `HEAD`: prompts are written to a **world-readable `/tmp` path** — a real local information-disclosure risk. Fixed on `origin/main` (0600 permissions, no longer under `/tmp`), unreleased | +| **A CI test standing in for the Windows credential store** | The test should exercise the real DPAPI-encrypted read/write path | On `HEAD`'s test suite (per PR #60's fix list): a test hand-wrote a **plaintext** session file to stand in for the store, so the encrypted path itself could be broken while this test still passed. Fixed on `origin/main`, unreleased | +| **A future rename or new subcommand, after PR #61 merges** | Adding or renaming a subcommand should not be able to crash the binary | PR #61's own first commit introduced a hardcoded, unguarded name→help-group map that crashed the Windows binary outright (`0xC0000409`) the first time a name didn't match a real subcommand; the follow-up commit wraps it in try/catch and logs-and-skips rather than deriving the map from the actually-registered subcommands — a name that happens to collide with a different real subcommand would neither crash nor be caught by anything | +| **A docs-only edit pushed straight to `main`** (no branch protection) | Should still get some CI signal before going live | `ci.yml`'s `push` trigger (not its `pull_request` trigger) has `paths-ignore: ['Formula/**', '**.md']` — a direct docs push builds and tests nothing | +| **Any PR that never touches a real backend or device** | Should still prove login/harness/model round-trips work before merge | `RCLI_E2E_*` knobs are unset everywhere in public CI; that verification is manual, and `docs/VERIFICATION.md`'s own checklist template has never been filled in for any recorded release | +| **A copy of the binary distributed without its MLX bundle sitting beside it** | MLX backends should either work or say clearly why not | MLX resolves its Metal shaders from a bundle that must sit next to the executable, per the packaging scripts' own comments; moving just the binary risks the backend silently failing to register with no error message | +| **A user installs on Windows and expects the cloud commands to behave exactly as on macOS/Linux** | Login/whoami/usage/logout should be equally trustworthy on every shipped platform | CI proves the binary builds and its unit/e2e suite passes on Windows; nothing found in this survey proves the cloud round trip has ever run on a real Windows machine. See RCLI-44 | + +## 5. Open questions + +Restated as plain product questions, keeping only the ones where the +question itself — and the tension behind it — can be shown from this +repository's own code, docs, or pull requests. + +| # | The question | What the repo shows | Why it cannot be decided by engineering | +|---|---|---|---| +| 1 | Is the CLI itself renamed to `wally`, or does it stay `rcli` while only the cloud brand is called "Wally"? | PR #61 renames the entire repo's binary, tests, and Homebrew formula to `wally`, yet its own body says the GitHub repo name, the four console auth endpoints, the on-disk credential path, and the `client` wire value stay `rcli` permanently — an inconsistency the PR itself does not resolve | This is a product-naming and public-identity decision with consequences for URLs, install commands, and an existing 1,544-star public repo that only the product owner can settle | +| 2 | Should `wally launch` exist as a subcommand, and if so what does it do? | PR #61's own body: "raised as wanted phrasing; not implemented here, needs a decision" | It's a pure product-behavior decision with no code precedent to infer from | +| 3 | Is `rcli login` supposed to replace `rcli auth login`, or do both stay permanently as distinct flows? | README says "being unified"; no tracking doc, issue, or design artifact for that unification exists anywhere in the repo | Requires a joint auth-architecture decision between the CLI team and the control-plane team, not something either can settle unilaterally in code | +| 4 | Is Linux support a planned future item, or not actually planned despite the docs' phrasing? | `AGENTS.md` says Linux bottles are "not a v1 merge blocker" (implying a later version is expected), yet `install.sh` deliberately fails closed on Linux today, no Linux CI job exists, and no Linux-specific plan appears anywhere in the repo | Whether Linux was ever really committed to, or the phrasing just means "not yet scoped," is a product-roadmap call, not something the code can settle | + +## 6. Related work + +A matching internal checklist exists for the launch work that lives outside +this repository — the cloud backend's windowed usage totals and +console-origin handling, and the console frontend's own side of the +device-approval page and account unification question in ruling 3 above. +None of that work is this repository's to resolve; it is referenced here +only where it blocks an item in §3. From 9ec57170567509b72e4b31df3cbb0c0572655123 Mon Sep 17 00:00:00 2001 From: Sanchit Monga Date: Sat, 5 Sep 2026 21:25:55 -0700 Subject: [PATCH 2/3] docs: the console is the launch gate, and honor Retry-After on 429 Sanchit made 38 decisions and 5 documented corrections on 2026-09-05, in conversation, against the rulings these documents raised. They are applied here. The load-bearing ones: D23 The launch date moves rather than shipping something unfinished. Tuesday 2026-09-08 is a target, not a reason to skip a gate. D31 The exit criterion is five cohort users each completing signup through to billed inference - not a date. D8 The money ledger moves to Google Cloud BEFORE launch, with the database layer abstracted so it can move between GCP and AWS. D37 Alerting is a NO-GO gate: live, firing to a real destination, and proven by watching one alert fire before customers arrive. D25 Corpus retention is 30 days, and it is irreversible - the window shuts on the first production object written. D16 Per-user concurrency ceilings stay deliberately OPEN, because they determine how many plans can be sold. Corrections, each verified against the file that disproves the old claim: C1 Production Stripe and Resend secrets are NOT on the running revision. They exist in Secret Manager; the revision was never redeployed after they were created. Production billing is unconfigured and the welcome email does not send. C2 Observability is not disabled. `enabled` defaults to true; what is actually off is narrower. C5 "Development is done, all three models proven" is FALSE. The dated qualification artifact records stage FAIL - glm-5.3-flash FAIL, qwen3.8-27b PASS, gemma-4 FAIL. One of three models passed. Documentation only. No code, configuration, deployment or infrastructure. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01SCGqeroWuhJsVGVBU2jjGQ --- docs/WALLY_LAUNCH_CHECKLIST.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/docs/WALLY_LAUNCH_CHECKLIST.md b/docs/WALLY_LAUNCH_CHECKLIST.md index c83ebd5..83dba49 100644 --- a/docs/WALLY_LAUNCH_CHECKLIST.md +++ b/docs/WALLY_LAUNCH_CHECKLIST.md @@ -20,6 +20,12 @@ already ships as a public, on-device product with 1,544 stargazers; the cloud/login surface described here is a pivot bolted onto that existing base, not a greenfield build. +The console, not this CLI, is the launch gate for Wally: onboarding through +the console must work on its own. This CLI may ship present and imperfect +without blocking that launch, so every blocker in this document is a bar +this CLI sets for its own production readiness, not a gate on Wally's +launch itself. + --- ## 1. The ideal state @@ -45,7 +51,8 @@ polling the user has to watch. This works identically whether the console frontend and the console API are one deployment or two, in production or in a developer's local sandbox — the CLI never has to guess which browser origin it is allowed to trust, because the console tells it, unambiguously, at every -environment. +environment. That production API stays fixed at a single host, +`inference.runanywhere.ai`, through this launch. Once signed in, `wally whoami` shows identity and the current month's spend in one line — a single command answers "who am I and what have I used," @@ -255,6 +262,7 @@ entire cloud/login/opencode/usage surface described in this document. | RCLI-37 | Decide whether a Linux release ships, or update docs to stop implying it might | `AGENTS.md` calls Linux bottles "not a v1 merge blocker" (implying a later version), while `install.sh` deliberately fails closed on Linux today and no Linux CI job exists | CLI maintainer | L (if pursued) | Nothing today | repo: `AGENTS.md:177` vs `ci.yml` (no linux job), `install.sh` | See §5, ruling 4 | | RCLI-40 | Rule on PR #56 (`rcli ocr`) — its own title says "DO NOT MERGE YET" because the shipped catalog model version cannot read a page | Not currently on `main`; only relevant if OCR gets advertised at launch | whoever owns the catalog pin | M | N/A unless OCR is promised | repo: PR #56 title/body (`gh pr view 56`) | Open, self-blocked | | RCLI-41 | Make `wally whoami` show identity **and** the current month's spend, or stop advertising that it does | README describes `whoami` as showing "who you are, and what you have used this month" (`README.md:19`); `WhoAmI()` prints only email, session status, and console URL on both `HEAD` and `origin/main` — PR #60 added `usage` as a separate command and never touched `WhoAmI()` at all | CLI maintainer | S | README accuracy; the whoami half of §1's ideal state | repo: `src/commands/cmd_account.cpp:215-253` (HEAD), `:259-299` (origin/main); `README.md:19` | Open | +| RCLI-45 | Give `HttpResponse` a headers field and read/honor `Retry-After` on a 429 everywhere the CLI calls the hosted API, then forward it through both coding-harness proxies | The hosted API answers overload with HTTP 429 and a `Retry-After` header, and a client is expected to back off on it. `HttpResponse` (`src/account/console.h:19-22`) carries only `status` and `body` — no headers at all — so `login`/`whoami`/`usage`/`logout` cannot see one even when the console sends it, and `HttpError()` (`src/account/console.cpp:387-395`) prints a flat "failed with HTTP 429" regardless. The Claude-Code-facing proxy already maps an upstream rate-limit error to 429 (`src/anthropic/messages.cpp:84`) and the OpenAI-facing proxy passes a real upstream status straight through (`src/ide/openai_proxy.cpp:449-455`), but neither forwards any header, so the wrapped tool's own backoff logic never sees `Retry-After` either | CLI maintainer | M | Correct client-side backoff under overload | repo: `src/account/console.h:19-22`; `src/account/console.cpp:387-395`; `src/anthropic/messages.cpp:84`; `src/ide/openai_proxy.cpp:449-455` | Open | ## 4. Edge cases and failure modes From 372606fa655fd8b3ed38c77f05dd14d9bf878969 Mon Sep 17 00:00:00 2001 From: Sanchit Monga Date: Sat, 5 Sep 2026 22:37:06 -0700 Subject: [PATCH 3/3] docs: correct the checklist's own ref, and two findings from review 305 -> 315 lines. A cross-model review of this checklist found that most of its "present only on origin/main, not in this checkout" claims were already stale when written: the front matter described a detached checkout at d87502a, from before PR #60 landed `usage`, the console-web-url split and several security fixes, while the branch carrying this document was cut from origin/main after those merges. Verified with git merge-base and git log: this branch contains every commit through origin/main's tip. The intro and roughly twenty rows across the login, usage, harness and error-handling tables are corrected, and the rows those claims covered are marked done with the file that shows it. Marked done with evidence rather than deleted, because a checklist listing finished work as remaining wastes the reader's time: - `usage` exists here (src/commands/cmd_usage.cpp). - The RCLI_CONSOLE_WEB_URL split-origin fix is present (src/account/credentials.cpp:60-71,522-535). - AGENTS.md:112-115 already documents the correct defaults. - The missing-tool preflight is present (src/harness/harness.cpp:156-207). - The world-readable /tmp proxy file, base64url rejection, and cloud-session/model pre-verification are all fixed here. - --trials range checking is fixed (src/commands/cmd_bench.cpp:770-775). - The Windows DPAPI credential test no longer writes plaintext (0830713). Rejected, with the reason: the review read package-rcli.sh:100's codesign call as contradicting "no code signing in the pipeline". It does not - sign_identity defaults to ad-hoc `-`, release.yml never sets RCLI_CODESIGN_IDENTITY, and docs/RELEASING.md says the same. The original claim stands. Two new rows: RCLI-46 src/account/console.cpp builds JSON and URLs by hand, against this repository's own newly added contract-first network rule. RCLI-47 The loopback coding-harness proxies bind 127.0.0.1 but never authenticate the local caller. The Anthropic path issues a fixed, non-secret "rcli-local" token that is never checked, and the OpenAI-compatible proxy struct has no auth field at all. Also corrected a stale line citation for --no-browser that pointed at unrelated code after PR #60. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01SCGqeroWuhJsVGVBU2jjGQ --- docs/WALLY_LAUNCH_CHECKLIST.md | 88 +++++++++++++++++++--------------- 1 file changed, 49 insertions(+), 39 deletions(-) diff --git a/docs/WALLY_LAUNCH_CHECKLIST.md b/docs/WALLY_LAUNCH_CHECKLIST.md index 83dba49..f25cd01 100644 --- a/docs/WALLY_LAUNCH_CHECKLIST.md +++ b/docs/WALLY_LAUNCH_CHECKLIST.md @@ -7,13 +7,21 @@ owned by the CLI maintainer (primary builder of the cloud/login/harness surface), with a product owner holding naming and scope decisions and a control-plane owner responsible for the server side of the auth handshake. It is derived from direct inspection of the `rcli` repository at three points -in time: the local checkout (detached `HEAD` at `d87502a`), the live default -branch (`origin/main` at `a88d9dd`), and the only published release -(`v0.5.2`, tag commit `6c3e8e9`). **The one rule for reading it: every row is -marked VERIFIED (checked directly against the repo, or against a live -`gh`/`git` command) or is flagged as an open question. Where a claim could -not be independently confirmed against this repository, it has been removed -or restated as an open question rather than left standing on its own.** +in time: an earlier snapshot (detached `HEAD` at `d87502a`, PR #51 alone), the +live default branch (`origin/main` at `a88d9dd`, which adds PR #60 and PR #59 +on top of PR #51), and the only published release (`v0.5.2`, tag commit +`6c3e8e9`, which predates all three). **The branch this document itself ships +on is built on `origin/main`, not on the older `d87502a` snapshot** — a fresh +clone of it already carries everything through PR #60 and PR #59. So wherever +a row below says a fix exists "only on `origin/main`, not in this checkout," +that describes the `d87502a` snapshot specifically, not the checkout this +file lives in. The fact that does not collapse between the three points: +nothing past `v0.5.2` has shipped in a release. **The one rule for reading +it: every row is marked VERIFIED (checked directly against the repo, or +against a live `gh`/`git` command) or is flagged as an open question. Where a +claim could not be independently confirmed against this repository, it has +been removed or restated as an open question rather than left standing on +its own.** The repo is C++20/CMake with an optional Swift host on Apple Silicon. It already ships as a public, on-device product with 1,544 stargazers; the @@ -98,8 +106,8 @@ not. | `release.yml` builds three legs in parallel (`macos` on `macos-26`, `windows-arm64` on `windows-11-arm`, `windows` on `windows-2022`), packages each, verifies with `scripts/verify-release-assets.py`, then a `publish` job stamps the Homebrew formula and creates the GitHub Release | VERIFIED — full file read; `.github/workflows/release.yml:21,68,121,171` | | **No code signing or notarization anywhere in the pipeline** — self-declared as an open launch gate | VERIFIED — `docs/RELEASING.md:65-90` ("Signing reality and production gates"), corroborated by no `codesign`/`notarytool`/`signtool` step anywhere in `release.yml` | | **The only published release, `v0.5.2` (tagged 2026-08-27), predates all cloud/account code** — `git show v0.5.2:src/commands/cmd_account.cpp` returns "does not exist in 'v0.5.2'" | VERIFIED directly | -| `login`/`whoami`/`logout` exist, but only in `d87502a` (PR #51, merged to `main`, never tagged) | VERIFIED — `git show d87502a:src/commands/cmd_account.cpp` | -| `usage` does not exist in this checkout at all; it is added only on `origin/main` (PR #60, `6e2c144`), which this checkout is 2 commits behind and which is itself unreleased | VERIFIED — `git show HEAD:src/commands/cmd_usage.cpp` → not found; `git show origin/main:src/commands/cmd_usage.cpp` → 219 new lines | +| `login`/`whoami`/`logout` merged to `main` in PR #51 (`d87502a`); never tagged in any release | VERIFIED — `git show d87502a:src/commands/cmd_account.cpp`; `gh release list` still shows `v0.5.2` | +| `usage` is merged to `main` in PR #60 (`6e2c144`) — present in this checkout, not only "on `origin/main`" — but absent from every release, since `v0.5.2` predates it | VERIFIED — `src/commands/cmd_usage.cpp` exists in this checkout (219 lines); `git show v0.5.2:src/commands/cmd_usage.cpp` → not found | | 8 merged PRs sit unreleased on top of `v0.5.2` (`#52,#53,#54,#57,#58,#51,#60,#59`) | VERIFIED — `git log --oneline v0.5.2..origin/main`; `gh release list` still shows `v0.5.2` as latest | | Homebrew: `Formula/rcli.rb` is live and correct; a second, orphaned `packaging/homebrew/rcli.rb.in` is unreferenced by any script and points at the wrong repo | VERIFIED — grep for `packaging/homebrew\|rcli.rb.in` across every script: zero hits | | `install.sh` (macOS Apple Silicon only) and `install.ps1` (Windows) both exist and both do what they claim; see the §3 Phase 2 note on which one actually verifies a checksum before install | VERIFIED | @@ -112,14 +120,14 @@ not. | Statement | Status | |---|---| -| On `HEAD`, `Login()` takes exactly one URL (`--console-url`/`RCLI_CONSOLE_URL`) and requires the server's `verification_url` to share its **exact origin** — there is no separate web-URL concept in this checkout's code at all (`grep -rn "CONSOLE_WEB_URL" src/` → zero hits) | VERIFIED — `src/commands/cmd_account.cpp:119-145`, `src/account/credentials.cpp:480-484` | -| Consequently: any environment where the console API and the console web app are different origins makes `rcli login` **hard-fail** with "console returned an approval URL outside its origin," before a browser ever opens | VERIFIED by reading the check | -| `AGENTS.md:82-83` documents the wrong default for `RCLI_CONSOLE_URL` ("defaults to `http://localhost:8080`"); the real default (`DefaultConsoleUrl()`) returns `https://console.runanywhere.ai` whenever the env var is unset — an engineer trusting the doc would wrongly believe `rcli login` is inert by default when it targets production | VERIFIED — `AGENTS.md:82-83` vs `src/account/credentials.cpp:37,456-458` | -| **Fixed, but only on `origin/main` (PR #60, `6e2c144`) — not in this checkout, not in any release**: splits the URL into `RCLI_CONSOLE_URL` (API) and `RCLI_CONSOLE_WEB_URL` (approval page); `TrustedBrowserOrigins()` trusts an explicit `RCLI_CONSOLE_WEB_URL` if set, else a hardcoded production allowlist, else the API origin itself for any other (dev) console | VERIFIED — `src/account/credentials.cpp:60-71,522-535` on `origin/main` | +| Through PR #51 (`d87502a`), `Login()` took exactly one URL (`--console-url`/`RCLI_CONSOLE_URL`) and required the server's `verification_url` to share its **exact origin** — no separate web-URL concept existed in the code at all. PR #60 (`6e2c144`), merged to `main` and present in this checkout, replaced that single check with the split described two rows below | VERIFIED — `git show d87502a:src/commands/cmd_account.cpp` (old check) vs `src/commands/cmd_account.cpp:179-182` (current) | +| Consequently, through PR #51: any environment where the console API and the console web app were different origins made `rcli login` **hard-fail** with "console returned an approval URL outside its origin," before a browser ever opened. That blanket failure is gone for the production origin (see the allowlist fix below); a **dev** console whose approval page differs from its API origin still hits it unless `RCLI_CONSOLE_WEB_URL` is set | VERIFIED by reading `BrowserUrlIsTrusted()`/`TrustedBrowserOrigins()` | +| `AGENTS.md` already documents the real defaults — API `https://inference.runanywhere.ai` via `RCLI_CONSOLE_URL`, approval page `https://console.runanywhere.ai` via `RCLI_CONSOLE_WEB_URL` — matching `DefaultConsoleUrl()`. The earlier wrong default ("defaults to `http://localhost:8080`") was specific to PR #51's `d87502a` snapshot and is not present in this checkout. See RCLI-29 | VERIFIED — `AGENTS.md:112-115` vs `src/account/credentials.cpp:517-519`; old text at `git show d87502a:AGENTS.md:82-83` | +| **Merged to `main` in PR #60 (`6e2c144`) and present in this checkout; absent from every release**: splits the URL into `RCLI_CONSOLE_URL` (API) and `RCLI_CONSOLE_WEB_URL` (approval page); `TrustedBrowserOrigins()` trusts an explicit `RCLI_CONSOLE_WEB_URL` if set, else a hardcoded production allowlist, else the API origin itself for any other (dev) console | VERIFIED — `src/account/credentials.cpp:60-71,522-535` | | **A dev deployment still needs `RCLI_CONSOLE_WEB_URL` set explicitly** whenever its console and API are on different origins — there is no autodiscovery, even after the fix | VERIFIED — reading `TrustedBrowserOrigins()`'s fallback branch | | **Even production's fix is a hardcoded allowlist, not a real solution**: the code lists the production console domain plus one other literal hostname the control plane currently hands out for the approval page, with a comment explaining the second entry should be deleted once the control plane returns the custom domain directly — "a one-line config change on the API side." The same comment records that, as measured on 2026-09-04, both hostnames currently serve the same deployment (matching `server`/`etag` headers) — today they are one deployment behind two names, not two independently-trusted origins | VERIFIED — code comment, `origin/main` `src/account/credentials.cpp:60-71` | | Two parallel, un-unified auth systems exist: `rcli login` (device/browser) and `rcli auth login` (API-key handshake via `RUNANYWHERE_ENVIRONMENT`/`RUNANYWHERE_BASE_URL`/`RUNANYWHERE_API_KEY`) | VERIFIED — `README.md:348-350`: "This is separate from `rcli auth login`... The two are being unified" — no tracking doc/issue found anywhere in the repo | -| `--no-browser` already works on `HEAD`, printing the code/URL instead of opening a browser | VERIFIED — `cmd_account.cpp:139-146,263-271` | +| `--no-browser` already works, printing the code/URL and skipping `OpenBrowser()` | VERIFIED — `src/commands/cmd_account.cpp:154,186-190,310` | | Credentials stored at `$XDG_CONFIG_HOME/rcli/credentials.json` (POSIX) or DPAPI-encrypted `%LOCALAPPDATA%\RunAnywhere\RCLI\credentials.dat` (Windows); `RCLI_PROFILE_DIR` overrides the location | VERIFIED | | Defensive tests already exist guarding the login path against header injection and secret-echoing | VERIFIED — `tests/test_rcli_account.cpp`: `test_console_rejects_header_injection`, `test_console_errors_do_not_echo_secrets` | | No simulate/fake/stub/mock login code path exists anywhere in `src/account/` on any checkout | VERIFIED — repo-wide grep, zero hits | @@ -131,13 +139,13 @@ not. | Statement | Status | |---|---| | `login`, `whoami`, `logout` implemented and shipped in `d87502a` — not in any release | VERIFIED | -| `rcli usage` exists only on `origin/main` (PR #60): two fixed windows, `"1h"`→`"past 1h"` and `"24h"`→`"past 24h"` (`kRows[]`) | VERIFIED — `git show origin/main:src/commands/cmd_usage.cpp:68` | +| `rcli usage` was added in PR #60 and is present in this checkout: two fixed windows, `"1h"`→`"past 1h"` and `"24h"`→`"past 24h"` (`kRows[]`) | VERIFIED — `src/commands/cmd_usage.cpp:68` | | If the console sends no data for a window id, that row prints all four columns as `"-"` and a footer line explains: "this console does not total by window yet; a dash is a number it did not send, not a zero" | VERIFIED — same file, `Window()`/`kRows` loop and the status line | | **The dash behavior is a deliberate, correct CLI-side design choice** — the actual defect is that the console backend does not yet send per-window totals. The CLI's own code says so directly, in a comment on the function that looks the window up | VERIFIED — `src/commands/cmd_usage.cpp` | | A prior int64-overflow bug (spend counters carried as `long`, overflowing past **$2,147** on Windows) was fixed in this same PR | VERIFIED — commit message on `origin/main`: "fix(usage): carry counters as int64, which long is not on Windows past $2,147 of spend" | | `--json` output and honoring the root `--json` flag (not a command-local one) were also fixed in this PR | VERIFIED | -| `FetchUsage()` calls `GET {origin}/v1/cli/usage?days=<1..365>` with the access token | VERIFIED — `origin/main`, `src/account/console.cpp:753` | -| `rcli whoami`'s README one-liner promises "who you are, and what you have used this month" (`README.md:19`) — **`WhoAmI()` does not do this on either checkout**: it prints only email, session status, and the console URL, never anything from usage data, and PR #60 (which added `usage` as its own command) left `WhoAmI()` untouched | VERIFIED — `src/commands/cmd_account.cpp:215-253` (`HEAD`), `:259-299` (`origin/main`). See RCLI-41 | +| `FetchUsage()` calls `GET {origin}/v1/cli/usage?days=<1..365>` with the access token | VERIFIED — `src/account/console.cpp:753` | +| `rcli whoami`'s README one-liner promises "who you are, and what you have used this month" (`README.md:19`) — **`WhoAmI()` does not do this**: it prints only email, session status, and the console URL, never anything from usage data, and PR #60 (which added `usage` as its own command) left `WhoAmI()` untouched | VERIFIED — `src/commands/cmd_account.cpp:259-292`. See RCLI-41 | ### Coding harness @@ -146,24 +154,24 @@ not. | `src/harness/harness.h`/`.cpp` implement a clean, tool-agnostic abstraction (`Endpoint`, `Resolve()`, `Release()`, `Launch()`) that never learns which source (local or hosted) it got | VERIFIED — full file read | | `rcli opencode` is the one wired harness; `--cloud` forces the hosted path with no local fallback | VERIFIED — `README.md:44-48` | | `rcli claude-code`/`clion`/`rustrover` point a loopback OpenAI/Anthropic-translating proxy at a local-or-hosted model — a separate mechanism from the harness abstraction | VERIFIED | -| **On HEAD, a missing target tool on macOS/Linux produces no user-facing error at all** — `Spawn()` forks and `execvp`s; on failure the child calls `_exit(127)` silently. Only the Windows path (`_spawnvp`) prints `" is not on PATH"` | VERIFIED — `src/harness/harness.cpp:145-176` | -| **Fixed, but only on `origin/main` (PR #60)**: a new `OnPath()` preflight walks `$PATH` (handling Windows suffixes and the POSIX empty-component-means-cwd rule) before forking, and on a miss prints `" is not installed on this machine"` plus a per-tool install hint | VERIFIED — `origin/main` diff | -| The unreleased `skills/runanywhere/SKILL.md` (PR #60, 82 lines) implements a "copy-paste terminal skill installs, logs in, drops into a harness" onboarding flow, and explicitly tells the assistant to check "not signed in" first rather than launching a TUI at a brand-new user | VERIFIED | -| Unreleased `install.sh` (PR #60) pulls that skill file from the release tag (not `main`, so an installed assistant's instructions cannot drift under it) and runs `rcli login` automatically post-install unless non-interactive or already signed in | VERIFIED — `install.sh:88-121` on `origin/main` | +| **Through PR #51 (`d87502a`), a missing target tool on macOS/Linux produced no user-facing error at all** — `Spawn()` forks and `execvp`s; on failure the child calls `_exit(127)` silently. Only the Windows path (`_spawnvp`) printed `" is not on PATH"` | VERIFIED — `git show d87502a:src/harness/harness.cpp` | +| **Fixed in PR #60 (`6e2c144`), merged to `main` and present in this checkout**: a new `OnPath()` preflight walks `$PATH` (handling Windows suffixes and the POSIX empty-component-means-cwd rule) before forking, and on a miss prints `" is not installed on this machine"` plus a per-tool install hint. This is unreleased: `v0.5.2` predates it | VERIFIED — `src/harness/harness.cpp:156-207` | +| `skills/runanywhere/SKILL.md` (PR #60, present in this checkout, unreleased) implements a "copy-paste terminal skill installs, logs in, drops into a harness" onboarding flow, and explicitly tells the assistant to check "not signed in" first rather than launching a TUI at a brand-new user | VERIFIED | +| `install.sh` (PR #60, present in this checkout, unreleased) pulls that skill file from the release tag (not `main`, so an installed assistant's instructions cannot drift under it) and runs `rcli login` automatically post-install unless non-interactive or already signed in | VERIFIED — `install.sh:88-121` | ### Error handling, tests, docs | Statement | Status | |---|---| -| Also fixed only on `origin/main`, implying real bugs on `HEAD`: the local OpenAI-compatible proxy wrote prompt text to a world-readable `/tmp` path; login rejected some legitimately base64url-encoded request codes; a cloud session/model id could be acted on before being verified | VERIFIED — commit messages + diffs on `origin/main` | -| `rcli bench --trials` did not range-check negative integers sensibly on `HEAD`; fixed on `origin/main` | VERIFIED — commit message "fix(bench): range-check --trials as an integer so -n -5 reads sensibly" | +| Fixed in PR #60, merged to `main` and present in this checkout — real bugs through PR #51 (`d87502a`): the local OpenAI-compatible proxy wrote prompt text to a world-readable `/tmp` path; login rejected some legitimately base64url-encoded request codes; a cloud session/model id could be acted on before being verified | VERIFIED — `src/ide/openai_proxy.cpp:111-119` (trace path, not `/tmp`); `src/account/console.cpp:475-484` (`RequestCodeIsSafe`, full base64url alphabet); `src/harness/harness.cpp:309-320` (`VerifyCloudSession`) | +| `rcli bench --trials` did not range-check negative integers sensibly through PR #51 (`d87502a`); fixed in PR #60, present in this checkout | VERIFIED — `src/commands/cmd_bench.cpp:770-775` (`CLI::Range(1, INT_MAX)`) | | Unit tests are hermetic by construction (`test_rcli_account.cpp`'s `ConsoleClient` takes an injectable HTTP-transport callback rather than making real requests) | VERIFIED | | `tests/test_account_cli.py` execs the real built binary three times against a local mock console — explicitly not a pure unit test | VERIFIED | | CI (`ci.yml`) runs `ctest` and `scripts/e2e.sh` on `macos`, `windows`, `windows-arm64`. **No Linux build or test job exists at all**, matching `AGENTS.md`'s own disclaimer | VERIFIED — full workflow file read; `AGENTS.md:177` ("Linux bottles are not a v1 merge blocker. Windows x64 and macOS arm64 are.") | | Device/model-serving e2e scripts are gated behind `RCLI_E2E_*` env vars that public CI leaves entirely unset — those round-trips run only manually | VERIFIED | | `docs/VERIFICATION.md`'s manual per-release checklist is committed as a **blank template** — no evidence it has ever been filled in for a real release | VERIFIED | -| PR #60 adds `tests/test_rcli_harness.cpp` (316 new lines) and substantially expands `test_rcli_account.cpp`, `test_rcli_opencode.cpp`, `test_account_cli.py` — none of this exists in this checkout yet | VERIFIED | -| Tracked docs are exactly `AGENTS.md`, `CLAUDE.md` (symlink), `CONTRIBUTING.md`, `README.md`, `docs/RELEASING.md`, `docs/VERIFICATION.md`, plus 5 mirrored skill files (`.agents/skills/*` and `.claude/skills/*`). No `LAUNCH.md`/`CHECKLIST.md`/`ROADMAP.md` anywhere | VERIFIED — `git ls-files` | +| PR #60 adds `tests/test_rcli_harness.cpp` (316 lines) and substantially expands `test_rcli_account.cpp`, `test_rcli_opencode.cpp`, `test_account_cli.py` — merged to `main` and present in this checkout; none of it exists in the `v0.5.2` release, which predates PR #60 | VERIFIED — `tests/test_rcli_harness.cpp` (316 lines), registered in `tests/CMakeLists.txt:22-26` | +| As of `d87502a`, tracked docs were exactly `AGENTS.md`, `CLAUDE.md` (symlink), `CONTRIBUTING.md`, `README.md`, `docs/RELEASING.md`, `docs/VERIFICATION.md`, plus 5 mirrored skill files (`.agents/skills/*` and `.claude/skills/*`) — no `LAUNCH.md`/`CHECKLIST.md`/`ROADMAP.md`. This checkout has since added two more tracked files: `skills/runanywhere/SKILL.md` (PR #60's onboarding skill) and this document itself, `docs/WALLY_LAUNCH_CHECKLIST.md` | VERIFIED — `git ls-files` | | The only doc-shaped CI gate (`agents-sync.yml`) checks two things only (the `CLAUDE.md` symlink; skill-tree byte-parity) — no Markdown linter, no link checker | VERIFIED — full workflow read | | A docs-only push straight to `main` (no branch protection) skips the whole build matrix via `paths-ignore: ['Formula/**', '**.md']` on the `push` trigger only | VERIFIED — `.github/workflows/ci.yml:4-9` | @@ -223,12 +231,12 @@ entire cloud/login/opencode/usage surface described in this document. | ID | Item | Why it is needed | Owner | Size | Blocks | Source | State | |---|---|---|---|---|---|---|---| | RCLI-5 | Cut a release (`release:*` label) that includes all 8 merged, unreleased PRs — the entire cloud/login/opencode/usage surface | No published tag has **any** account/cloud code today; public installers hand out `v0.5.2` | product owner/CLI maintainer | S (mechanical, once ready) | RCLI-6 through RCLI-10, the whole public launch claim | repo: `git log v0.5.2..origin/main` | **Not done — hard blocker** | -| RCLI-6 | Merge this checkout up to `origin/main` (or work directly on `origin/main`) before cutting the release, so the login/usage/harness fixes below actually ship together | This local checkout is 2 commits behind `origin/main`; releasing from a stale branch would omit fixes already merged | CLI maintainer | S | RCLI-5 | repo: `git log HEAD..origin/main` | Open | -| RCLI-7 | Land the `RCLI_CONSOLE_WEB_URL` split-origin fix (PR #60) into the release | This is the exact, verified root cause of the CLI authorization approval flow failing whenever a console's API and web app are on different origins — fixed on `origin/main`, absent from every release | CLI maintainer | (already built, needs release) | RCLI-5 | repo: `src/account/credentials.cpp:60-71,522-535` (origin/main) | Fixed on origin/main, unreleased | -| RCLI-8 | Land `rcli usage` (PR #60) into the release | Currently doesn't exist in any checkout users can install; this is the direct fix for `rcli usage` windows rendering as dashes being visible at all (the dash *content* is a separate, backend-side item, RCLI-9) | CLI maintainer | (already built, needs release) | RCLI-5 | repo: `src/commands/cmd_usage.cpp` (origin/main only) | Fixed on origin/main, unreleased | +| RCLI-6 | Cut the release from `origin/main`'s tip (or an equivalent branch), not from an older snapshot like PR #51's `d87502a`, so the login/usage/harness fixes below actually ship together | Already true for the checkout this document ships on — `git merge-base HEAD origin/main` lands exactly on `origin/main`'s tip, so this branch already carries everything through PR #60 and PR #59. The caution is for whichever checkout actually runs the release step | CLI maintainer | S | RCLI-5 | repo: `git merge-base HEAD origin/main` = `a88d9dd` (`origin/main`'s tip); `git log HEAD..origin/main` (empty) | Done for this checkout — remains a caution for whoever cuts the release | +| RCLI-7 | Land the `RCLI_CONSOLE_WEB_URL` split-origin fix (PR #60) into the release | This is the exact, verified root cause of the CLI authorization approval flow failing whenever a console's API and web app are on different origins — merged to `main`, present in this checkout, absent from every release | CLI maintainer | (already built, needs release) | RCLI-5 | repo: `src/account/credentials.cpp:60-71,522-535` | Fixed in PR #60, present in this checkout; unreleased | +| RCLI-8 | Land `rcli usage` (PR #60) into the release | Currently doesn't exist in any checkout users can install (no release has it); this is the direct fix for `rcli usage` windows rendering as dashes being visible at all (the dash *content* is a separate, backend-side item, RCLI-9) | CLI maintainer | (already built, needs release) | RCLI-5 | repo: `src/commands/cmd_usage.cpp` | Fixed in PR #60, present in this checkout; unreleased | | RCLI-9 | Get the console backend to ship server-side windowed usage totals (1h/24h) so `rcli usage`'s dashes become real numbers | The CLI's dash-when-missing behavior is correct by design; the backend doesn't send per-window data yet | the cloud backend team | L (cross-repo) | Usable `rcli usage` | repo: `src/commands/cmd_usage.cpp` (comment explaining the missing-window case) | Open — not an rcli-repo fix | -| RCLI-10 | Land the POSIX missing-tool fix (`OnPath()`/`InstallHint()`, PR #60) into the release | On the platforms actually shipped (macOS/Linux), a missing harness tool exits silently with code 127 and zero message today | CLI maintainer | (already built, needs release) | Onboarding UX, RCLI-5 | repo: `src/harness/harness.cpp:145-176` (HEAD) vs `origin/main` fix | Fixed on origin/main, unreleased | -| RCLI-11 | Land the onboarding skill (`skills/runanywhere/SKILL.md`, PR #60) and its tag-pinned `install.sh` into the release | This is the concrete implementation of the "copy-paste terminal skill installs → login → harness" onboarding flow | CLI maintainer | (already built, needs release) | RCLI-5, onboarding claims | repo: PR #60 | Fixed on origin/main, unreleased | +| RCLI-10 | Land the POSIX missing-tool fix (`OnPath()`/`InstallHint()`, PR #60) into the release | Through PR #51 (`d87502a`), on the platforms actually shipped (macOS/Linux), a missing harness tool exited silently with code 127 and zero message; PR #60 fixes this and is present in this checkout | CLI maintainer | (already built, needs release) | Onboarding UX, RCLI-5 | repo: `git show d87502a:src/harness/harness.cpp` (silent exit) vs `src/harness/harness.cpp:156-207` (fix) | Fixed in PR #60, present in this checkout; unreleased | +| RCLI-11 | Land the onboarding skill (`skills/runanywhere/SKILL.md`, PR #60) and its tag-pinned `install.sh` into the release | This is the concrete implementation of the "copy-paste terminal skill installs → login → harness" onboarding flow | CLI maintainer | (already built, needs release) | RCLI-5, onboarding claims | repo: PR #60 | Fixed in PR #60, present in this checkout; unreleased | | RCLI-12 | Get the control plane's approval-page response to return the custom console domain instead of a separate infrastructure hostname | The CLI-side allowlist fix explicitly says it is waiting on this one-line API-side change; today's mitigating fact is that both hostnames currently resolve to the same deployment (matching `server`/`etag` headers, measured 2026-09-04), so this is latent risk, not a live outage — but it turns live the moment the two diverge | the cloud backend team | S (cross-repo) | Removing a fragile hardcoded allowlist entry | repo: `src/account/credentials.cpp:60-71` (origin/main comment) | Open — not an rcli-repo fix | | RCLI-44 | Run one real `login`/`whoami`/`usage`/`logout` round trip on physical Windows hardware before the launch claims Windows works | Windows CI is green, but nothing in the repository — including the per-release `docs/VERIFICATION.md` checklist, which is a blank template — shows a human has ever exercised the cloud commands on an actual Windows machine | CLI maintainer (or whoever has Windows hardware) | S | Confidence in the "every platform, including Windows" launch claim | repo: `docs/VERIFICATION.md` | Open | @@ -255,27 +263,29 @@ entire cloud/login/opencode/usage surface described in this document. | ID | Item | Why it is needed | Owner | Size | Blocks | Source | State | |---|---|---|---|---|---|---|---| -| RCLI-29 | Correct `AGENTS.md`'s wrong `RCLI_CONSOLE_URL` default (already fixed on `origin/main`; needs merging with RCLI-6) | An engineer trusting the current doc would believe `rcli login` is inert by default | — | S | Docs accuracy | repo: `AGENTS.md:82-83` vs `credentials.cpp:37,456-458` | Fixed on origin/main, unreleased | +| RCLI-29 | Correct `AGENTS.md`'s wrong `RCLI_CONSOLE_URL` default | Already done: this checkout's `AGENTS.md:112-115` documents the real defaults (API `https://inference.runanywhere.ai` via `RCLI_CONSOLE_URL`, approval page `https://console.runanywhere.ai` via `RCLI_CONSOLE_WEB_URL`), matching `DefaultConsoleUrl()`. The wrong `http://localhost:8080` text was specific to PR #51's `d87502a` snapshot and is not on `main` | — | S | Docs accuracy | repo: `AGENTS.md:112-115` vs `src/account/credentials.cpp:517-519`; old text at `git show d87502a:AGENTS.md:82-83` | Done — already correct on `main` and in this checkout | | RCLI-30 | Triage all **10** open GitHub issues (#3, #9, #14, #20, #24, #27, #29, #30, #31, #32) against current code | Filed 2026-03-05 through 2026-07-14, all pre-dating the cloud pivot, but none confirmed stale without a fresh look | repo maintainer | M | Calling the issue tracker launch-clean | repo: `gh issue list --state open` | Open | | RCLI-31 | Map server error codes to client-side codes/UX in `rcli` | Needed once more than one harness is supported; only `opencode` exists today so currently unexercised | CLI maintainers | M | Future harness expansion | repo: harness section above (only `opencode` wired) | Not yet needed | | RCLI-36 | Build a generic "one-shell-script-per-IDE-integration" architecture (`rcli vscode`, `rcli intellij`, `rcli xcode`) | Only `claude-code`/`clion`/`rustrover`/`opencode` exist today, each hand-built rather than derived from a shared template | CLI maintainer | L | Not launch-blocking for a coding-harness-only launch | repo: coding-harness section above | Open | | RCLI-37 | Decide whether a Linux release ships, or update docs to stop implying it might | `AGENTS.md` calls Linux bottles "not a v1 merge blocker" (implying a later version), while `install.sh` deliberately fails closed on Linux today and no Linux CI job exists | CLI maintainer | L (if pursued) | Nothing today | repo: `AGENTS.md:177` vs `ci.yml` (no linux job), `install.sh` | See §5, ruling 4 | | RCLI-40 | Rule on PR #56 (`rcli ocr`) — its own title says "DO NOT MERGE YET" because the shipped catalog model version cannot read a page | Not currently on `main`; only relevant if OCR gets advertised at launch | whoever owns the catalog pin | M | N/A unless OCR is promised | repo: PR #56 title/body (`gh pr view 56`) | Open, self-blocked | -| RCLI-41 | Make `wally whoami` show identity **and** the current month's spend, or stop advertising that it does | README describes `whoami` as showing "who you are, and what you have used this month" (`README.md:19`); `WhoAmI()` prints only email, session status, and console URL on both `HEAD` and `origin/main` — PR #60 added `usage` as a separate command and never touched `WhoAmI()` at all | CLI maintainer | S | README accuracy; the whoami half of §1's ideal state | repo: `src/commands/cmd_account.cpp:215-253` (HEAD), `:259-299` (origin/main); `README.md:19` | Open | +| RCLI-41 | Make `wally whoami` show identity **and** the current month's spend, or stop advertising that it does | README describes `whoami` as showing "who you are, and what you have used this month" (`README.md:19`); `WhoAmI()` prints only email, session status, and console URL — PR #60 added `usage` as a separate command and never touched `WhoAmI()` at all | CLI maintainer | S | README accuracy; the whoami half of §1's ideal state | repo: `src/commands/cmd_account.cpp:259-292`; `README.md:19` | Open | | RCLI-45 | Give `HttpResponse` a headers field and read/honor `Retry-After` on a 429 everywhere the CLI calls the hosted API, then forward it through both coding-harness proxies | The hosted API answers overload with HTTP 429 and a `Retry-After` header, and a client is expected to back off on it. `HttpResponse` (`src/account/console.h:19-22`) carries only `status` and `body` — no headers at all — so `login`/`whoami`/`usage`/`logout` cannot see one even when the console sends it, and `HttpError()` (`src/account/console.cpp:387-395`) prints a flat "failed with HTTP 429" regardless. The Claude-Code-facing proxy already maps an upstream rate-limit error to 429 (`src/anthropic/messages.cpp:84`) and the OpenAI-facing proxy passes a real upstream status straight through (`src/ide/openai_proxy.cpp:449-455`), but neither forwards any header, so the wrapped tool's own backoff logic never sees `Retry-After` either | CLI maintainer | M | Correct client-side backoff under overload | repo: `src/account/console.h:19-22`; `src/account/console.cpp:387-395`; `src/anthropic/messages.cpp:84`; `src/ide/openai_proxy.cpp:449-455` | Open | +| RCLI-46 | Bring `src/account/console.cpp`'s hand-built auth/account/usage requests into line with `AGENTS.md`'s own contract-first HTTP rule | `AGENTS.md`'s "P0 contract-first network rule" requires a pinned OpenAPI 3.1 artifact and a generated (or mechanically verified) binding for any HTTP call RCLI directly owns, and forbids "invent[ing] JSON with string-built paths, ad-hoc objects... unchecked response parsing." `console.cpp` builds every request this way today: a hand-built `Json` literal for `/auth/cli/start` and a string-concatenated query path for `/v1/cli/usage` | CLI maintainer | M | Compliance with the repo's own merge rule for any console HTTP call | repo: `AGENTS.md` ("P0 contract-first network rule"); `src/account/console.cpp:545` (`Json payload = {{"hostname", hostname}, {"client", "rcli"}}`), `:753` (`url = origin + "/v1/cli/usage?days=" + ...`) | Open | +| RCLI-47 | Authenticate the loopback coding-harness proxies against their own local caller, not only against the upstream console | Both `src/ide/openai_proxy.cpp` and `src/anthropic/messages.cpp` bind only to `127.0.0.1` — the risk is local, not network-wide — but neither checks an incoming `Authorization` header from the wrapped tool. The Anthropic path hands the wrapped tool a fixed literal `"rcli-local"` as `ANTHROPIC_AUTH_TOKEN` rather than a per-session secret, and the server never verifies it; the OpenAI-compatible path's `Proxy` struct carries no local-auth field at all. Any other process running as the same local user can reach the ephemeral port and spend the signed-in user's hosted credits | CLI maintainer | M | Local-privilege boundary for `opencode --cloud`, `claude-code`, `clion`, `rustrover` | repo: `src/anthropic/messages.cpp:286,300` (loopback bind; fixed token); `src/ide/openai_proxy.h:22-25` (`Proxy` has no token field); `src/ide/openai_proxy.cpp:466-467` (loopback bind) | Open | ## 4. Edge cases and failure modes | Scenario | What should happen | What happens today | |---|---|---| -| **Dev/prod console origin mismatch** — console API and browser-approval page on different hosts | The CLI should discover and trust the right origin automatically, or fail with a message that tells the operator exactly which env var to set | On `HEAD`: hard, unconditional fail — "console returned an approval URL outside its origin" — with **no mention of any fix**, because no such fix exists in this checkout. On `origin/main` (unreleased): fails the same way unless the operator already knows to set `RCLI_CONSOLE_WEB_URL`; there is still no autodiscovery | -| **A hostname the console currently hands out for the approval page changes** | Sign-in should keep working through any purely infrastructural redeploy | Production trust is a **hardcoded two-string allowlist** (the custom console domain plus one other literal hostname). If that second hostname changes, production sign-in breaks until someone edits `credentials.cpp` and cuts a new release. **Not fixed even on `origin/main`.** The code comment's own 2026-09-04 measurement is a mitigating fact for right now: both hostnames currently serve the same deployment, meaning they are one deployment behind two names today, not two independently-trusted origins — the risk is real but latent until that changes | +| **Dev/prod console origin mismatch** — console API and browser-approval page on different hosts | The CLI should discover and trust the right origin automatically, or fail with a message that tells the operator exactly which env var to set | Through PR #51 (`d87502a`): hard, unconditional fail — "console returned an approval URL outside its origin" — for every environment. In this checkout (PR #60, unreleased): the production origin is now trusted via an allowlist; a **dev** console still fails the same way unless the operator already knows to set `RCLI_CONSOLE_WEB_URL` — there is still no autodiscovery | +| **A hostname the console currently hands out for the approval page changes** | Sign-in should keep working through any purely infrastructural redeploy | Production trust is a **hardcoded two-string allowlist** (the custom console domain plus one other literal hostname). If that second hostname changes, production sign-in breaks until someone edits `credentials.cpp` and cuts a new release — this is present in this checkout, not just "on `origin/main`," and it is also not released yet. The code comment's own 2026-09-04 measurement is a mitigating fact for right now: both hostnames currently serve the same deployment, meaning they are one deployment behind two names today, not two independently-trusted origins — the risk is real but latent until that changes | | **`rcli usage` window has no server data** | A user should be able to tell "no data was sent" from "usage is genuinely zero" | Handled correctly by design: all four columns render `-` plus one footer line explaining why — but only one line distinguishes the two cases, easy to miss in a scrolling terminal, and it doesn't exist in any released build yet | -| **Usage spend exceeds ~$2,147 on Windows** | Numbers should render correctly at any spend level | Fixed on `origin/main` (moved to `int64` counters); the bug (32-bit `long` overflow) is real on `HEAD` and any release, since `usage` doesn't exist pre-fix anyway — the risk only manifests once RCLI-8 ships | -| **Missing coding-tool binary at harness launch time** | A clear, actionable message (an install command, then "run this again") | On Windows: message printed. On **macOS/Linux — the platforms this product actually ships on**: `execvp` fails and the child calls `_exit(127)` with zero output anywhere. Fixed on `origin/main` via a `PATH`-walking preflight, unreleased | -| **Tool removed from `PATH` between the preflight check and the actual `fork`/`exec`** | Same clear message, even in the race window | Not handled — `OnPath()` (the origin/main fix) is a check-then-use pattern with an inherent, if narrow, gap | -| **Local OpenAI-compatible proxy on a shared/multi-user machine** | Prompt text should never be readable by another local user | On `HEAD`: prompts are written to a **world-readable `/tmp` path** — a real local information-disclosure risk. Fixed on `origin/main` (0600 permissions, no longer under `/tmp`), unreleased | -| **A CI test standing in for the Windows credential store** | The test should exercise the real DPAPI-encrypted read/write path | On `HEAD`'s test suite (per PR #60's fix list): a test hand-wrote a **plaintext** session file to stand in for the store, so the encrypted path itself could be broken while this test still passed. Fixed on `origin/main`, unreleased | +| **Usage spend exceeds ~$2,147 on Windows** | Numbers should render correctly at any spend level | Fixed in PR #60 (moved to `int64` counters), present in this checkout; the bug (32-bit `long` overflow) was real only through PR #51 (`d87502a`) and in `v0.5.2`, where `usage` does not exist at all | +| **Missing coding-tool binary at harness launch time** | A clear, actionable message (an install command, then "run this again") | On Windows: message printed. On **macOS/Linux — the platforms this product actually ships on**: through PR #51, `execvp` failed and the child called `_exit(127)` with zero output anywhere. Fixed in PR #60 via a `PATH`-walking preflight, present in this checkout, unreleased | +| **Tool removed from `PATH` between the preflight check and the actual `fork`/`exec`** | Same clear message, even in the race window | Not handled — `OnPath()` (the PR #60 fix) is a check-then-use pattern with an inherent, if narrow, gap | +| **Local OpenAI-compatible proxy on a shared/multi-user machine** | Prompt text should never be readable by another local user | Through PR #51: prompts were written to a **world-readable `/tmp` path** — a real local information-disclosure risk. Fixed in PR #60 (0600 permissions under the credential profile directory, no longer under `/tmp`), present in this checkout, unreleased | +| **A CI test standing in for the Windows credential store** | The test should exercise the real DPAPI-encrypted read/write path | Through PR #51's test suite: a test hand-wrote a **plaintext** session file to stand in for the store, so the encrypted path itself could be broken while this test still passed. Fixed in PR #60 (commit `0830713`), present in this checkout: the current Windows test instead asserts that a file which cannot decrypt must fail to load, unreleased | | **A future rename or new subcommand, after PR #61 merges** | Adding or renaming a subcommand should not be able to crash the binary | PR #61's own first commit introduced a hardcoded, unguarded name→help-group map that crashed the Windows binary outright (`0xC0000409`) the first time a name didn't match a real subcommand; the follow-up commit wraps it in try/catch and logs-and-skips rather than deriving the map from the actually-registered subcommands — a name that happens to collide with a different real subcommand would neither crash nor be caught by anything | | **A docs-only edit pushed straight to `main`** (no branch protection) | Should still get some CI signal before going live | `ci.yml`'s `push` trigger (not its `pull_request` trigger) has `paths-ignore: ['Formula/**', '**.md']` — a direct docs push builds and tests nothing | | **Any PR that never touches a real backend or device** | Should still prove login/harness/model round-trips work before merge | `RCLI_E2E_*` knobs are unset everywhere in public CI; that verification is manual, and `docs/VERIFICATION.md`'s own checklist template has never been filled in for any recorded release |