Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ On a release, the maintainer moves the accumulated `[Unreleased]` entries into a

## [Unreleased]

- **P15-completion-docs** — Reconcile the canonical task, provider-roadmap, and architecture docs after P15-T13: Phase 15 is complete, the hermetic golden eval is shipped, and the documented I7 boundary now matches the implemented native drop-on-decode plus client-side untrusted-data fence. _Owns:_ `docs/{TASKS,ROADMAP-PROVIDERS,ARCHITECTURE}.md`, `CHANGELOG.md`. _(Phase 15 documentation)_

- **P15-T13** — Add a hermetic, golden-transcript provider compatibility evaluation covering custom OpenAI-compatible endpoints, reasoning token caps, strict JSON Schema output, OpenRouter routing/extras/attribution, native search rendering and drop-on-decode containment, plus injection-flagged and untrusted-data-fenced client search fallback. _Owns:_ `eval/provider-compat/`, `CHANGELOG.md`. _(Phase 15)_

- **frontdoor-parity** — Centralize the native runtime capabilities shared by run/watch/resume/swarm, chat/TUI, and serve (advisor budget, repo orientation, proactive context window, live code intelligence, memory/lessons, and trusted steering) behind one configurator, with a constructor-level parity test across all three real builders so a one-door omission cannot ship inert again. _Owns:_ `cmd/nilcore/{native_runtime.go,native_runtime_test.go,chat.go,main.go}`, `CHANGELOG.md`. _(architecture hardening)_
Expand Down
2 changes: 1 addition & 1 deletion docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ The native loop, Codex, and Claude Code are interchangeable behind this. Adding

**Agentic browser + computer use (Phase 14 / Phase CU, additive, opt-in, contract-untouched).** Two GUI-agent loops drive a *persistent* session over many turns — observe → plan → act → verify — atop the same native loop, egress proxy, sandbox, capguard Rule-of-Two gate, and verifier. **Browser** (`nilcore browse`, `docs/ROADMAP-BROWSER-USE.md`): an accessibility **Set-of-Marks** snapshot (`internal/cdp` stamps `data-nilref`) the model acts on by ref; the browser runs in ONE long-lived in-sandbox `nilcore-browser --serve` Exec (I4) reached over a file-queue on `/work`; `internal/browsersession` version-stamps refs (stale-ref guard) and does host-side `{{secret}}` substitution so a credential never enters the model context (I3); `record_finding` writes typed `artifact.Claim`s re-verified in-box (I2). **Desktop** (`nilcore desktop`, `docs/ROADMAP-COMPUTER-USE.md`): a contained virtual X11 desktop driven through a **3-rung Set-of-Marks ladder** (AT-SPI refs → SoM-marked screenshot → raw coordinate), the fat driver shelling to image-baked tools (`internal/{desktopwire,desktopsession,desktopagent,som,desktop}`). Two paths: generic **Path B** (default, provider-agnostic) and native Anthropic **Path A** (opt-in, via the lone `model.Tool.Builtin` contract addition — byte-identical for every normal tool). A **native-macOS** tier (`docs/ROADMAP-COMPUTER-USE-DARWIN.md`) adds a pure-Go darwin driver (`screencapture`+`cliclick`, Rungs 2/3) and a `--mac-host` host-control mode — the one path where I4's sandbox boundary is *deliberately* relaxed, behind a separate opt-in + forced gate + kill-switch/allowlist (see CLAUDE.md §2 I4). Both default to byte-identical when their env gate is unset; **no module added** (I6).

**Provider upgrade — OpenAI / OpenRouter / openai-compatible (Phase 15, additive, contract-untouched).** The `model.Provider` seam now backs a configurable-BaseURL OpenAI adapter + a generic **`openai-compatible`** vendor (vLLM/Ollama/Groq/Fireworks/Azure/…) with an auth descriptor (Bearer/Azure/None), a single-key `max_tokens`/`max_completion_tokens` marshal (gpt-5.x/o-series), SOTA request fields (reasoning effort, structured outputs, tool_choice, service tier), OpenRouter typed routing/`models[]`/plugins + attribution headers, a typed `model.APIError` (terminal-vs-retryable + `Retry-After`) that `resilience.go` consumes, and prompt-cache/reasoning-token + authoritative-cost accounting in the meter. Anti-exfiltration (I3): a compat BaseURL **rejects** a canonical key-env with a key-free error. Every new request field is `omitempty` set once in `newRequest`, so a zero-valued config is **byte-identical** (`docs/ROADMAP-PROVIDERS.md`). `backend.CodingBackend` untouched (I1); stdlib-only (I6). Web search (native `BuiltinTool` + the client-side fenced fallback) is the remaining Phase-15 wave.
**Provider upgrade — OpenAI / OpenRouter / openai-compatible (Phase 15, complete, contract-untouched).** The `model.Provider` seam now backs a configurable-BaseURL OpenAI adapter + a generic **`openai-compatible`** vendor (vLLM/Ollama/Groq/Fireworks/Azure/…) with an auth descriptor (Bearer/Azure/None), a single-key `max_tokens`/`max_completion_tokens` marshal (gpt-5.x/o-series), SOTA request fields (reasoning effort, structured outputs, tool_choice, service tier), OpenRouter typed routing/`models[]`/plugins + attribution headers, a typed `model.APIError` (terminal-vs-retryable + `Retry-After`) that `resilience.go` consumes, and prompt-cache/reasoning-token + authoritative-cost accounting in the meter. Anti-exfiltration (I3): a compat BaseURL **rejects** a canonical key-env with a key-free error. Every new request field is `omitempty` set once in `newRequest`, so a zero-valued config is **byte-identical**. Native web search rides `BuiltinTool`; Anthropic raw server-tool blocks are dropped at decode while client fallback output stays `guard.Wrap`-fenced. `eval/provider-compat/` pins these request and safety contracts with hermetic golden transcripts (`docs/ROADMAP-PROVIDERS.md`). `backend.CodingBackend` untouched (I1); stdlib-only (I6).

**Earned backend selection — `-backend auto` + preferred-backend (additive, Phase 13).** The default backend is no longer hard-wired to native. `-backend auto` (and config `backend: auto`, mapped by `applyConfigDefaults`) resolves to a single concrete backend at run time via `resolveAutoBackend`: `availableBackends(cfg, cred)` reports which of {native, codex, claude-code} actually have their CLI + key present on the host, and the survivors are ordered PREFERENCE-FIRST — the one-run `-prefer-backend` flag, else durable config `preferred_backend` (a CONCRETE backend, validated never to be "auto") — then re-ordered by the verifier-judged Trust Ledger (`trust.Replay` → `Order`), so a cold install honors the stated preference and accumulated evidence overtakes it. `-backends auto` expands the same available set into the multi-backend race (the `Selector` seam above). The selector and the preference only ORDER which backend RUNS; the verifier still judges every result (I2), and per-backend creds resolve through the SecretStore seam (I3).

Expand Down
15 changes: 8 additions & 7 deletions docs/ROADMAP-PROVIDERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ without adding a Go module, and keeping the default binary byte-identical until
Read order: `CLAUDE.md` §2 (invariants) → `docs/ARCHITECTURE.md` (providers) → this file → `docs/TASKS.md`
(the Phase-15 queue rows + specs).

> **Status: SHIPPED.** The provider upgrade's non-web waves (T01–T06, T10–T12) merged in PR #61 (this Phase-15 DAG runs T01–T14 — there is no T15/T16).
> **Status: COMPLETE.** The provider upgrade's non-web waves (T01–T06, T10–T12) merged in PR #61 (this Phase-15 DAG runs T01–T14 — there is no T15/T16).
> **Web search (T07 · T08 · T09) is now shipped too** — the `model.BuiltinTool` foundation it
> depended on landed via the computer-use work (#60). Native render: Anthropic tools-entry
> (`web_search_20250305`), OpenAI top-level `web_search_options`, OpenRouter `web` plugin —
Expand All @@ -18,8 +18,9 @@ Read order: `CLAUDE.md` §2 (invariants) → `docs/ARCHITECTURE.md` (providers)
> both. The **I7 fence** holds by construction — provider web results are the model's own
> synthesized text (OpenAI) or distinct non-text blocks dropped from re-injection (Anthropic),
> and the client path stays `guard.Wrap`'d; raw provider result blocks never re-enter as trusted
> instructions. Remaining: the optional `eval/provider-compat` coverage harness (T13) and a live
> end-to-end check against real provider keys (hermetic shape/byte-identity tests gate it today).
> instructions. The hermetic `eval/provider-compat` golden-transcript suite (T13, PR #105) now gates generic
> endpoints, reasoning, structured output, OpenRouter extras, and both web-search safety paths without network
> access. A live check against real provider keys remains an operator-only validation, never a CI requirement.

---

Expand Down Expand Up @@ -192,8 +193,8 @@ independently. Sequence the web-search waves after the `BuiltinTool` foundation
| **T10** | Onboarding config + wizard for compat vendor | `internal/onboard/*` (+tests) | T02 | M |
| **T11** | Metering/pricing for new ids + authoritative `usage.cost` | `meter/pricer.go` (+test) | T03 | M |
| **T12** | Egress allowlist extensibility (sandbox only) | `policy/egress.go` (+test) | — | S |
| **T13** | Eval coverage (compat, reasoning, structured, native search) — **NOT BUILT** (`eval/provider-compat/` does not exist; hermetic shape/byte-identity tests gate the feature today) | `eval/provider-compat/` | T06,T08,T09 | M |
| **T14** | 📄 Docs: PREREQUISITES + ARCHITECTURE + TASKS *(contract, serialised)* | `docs/{PREREQUISITES,ARCHITECTURE,TASKS}.md` | T02,T03,T06,T08,T09,T10,T11,T12 | M |
| **T13** | Eval coverage (compat, reasoning, structured, native search) — **SHIPPED in #105** (hermetic per-provider servers + golden transcripts) | `eval/provider-compat/` | T06,T08,T09 | M |
| **T14** | 📄 Docs: PREREQUISITES + ARCHITECTURE + TASKS *(contract, serialised)* | `docs/{PREREQUISITES,ARCHITECTURE,TASKS}.md` | T02,T03,T06,T08,T09,T10,T11,T12,T13 | M |

## 7. Parallel execution waves

Expand All @@ -205,9 +206,9 @@ Each wave's `Owns` sets are pairwise-disjoint; every dependency resolves to a st
| 2 | T02 · T04 | `provider.go` ‖ new `openai_maxtokens.go` |
| 3 | T05 *(alone)* | sole owner of `openai.go` |
| 4 | T06 · T07 · T10 · T11 | `openrouter_extras.go` ‖ `builtin.go`+`openai_websearch.go` ‖ `onboard/*` ‖ `meter/*` |
| 5 | T08 *(alone)* | co-edits `openai.go`+`native.go`+`model.go` |
| 5 | T08 *(alone)* | `provider/anthropic.go` (+tests), as shipped |
| 6 | T09 *(alone)* | new `cmd/nilcore/webcap.go` |
| 7 | T13 *(alone — NOT BUILT)* | new `eval/provider-compat/` |
| 7 | T13 *(alone — shipped #105)* | `eval/provider-compat/` |
| 8 | T14 *(alone)* | serialised contract docs |

**Critical path:** T01 → T05 → T07 → T08 → T09 → T13 → T14. Peak fan-out: Wave 1 (3) and Wave 4 (4).
Expand Down
Loading
Loading