diff --git a/CHANGELOG.md b/CHANGELOG.md index aac9941..e6eef40 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,9 @@ # Changelog +## 0.5.0 - 2026-09-20 + +- Added label scoring through OpenRouter with `choosekit/openrouter`. + ## 0.4.2 - Made the request cancellation signal available to custom prompt formatters. diff --git a/README.md b/README.md index 19d9252..94d75fc 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # choosekit -`choosekit` scores a finite set of choices with a language model you already run and returns a typed decision with a probability distribution. +`choosekit` scores a finite set of choices with a language model and returns a typed decision with a probability distribution. It supports local llama.cpp models and an optional OpenRouter backend. ```sh npm install choosekit @@ -18,7 +18,7 @@ Agents often need to choose from known options: `choosekit` scores choices using the model's conditional log probabilities at the token branches that distinguish them. -The project was inspired by [Jev and the System One model interface](https://typesafe.ai/blog/introducing-system-one-models-and-jev): application state in, typed probabilistic decisions out. Jev is a specialized hosted model. `choosekit` explores the same useful interface with a model you control. Application state stays on infrastructure you choose, and the decision path can use a model already running inside an existing deployment. +The project was inspired by [Jev and the System One model interface](https://typesafe.ai/blog/introducing-system-one-models-and-jev): application state in, typed probabilistic decisions out. Jev is a specialized hosted model. `choosekit` explores the same useful interface with a model you control. The llama.cpp backend keeps application state on infrastructure you choose; OpenRouter is available when a hosted model is more convenient. `choosekit` is an independent project with no affiliation to TypeSafe or Jev. @@ -46,7 +46,7 @@ console.log(decision.choice); // "no" console.log(decision.distribution); // { yes: ..., no: ... } ``` -Compatibility requires the native llama.cpp `/tokenize` and `/completion` endpoints with raw pre-sampling log probabilities and returned token IDs. An OpenAI-compatible `/v1` endpoint alone lacks these capabilities. +The llama.cpp backend requires its native `/tokenize` and `/completion` endpoints. `minimal-prefix` is available only with this backend. The library has no telemetry. @@ -54,12 +54,35 @@ The library has no telemetry. [`choosekit-mcp`](packages/choosekit-mcp/README.md) exposes the same local llama.cpp decision interface as a read-only stdio tool for Claude Code, Codex, and OpenCode. Configure the llama.cpp endpoint, model, and scoring mode with environment variables when starting the MCP server. Every `choose` call uses this configuration. +## OpenRouter + +```ts +import { fromOpenRouter } from "choosekit/openrouter"; + +const choose = fromOpenRouter({ + apiKey: process.env.OPENROUTER_API_KEY!, + model: "qwen/qwen3.8-27b", +}); +``` + +The OpenRouter backend supports models and providers that return first-token `top_logprobs`, with up to 20 choices. Unlike llama.cpp, this backend sends the prompt to OpenRouter. It requests reasoning to be disabled. Choices omitted from `top_logprobs` receive zero probability. Returned probabilities are normalized across the supplied choices and are not calibrated correctness estimates. + +OpenRouter may route the same model through different providers. Set `provider` to an OpenRouter provider slug to use only that provider and disable fallback: + +```ts +const choose = fromOpenRouter({ + apiKey: process.env.OPENROUTER_API_KEY!, + model: "qwen/qwen3.8-27b", + provider: process.env.OPENROUTER_PROVIDER!, +}); +``` + ## Scoring modes | Mode | Candidate representation | Use when | |---|---|---| -| `labels` | `A`, `B`, `C`, ... | Default. The choice set has at most 26 entries. | -| `minimal-prefix` | Original JSON-quoted keys | The key names should influence the decision, or the set has more than 26 entries. | +| `labels` | `A`, `B`, `C`, ... | Default. Up to 26 choices with llama.cpp or 20 with OpenRouter. | +| `minimal-prefix` | Original JSON-quoted keys | llama.cpp only. Use when key names should influence the decision. | In `labels` mode, choices are shown to the model as `A`, `B`, `C` instead of their original keys. For example, `refund: "Issue the refund"` is shown as `"A": "Issue the refund"`. Each description must therefore make the option clear. `choosekit` maps the selected label back to the original key. @@ -125,18 +148,7 @@ Most distributions are similar. Some differ substantially: the systems select di `context` is copied unchanged to the start of the scoring prompt. The default formatter then appends the question, choice descriptions, and an answer marker. -For chat models, `context` should be the model's normal serialized chat prefix. Use `formatPrompt` when the decision turn needs a particular template. This example uses Qwen's chat markers: - -```ts -const choose = fromLlamaCpp({ - baseURL: "http://127.0.0.1:8080/", - mode: "minimal-prefix", - formatPrompt: ({ context, instruction }) => - `${context}<|im_start|>user\n${instruction}<|im_end|>\n<|im_start|>assistant\n`, -}); -``` - -The formatted prompt must start with `context` unchanged so an existing server-side prefix cache can still be reused. +Use `formatPrompt` only when you need custom prompt formatting. The result must preserve `context` as an unchanged prefix so an existing server-side prefix cache can still be reused. ## Custom scorer diff --git a/TODO.md b/TODO.md index f1ac46d..fecc4a3 100644 --- a/TODO.md +++ b/TODO.md @@ -4,4 +4,3 @@ - Tokenize the prompt and prompt-plus-candidate inputs with bounded parallelism in the `llama.cpp` adapter instead of awaiting every `/tokenize` request sequentially. Preserve input order and exact boundary-aware tokenization; make concurrency configurable and measure transport overhead separately from server processing. Consider a future native batch-tokenization endpoint returning one token array per input, because stock `/tokenize` does not provide that response shape. - Add native `llama.cpp` support for requesting raw log probabilities for multiple arbitrary token IDs in one request (`logprob_token_ids`). Prefer zero-token generation (`n_predict: 0`) and calculate every requested token's log probability from the full-vocabulary softmax. Keep the current top-N plus exact forced-token fallback path for servers without the extension; an explicitly selected bulk mode must fail clearly when unsupported instead of silently changing protocols. -- Add an OpenRouter scorer behind the existing `Scorer` and `Chooser` contracts. Support label scoring only: request `logprobs` and up to 20 `top_logprobs` from a compatible provider, require every choice label to be present, and fail clearly instead of treating a missing label as zero probability. Do not require a local tokenizer or expose `minimal-prefix` for this backend. diff --git a/benchmarks/run-semif.mjs b/benchmarks/run-semif.mjs index 6d44f52..67fa26a 100644 --- a/benchmarks/run-semif.mjs +++ b/benchmarks/run-semif.mjs @@ -172,7 +172,7 @@ for (let index = 0; index < rows.length; index++) { totalRows: allRows.length, selectedRows: rows.length, }, - runtime: { baseURL, model, mode, adapter: "choosekit/llama-cpp", packageVersion: "0.4.2" }, + runtime: { baseURL, model, mode, adapter: "choosekit/llama-cpp", packageVersion: "0.5.0" }, interpretation: mode === "labels" ? "Package A/B/C label prompt and distinguishing-token likelihoods." : "Package original-key prompt and minimal distinguishing-prefix likelihoods.", diff --git a/package-lock.json b/package-lock.json index 0524abf..8fcdebf 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "choosekit", - "version": "0.4.2", + "version": "0.5.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "choosekit", - "version": "0.4.2", + "version": "0.5.0", "license": "Apache-2.0", "devDependencies": { "typescript": "5.8.3" diff --git a/package.json b/package.json index 0d96e4b..f6a302f 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "choosekit", - "version": "0.4.2", + "version": "0.5.0", "description": "Typed choices from your existing language model scorer.", "license": "Apache-2.0", "author": { @@ -40,6 +40,16 @@ "default": "./dist/cjs/llama-cpp.js" } }, + "./openrouter": { + "import": { + "types": "./dist/esm/openrouter.d.ts", + "default": "./dist/esm/openrouter.js" + }, + "require": { + "types": "./dist/cjs/openrouter.d.ts", + "default": "./dist/cjs/openrouter.js" + } + }, "./package.json": "./package.json" }, "files": [ @@ -69,6 +79,7 @@ "choices", "logprobs", "llama.cpp", + "openrouter", "typescript" ] } diff --git a/src/openrouter.ts b/src/openrouter.ts new file mode 100644 index 0000000..ce517b8 --- /dev/null +++ b/src/openrouter.ts @@ -0,0 +1,178 @@ +import { createFormattedChooser } from "./internal-chooser.js"; +import type { Chooser, ChooserOptions, Scorer, Usage } from "./types.js"; +import { isCount, isRecord, requireText, ScoringError } from "./validation.js"; + +const ENDPOINT = "https://openrouter.ai/api/v1/chat/completions"; +const MAX_CANDIDATES = 20; +const CLAMPED_LOGPROB = -9999; + +export interface OpenRouterOptions extends ChooserOptions { + readonly apiKey: string; + readonly model: string; + readonly fetch?: typeof globalThis.fetch; + /** Pin one OpenRouter provider and disable provider fallback. */ + readonly provider?: string; +} + +async function post(fetchImpl: typeof globalThis.fetch, apiKey: string, body: unknown, + signal?: AbortSignal): Promise { + signal?.throwIfAborted(); + let response: Response; + try { + response = await fetchImpl(ENDPOINT, { + method: "POST", + headers: { + authorization: `Bearer ${apiKey}`, + "content-type": "application/json", + }, + body: JSON.stringify(body), + ...(signal ? { signal } : {}), + }); + } catch { + signal?.throwIfAborted(); + throw new ScoringError("OpenRouter request failed."); + } + signal?.throwIfAborted(); + if (!response.ok) { + throw new ScoringError(`OpenRouter returned HTTP ${response.status} for /chat/completions.`); + } + try { + const value: unknown = await response.json(); + signal?.throwIfAborted(); + return value; + } catch { + signal?.throwIfAborted(); + throw new ScoringError("OpenRouter returned invalid JSON for /chat/completions."); + } +} + +function parseUsage(value: unknown): Usage | undefined { + if (value === undefined || value === null) return undefined; + if (!isRecord(value) + || !isCount(value.prompt_tokens) + || !isCount(value.completion_tokens)) { + throw new ScoringError("OpenRouter returned invalid token usage."); + } + let cachedTokens: number | null = null; + if (value.prompt_tokens_details !== undefined && value.prompt_tokens_details !== null) { + if (!isRecord(value.prompt_tokens_details)) { + throw new ScoringError("OpenRouter returned invalid cached-token usage."); + } + const cached = value.prompt_tokens_details.cached_tokens; + if (cached !== undefined && cached !== null) { + if (!isCount(cached) || cached > value.prompt_tokens) { + throw new ScoringError("OpenRouter returned invalid cached-token usage."); + } + cachedTokens = cached; + } + } + return Object.freeze({ + promptTokens: value.prompt_tokens, + cachedTokens, + completionTokens: value.completion_tokens, + requests: 1, + }); +} + +function hasRefusal(choice: Record): boolean { + if (choice.finish_reason === "content_filter") return true; + const message = choice.message; + return isRecord(message) && message.refusal !== undefined && message.refusal !== null; +} + +function parseScores(value: unknown, candidates: readonly string[]): { + readonly logprobs: readonly number[]; + readonly usage?: Usage; +} { + if (!isRecord(value) || !Array.isArray(value.choices) || value.choices.length === 0) { + throw new ScoringError("OpenRouter returned an invalid completion."); + } + const choice: unknown = value.choices[0]; + if (!isRecord(choice)) { + throw new ScoringError("OpenRouter returned an invalid completion."); + } + if (hasRefusal(choice)) { + throw new ScoringError("OpenRouter refused or filtered the scoring request."); + } + if (!isRecord(choice.logprobs)) { + throw new ScoringError("OpenRouter did not return logprobs."); + } + if (Array.isArray(choice.logprobs.refusal) && choice.logprobs.refusal.length > 0) { + throw new ScoringError("OpenRouter refused or filtered the scoring request."); + } + const content = choice.logprobs.content; + if (!Array.isArray(content) || content.length !== 1 || !isRecord(content[0])) { + throw new ScoringError("OpenRouter did not return exactly one scored token position."); + } + const top = content[0].top_logprobs; + if (!Array.isArray(top) || top.length === 0) { + throw new ScoringError("OpenRouter did not return top logprobs."); + } + + const expected = new Set(candidates); + const found = new Map(); + for (const entry of top) { + if (!isRecord(entry) || typeof entry.token !== "string") { + throw new ScoringError("OpenRouter returned an invalid top-logprob entry."); + } + if (!expected.has(entry.token)) continue; + if (found.has(entry.token)) { + throw new ScoringError(`OpenRouter returned duplicate logprobs for label ${entry.token}.`); + } + if (typeof entry.logprob !== "number" || !Number.isFinite(entry.logprob) + || entry.logprob > 0 || entry.logprob <= CLAMPED_LOGPROB) { + throw new ScoringError(`OpenRouter returned an invalid or clamped logprob for label ${entry.token}.`); + } + if (entry.bytes !== undefined && entry.bytes !== null) { + if (!Array.isArray(entry.bytes) || entry.bytes.length !== 1 + || entry.bytes[0] !== entry.token.charCodeAt(0)) { + throw new ScoringError(`OpenRouter returned invalid bytes for label ${entry.token}.`); + } + } + found.set(entry.token, entry.logprob); + } + + if (found.size === 0) { + throw new ScoringError("OpenRouter did not return logprobs for any choice label."); + } + const usage = parseUsage(value.usage); + return { + logprobs: Object.freeze(candidates.map((candidate) => found.get(candidate) ?? -Infinity)), + ...(usage === undefined ? {} : { usage }), + }; +} + +export function fromOpenRouter(options: OpenRouterOptions): Chooser { + if (!isRecord(options)) throw new TypeError("options must be an object."); + const { apiKey, model, provider, formatPrompt } = options; + requireText(apiKey, "apiKey"); + requireText(model, "model"); + if (provider !== undefined) requireText(provider, "provider"); + if (options.fetch !== undefined && typeof options.fetch !== "function") { + throw new TypeError("fetch must be a function."); + } + const fetchImpl = options.fetch ?? globalThis.fetch; + if (typeof fetchImpl !== "function") throw new TypeError("A fetch implementation is required."); + + const score: Scorer = async ({ prompt, candidates, signal }) => { + if (candidates.length > MAX_CANDIDATES) { + throw new TypeError(`OpenRouter supports at most ${MAX_CANDIDATES} choices.`); + } + const response = await post(fetchImpl, apiKey, { + model, + messages: [{ role: "user", content: prompt }], + max_tokens: 1, + stream: false, + temperature: 1, + top_p: 1, + logprobs: true, + top_logprobs: MAX_CANDIDATES, + reasoning_effort: "none", + ...(provider === undefined ? {} : { + provider: { only: [provider], allow_fallbacks: false }, + }), + }, signal); + return parseScores(response, candidates); + }; + return createFormattedChooser(score, formatPrompt === undefined ? {} : { formatPrompt }, "labels"); +} diff --git a/tests/openrouter.test.mjs b/tests/openrouter.test.mjs new file mode 100644 index 0000000..cdd95c0 --- /dev/null +++ b/tests/openrouter.test.mjs @@ -0,0 +1,233 @@ +import assert from "node:assert/strict"; +import { test } from "node:test"; +import { fromOpenRouter } from "../dist/esm/openrouter.js"; +import { ScoringError } from "../dist/esm/index.js"; + +const request = Object.freeze({ + context: "A customer reports that the same invoice was charged twice.", + question: "Which team should handle this message?", + choices: Object.freeze({ + sales: "Pricing, upgrades, and new accounts.", + technical: "Bugs, outages, and integrations.", + billing: "Payments, invoices, and refunds.", + security: "Account compromise and suspicious activity.", + support: "General product questions.", + }), +}); + +const unorderedTopLogprobs = Object.freeze([ + { token: "A", bytes: [65], logprob: -1.2338635921 }, + { token: "C", bytes: [67], logprob: -1.1088635921 }, + { token: "E", bytes: [69], logprob: -1.7338635921 }, + { token: "B", bytes: [66], logprob: -2.2338635921 }, + { token: "D", bytes: [68], logprob: -2.3588635921 }, + { token: "c", bytes: [99], logprob: -8 }, +]); + +function scoredPosition(topLogprobs = unorderedTopLogprobs) { + return { + token: "A", bytes: [65], logprob: -1.2338635921, top_logprobs: topLogprobs, + }; +} + +function response(topLogprobs = unorderedTopLogprobs, overrides = {}) { + return { + choices: [{ + finish_reason: "length", + message: { role: "assistant", content: "A" }, + logprobs: { content: [scoredPosition(topLogprobs)] }, + ...overrides, + }], + }; +} + +function json(value, status = 200) { + return new Response(JSON.stringify(value), { + status, headers: { "content-type": "application/json" }, + }); +} + +function fixture(value = response(), status = 200) { + const calls = []; + return { + calls, + fetch: async (url, init) => { + calls.push({ url, init, body: JSON.parse(init.body) }); + return json(value, status); + }, + }; +} + +function chooser(f, extra = {}) { + return fromOpenRouter({ + apiKey: "test-secret", model: "test/model", fetch: f.fetch, ...extra, + }); +} + +test("uses first-position label logprobs instead of the sampled token", async () => { + const value = response(); + value.usage = { + prompt_tokens: 180, + completion_tokens: 1, + prompt_tokens_details: { cached_tokens: 128 }, + }; + const f = fixture(value); + const decision = await chooser(f)(request); + + assert.equal(decision.choice, "billing"); + assert.deepEqual(decision.scores, { + sales: -1.2338635921, + technical: -2.2338635921, + billing: -1.1088635921, + security: -2.3588635921, + support: -1.7338635921, + }); + for (const [key, percent] of Object.entries({ + sales: 29.14, technical: 10.72, billing: 33.02, security: 9.46, support: 17.67, + })) { + assert.ok(Math.abs(decision.distribution[key] * 100 - percent) < 0.01, key); + } + assert.deepEqual(decision.usage, { + promptTokens: 180, cachedTokens: 128, completionTokens: 1, requests: 1, + }); + + assert.equal(f.calls.length, 1); + const call = f.calls[0]; + assert.equal(call.url, "https://openrouter.ai/api/v1/chat/completions"); + assert.equal(call.init.method, "POST"); + const headers = new Headers(call.init.headers); + assert.equal(headers.get("authorization"), "Bearer test-secret"); + assert.equal(headers.get("content-type"), "application/json"); + const { messages, ...body } = call.body; + assert.deepEqual(body, { + model: "test/model", + max_tokens: 1, + stream: false, + temperature: 1, + top_p: 1, + logprobs: true, + top_logprobs: 20, + reasoning_effort: "none", + }); + assert.equal(messages.length, 1); + assert.equal(messages[0].role, "user"); + assert.ok(messages[0].content.startsWith(request.context)); + assert.ok(messages[0].content.includes(JSON.stringify(request.question))); + for (const [index, description] of Object.values(request.choices).entries()) { + const label = String.fromCharCode(65 + index); + assert.ok(messages[0].content.includes(`"${label}": "${description}"`)); + } + assert.ok(messages[0].content.endsWith("Answer: ")); +}); + +test("pins an optional provider without fallback", async () => { + const f = fixture(); + await chooser(f, { provider: "reka" })(request); + assert.deepEqual(f.calls[0].body.provider, { + only: ["reka"], allow_fallbacks: false, + }); +}); + +test("assigns zero probability to labels omitted from top logprobs", async () => { + const f = fixture(response(unorderedTopLogprobs.filter(({ token }) => token !== "C"))); + const decision = await chooser(f)(request); + + assert.equal(decision.choice, "sales"); + assert.equal(decision.scores.billing, -Infinity); + assert.equal(decision.distribution.billing, 0); + assert.ok(Math.abs(Object.values(decision.distribution) + .reduce((sum, probability) => sum + probability, 0) - 1) < 1e-12); +}); + +const malformed = [ + ["null logprobs", response(undefined, { logprobs: null }), /logprobs/i], + ["no scored position", response(undefined, { logprobs: { content: [] } }), /exactly one/i], + ["multiple scored positions", response(undefined, { + logprobs: { content: [scoredPosition(), scoredPosition()] }, + }), /exactly one/i], + ["no choice labels", response([{ token: "x", bytes: [120], logprob: -0.1 }]), + /any choice label/i], + ["duplicate required label", response([...unorderedTopLogprobs, unorderedTopLogprobs[0]]), /duplicate.*A|A.*duplicate/i], + ["invalid logprob", response(unorderedTopLogprobs.map((entry) => + entry.token === "C" ? { ...entry, logprob: 0.1 } : entry)), /logprob/i], + ["clamped logprob", response(unorderedTopLogprobs.map((entry) => + entry.token === "C" ? { ...entry, logprob: -9999 } : entry)), /clamped/i], + ["mismatched bytes", response(unorderedTopLogprobs.map((entry) => + entry.token === "C" ? { ...entry, bytes: [99] } : entry)), /bytes/i], + ["content filtering", response(unorderedTopLogprobs, { finish_reason: "content_filter" }), + /refused|filtered/i], + ["message refusal", response(undefined, { + message: { role: "assistant", content: null, refusal: "Cannot answer." }, + }), /refused|filtered/i], + ["logprob refusal", response(undefined, { + logprobs: { content: [scoredPosition()], refusal: [{ token: "refusal" }] }, + }), /refused|filtered/i], +]; + +for (const [name, value, pattern] of malformed) { + test(`rejects ${name}`, async () => { + const f = fixture(value); + await assert.rejects(chooser(f)(request), + (error) => error instanceof ScoringError && pattern.test(error.message)); + }); +} + +test("rejects more than 20 choices before sending a request", async () => { + const f = fixture(); + const choices = Object.fromEntries(Array.from({ length: 21 }, (_, index) => + [`choice_${index}`, `Choice ${index}`])); + await assert.rejects(chooser(f)({ ...request, choices }), /at most 20/i); + assert.equal(f.calls.length, 0); +}); + +test("accepts exactly 20 choices", async () => { + const labels = [..."ABCDEFGHIJKLMNOPQRST"]; + const choices = Object.fromEntries(labels.map((label) => [label, `Choice ${label}`])); + const top = labels.map((token, index) => ({ + token, bytes: [token.charCodeAt(0)], logprob: -(index + 1), + })); + const f = fixture(response(top)); + + const decision = await chooser(f)({ ...request, choices }); + + assert.equal(decision.choice, "A"); + assert.equal(Object.keys(decision.distribution).length, 20); + assert.equal(f.calls.length, 1); +}); + +test("passes AbortSignal to fetch", async () => { + const controller = new AbortController(); + const f = fixture(); + f.fetch = async (_url, init) => { + assert.equal(init.signal, controller.signal); + controller.abort(); + throw controller.signal.reason; + }; + await assert.rejects(chooser(f)({ ...request, signal: controller.signal }), + (error) => error === controller.signal.reason); +}); + +test("does not include the API key or prompt in HTTP errors", async () => { + const f = fixture({ error: { message: `bad request: test-secret ${request.context}` } }, 400); + await assert.rejects(chooser(f)(request), (error) => { + assert.match(error.message, /400/); + assert.doesNotMatch(error.message, /test-secret|charged twice/); + return true; + }); + assert.equal(f.calls.length, 1); +}); + +test("does not retry invalid JSON responses", async () => { + let calls = 0; + const choose = fromOpenRouter({ + apiKey: "test-secret", + model: "test/model", + fetch: async () => { + calls++; + return new Response("{", { status: 200 }); + }, + }); + + await assert.rejects(choose(request), /invalid JSON/i); + assert.equal(calls, 1); +}); diff --git a/tests/package.test.mjs b/tests/package.test.mjs index 28a3152..cbccead 100644 --- a/tests/package.test.mjs +++ b/tests/package.test.mjs @@ -14,6 +14,8 @@ test("ESM and CommonJS public exports implement the same API", async () => { assert.deepEqual(a, b); assert.equal(typeof (await import("choosekit/llama-cpp")).fromLlamaCpp, "function"); assert.equal(typeof createRequire(import.meta.url)("choosekit/llama-cpp").fromLlamaCpp, "function"); + assert.equal(typeof (await import("choosekit/openrouter")).fromOpenRouter, "function"); + assert.equal(typeof createRequire(import.meta.url)("choosekit/openrouter").fromOpenRouter, "function"); }); test("has no runtime dependencies, install hooks or executable", () => { @@ -32,6 +34,7 @@ test("importing public entrypoints does not call fetch or log anything", () => { globalThis.fetch = () => { throw new Error("Unexpected network call"); }; await import("choosekit"); await import("choosekit/llama-cpp"); + await import("choosekit/openrouter"); `; const result = spawnSync(process.execPath, ["--input-type=module", "-e", code], { cwd: new URL("../", import.meta.url), encoding: "utf8" }); diff --git a/tests/types/api.cts b/tests/types/api.cts index 66cd24f..18c78ea 100644 --- a/tests/types/api.cts +++ b/tests/types/api.cts @@ -1,6 +1,8 @@ import { createChooser, type Scorer } from "choosekit"; import { fromLlamaCpp } from "choosekit/llama-cpp"; +import { fromOpenRouter } from "choosekit/openrouter"; const score: Scorer = ({ candidates }) => ({ logprobs: candidates.map(() => -1) }); const choose = createChooser(score); void choose({ context: "", question: "Next?", choices: { test: "Test", done: "Done" } }); void fromLlamaCpp({ baseURL: "http://127.0.0.1:8080" }); +void fromOpenRouter({ apiKey: "test-key", model: "test/model" }); diff --git a/tests/types/api.mts b/tests/types/api.mts index d7f6a9f..290b0fe 100644 --- a/tests/types/api.mts +++ b/tests/types/api.mts @@ -1,5 +1,6 @@ import { createChooser, type Scorer, type Decision } from "choosekit"; import { fromLlamaCpp } from "choosekit/llama-cpp"; +import { fromOpenRouter } from "choosekit/openrouter"; const scorer: Scorer = async ({ candidates, signal }) => { signal?.throwIfAborted(); @@ -30,6 +31,8 @@ void numericKey; const local = fromLlamaCpp({ baseURL: "http://127.0.0.1:8080", mode: "labels" }); void local({ context: "", question: "?", choices: { yes: "Yes", no: "No" } }); +const remote = fromOpenRouter({ apiKey: "test-key", model: "test/model" }); +void remote({ context: "", question: "?", choices: { yes: "Yes", no: "No" } }); // @ts-expect-error The old implicit agent-state input is not part of this API. choose({ state: "Changed", question: "Next?", choices: { yes: "Yes", no: "No" } });