Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Guidance for agents working in this repository.
## What this is

An OpenCode V2 plugin (`context-limit.ts`) that sets a per-model working context
budget by lowering the model's `limit.context` through a catalog transform. No
budget by lowering the model's `limit.context` through a model transform. No
build step, no dependencies, MIT.

## Local development
Expand All @@ -24,9 +24,9 @@ grep context-limit ~/.local/share/opencode/log/opencode.log | tail

## Spike result (T0)

A catalog transform can lower a model's context window at runtime. A probe set
A model transform can lower a model's context window at runtime. A probe set
`deepseek/deepseek-flash` from 1,000,000 to 123,456 through
`ctx.catalog.transform`, and a re-read showed 123,456. Compaction's default
`ctx.model.transform`, and a re-read showed 123,456. Compaction's default
threshold follows the model's usable input budget, so this is the mechanism. No
config edit is needed.

Expand All @@ -40,19 +40,19 @@ config edit is needed.

## API notes

- `ctx.catalog.transform((catalog) => catalog.model.update(providerID, modelID, (model) => { model.limit = { ...model.limit, context: n } }))`
lowers a window. Call `ctx.catalog.reload()` after changing the rules.
- Model entries from `ctx.catalog.model.list()` carry `providerID`, `id`, and
- `ctx.model.transform((editor) => editor.update(providerID, modelID, (model) => { model.limit = { ...model.limit, context: n } }))`
lowers a window. Call `ctx.model.reload()` after changing the rules.
- Model entries from `ctx.model.list()` carry `providerID`, `id`, and
`limit.context`.
- Rules live in `ctx.storage` under `context-limit`.

## Layout

- `parseBudget` - parses tokens, `128K`, `1M`, and `50%`, with clamping.
- `matchPattern`, `longestMatch`, `resolveBudget` - rule matching.
- `applyBudget` - the catalog transform body, exported for tests.
- `applyBudget` - the model transform body, exported for tests.
- `setup` - registers the command and the transform.
- `context-limit.test.ts` - tests with a fake catalog and ctx.
- `context-limit.test.ts` - tests with a fake model editor and ctx.

## Releasing

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,9 @@ matches everything. The longest matching pattern wins.

## How it works

The plugin registers a catalog transform that lowers the matched models'
The plugin registers a model transform that lowers the matched models'
`limit.context`. Compaction's default threshold follows the model's usable input
budget, so compaction fires earlier. A change calls `ctx.catalog.reload()` and
budget, so compaction fires earlier. A change calls `ctx.model.reload()` and
applies at once. Nothing is written to `opencode.json`.

## Tests
Expand Down
62 changes: 24 additions & 38 deletions context-limit.test.ts
Original file line number Diff line number Diff line change
@@ -1,28 +1,21 @@
import { describe, expect, test } from "bun:test"
import plugin, { applyBudget, longestMatch, matchPattern, parseBudget, resolveBudget, VERSION } from "./context-limit.ts"

function makeCatalog(models: Array<{ providerID: string; id: string; context: number }>) {
function makeEditor(models: Array<{ providerID: string; id: string; context: number }>) {
const entries = models.map((model) => ({
providerID: model.providerID,
id: model.id,
limit: { context: model.context, output: 1000 },
}))
const providers = new Map<string, { providerID: string; models: Map<string, any> }>()
for (const entry of entries) {
if (!providers.has(entry.providerID)) providers.set(entry.providerID, { providerID: entry.providerID, models: new Map() })
providers.get(entry.providerID)!.models.set(entry.id, entry)
}
const catalog: any = {
provider: { list: () => [...providers.values()] },
model: {
update: (providerID: string, modelID: string, change: (model: any) => void) => {
const model = providers.get(providerID)?.models.get(modelID)
if (model) change(model)
},
const editor: any = {
list: () => entries,
update: (providerID: string, modelID: string, change: (model: any) => void) => {
const model = entries.find((entry) => entry.providerID === providerID && entry.id === modelID)
if (model) change(model)
},
entries,
}
return catalog
return editor
}

function makeCtx(options: { model?: any; models?: any[] } = {}) {
Expand All @@ -36,22 +29,15 @@ function makeCtx(options: { model?: any; models?: any[] } = {}) {
]
).map((model) => structuredClone(model))

// Rebuild the catalog from the registered transforms on every read, the way
// the runtime replays transforms onto a fresh value.
// Rebuild the model list from the registered transforms on every read, the
// way the runtime replays transforms onto a fresh value.
const rebuild = () => {
const entries = structuredClone(baseModels)
const providers = new Map<string, { providerID: string; models: Map<string, any> }>()
for (const entry of entries) {
if (!providers.has(entry.providerID)) providers.set(entry.providerID, { providerID: entry.providerID, models: new Map() })
providers.get(entry.providerID)!.models.set(entry.id, entry)
}
const editor = {
provider: { list: () => [...providers.values()] },
model: {
update: (providerID: string, modelID: string, change: (model: any) => void) => {
const model = providers.get(providerID)?.models.get(modelID)
if (model) change(model)
},
list: () => entries,
update: (providerID: string, modelID: string, change: (model: any) => void) => {
const model = entries.find((entry) => entry.providerID === providerID && entry.id === modelID)
if (model) change(model)
},
}
for (const transform of transforms) transform(editor)
Expand All @@ -66,10 +52,10 @@ function makeCtx(options: { model?: any; models?: any[] } = {}) {
session: {
get: async () => ({ model: options.model ?? { providerID: "opencode-go", id: "deepseek-v4.1-flash" } }),
},
catalog: {
model: {
transform: async (callback: any) => void transforms.push(callback),
reload: async () => void (reloads += 1),
model: { list: async () => ({ data: rebuild() }) },
list: async () => ({ data: rebuild() }),
},
command: { transform: (callback: any) => callback({ add: (definition: any) => commands.push(definition) }) },
}
Expand Down Expand Up @@ -128,20 +114,20 @@ describe("longestMatch and resolveBudget", () => {

describe("applyBudget", () => {
test("lowers only matched models", () => {
const catalog = makeCatalog([
const editor = makeEditor([
{ providerID: "opencode-go", id: "x", context: 1_000_000 },
{ providerID: "deepseek", id: "y", context: 1_000_000 },
])
applyBudget(catalog, [{ pattern: "opencode-go/*", value: 128_000, unit: "tokens" }])
const byKey = Object.fromEntries(catalog.entries.map((entry: any) => [`${entry.providerID}/${entry.id}`, entry.limit.context]))
applyBudget(editor, [{ pattern: "opencode-go/*", value: 128_000, unit: "tokens" }])
const byKey = Object.fromEntries(editor.entries.map((entry: any) => [`${entry.providerID}/${entry.id}`, entry.limit.context]))
expect(byKey["opencode-go/x"]).toBe(128_000)
expect(byKey["deepseek/y"]).toBe(1_000_000)
})

test("does nothing without rules", () => {
const catalog = makeCatalog([{ providerID: "a", id: "b", context: 1000 }])
applyBudget(catalog, [])
expect(catalog.entries[0].limit.context).toBe(1000)
const editor = makeEditor([{ providerID: "a", id: "b", context: 1000 }])
applyBudget(editor, [])
expect(editor.entries[0].limit.context).toBe(1000)
})
})

Expand Down Expand Up @@ -208,9 +194,9 @@ describe("setup", () => {
expect(commands.map((entry) => entry.name)).toEqual(["context-limit"])
expect(transforms).toHaveLength(1)

const catalog = makeCatalog([{ providerID: "opencode-go", id: "x", context: 1_000_000 }])
transforms[0](catalog)
expect(catalog.entries[0].limit.context).toBe(128_000)
const editor = makeEditor([{ providerID: "opencode-go", id: "x", context: 1_000_000 }])
transforms[0](editor)
expect(editor.entries[0].limit.context).toBe(128_000)
})
})

Expand Down
34 changes: 16 additions & 18 deletions context-limit.ts
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
// OpenCode V2 context-limit plugin.
//
// Sets a working context budget per model by lowering the model's
// `limit.context` through a catalog transform. Compaction's default threshold
// `limit.context` through a model transform. Compaction's default threshold
// follows the model's usable input budget, so a lower window makes compaction
// fire earlier. A budget can only lower a window, never raise it.
//
// The runtime does not resolve @opencode/plugin, so this file exports a plain
// { id, setup } object.

const VERSION = "0.1.2"
const VERSION = "0.1.3"

type Unit = "tokens" | "percent"

Expand Down Expand Up @@ -60,19 +60,17 @@ function resolveBudget(rules: Rule[], key: string, catalogContext: number): numb
return rule ? budgetFor(rule, catalogContext) : undefined
}

// The catalog transform body. `catalog` is a catalog editor: it exposes
// provider.list() and model.update(providerID, modelID, change).
function applyBudget(catalog: any, rules: Rule[]): void {
// The model transform body. `editor` is a model editor: it exposes
// list() and update(providerID, modelID, change).
function applyBudget(editor: any, rules: Rule[]): void {
if (rules.length === 0) return
for (const record of catalog.provider.list()) {
for (const model of record.models.values()) {
const key = `${model.providerID}/${model.id}`
const budget = resolveBudget(rules, key, model.limit.context)
if (budget === undefined) continue
catalog.model.update(model.providerID, model.id, (entry: any) => {
entry.limit = { ...entry.limit, context: budget }
})
}
for (const model of editor.list()) {
const key = `${model.providerID}/${model.id}`
const budget = resolveBudget(rules, key, model.limit.context)
if (budget === undefined) continue
editor.update(model.providerID, model.id, (entry: any) => {
entry.limit = { ...entry.limit, context: budget }
})
}
}

Expand All @@ -86,7 +84,7 @@ async function saveRules(ctx: any, rules: Rule[]): Promise<void> {
}

async function modelContext(ctx: any, key: string): Promise<number | undefined> {
const result = await ctx.catalog.model.list()
const result = await ctx.model.list()
const data: any[] = Array.isArray(result) ? result : (result?.data ?? [])
const found = data.find((model) => `${model.providerID}/${model.id}` === key)
return found?.limit?.context
Expand All @@ -104,7 +102,7 @@ const plugin = {
async setup(ctx: any) {
let rules = await loadRules(ctx)

await ctx.catalog.transform((catalog: any) => applyBudget(catalog, rules))
await ctx.model.transform((editor: any) => applyBudget(editor, rules))

await ctx.command.transform((editor: any) => {
editor.add({
Expand Down Expand Up @@ -163,7 +161,7 @@ const plugin = {
if (clears) {
rules = rules.filter((rule) => rule.pattern !== pattern)
await saveRules(ctx, rules)
await ctx.catalog.reload()
await ctx.model.reload()
return
}
if (!parsed || (parsed.unit === "tokens" && parsed.value <= 0)) {
Expand All @@ -172,7 +170,7 @@ const plugin = {
rules = rules.filter((rule) => rule.pattern !== pattern)
rules.push({ pattern, value: parsed.value, unit: parsed.unit })
await saveRules(ctx, rules)
await ctx.catalog.reload()
await ctx.model.reload()
},
})
})
Expand Down
18 changes: 9 additions & 9 deletions docs/compose/spec/context-limit.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,11 @@ commits: 5e865ff..4647039

## Report

**T0 spike result** - The mechanism is a catalog transform:
**T0 spike result** - The mechanism is a model transform:

```ts
ctx.catalog.transform((catalog) =>
catalog.model.update(providerID, modelID, (model) => {
ctx.model.transform((editor) =>
editor.update(providerID, modelID, (model) => {
model.limit = { ...model.limit, context: n }
}),
)
Expand All @@ -27,9 +27,9 @@ usable input budget, so this is the mechanism, and no config edit is needed.
**What was built** - A single-file OpenCode V2 plugin that sets a working
context budget per model. `/context-limit` shows the budget, `/context-limit
128K` or `50%` sets it for the current model, `/context-limit <pattern>
<value>` sets a rule, and `0` clears. Rules live in storage. A catalog transform
<value>` sets a rule, and `0` clears. Rules live in storage. A model transform
lowers the matched models' `limit.context`, and a change calls
`ctx.catalog.reload()`. Budgets clamp to the catalog window and never raise it.
`ctx.model.reload()`. Budgets clamp to the catalog window and never raise it.

**Verification** - `bun test`: 14 pass, 0 fail, 42 assertions. Live: `128K`
lowered the effective window to 128000; `50%` reported window 500000 with
Expand All @@ -38,14 +38,14 @@ blocking items plus a medium and lows; all are resolved.

**Journey log**

1. The spike proved the mechanism: a catalog transform lowered
1. The spike proved the mechanism: a model transform lowered
`deepseek/deepseek-flash` from 1,000,000 to 123,456, so no config edit was
needed.
2. The show path re-resolved a percent rule against the already-lowered window,
so a 50% rule printed Budget 250000. It now reports the stored rule, and a
token rule above the window prints the clamped number.
3. The show test used a fixed catalog, which hid that bug. The fake now rebuilds
the catalog from the registered transforms, the way the runtime replays them.
3. The show test used a fixed model list, which hid that bug. The fake now rebuilds
the list from the registered transforms, the way the runtime replays them.

## [S1] Problem

Expand All @@ -68,7 +68,7 @@ compaction.
such as `opencode-go/*` or `*`, and the longest matching pattern wins. The map
lives in `opencode.json` under `compaction.max_context` when the config
supports it, otherwise in plugin storage.
- A spike task decides where the budget can take effect: a catalog transform on
- A spike task decides where the budget can take effect: a model transform on
the model limit, or the config compaction threshold. The chosen path is
recorded in the spec before the command is built.

Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "opencode-context-limit",
"version": "0.1.2",
"version": "0.1.3",
"description": "OpenCode V2 plugin that sets a per-model working context budget",
"type": "module",
"license": "MIT",
Expand Down