diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json new file mode 100644 index 0000000..a039616 --- /dev/null +++ b/.claude-plugin/marketplace.json @@ -0,0 +1,12 @@ +{ + "name": "cef", + "owner": { "name": "Cere / CEF-AI" }, + "description": "Tooling for authoring CEF agents", + "plugins": [ + { + "name": "cef", + "source": "./plugins/cef", + "description": "Author, test, build, push, and deploy CEF agents" + } + ] +} diff --git a/README.md b/README.md index b3370a8..fb2de5d 100644 --- a/README.md +++ b/README.md @@ -1 +1,28 @@ -# cef-plugins \ No newline at end of file +# cef-plugins + +Claude Code plugin marketplace for **CEF agent development**. + +## Install + +``` +/plugin marketplace add CEF-AI/cef-plugins +/plugin install cef@cef +``` + +Projects scaffolded with `cef init` auto-wire this plugin via +`.claude/settings.json` — trust the folder and Claude Code installs and keeps +it updated. + +## Plugins + +### `cef` + +Author, test, build, push, and deploy CEF agents. Skills: + +- **`cef:develop`** — author, extend, and debug agents (engagements, cubbies, + migrations, widgets, tests). Loads automatically when you work on agent code. +- **`cef:deploy`** — the `build → push → publish → deploy` workflow; confirms + before outward-facing steps and guides the human-only ROC / credentials + actions. + +Depth lives in each skill's `references/`. diff --git a/plugins/cef/.claude-plugin/plugin.json b/plugins/cef/.claude-plugin/plugin.json new file mode 100644 index 0000000..0dd976d --- /dev/null +++ b/plugins/cef/.claude-plugin/plugin.json @@ -0,0 +1,7 @@ +{ + "name": "cef", + "description": "Author, test, build, push, and deploy CEF agents", + "version": "0.1.0", + "author": { "name": "Cere / CEF-AI" }, + "homepage": "https://github.com/CEF-AI/cef-plugins" +} diff --git a/plugins/cef/skills/deploy/SKILL.md b/plugins/cef/skills/deploy/SKILL.md new file mode 100644 index 0000000..b0e9cbb --- /dev/null +++ b/plugins/cef/skills/deploy/SKILL.md @@ -0,0 +1,127 @@ +--- +name: deploy +description: Build, push, and deploy a CEF agent so it runs live (including in a sandbox); marketplace publish is optional and last. Use when shipping/releasing an agent or when the user asks to build/push/deploy/publish. +--- + +# Deploy a CEF agent + +Ship an agent through this order: **`cef init` → `cef build` → `cef push` → `cef deploy`**. + +**`cef deploy` is what makes an agent live and testable** (including in a +sandbox). **`cef publish` — the marketplace listing — is OPTIONAL and the LAST +step.** Publish only places the agent's card in the marketplace for discovery; +skip it entirely for debug, test, or sandbox use. An agent runs because it is +deployed, not because it is published. + +In a scaffolded project run `cef` via `npx cef …` (it's a local dev-dep). + +```bash +npx cef init # scaffold a project (ships a deployments/ folder) +npx cef build # local: compile dist//{bundle.js,manifest.json} +npx cef push --bucket --as-pubkey # OUTWARD: upload bundle to the DDC registry +npx cef deploy --endpoint --as-pubkey # OUTWARD: apply deployments/ — makes the agent live +npx cef publish --keystore --as-pubkey # OPTIONAL, LAST: list the card in the marketplace +``` + +## Non-negotiable rules + +- **`push`, `deploy`, and `publish` are outward-facing.** Never run any of them + without confirming with the user first (see checklist below). `build` and + `init` are local and safe — run them freely. +- **Two setup steps remain human-only, done in the ROC console:** creating the + Agent Service (which yields the AS pubkey) and minting a DDC registry access + token. Hand those back as steps; never fake them. The ROC **Deploy** button + does the same thing as `cef deploy`. +- **Never invent flags or commands.** Everything here is grounded in the + references. + +## The `deployments/` folder is the source of truth + +An agent project declares its deployment records as files — one record per +`deployments/.jsonc`, the filename stem being the record `name` (the same +convention as `migrations//` and `widgets//`). `cef deploy` reads the +whole folder, assembles the deployment SET, and applies it. + +A record's key fields: `priority` (int, REQUIRED, **lower wins**), `targeting` +(string, REQUIRED — a CEL expression over `vault.id` / `vault.scope` / +`event.type`; `""` means always eligible), `version` (REQUIRED — a semver or the +literal `"latest"`), and `weight` (int ≥ 1, default 1, splits traffic within a +priority tier for A/B). Full field list and SET rules: +[references/roc-deploy.md](./references/roc-deploy.md). + +The folder must contain **exactly one** record with empty `targeting` — the +mandatory default/fallthrough. Give it a HIGH priority number (e.g. 99) so more +specific, lower-priority records win. `version: "latest"` fails closed if +`latest` was never pushed — pin a semver for anything that must run. + +## `cef deploy` — the command surface + +```bash +npx cef deploy [--version ] [--endpoint ] \ + [--as-pubkey ] [--author ] [--note ] [--dry-run] +``` + +- Reads `deployments/`; **errors if the folder is absent or empty**. +- `--endpoint` (or `$CEF_ENDPOINT`) selects WHERE to apply, defaulting to the + dev environment. It is **orthogonal** to the files: the same `deployments/` + folder applies to any environment, so never put environment names inside the + files. +- `--version` overrides the record `version` without editing files — the + everyday "pushed a new build, roll it out" step. +- `--deployments ` points at the records: a directory (default + `deployments/`, one record per file) or a single set/record file — like + `kubectl apply -f `. +- The whole set is applied at once (declarative desired state) — records removed + locally disappear remotely on the next deploy. +- `--dry-run` shows what would be applied without applying it. + +## Before push / deploy / publish — confirm identity, bucket, target + +Surface these to the user and get an explicit go-ahead before any outward call: + +1. **AS pubkey** (`--as-pubkey `) — the Agent Service routing identity from + ROC. Same value on push, deploy, and publish. A wrong pubkey means the agent + is unroutable (`connect` rejects on an agentId prefix mismatch). +2. **Bucket** (`--bucket `, push only) — the AS's DDC registry bucket, + in decimal. +3. **DDC auth** (push only) — exactly one of `--access-token` / + `$CEF_DDC_ACCESS_TOKEN` (ROC-minted, the common team case) or + `--secret-phrase` / `$CEF_DDC_SECRET_PHRASE` (bucket owner). Both = error; + neither = error. +4. **Endpoint** (deploy) — echo the resolved `--endpoint` / `$CEF_ENDPOINT` so + the user knows which environment goes live. +5. **AS keypair** (publish) — resolved via `--privkey/--pubkey`, `CEF_AS_*` env, + `--keystore`, or a `~/.config/cef/credentials` profile. + +Echo the resolved AS pubkey, bucket, endpoint, and (for publish) marketplace +URL, then ask to proceed. See +[references/credentials.md](./references/credentials.md) for the two-identity +model and resolution order. + +## Detect the missing prerequisite → hand back the exact next step + +Diagnose *which* gate is unmet and return the one precise command or ROC click. +The CLI's own error strings name the gate — surface them verbatim, then act. + +| Symptom / state | Missing | Hand back | +|---|---|---| +| No AS keypair anywhere (`CEF_AS_*`, keystore, credentials profile) | Publisher identity | `cef keypair generate --out ~/.cef/my-agent.key` (once; share via a secrets manager, not per-dev) | +| No AS pubkey to pass `--as-pubkey` | ROC setup step 1 | "Open ROC → your Agent Service → copy the AS pubkey (hex)" | +| `push`: "no DDC authorization" | ROC setup step 2 | "ROC → Agent Service → Access → generate token; `export CEF_DDC_ACCESS_TOKEN=…`" (note the `bucketId`) | +| `push`: "both … provided" | Ambiguous auth | Unset one of `$CEF_DDC_SECRET_PHRASE` / `$CEF_DDC_ACCESS_TOKEN` | +| `push`: bundle/version not found | Build not run | `cef build` first | +| `push`: missing `@cere-ddc-sdk/ddc-client` | Optional dep | `pnpm add @cere-ddc-sdk/ddc-client` | +| `deploy`: "no deployments/ folder" / empty folder | No records | Create `deployments/default.jsonc` (see reference), then re-run | +| Pushed, but a user's `connect` never gets served | Not deployed | `npx cef deploy --endpoint --as-pubkey ` (or ROC → the agent → **Deploy**) | +| `connect` rejected (agentId prefix mismatch) | Wrong AS pubkey | Re-push/deploy with the **provisioned** AS pubkey | +| Card missing from the marketplace (agent still runs) | Not published | `npx cef publish …` — optional, only for discovery | + +Rule of thumb: don't guess past the current gate — resolve it, then re-run. + +## Shipping a new version + +Bump `version` in `cef.config.ts`, then re-run build → push → `cef deploy` +(pass `--version ` to roll the new build out without editing files, or +deploy `latest`). Add a lower-priority or weighted record in `deployments/` for +a controlled rollout. Re-run `cef publish` only if the marketplace card needs +refreshing — it is not required for the new version to run. diff --git a/plugins/cef/skills/deploy/references/credentials.md b/plugins/cef/skills/deploy/references/credentials.md new file mode 100644 index 0000000..6c32931 --- /dev/null +++ b/plugins/cef/skills/deploy/references/credentials.md @@ -0,0 +1,135 @@ +# Credentials, keypair & DDC bucket + +Two independent identities are involved in shipping an agent, and they are +easy to confuse: + +| Command | Identity | Key type | Where it goes | +| --- | --- | --- | --- | +| `cef push` | **DDC bucket owner** (or a token from them) | Sr25519 (substrate) | authorizes writes to the content-addressed bucket | +| `cef publish` | **Agent Service (AS)** | Ed25519 | signs the marketplace publish envelope | + +They are unrelated keys. `cef push` signs DDC writes with the bucket-owner +key, which "can be anything with write access" — so the AS identity **cannot +be derived** from it and must be passed in explicitly with `--as-pubkey`. + +## AS Ed25519 keypair (`cef publish`, and `--as-pubkey` on push) + +The AS keypair is the **Agent Service identity** — the marketplace-visible +publisher of the agent — and is **typically shared across a team**, not +unique per developer. The CLI never auto-generates it; you provision it +once with `cef keypair generate` and share it through your secrets channel. +Generating a fresh one per developer fragments the agent's marketplace +history under different identities. + +### Generate a keypair + +`cef keypair generate` is the only sanctioned path to a new AS keypair: + +```bash +# JSON to stdout — pipe to a secrets manager +cef keypair generate + +# Write a 0600 file (refuses an existing file unless --force) +cef keypair generate --out keypair.json --force + +# Merge into the shared credentials file under [staging] +# (atomic 0600 write; refuses an existing section unless --force) +cef keypair generate --profile staging +``` + +Keypair shape: `pubkey` is 64 lowercase hex chars (32 bytes); the private +key is a 32-byte Ed25519 **seed**, also 64 hex chars. + +### Resolution order + +`cef publish` resolves the AS keypair AWS-CLI-style — flags beat env, env +beats files, files beat the shared INI. The **first** source that yields a +complete keypair wins: + +| # | Source | How | +| --- | --- | --- | +| 1 | CLI flags | `--privkey --pubkey ` | +| 2 | Env vars | `CEF_AS_PRIVATE_KEY= CEF_AS_PUBLIC_KEY=` | +| 3 | Keystore via flag | `--keystore ` — JSON `{ pubkey, privkey }` | +| 4 | Keystore via env | `CEF_AS_KEYSTORE=` — same shape | +| 5 | Profile in shared file | `~/.config/cef/credentials`, section via `--profile ` / `CEF_PROFILE`, default `default` | + +Gotchas: + +- **Pairs must be complete.** Setting only one of `--privkey`/`--pubkey` + (or only one of the two env vars) is a hard error, not a fall-through. +- **A named path or profile must exist.** `--keystore `, + `CEF_AS_KEYSTORE`, and an explicit `--profile ` all error if the + file/section is missing — the miss is not silently skipped (that would + mask a typo). Only a *default* profile with no file present counts as + "no credentials". +- With no source at all, the CLI prints multi-line guidance listing every + source and pointing at `cef keypair generate`, then exits non-zero. + +### Shared credentials file + +Lives at `~/.config/cef/credentials` (or `$XDG_CONFIG_HOME/cef/credentials`), +mode `0600`, minimal INI. Multiple profiles hold identities for several +environments on one workstation: + +```ini +[default] +pubkey = abc... +privkey = abc... + +[staging] +pubkey = def... +privkey = def... +``` + +## DDC bucket + credentials (`cef push`) + +`cef push` uploads `dist//bundle.js` to DDC as a content-addressed +DAG and rewrites `manifest.json` so `bundle` is `{ bucketId, cid }` instead +of embedded inline. It requires: + +1. **`--bucket `** (REQUIRED) — decimal DDC bucket id. The + substrate signer must own it. The bucket *is* the agent registry + (`CNS "agents" → directory → per-agent record → per-version node`). +2. **Exactly one** DDC write authorization: + - `--secret-phrase ` / `$CEF_DDC_SECRET_PHRASE` — the Sr25519 + phrase that **owns** the bucket. + - `--access-token ` / `$CEF_DDC_ACCESS_TOKEN` — a base58 token + **granted by the bucket owner** (e.g. minted in ROC by the wallet that + owns the AS's bucket). Use this when you don't hold the bucket's seed; + the client is built with a throwaway signer and the token carries the + authority. Providing **both** phrase and token is an error; providing + **neither** is an error. +3. **`--as-pubkey `** — the AS pubkey (exactly as shown in ROC). + Because push's signing key is unrelated to the AS identity, this is how + the manifest gets stamped: `agentServicePubkey = ` and + `agentId = :`. With it stamped, bucket consumers (the + dashboard registry read, the `connect` check) see a fully-identified + manifest immediately — no need to wait for `cef publish`. + +```bash +# Owner pushes with the bucket's secret phrase +cef push --bucket 1234 \ + --secret-phrase "$CEF_DDC_SECRET_PHRASE" \ + --as-pubkey + +# Teammate pushes with a ROC-minted access token (no bucket seed) +cef push --bucket 1234 \ + --access-token "$CEF_DDC_ACCESS_TOKEN" \ + --as-pubkey --preset TESTNET +``` + +Other push flags: `--preset MAINNET|TESTNET|DEVNET` (default MAINNET), +`--endpoint ` (overrides preset), `--agent ` (when `dist/` holds +more than one), `--out ` (default `dist`). + +Gotchas: + +- Run `cef build` first — push errors if `dist//{bundle.js,manifest.json}` + or the manifest `version` is missing. +- `cef push` needs the optional dep `@cere-ddc-sdk/ddc-client`; install it + (`pnpm add @cere-ddc-sdk/ddc-client`) or push throws an actionable error. +- `bucketId` is recorded in the manifest as a **decimal string**. +- On a bucket's first push the `agents` CNS root doesn't exist yet; push + logs that it's creating the registry root (the internal resolve miss is + expected, not a failure). Set `CEF_DEBUG` for full ddc-client logs. diff --git a/plugins/cef/skills/deploy/references/roc-deploy.md b/plugins/cef/skills/deploy/references/roc-deploy.md new file mode 100644 index 0000000..10c8472 --- /dev/null +++ b/plugins/cef/skills/deploy/references/roc-deploy.md @@ -0,0 +1,192 @@ +# Deploying: the `deployments/` folder, `cef deploy`, and ROC setup + +`cef build` → `cef push` gets your agent's manifest and bundle into the DDC +registry, but the CEF platform **does not select or serve it until a deployment +targets it**. `cef deploy` is the step that makes an agent live and testable +(including in a sandbox). Marketplace **publish is optional and comes last** — it +only lists the agent's card for discovery and can be skipped entirely for debug, +test, or sandbox use. + +Two setup actions still happen in the ROC console (they can't be done from the +CLI): creating the Agent Service and minting a DDC registry access token. This +file covers those, the `deployments/` folder, the `cef deploy` command, and what +the skill should do when a prerequisite is missing. + +## The `deployments/` folder is the source of truth + +An agent project declares its deployment records as files: one record per +`deployments/.jsonc`, where the filename stem is the record `name`. This +mirrors the `migrations//` and `widgets//` conventions. `cef deploy` +reads the whole folder, assembles the deployment SET, and applies it to the +target environment. + +### Record fields (in-file) + +| Field | Type | Required | Meaning | +|---|---|---|---| +| `priority` | int | yes | **Lower wins.** Selects which record applies when several match. | +| `targeting` | string | yes | A CEL expression over `vault.id` / `vault.scope` / `event.type`. Empty string `""` = always eligible. | +| `version` | string | yes | A semver (e.g. `1.4.0`) or the literal `"latest"`. | +| `weight` | int ≥ 1 | no (default 1) | Splits traffic **within a priority tier** — this is how you run an A/B. | +| `enabled` | bool | no (default true) | Set false to keep a record on disk but inactive. | +| `params` | object | no | Parameters passed to the agent for this record. | +| `engagements` | object | no | Per-engagement overrides. | +| `expiresAt` | RFC3339 | no | Drops the record after this time — handy for sandbox cleanup. | + +### SET rules (the folder must satisfy) + +- At least one record. +- **Exactly one** record with empty `targeting` — the mandatory + default/fallthrough, so selection is never empty. +- Unique lowercase-slug `name`s. +- `weight >= 1` on every record. +- A non-empty `version` on every record. + +### Audiences & A/B + +`targeting` decides *who* a record applies to. Among the records that match a +given vault/event, the **lowest `priority` tier wins**; if that tier holds more +than one record, traffic splits between them by `weight` (an A/B split). Give the +default (empty-`targeting`) record a HIGH priority number — e.g. `99` — so it is +the fallthrough and more specific, lower-priority records win when present. + +> `version: "latest"` **fails closed** if `latest` was never pushed. Prefer a +> pinned semver for anything that must run. + +### Example files a scaffolded project ships + +```jsonc +// deployments/default.jsonc — the mandatory fallthrough +{ + "priority": 99, + "targeting": "", + "version": "latest", + "weight": 1 +} +``` + +```jsonc +// deployments/sandbox-canary.jsonc — a pinned build for sandbox vaults +{ + "priority": 10, + "targeting": "vault.scope == 'sandbox'", + "version": "1.4.0", + "weight": 1 +} +``` + +Here a sandbox-scoped vault matches both records; priority 10 beats priority 99, +so it gets the pinned `1.4.0`. Everyone else falls through to the default. + +## `cef deploy` — the command + +```bash +npx cef deploy [--version ] [--endpoint ] \ + [--as-pubkey ] [--author ] [--note ] [--dry-run] +``` + +- Reads the records and applies the assembled SET. **Errors if there are none.** + `--deployments ` points at a directory (default `deployments/`, one + record per file) or a single set/record file — like `kubectl apply -f`. +- `--endpoint` (or `$CEF_ENDPOINT`) selects WHERE to apply, defaulting to the dev + environment. It is **orthogonal** to the deployment files: the same + `deployments/` folder applies to any environment. Do **not** put environment + names inside deployment files. +- `--version ` overrides the `version` in the applied records without + editing files — the everyday "pushed a new build, roll it out" step. +- The whole folder is applied as one set (declarative desired state): a record + deleted locally disappears remotely on the next deploy. +- `--author` / `--note` annotate the applied set for audit. +- `--dry-run` prints what would be applied without applying it. Use it to confirm + the assembled set before a real deploy. + +The ROC console **Deploy** button applies the same deployment set; either path +works. + +## ROC setup step 1 — Agent Service + AS pubkey + +`cef push --as-pubkey `, `cef deploy --as-pubkey `, and +`cef publish --as-pubkey ` all require the on-platform **Agent Service +pubkey**. This is the *routing* identity the platform looks up to deliver events; +it is provisioned by the platform and shown in the ROC console. It is distinct +from the AS Ed25519 keypair that signs the publish envelope (`cef keypair +generate`). + +In ROC: open (or create) your **Agent Service**; its details dialog shows the AS +pubkey (hex). Copy it — you pass the same value to every `--as-pubkey` flag. + +> Gotcha: an agent can `cef publish` fine yet be **unroutable** if its AS pubkey +> does not match a provisioned Agent Service. The `connect` check enforces that +> the `agentId` begins with the provisioned AS pubkey — so always use the real +> provisioned pubkey, not just any local keypair, for anything that must run. + +## ROC setup step 2 — DDC registry access token + +`cef push` writes the manifest + bundle into your Agent Service's **DDC registry +bucket** (the bucket *is* the registry). To authorize the write without handing +out the bucket's raw secret phrase, mint a token in ROC: + +1. Open the **ROC console** → your **Agent Service** → **Access** tab. +2. It shows the AS's registry `bucketId` and a "generate access token" control + with a selectable validity window. +3. Generate and **copy immediately — the token is shown once.** Note the + `bucketId` too — you pass it to `cef push --bucket`. + +The token is a base58, full-access (GET/PUT/DELETE) DDC token scoped to that +bucket, signed by the bucket-owning wallet. It is a **bearer** token any holder +can use until it expires; it is stateless and **cannot be individually +revoked** — it only expires (default ~30 days). Prefer a short window and +regenerate on exposure. + +```bash +export CEF_DDC_ACCESS_TOKEN="" +cef push --bucket --as-pubkey +``` + +`cef push` requires **exactly one** of `--access-token` / +`$CEF_DDC_ACCESS_TOKEN` or `--secret-phrase` / `$CEF_DDC_SECRET_PHRASE`. Neither +→ hard error; both → hard error. The token path is the common team case (you +don't hold the phrase). + +## Detect the missing prerequisite → hand back the exact next step + +Diagnose *which* gate is unmet and return the precise command or ROC step. In +flow order: + +| Symptom / state | What's missing | Hand back | +|---|---|---| +| No AS keypair (`~/.cef/*.key`, `~/.config/cef/credentials`, `$CEF_AS_*`) | Publisher identity | `cef keypair generate --out ~/.cef/my-agent.key` (once; share via a secrets manager) | +| No AS pubkey to pass `--as-pubkey` | ROC setup step 1 | "Open ROC → your Agent Service → copy the AS pubkey (hex)" | +| `push` errors "no DDC authorization" | ROC setup step 2 | "Open ROC → Agent Service → Access → generate token; `export CEF_DDC_ACCESS_TOKEN=…`" (note the `bucketId` for `--bucket`) | +| `push` errors "both … provided" | Ambiguous auth | Unset one of `$CEF_DDC_SECRET_PHRASE` / `$CEF_DDC_ACCESS_TOKEN` (or drop one flag) | +| `push` errors "bundle not found" / "no built agents" / "no version" | Build not run | `cef build` first | +| `deploy` errors "no deployments" / empty folder | No records to apply | Create `deployments/default.jsonc` (priority 99, targeting `""`), then re-run | +| Pushed but a user's `connect` never gets served | Not deployed | `npx cef deploy --endpoint --as-pubkey ` (or ROC → the agent → **Deploy**) | +| `connect` rejected (agentId prefix mismatch) | Wrong AS pubkey | Re-push/deploy with the **provisioned** AS pubkey so `agentId` starts with it | +| Card absent from the marketplace, but the agent runs | Not published | `npx cef publish …` — optional, discovery only | + +Rule of thumb: the CLI's own error strings name the missing gate ("run +`cef build` first", "no DDC authorization"). Surface that string verbatim, then +translate it into the one next command or ROC click — don't guess further ahead +than the current gate. + +## The full flow (for reference) + +```bash +# one-time: publisher identity (share via a secrets manager, not per-dev) +cef keypair generate --out ~/.cef/my-agent.key +# one-time per token window: copy from ROC → Agent Service → Access +export CEF_DDC_ACCESS_TOKEN="" + +npx cef init # scaffold, incl. deployments/ +npx cef build +npx cef push --bucket --as-pubkey +npx cef deploy --endpoint --as-pubkey # makes it live +# optional, last — only if you want a marketplace listing: +npx cef publish --keystore ~/.cef/my-agent.key --as-pubkey +``` + +The `--endpoint` (or `$CEF_ENDPOINT`) defaults to the dev environment; point it +at another environment to deploy there — the same `deployments/` folder applies +either way. Select the DDC network for `push` with `--preset +MAINNET|TESTNET|DEVNET` (default `MAINNET`) or `--endpoint`. diff --git a/plugins/cef/skills/develop/SKILL.md b/plugins/cef/skills/develop/SKILL.md new file mode 100644 index 0000000..cadf0e9 --- /dev/null +++ b/plugins/cef/skills/develop/SKILL.md @@ -0,0 +1,106 @@ +--- +name: develop +description: Author, extend, and debug a CEF agent — add engagements/event handlers, cubbies and migrations, widgets, and tests. Use when writing or changing agent code in a CEF agent project. +--- + +# Develop a CEF agent + +A CEF agent is a TypeScript project: a declarative manifest (`cef.config.ts`) +plus a default-exported **engagement** class (`src/agent.ts`) whose methods +handle events and lifecycle. `cef build` lints + bundles it into +`dist//{bundle.js, manifest.json}`; the Cere Agent Runtime executes the +bundle against Web globals plus one CEF surface, `ctx`. Everything you write +lives inside that sandbox — the constraints below are hard rules, not style. + +## OVERVIEW — the loop + +1. **Understand the project.** Read `cef.config.ts` (identity, engagements, + cubbies, models, widgets, params) and the entry class it points at. The + config is the source of truth the manifest and lints are derived from. +2. **Author the agent class.** Add `@OnEvent("type")` methods and `@OnStart` / + `@OnClose` lifecycle hooks on the `@Engagement`-decorated default export. +3. **Add state.** Declare a cubby in `cubbies[]`, write SQL migrations, read/write + via `ctx.cubby("alias")`. +4. **Add UI (optional).** Ship a self-contained widget dir declared in `widgets[]`. +5. **Test.** `@cef-ai/testing` (`testAgent` / `testPlatform`) under Vitest, with + the same cubby aliases + migration dirs as the config. +6. **Respect the runtime constraints on every edit** — no Node built-ins, string + literals for `@OnEvent` / `ctx.publish` / `ctx.cubby`, declared aliases only. + +After changing `cef.config.ts`, run `cef typegen` so `@OnEvent` / `ctx.publish` +literals autocomplete and type their payloads. Then `cef build` to lint + bundle. + +## ROUTING — read the reference before you edit that layer + +| You are… | Read | Covers | +|---|---|---| +| Orienting in an unfamiliar project | [anatomy.md](./references/anatomy.md) | file layout, `defineAgent` fields, manifest/`__handlers` model, build→push→publish→deploy | +| Writing handlers / lifecycle / multi-engagement | [engagements.md](./references/engagements.md) | `@Engagement`/`@OnEvent`/`@OnStart`/`@OnClose`, `Event

`, the `ctx` surface, selection decorators | +| Adding or querying persistent state | [cubbies.md](./references/cubbies.md) | `cubbies[]` decl, SQL migration ordering, `.query`/`.exec`, alias lint | +| Shipping a static UI | [widgets.md](./references/widgets.md) | `WidgetDecl`, self-contained dirs, `cef push` DAG, `window.WidgetRuntime` | +| Writing tests | [testing.md](./references/testing.md) | `testAgent`, `testPlatform`, model mocks, `dispatch`/`published`, harness members | +| Debugging a build failure or lint error | [constraints.md](./references/constraints.md) | banned imports, string-literal + declared-alias rules, ESLint wiring | + +## The shape at a glance + +```ts +// cef.config.ts +import { defineAgent } from "@cef-ai/agent-sdk/config"; +export default defineAgent({ + id: "echo", + version: "0.1.0", + entry: "./src/agent.ts", // OR engagements: [...] + cubbies: [{ alias: "history", migrations: "./migrations/history" }], +}); +``` + +```ts +// src/agent.ts +import { Engagement, OnEvent, OnStart, type Context, type Event } from "@cef-ai/agent-sdk"; + +@Engagement({ id: "default", goal: "echo back any user message" }) +export default class Echo { + @OnStart async onStart() { console.info("started"); } + + @OnEvent("user_message") + async onMessage(event: Event<{ text: string }>, ctx: Context) { + await ctx.cubby("history").exec( + "INSERT INTO messages(id, text, ts) VALUES (?, ?, ?)", + [Date.now(), event.payload.text, event.ts], + ); + await ctx.publish("ack", { for: event.id }); + } +} +``` + +## Runtime constraints (always in force — see constraints.md) + +- **No Node built-ins, no sync DB/network libs.** `fetch` for HTTP, + `ctx.cubby` for state, `globalThis.crypto` (WebCrypto) instead of `node:crypto`. + `cef build` bans `fs`/`net`/`http`/`child_process`/`pg`/`ws`/`axios`/… . +- **Literals, statically read into the manifest.** `@OnEvent("...")`, + `ctx.publish("...", …)`, and `ctx.cubby("...")` first args must be inline string + literals; the cubby alias (and any `ctx.models.X`) must be **declared** in + `cef.config.ts`. +- **Keep the context param named `ctx`** — the lints key off that identifier. +- **Fresh instance + fresh `ctx` per call** — never stash per-request state on `this`. +- **Use `console.*` for logging** (runtime captures it); `ctx` is CEF-only — + there is no `ctx.log` / `ctx.fetch`. +- **One engagement owns each lifecycle hook**; `@OnStart`/`@OnClose` collisions + fail the build. Event method names may repeat across engagements (namespaced). + +## Dev commands + +In a scaffolded project `cef` is a local dev-dependency, so run it via `npx cef …` +(bare `cef` is not on `PATH`). A `cef init` project also aliases the common ones +as npm scripts (`pnpm run build`, `pnpm test`). + +```sh +npx cef typegen # after editing cef.config.ts: refresh .cef/generated.d.ts + cef.lock.json +npx cef build # lint entry + bundle → dist//{bundle.js, manifest.json} +npx cef inspect dist/ # verify exposed __handlers match manifest routing +npx vitest run # run the @cef-ai/testing simulator tests (or: pnpm test) +``` + +`cef push` (upload bundle + widget DAGs) and `cef publish` (sign + register) come +after build; see anatomy.md for the full pipeline. diff --git a/plugins/cef/skills/develop/references/anatomy.md b/plugins/cef/skills/develop/references/anatomy.md new file mode 100644 index 0000000..a4e1da3 --- /dev/null +++ b/plugins/cef/skills/develop/references/anatomy.md @@ -0,0 +1,192 @@ +# Project anatomy & runtime model + +A CEF agent is authored as a TypeScript project, compiled by `cef build` into a +bundle + manifest, and executed by the Cere Agent Runtime. This +reference maps the files you'll touch to what the runtime and platform +actually consume. + +## What a project contains + +``` +my-agent/ + cef.config.ts # declarative agent manifest source (defineAgent) + src/agent.ts # default-exported engagement class w/ decorators + migrations/ # per-cubby SQL migration dirs (one dir per cubby alias) + history/001-init.sql + widgets/ # optional: built, self-contained static web UIs + test/*.test.ts # vitest + @cef-ai/testing simulator tests + package.json # depends on @cef-ai/agent-sdk (+ -D @cef-ai/cli) + .cef/generated.d.ts # emitted by `cef typegen` — KnownEventTypes autocompletion + cef.lock.json # typegen snapshot; VCS diffs surface drift + dist// # `cef build` output (bundle.js + manifest.json) +``` + +- **`cef.config.ts`** — the single source of declarative config. Exports + `defineAgent({...})` as default. Declares identity, engagements, cubbies, + widgets, models, settings, params. `cef build` turns it into the manifest. +- **`src/agent.ts`** — the code. A default-exported class decorated with + `@Engagement`, `@OnEvent`, `@OnStart`, `@OnClose`. This is the *entry* the + config points at (`entry` or `engagements[].entry`). +- **`migrations//*.sql`** — SQL schema for a cubby. A `CubbyDecl` in the + config (`{ alias, migrations: "./migrations/history" }`) points at the dir; + `ctx.cubby("history")` at runtime reads/writes that SQLite-backed store. +- **`widgets/`** — optional built static UIs (entry HTML + sibling assets using + `./relative` refs). `cef push` uploads each `WidgetDecl.dir` to DDC as ONE + content-addressed directory DAG; the manifest carries `{ bucketId, cid, entry }` + and consumers resolve at `///`. The `dir` must + already be built before push — the CLI copies, it does not bundle widgets. + +### Minimal example + +```ts +// cef.config.ts +import { defineAgent } from "@cef-ai/agent-sdk/config"; + +export default defineAgent({ + id: "echo", + version: "0.1.0", + entry: "./src/agent.ts", + cubbies: [{ alias: "history", migrations: "./migrations/history" }], +}); +``` + +```ts +// src/agent.ts +import { Engagement, OnEvent, OnStart, type Context, type Event } from "@cef-ai/agent-sdk"; + +@Engagement({ id: "default", goal: "echo back any user message" }) +export default class Echo { + @OnStart async onStart() { console.info("started"); } + + @OnEvent("user_message") + async onMessage(event: Event<{ text: string }>, ctx: Context) { + await ctx.cubby("history").exec( + "INSERT INTO messages(id, text, ts) VALUES (?, ?, ?)", + [Date.now(), event.payload.text, event.ts], + ); + await ctx.publish("ack", { for: event.id }); + } +} +``` + +## What `defineAgent` accepts + +`defineAgent(c: T): T` is just an identity function that +preserves literal types. Key `AgentConfig` fields: + +- **`id`** (required) — the agent **alias**. The composite `agentId` is + `"{agentServicePubkey}:{alias}"` (the AS pubkey plus this alias); `id` is the + alias half, not the whole composite. +- **`version`** (required) — semver string. +- **`entry`** OR **`engagements[]`** — *exactly one*. `entry: "./src/agent.ts"` + is shorthand for `engagements: [{ id: "default", entry }]`. Multi-engagement + form: each `{ id, entry, goal?, condition?, priority?, weight?, limit?, + params?, enabled? }` (on-declaration selection fields, mirroring the + `@Condition`/`@Priority`/`@Weight`/`@Limit`/`@Params` class decorators). +- **`cubbies`** — `[{ alias, migrations? }]`. `alias` must match every + `ctx.cubby("alias")` call (build lints this). +- **`models`** — `Record`; used via `ctx.models.` (lint + requires declared aliases). +- **`params`** — `Record` (`type`, `default`, `min`/`max`/`enum`; + `"modelAlias"` must enumerate `models` keys). Resolved per Job as + `manifest defaults ⊕ deployment ⊕ engagement` and surfaced as `ctx.params`. +- **`settings`** — declared config knobs (`string|number|boolean|url|secret`). +- **`widgets`** — `WidgetDecl[]` (see above). +- **`requiredScopes`** — vault scopes needed at connection (default `["default"]`). +- **`idleTimeout`** — duration string (`"30m"` default; `"0s"` disables). Parsed + to ms into `manifest.idleTimeout`; on expiry the platform marks the Job + terminal and dispatches the synthetic close event. +- **`eventSchemas`** — JSON Schemas for events this agent *owns* (see "Typed + event payloads" below). +- **`agentServicePubkey`** — optional; stamps full identity at build time. +- **`targeting`** (the agent-config engagement table) — DEPRECATED: put + selection knobs on each engagement instead (see engagements.md). Distinct from + a *deployment's* `targeting` (audience CEL), which is current. + +## Typed event payloads (`eventSchemas` + `cef typegen`) + +`@OnEvent("t")` and `ctx.publish("t", …)` accept any string by default, so you +can ship without declaring schemas. To get **autocomplete + payload type +checking** on your own events, declare their JSON Schema in `eventSchemas` +(`Record`) and run `cef typegen`, which folds them into +`KnownEventTypes` so `Event

` and the `publish` payload are typed. + +```ts +// cef.config.ts +export default defineAgent({ + id: "echo", + version: "0.1.0", + entry: "./src/agent.ts", + eventSchemas: { + feedback: { + type: "object", + properties: { rating: { type: "number" } }, + required: ["rating"], + }, + feedback_ack: { + type: "object", + properties: { rating: { type: "number" } }, + required: ["rating"], + }, + }, +}); +``` + +Then `cef typegen` (writes `.cef/generated.d.ts` + `cef.lock.json`) makes +`@OnEvent("feedback")` give `event.payload.rating: number` and type-checks +`ctx.publish("feedback_ack", { rating })`. Declare schemas only for events this +agent owns; peer-published event schemas are pulled from the peer's manifest. + +## Runtime & manifest model + +The runtime is **CEF-agnostic**: it exposes only Web-platform globals plus one +opaque `globalThis.context`, and dispatches +`globalThis.__handlers[name](event, context)`. Two build artifacts encode this: + +- **`bundle.js`** — an IIFE (esbuild output) that imports the SDK runtime shims + (`ctx.cubby`/`.publish`/`.models`/`.close`, reading endpoints from `context`) + and **installs `globalThis.__handlers`**: a *flat* map of handler functions. + Event handlers are keyed `"::"` (namespaced to stay + collision-free across engagements); lifecycle hooks live under the fixed, + NON-namespaced keys `onStart` / `onClose`. +- **`manifest.json`** — the **routing table** the platform consults. Each + engagement carries `handles{eventType -> handlerName}` (namespaced) and + `hooks[]` (`"start"`/`"close"`). The platform looks up those handler names + directly on `globalThis.__handlers` to dispatch each event. + +Gotchas: +- Only one engagement per agent may own each lifecycle hook — `cef build` errors + on an `onStart`/`onClose` collision (hooks aren't namespaced). Event method + names *may* be reused across engagements (they are namespaced). +- No Node built-ins. Use global `fetch` for HTTP, `ctx.cubby` for state, + `globalThis.crypto` (WebCrypto) instead of `node:crypto`. `cef build` bans + `fs`/`net`/`http`/`child_process` and sync DB/network packages. +- Use `console.*` for plain logging (the runtime captures it); `ctx` is only for + CEF-specific calls. + +## build → push → publish → deploy + +1. **`cef build`** — lints the entry (see below), bundles with esbuild, emits + `dist//{bundle.js, manifest.json}`. Identity/signature fields stay + blank unless `agentServicePubkey` is set. +2. **`cef push`** — uploads the bundle and each widget dir to DDC as + content-addressed objects/DAGs; stamps `{ bucketId, cid }` refs into the + manifest. (`--as-pubkey ` overrides identity.) +3. **`cef deploy`** — apply the `deployments/` folder so a version is live for + connected users; the platform resolves the manifest from the bucket-backed + registry and dispatches Jobs against `__handlers`. This is what makes an + agent run. +4. **`cef publish`** — OPTIONAL, last: places the agent **card** on the + marketplace for discovery (a signed card envelope). The manifest is NOT sent + to the marketplace — it lives in the developer's DDC bucket (from `cef push`). + Skip publish for debug/test. + +Build-time lints (mirror `@cef-ai/eslint-plugin`): `@OnEvent(...)` and +`ctx.publish(t, ...)` first args must be string literals; `ctx.cubby("alias")` +must be a literal AND a declared alias; `ctx.models.X` must reference a declared +model alias; banned imports abort the build. + +Support commands: **`cef typegen`** regenerates `.cef/generated.d.ts` +(populates `KnownEventTypes` for `@OnEvent`/`ctx.publish` autocomplete) and +`cef.lock.json`. **`cef inspect `** runs the built bundle in `node:vm` +and reports exposed `__handlers` names vs. manifest routing targets. diff --git a/plugins/cef/skills/develop/references/constraints.md b/plugins/cef/skills/develop/references/constraints.md new file mode 100644 index 0000000..0391972 --- /dev/null +++ b/plugins/cef/skills/develop/references/constraints.md @@ -0,0 +1,162 @@ +# Runtime constraints & lints + +`cef build` runs a **syntactic lint pass over the agent's entry file before +bundling**. Any violation aborts the build with a single, actionable message +(`[cef build] :: — `). There is no error-collection +mode — the *first* violation throws. `@cef-ai/eslint-plugin` (`recommended` +config) mirrors every rule at edit time, so your editor agrees with CI. + +These are hard rules of the agent sandbox, not style preferences. Fix them at +the source; there is no override flag. + +> Scope note: v1 lints the **entry file only**. Helper modules imported via +> `./` paths are not scanned syntactically, but banned imports are still +> caught at bundle time when esbuild resolves them. Keep the rules in mind +> everywhere, not just the entry. + +## 1. Banned imports + +Agents run in a sandbox with **no Node built-ins and no sync DB/network +libraries**. Importing any of these — via `import`, `require(...)`, +`import(...)`, or `export ... from` — fails the build. + +Banned Node built-ins (matched with or without the `node:` prefix, including +subpaths like `node:fs/promises`): + +``` +fs, fs/promises, path, net, tls, dgram, dns, http, https, http2, +child_process, worker_threads, cluster, os, process, readline, repl, +vm, stream, zlib, buffer, crypto +``` + +Banned npm packages (sync DB / raw network clients): + +``` +pg, pg-native, mysql2, mysql, mongodb, redis, ioredis, sqlite3, +better-sqlite3, ws, node-fetch, axios +``` + +### What to use instead + +| Instead of… | Use… | +| --- | --- | +| `http` / `https` / `node-fetch` / `axios` | the global `fetch` (`await fetch(url)`) | +| `fs`, `pg`, `redis`, `mongodb`, … (state/storage) | `ctx.cubby("alias")` (declared persistent state) | +| `node:crypto` | `globalThis.crypto` — the WebCrypto API | +| `net` / `tls` / `ws` (raw sockets) | `fetch` (HTTP only; raw sockets are not available) | +| `buffer` / `stream` | Web standards: `Uint8Array`, `TextEncoder`/`TextDecoder`, `ReadableStream` | + +```ts +// BAD — build fails: banned import "node:crypto" +import { randomUUID } from "node:crypto"; +const id = randomUUID(); + +// GOOD — WebCrypto is on the global +const id = globalThis.crypto.randomUUID(); +const digest = await globalThis.crypto.subtle.digest("SHA-256", bytes); +``` + +```ts +// BAD — banned import "axios" +import axios from "axios"; +const res = await axios.get(url); + +// GOOD — global fetch +const res = await fetch(url); +const data = await res.json(); +``` + +## 2. `@OnEvent(...)` must be a string literal + +The event-type argument is read statically to build the manifest routing +table, so it cannot be a variable or expression. + +```ts +// BAD — @OnEvent argument must be a string literal +const EVT = "user_message"; +@OnEvent(EVT) + +// GOOD +@OnEvent("user_message") +``` + +## 3. `ctx.publish(t, ...)` first arg must be a string literal + +The literal is collected into the manifest's `publishes[]` (when not declared +in `cef.config.ts`), so it must be inline. + +```ts +// BAD — ctx.publish first argument must be a string literal +ctx.publish(topic, payload); + +// GOOD +ctx.publish("assistant_message", payload); +``` + +## 4. `ctx.cubby("alias")` — literal AND declared + +The first arg must be a string literal **and** a `cubbies[].alias` declared in +`cef.config.ts`. An undeclared alias fails with the list of declared aliases. + +```ts +// cef.config.ts declares: cubbies: [{ alias: "history", ... }] + +// BAD — literal but not declared: +// ctx.cubby alias "chat" is not declared in cef.config.ts cubbies[] +const c = ctx.cubby("chat"); + +// BAD — not a literal: +// ctx.cubby first argument must be a string literal +const c = ctx.cubby(name); + +// GOOD +const c = ctx.cubby("history"); +``` + +Fix: add the alias to `cubbies[]` in `cef.config.ts`, or use an already-declared +one. + +## 5. `ctx.models.X` must be a declared model alias + +Member access on `ctx.models` must name a model alias declared under `models` +in `cef.config.ts`. + +```ts +// cef.config.ts declares: models: { llm: ... } + +// BAD — ctx.models.gpt is not declared in cef.config.ts models +const out = await ctx.models.gpt.complete(prompt); + +// GOOD +const out = await ctx.models.llm.complete(prompt); +``` + +## Keeping editor and CI in sync + +Enable `@cef-ai/eslint-plugin` `recommended` to catch all of the above at edit +time. The declared-alias/type rules need the sets passed as options; wire them +from `cef.config.ts`: + +```js +// .eslintrc.cjs +module.exports = { + plugins: ["@cef-ai"], + extends: ["plugin:@cef-ai/recommended"], + rules: { + "@cef-ai/cubby-declared-alias": ["error", { aliases: ["history"] }], + "@cef-ai/model-declared-alias": ["error", { aliases: ["llm"] }], + "@cef-ai/publish-declared-type": [ + "error", + { knownEventTypes: ["assistant_message"] }, + ], + }, +}; +``` + +`cef typegen` refreshes `KnownEventTypes` (in `.cef/generated.d.ts`) so the +string-literal args autocomplete; run it after changing `cef.config.ts`. + +> Gotcha: the lint identifies context calls by the **head identifier `ctx`**, +> by convention. Rename the context parameter (e.g. `context.cubby(...)`) and +> the alias/literal checks silently stop firing at build time — keep the +> parameter named `ctx`. diff --git a/plugins/cef/skills/develop/references/cubbies.md b/plugins/cef/skills/develop/references/cubbies.md new file mode 100644 index 0000000..bd0f7d0 --- /dev/null +++ b/plugins/cef/skills/develop/references/cubbies.md @@ -0,0 +1,183 @@ +# State: cubbies & migrations + +A **cubby** is a per-agent, per-alias SQLite store. You declare it in +`cef.config.ts`, define its schema with SQL migrations, and read/write it from +handlers via `ctx.cubby(alias)`. Each declared alias is an isolated database +scoped to the agent instance. + +## 1. Declare the cubby in `cef.config.ts` + +Every cubby is an entry in the `cubbies[]` array: an `alias` plus an optional +`migrations` directory (path relative to `cef.config.ts`). + +```ts +// cef.config.ts +import { defineAgent } from "@cef-ai/agent-sdk/config"; + +export default defineAgent({ + id: "echo", + version: "0.1.0", + entry: "./src/agent.ts", + cubbies: [{ alias: "history", migrations: "./migrations/history" }], +}); +``` + +`CubbyDecl` shape: + +```ts +interface CubbyDecl { + alias: string; + /** Directory of *.sql migrations, relative to cef.config.ts. */ + migrations?: string; +} +``` + +## 2. Write SQL migrations + +Put one or more `*.sql` files in the migrations directory. They are read, +**sorted lexicographically**, and applied in order — so use a numeric prefix +(`001-init.sql`, `002-add-index.sql`) to pin ordering. A migration file may +contain multiple statements (applied via SQLite `db.exec`). + +```sql +-- migrations/history/001-init.sql +CREATE TABLE messages ( + id INTEGER PRIMARY KEY, + text TEXT NOT NULL, + ts INTEGER NOT NULL +); +``` + +Migrations are additive and forward-only: add a **new** numbered file for each +schema change rather than editing an already-applied one. + +## 3. Read / write from handlers + +`ctx.cubby(alias)` returns a `CubbyHandle`: + +```ts +interface CubbyHandle { + query(sql: string, params?: unknown[]): Promise; + exec( + sql: string, + params?: unknown[], + ): Promise<{ changes: number; lastInsertRowid: number | bigint }>; +} +``` + +- **`.exec(sql, params)`** — a single DML/DDL statement with bound params; + returns `{ changes, lastInsertRowid }`. +- **`.query(sql, params)`** — returns an array of row objects. The runtime + receives a columnar response (`{ columns, rows }`) and zips it into + `T[]` for you. + +Both are `async` — always `await`. Use `?` placeholders and pass values in the +`params` array; never string-concatenate SQL. + +```ts +// src/agent.ts +import { Engagement, OnEvent, type Context, type Event } from "@cef-ai/agent-sdk"; + +@Engagement({ id: "default", goal: "echo + persist to history" }) +export default class Echo { + @OnEvent("user_message") + async onMessage(event: Event<{ text: string }>, ctx: Context) { + await ctx + .cubby("history") + .exec("INSERT INTO messages(id, text, ts) VALUES (?, ?, ?)", [ + Date.now(), + event.payload.text, + event.ts, + ]); + await ctx.publish("ack", { for: event.id }); + } +} +``` + +Reading back: + +```ts +const rows = await ctx + .cubby("history") + .query<{ text: string }>("SELECT text FROM messages ORDER BY ts"); +``` + +## 4. The alias must be a declared string literal + +The `@cef-ai/cubby-declared-alias` ESLint rule enforces two things on +`ctx.cubby(...)`: + +1. The first argument **must be a string literal** — not a variable or + expression. `ctx.cubby(someVar)` is a lint error (`notLiteral`). +2. When configured with the declared aliases, the literal **must name a cubby + declared in `cef.config.ts` `cubbies[]`** — otherwise `undeclared`. + +```ts +ctx.cubby("history") // ok — declared literal +ctx.cubby(alias) // error: notLiteral +ctx.cubby("scratch") // error: undeclared (not in cubbies[]) +``` + +Keeping the alias a literal lets the toolchain statically verify that every +cubby you touch has a declared schema. + +## 5. Testing cubbies + +The test harness gives each alias an isolated in-memory SQLite database and +applies the same migration files, so tests exercise the real schema. Declare +the same alias + migrations dir you use in `cef.config.ts`: + +```ts +import { testAgent } from "@cef-ai/testing"; +import Echo from "../src/agent.js"; + +const HISTORY_MIGRATIONS = path.resolve(__dirname, "../migrations/history"); + +const h = testAgent(Echo, { + cubbies: [{ alias: "history", migrations: HISTORY_MIGRATIONS }], +}); +await h.dispatch({ type: "user_message", payload: { text: "hello" } }); + +const rows = await h + .cubby("history") + .query<{ text: string }>("SELECT text FROM messages"); +// rows -> [{ text: "hello" }] +``` + +## Adding a second cubby + +Cubbies are independent — add another entry to `cubbies[]` with its own +migrations dir, and declare **every** alias you touch in **both** places: + +```ts +// cef.config.ts +cubbies: [ + { alias: "history", migrations: "./migrations/history" }, + { alias: "ratings", migrations: "./migrations/ratings" }, // new +], +``` + +```ts +// the test harness must list the same aliases — cubby(alias) throws otherwise +h = testAgent(MyAgent, { + cubbies: [ + { alias: "history", migrations: HISTORY_MIGRATIONS }, + { alias: "ratings", migrations: RATINGS_MIGRATIONS }, + ], +}); +``` + +Forgetting to add the new alias to the test's `cubbies` is the usual trip-up — +`ctx.cubby("ratings")` then throws "alias not declared" at dispatch. + +## Gotchas + +- **`exec` is single-statement.** In handlers, one statement per `.exec()` + call (params bind via `prepare().run()`). Multi-statement scripts are only + for migration files. +- **Missing migrations dir throws.** In tests a non-existent `migrations` path + errors at cubby construction — point it at the real directory. +- **Lexicographic ordering.** `10-*.sql` sorts before `2-*.sql`; zero-pad + numeric prefixes. +- **Alias is scoped, not shared.** Two aliases are two separate databases; + there is no cross-alias query. diff --git a/plugins/cef/skills/develop/references/engagements.md b/plugins/cef/skills/develop/references/engagements.md new file mode 100644 index 0000000..2e05c59 --- /dev/null +++ b/plugins/cef/skills/develop/references/engagements.md @@ -0,0 +1,155 @@ +# Agent code: engagements, events, lifecycle + +A CEF agent is one or more **engagements** — goal-bound code units. Each +engagement is a class **default-exported** from its own module, decorated with +`@Engagement(...)`. Event and lifecycle handlers are methods on that class. All +decorators come from `@cef-ai/agent-sdk`; types are imported `type`-only. + +## The default-exported class + +```ts +import { + Engagement, + OnEvent, + OnStart, + OnClose, + type Context, + type Event, +} from "@cef-ai/agent-sdk"; + +interface UserMessage { text: string } + +@Engagement({ + id: "default", + goal: "echo back any user message and persist to history", +}) +export default class Echo { + @OnStart + async onStart(/* ctx: Context */) { + console.info("started"); // plain logging → sandbox console; runtime captures it + } + + @OnEvent("user_message") + async onMessage(event: Event, ctx: Context) { + await ctx.cubby("history").exec( + "INSERT INTO messages(id, text, ts) VALUES (?, ?, ?)", + [Date.now(), event.payload.text, event.ts], + ); + await ctx.publish("ack", { for: event.id }); + } + + @OnClose + async onClose(_ctx: Context, reason: string) { + console.info("closing", { reason }); + } +} +``` + +`@Engagement({ id, goal })` is required to mark the class as an engagement (`id` is +its stable identifier within the agent; `goal` is human-readable intent). Classes +using only the method decorators are transitional. + +## Decorators + +| Decorator | Applies to | Purpose | +|-----------|-----------|---------| +| `@Engagement({id, goal})` | class | Marks the engagement; sets id + goal. | +| `@OnEvent("type")` | method | Handler for inbound events of that type. | +| `@OnStart` / `@OnStart()` | method | Lifecycle: called once before any events. | +| `@OnClose` / `@OnClose()` | method | Lifecycle: called on shutdown. | + +`@OnStart`/`@OnClose` support both bare and factory (`()`) forms — both land the +same metadata. `@OnEvent` requires the call form with a string literal. + +**Handler invocation signatures** (the runtime calls with a **fresh instance and +fresh `ctx` per call** — never store per-request state on `this`): + +- Event: `method(event, ctx)` +- `@OnStart`: `method(ctx)` +- `@OnClose`: `method(ctx, reason)` — `reason` is a string, defaulting to + `"closed_by_agent"`. Semantic values (`OnCloseReason`): `"revoked"`, + `"idle_timeout"`, `"closed_by_agent"`, `"failed"`. + +**Gotcha — duplicate `@OnEvent` for the same type:** first handler wins, a +`console.warn` fires at runtime, and build tooling errors. One handler per event +type per engagement. + +## The `Event

` type + +```ts +interface Event

{ + id: string; // event id + type: string; // event type string + ts: number; // unix-millis timestamp + from: string; // publisher (agent id or external source) + payload: P; // typed body +} +``` + +Type the payload with your own interface: `Event`, then read +`event.payload.text`. + +## The `Context` (`ctx`) + +`ctx` is the **CEF-specific platform surface only**. Plain Web capabilities are NOT +mirrored — use sandbox globals directly: `console.*` for logging (runtime captures +it out-of-band) and `fetch` for HTTP. Do not expect `ctx.log`/`ctx.fetch`. + +- `ctx.cubby(alias)` → `CubbyHandle` with `query(sql, params?)` and `exec(sql, params?)` + (per-agent scoped SQLite-style storage; alias must be declared in `cef.config.ts`). +- `ctx.models` → model handles keyed by manifest-declared alias. Each is a + `ModelHandle` with `infer(input)` (single-shot) and `stream(input)` (async + iterable). Aliases are typed via `KnownModels` (filled by `cef typegen`): + ```ts + const reply = await ctx.models.llm.infer({ prompt: event.payload.text }); + await ctx.publish("assistant_message", { text: reply.text }); + ``` +- `ctx.publish(type, payload)` → publish an event (awaitable). +- `ctx.settings` → frozen `ConnectionSettings` the user supplied at connect time. +- `ctx.params` → frozen effective params (manifest ⊕ deployment ⊕ engagement, + resolved by the platform). +- `ctx.close(reason?)` → ask the runtime to close this agent. + +## String-literal rules for `@OnEvent` / `ctx.publish` + +Event types are plain strings and must **match the schemas declared in +`cef.config.ts`** (`eventSchemas` for inbound, publishable types). `cef typegen` +declaration-merges `KnownEventTypes`, so with typegen run you get autocompletion and +payload typing on both `@OnEvent("...")` and `ctx.publish("...", payload)`. Without +typegen, any string is accepted (untyped overload). Keep the literal in code exactly +equal to the manifest's type string. + +## Single vs multi-engagement + +**Single:** one file, one default-exported class (see `echo-agent`). + +**Multi:** one file per engagement, each with its own default-exported class. Wire +them in `cef.config.ts` and put the selection knobs **on each engagement** — +`priority` (lower wins), `weight` (split traffic within a tier), `limit`, and a +CEL `condition`: + +```ts +// src/engagements/simple.ts +@Engagement({ id: "simple", goal: "give a short conversational reply" }) +export default class Simple { + @OnEvent("user_message") + async onMessage(event: Event, ctx: Context) { /* ... */ } +} +``` + +```ts +// cef.config.ts +export default defineAgent({ + id: "multi-asst", + engagements: [ + { id: "advanced", entry: "./src/engagements/advanced.ts", priority: 1, limit: { n: 10, per: "day" } }, + { id: "simple", entry: "./src/engagements/simple.ts", priority: 2 }, + ], +}); +``` + +Equivalently, author the same knobs **on the class** with `@Condition` / +`@Priority` / `@Weight` / `@Limit` / `@Params` — `cef build` folds either form +into the manifest. The engagement `id` in the class decorator must match the +config entry's `id`. (A deployment can then override an engagement's +`priority`/`weight`/`limit` at deploy time.) diff --git a/plugins/cef/skills/develop/references/testing.md b/plugins/cef/skills/develop/references/testing.md new file mode 100644 index 0000000..1a5d375 --- /dev/null +++ b/plugins/cef/skills/develop/references/testing.md @@ -0,0 +1,192 @@ +# Testing with @cef-ai/testing + +`@cef-ai/testing` is a deterministic, in-process simulator for CEF agents. +Use `testAgent` for single-agent unit tests and `testPlatform` for +multi-agent / app-driven tests. Tests run under Vitest. + +```sh +pnpm add -D @cef-ai/testing +``` + +```ts +import { testAgent, testPlatform, createModelMock } from "@cef-ai/testing"; +``` + +## `testAgent(AgentClass, opts)` + +Constructs an in-memory harness around a single agent class. It wires +`ctx.cubby` to per-alias SQLite, `ctx.models[alias]` to stubs, and +`ctx.publish` to a collector you can assert on, and installs a URL-pattern +mock as the global `fetch` for the duration of each invocation. + +`opts` (all optional): `cubbies`, `models`, `settings`, `params`, +`clock: { now }`. A cubby spec is either `"alias"` (no schema) or +`{ alias, migrations }` where `migrations` is a directory of SQL files. + +### Minimal passing test + +```ts +import { describe, it, expect, afterEach } from "vitest"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; +import { testAgent, type TestHarness } from "@cef-ai/testing"; +import Echo from "../src/agent.js"; + +const __dirname = path.dirname(fileURLToPath(import.meta.url)); +const HISTORY_MIGRATIONS = path.resolve(__dirname, "../migrations/history"); + +describe("echo agent", () => { + let h: TestHarness | undefined; + afterEach(() => h?.dispose()); // release SQLite handles; idempotent + + it("acks user_message and persists to history", async () => { + h = testAgent(Echo, { + cubbies: [{ alias: "history", migrations: HISTORY_MIGRATIONS }], + }); + + // dispatch returns ONLY the publishes from this call + const events = await h.dispatch({ + type: "user_message", + payload: { text: "hello" }, + }); + expect(events.map((e) => e.type)).toContain("ack"); + + // assert on cubby rows + const rows = await h + .cubby("history") + .query<{ text: string }>("SELECT text FROM messages"); + expect(rows.map((r) => r.text)).toEqual(["hello"]); + }); +}); +``` + +### Asserting a published event's payload + +`dispatch()` returns the events published during that call. To assert a +specific event carried the right body, `find` it and read `.payload` (type it +with a cast, or declare the event in `eventSchemas` + run `cef typegen` for real +types): + +```ts +const events = await h.dispatch({ type: "feedback", payload: { rating: 5 } }); + +const ack = events.find((e) => e.type === "feedback_ack"); +expect(ack).toBeDefined(); +expect((ack!.payload as { rating: number }).rating).toBe(5); +``` + +### Lifecycle (`@OnStart` / `@OnClose`) + +`start()` runs the agent's `@OnStart` hook; `close(reason)` runs `@OnClose`. +Observe state via `h.lifecycle`. + +```ts +await h.start(); +expect(h.lifecycle.state).toBe("started"); +await h.close("idle_timeout"); +expect(h.lifecycle.state).toBe("closed"); +expect(h.lifecycle.terminalReason).toBe("idle_timeout"); +``` + +Note the two distinct "close" concerns: `h.close(reason)` drives the +lifecycle hook, while `h.dispose()` frees test resources (SQLite handles). + +### Other `TestHarness` members + +- `published` — cumulative log of every publish since construction (vs. + `dispatch()`'s return, which is scoped to that one call). +- `runInCubby(alias, (handle, rawDb) => …)` — run with both the + `CubbyHandle` and the raw `better-sqlite3` DB for synchronous asserts. +- `models` — the model-handle map attached to `ctx.models`. +- `fetchMock()` — the URL-pattern mock installed as the global `fetch`. +- `advanceTime(ms)` / `now()` — manipulate the injected clock. +- `snapshot()` / `restore(snap)` — capture/restore cubbies + clock + log. +- `replay(jsonlPath)` — feed a JSONL fixture of events through the harness. + +## Model mocks + +Pass either a plain `ModelHandle` (`{ infer, stream }`) or a +`createModelMock()` handle via `opts.models`. The mock matches inputs by +`JSON.stringify` deep-equality; register pairs with +`.expect(input).respond(output)`: + +```ts +const llm = createModelMock(); +llm.expect({ prompt: "hi" }).respond({ text: "hello back" }); + +h = testAgent(Assistant, { models: { llm } }); +// inside the agent, ctx.models.llm.infer({ prompt: "hi" }) resolves to +// { text: "hello back" }; llm.calls records every invocation. +``` + +`stream()` yields each element when the response is an array, otherwise +yields the single value once. An `infer`/`stream` with no matching +expectation throws — so a wrong or missing prompt fails loud. + +## `testPlatform(opts)` — multi-agent + +Lifts the harness into a multi-agent world: each agent runs in its own +simulator, a shared vault routes events between them, and the test drives +interactions through the vault (optionally as a specific user/wallet). +Use it when an agent talks to peers or when an app client publishes into +the vault. + +```ts +import { testPlatform, testWallet, createModelMock, type TestPlatform } from "@cef-ai/testing"; +import Assistant from "../agent/src/assistant.js"; +import { sendAndRead } from "../app/send-and-read.js"; + +let p: TestPlatform | undefined; +afterEach(() => p?.dispose()); // tears down every per-agent simulator + +const llm = createModelMock(); +llm.expect({ prompt: "hi" }).respond({ text: "hello back" }); + +p = testPlatform({ + users: { alice: { wallet: testWallet("alice") } }, + agents: { + assistant: { + source: Assistant, + cubbies: [{ alias: "history", migrations: HISTORY_MIGRATIONS }], + }, + }, + models: { llm }, +}); + +const alice = p.user("alice"); +await alice.vault.agents.connect({ agentId: "assistant" }); + +const reply = await sendAndRead(alice.vault, "default", "default", "hi"); +expect(reply).toBe("hello back"); + +// assert on a specific agent's cubby +const rows = await p.runInCubby("assistant", "history", (h) => + h.query<{ role: string; text: string }>( + "SELECT role, text FROM messages ORDER BY id", + ), +); +expect(rows).toEqual([ + { role: "user", text: "hi" }, + { role: "assistant", text: "hello back" }, +]); +``` + +- `p.vault` — simulated vault: `agents.connect`, event streams, and + scoped per-conversation channels. +- `p.user(id)` — the per-user accessor (`.vault` scoped to that wallet). +- `p.runInCubby(agentId, alias, fn)` — assert on one agent's cubby. +- `mockAgent(spec)` — a peer descriptor to stand in for an agent you don't + want to pull into the test bundle. (`fromMarketplace(url)` throws in v1.) + +## Gotchas + +- Agents call the **global `fetch`** and log with **`console.*`** — there is + no `ctx.fetch` or `ctx.log`. The harness swaps the global `fetch` only + during an invocation and restores it after (nesting-safe), so the stub + never leaks between dispatches. +- `cubby(alias)` throws if the alias wasn't declared in `opts.cubbies`; + duplicate aliases throw at construction. +- The agent class must carry `@OnEvent`/`@OnStart`/`@OnClose` decorators — + a bare class with no metadata throws at construction. +- `settings` and `params` are frozen when installed. +- Always `dispose()` in `afterEach` to release SQLite handles. diff --git a/plugins/cef/skills/develop/references/widgets.md b/plugins/cef/skills/develop/references/widgets.md new file mode 100644 index 0000000..b72c7c3 --- /dev/null +++ b/plugins/cef/skills/develop/references/widgets.md @@ -0,0 +1,106 @@ +# Widgets + +A **widget** is a self-contained static web UI (an entry HTML plus sibling JS/assets) that an agent ships alongside its bundle. You declare it in `cef.config.ts`, build it into a plain directory, and `cef push` uploads the whole directory to DDC as one content-addressed **directory DAG** — next to `bundle.js`. There is no baked gateway URL: consumers resolve the widget at `///`. + +## Declaring a widget + +Add a `widgets: [...]` array to `defineAgent(...)`. Each entry is a `WidgetDecl`. Minimal example: + +```ts +import { defineAgent } from "@cef-ai/agent-sdk/config"; + +export default defineAgent({ + id: "widgetized", + version: "1.0.0", + entry: "./src/agent.ts", + widgets: [ + { + id: "dashboard", // stable, unique within the agent; used as the dist subdir + name: "Hiring Dashboard", // optional display name + description: "Shows candidate pipeline", + cubbyAlias: "history", // optional: cubby the widget reads/queries against + kind: "panel", // optional free-form host hint + config: { theme: "dark" }, // optional; carried verbatim into the manifest + queries: [ // optional named SQL queries against the cubby + { id: "recent", label: "Recent", sql: "SELECT 1", timeoutMs: 5000 }, + ], + events: ["candidate_added"], // optional: event types the widget subscribes to + dir: "./widgets/dashboard", // path (relative to cef.config.ts) to the BUILT widget dir + entry: "dashboard.html", // entry file WITHIN dir + }, + ], +}); +``` + +### Field reference (`WidgetDecl`) + +| Field | Required | Meaning | +|---|---|---| +| `id` | yes | Stable id, unique within the agent. Used as the dist subdir name. | +| `dir` | yes | Path relative to `cef.config.ts` of the **already-built**, self-contained widget directory. | +| `entry` | yes | Entry file within `dir`, e.g. `"dashboard.html"`. Resolved at `///`. | +| `name` / `description` | no | Human-friendly display metadata. | +| `cubbyAlias` | no | Cubby alias the widget reads/queries against. | +| `kind` | no | Free-form widget-kind hint for the host. | +| `config` | no | Arbitrary widget-specific config, carried verbatim into the manifest. | +| `queries` | no | Named SQL queries: `{ id, label?, sql, timeoutMs? }[]`. | +| `events` | no | Event types the widget subscribes to. | + +All fields except `id`/`dir`/`entry` are metadata copied verbatim into the manifest. + +## Building a self-contained widget dir + +The directory must be **fully self-contained**: the entry HTML references its siblings via `./relative` paths only (no external CDNs, no absolute URLs), because everything is served under one directory CID. A "hello-world" widget needs no framework and no runtime — just HTML + JS. + +Real fixture (`packages/cli/test/fixtures/widget-agent/widgets/dashboard/`): + +`dashboard.html`: +```html + + +Hiring Dashboard +

+ +``` + +`app.js`: +```js +document.getElementById("root").textContent = "hello from widget"; +``` + +That's a complete, valid widget. Keep the starter widget plain like this — do not pull in the widget runtime until you actually need Vault access. + +> The CLI does NOT bundle or transform your widget. `dir` must already be built by the time you run `cef push`. If your widget needs a build step (bundler, TS), run it as the agent's own `build:widgets` script before `cef build`. + +## How `cef build` and `cef push` handle it + +- **`cef build`** copies whatever is in each `dir` into `dist//widgets//` (fresh each build — stale files are removed first). It also emits one `ManifestWidget` per declared widget with a **placeholder** `dag: { bucketId: "", cid: "" }`. If `dir` is missing, or the `entry` file isn't found inside it, build prints a WARNING but still emits the manifest entry. +- **`cef push`** reads each staged `dist//widgets//` directory, uploads it to DDC as one content-addressed directory DAG, and stamps the real `{ bucketId, cid }` into that widget's manifest entry. A widget with no staged directory (or an empty one) is skipped with a WARNING. + +So the flow is: `build:widgets` (your own) → `cef build` (stage into dist) → `cef push` (upload as DAG + stamp ref). + +## Optional: `window.WidgetRuntime` (Vault-native widgets) + +When a widget is loaded inside the Cere Vault host, an injected runtime (`@cef-ai/widget-runtime`, emerging) exposes `window.WidgetRuntime` for reading/writing the **viewing user's** own Vault as the connected agent — no baked secret, key material stays in the wallet iframe. Core surface: + +```ts +window.WidgetRuntime.query(ref, params?) // ref = a named query id (from queries[]) or raw SQL + // → { columns, rows, meta }; params are BOUND, never interpolated +window.WidgetRuntime.publish(type, payload, context?) // → { eventId } +window.WidgetRuntime.connectAgent() // the "install" flow: getAgent + vault.agents.connect() +window.WidgetRuntime.identity() / .connect() / .agentStatus() / .onIdentityChange(cb) +``` + +`query()` throws `AgentNotConnectedError` when the agent isn't connected for this user — catch it to render a "Connect / Install" CTA wired to `connectAgent()`. + +Gotchas: +- The runtime is injected by the upload/host step, not by `cef push`. Opened directly in a browser (no Vault host) it assigns `window.WidgetRuntime` synchronously but `query()`/`publish()` reject after an ~8s handshake deadline and render "Open this widget inside the Cere Vault." +- Keep starter/hello-world widgets plain and self-contained. Only reach for the runtime once the widget genuinely needs Vault data. + +## Gotchas checklist + +- `id`, `dir`, and `entry` are all required; `id` must be unique within the agent (build errors on duplicates). +- `dir` must be **built already** — the CLI copies, it does not compile or bundle. +- Reference siblings with `./relative` paths only; absolute/external URLs won't resolve under the directory CID. +- A missing `dir` or `entry` produces a WARNING (not an error) at build; `cef push` then skips that widget — so a "successful" build can still ship no widget. Check push output for `widget : dir DAG ( files)`. +- No gateway URL is baked in; consumers build `///` from the manifest ref.