Repository navigation
feat(mar-888): a versioned agent recipe behind every scaffolder, validated before any write - #358
Merged
Merged
Conversation
Three programs write a DASH agent folder and each assembled its own manifest v2 literal: the same manifest_version, safety_contract, monitoring block and runtime/trigger/locations/control skeleton, transcribed three times. Two had already drifted in ways nobody chose. `agent-kit/recipe.ts` holds one `AgentRecipe` (recipe_version 1) and one `planFromRecipe`. The Agent Kit's scaffolder, `lib/sample-agent.ts` and `tools/dash-mcp/src/scaffold.ts` are now adapters over it, each carrying only what actually differs between the agents they build: steps, sources, connections, what a run emits, permissions, and the panel. `validateRecipe` runs the recipe's own rules and then puts the manifest it would produce through `validateManifest` and `checkManifestConstraints` — DASH's own import verdict — before `planFromRecipe` returns a single file. It refuses an unusable id, a relative or traversing directory, a symlinked target, a non-empty folder, a write inside DASH's own agents root, and a dependency pinned to a range rather than a version. Everything is decided before anything is returned, so a refusal cannot leave half a project. The recipe travels with the agent as `agent.recipe.json`, and `checkRecipeAgainstManifest` reports where a hand-edited manifest has come apart from it. `create-dash-agent --check <folder>` and `dash_agent_validate` both run it. Every scaffold also gets `AGENT_BUILDER.md` — which files a coding assistant may edit and which it must not — and `evals/`, four acceptance checks that spawn the real agent under the runner protocol with a local fixture server and a broker double: a normal run, a run with nothing to read, a run where a source fails, and a run where every brokered request is refused. Node builtins only, no dependency, no network, no model. `lib/sample-agent.ts` amends the recipe rather than the finished manifest, so DASH's own sample does not create a folder its own drift check calls drifted. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…act this leaves the MCP The fixture is the recipe the MCP builder produces for a three-step competitor scout, generated from a finished interview draft rather than written by hand. tests/agent-recipe.test.ts walks it from validateRecipe through planFromRecipe, files on a real disk, validateManifest read back off those bytes, and a real Supervisor spawning what was written until it hands DASH a digest that passes the artifact contract. Also pinned: every refusal, one proving the real CLI leaves a folder with somebody's work in it exactly as it found it, that a hand-edited manifest which still validates is caught as drift, and that DASH's own sample is the Agent Kit's agent plus exactly two declarations. docs/foundation/mcp-recipe-contract.md says what the other repository's export_build_brief would have to emit, why there is no dash_agent_recipe tool, and — in its own section — that no live integration exists and the committed fixture is a local example rather than evidence of one. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ions What was verified and how: typecheck clean, 284 test files and 5383 tests green in one full run, the four generated evals passing against a real scaffolded agent under the real runner protocol, --check catching a hand-edited manifest that still validates, and 85 PASS / 0 FAIL on an installed-style shell smoke against a scratch store whose 6b proof lists agent.recipe.json, AGENT_BUILDER.md and evals/ travelling with the sample. Four deviations argued rather than buried: the recipe is deliberately not in AGENT_KIT_PROJECT_FILES, there is no dash_agent_recipe tool, the sample-refresh hook the brief made conditional does not exist, and four files outside the ownership list were touched. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Thebeatkicks
marked this pull request as ready for review
September 7, 2026 16:41
Its docblock still opened with "the whole folder, as data" and described a manifest this file no longer assembles. It now names what is left here — the one statement of how an agent this tool builds differs from the Agent Kit's — and keeps the record of why they differ, which is still true. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`unsafeRelativePath` returned a clause and the caller wrapped it, which read as "That build may not overwrite "../package.json" walks out of the project directory." A refusal is the one string somebody will actually be shown, so each branch now returns the whole sentence. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Orchestrator ruling on this packet's deviation 1. AGENT_KIT_PROJECT_FILES gains agent.recipe.json as an optional entry, beside the manifest it generated. Optional for a third reason, different from both already in that list. sources.json, README.md and .gitignore are optional because an agent can outgrow them (MAR-595 finding 9); the manifest, package, program, runtime and install script are required because losing one is a real incomplete build. The recipe is optional because every agent scaffolded before this packet has none, and requiring it would tell somebody their build was incomplete over a file missing only because of when their agent was made. It also removes an asymmetry with no reason behind it: the MCP's handoff walks the project directory and has always carried whatever is in it, so until now a kit-built agent reached DASH without its recipe while an MCP-built one reached it with one. Two tests: the handoff carries the recipe with the same bytes that are on disk and an agent id that still matches, and a project with no recipe still produces a handoff with the manifest and the program in it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… see Records the orchestrator's ruling on deviation 1, the two tests behind it, and why smoke proof 6b needed no change: electron/smoke.ts imports neither AGENT_KIT_PROJECT_FILES nor projectFiles nor the kit's writeHandoff, and 6b logs the sample plan's own file list, which already carried the recipe. Also records what merging lane F3's PR #357 actually did. merge-tree was clean and the merge produced no conflict, because the two branches changed two different files -- and they disagreed anyway: F3 bumped the runtime's SDK_VERSION to 1.1.0 while agent-kit/recipe.ts still pinned 1.0.0. That would have shipped agents running 1.1.0 whose own agent.recipe.json claimed 1.0.0. tests/agent-recipe.test.ts's first assertion reads the constant out of the runtime file and caught it, which is the whole reason it exists. Fixed here, the fixture regenerated to match, and the four evals re-proved against F3's instrumented runtime rather than assumed. And the honest version of the union's test results: the first full run had eight timeouts at the default 5s, all on the first freshStore() in six files, all passing alone, with a clean second full run. Contention, not behaviour -- flagged for the orchestrator because migration 38 raised what every fresh store costs. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Thebeatkicks
added a commit
that referenced
this pull request
Sep 7, 2026
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
MAR-888, stage 2 of MAR-886. Base:
codex/agent-foundationat821c239, whichalready carries stage 1's SDK (MAR-887, PR #356, now on
master). Full handoff:docs/foundation/stage-2-handoff.md.What this is
Three programs write a DASH agent folder —
agent-kit/scaffold.tsfor aterminal,
lib/sample-agent.tsfor "Try a sample agent", andtools/dash-mcp/src/scaffold.tsfor a coding assistant — and each one assembledits own manifest v2 literal: the same
manifest_version,safety_contract,monitoringblock andagent_domskeleton, transcribed three times. Two hadalready drifted in ways nobody chose. Stage 1 closed the fork in the runtime
those three write; this closes the fork in the document they write about it.
agent-kit/recipe.tsholds oneAgentRecipe(recipe_version: 1) and oneplanFromRecipe. The three scaffolders are adapters over it and carry only whatactually differs between the agents they build: the steps, the sources, the
connections, what a run emits, the permissions claimed, and the panel.
Validated before the first byte
validateReciperuns the recipe's own rules and then puts the manifest itwould write through
validateManifestandcheckManifestConstraints— DASH'sown import verdict — and
planFromRecipedecides everything before it returnsanything, so there is no state in which half a project exists. It refuses an
unusable id, a relative or traversing directory, a symlinked target, a folder
with somebody's files in it, a write inside DASH's own agents root, and a
dependency pinned to a range rather than an exact version.
Definition and manifest cannot drift unnoticed
The recipe travels with the agent as
agent.recipe.json, andcheckRecipeAgainstManifestreports where a hand-edited manifest has come apartfrom it.
create-dash-agent --check <folder>anddash_agent_validateboth runit. The check earns its keep because the edited manifest is usually still
valid — the test asserts that before asserting the drift is caught.
Instructions and evals
Every scaffold now carries
AGENT_BUILDER.md— which files a coding assistantmay edit and which it must not — and
evals/, four checks that spawn the realagent under the runner protocol against a loopback fixture server and a broker
double: a normal run, a run with nothing to read, a run where a source fails,
and a run where every brokered request is refused. Node builtins only, no
dependency, no network beyond loopback, no model and no key.
Evidence
pnpm typecheckclean.pnpm test: 284 files, 5383 passed, 13 skipped, exit 0. Nothing failed.tests/agent-recipe.test.ts: 31 new tests, including one that runs the realCLI against a folder with somebody's work in it and asserts the work is still
there and nothing else is.
pnpm build:agent-kit, then the real CLI into a scratch folder, thennode evals/run-evals.mjsagainst that agent: 4 of 4 checks passed.pnpm build:renderer,pnpm build:shell, then the installed-style shellsmoke on a scratch store: 85 PASS / 0 FAIL,
[smoke] all proofs passed.6b's detail showsagent.recipe.json,AGENT_BUILDER.mdandevals/travelling with the sample DASH creates on a first run. Runner retired
(status 202,
RETIRED).Evidence class: fixture tests, a real generated agent under a real
Supervisorand under its own eval harness, and an installed-style shell on ascratch store. Nothing proven on the installed build;
%APPDATA%\orchestratedashwas never opened.
Deviations, argued in the handoff
agent.recipe.jsonis deliberately not added toAGENT_KIT_PROJECT_FILES(a file this lane does not own, and the list thesmoke's
6breads).dash_agent_recipetool — documented indocs/foundation/mcp-recipe-contract.mdwith the reasoning.electron/sample-agent.tsandscripts/build-shell.mjsunder the lane'sconditional permission, plus one line each in
tests/agent-sdk.test.tsandtests/runner-telemetry.test.tsso they compile.No ADR, no migration, no contract change. Nothing in
runner/**,lib/store.ts,lib/db.ts,lib/views/**,app/**oragent-kit/template/dash-agent-sdk.mjs— lane F3's files are untouched.🤖 Generated with Claude Code