Skip to content

feat(guards): lesson guards — a recorded mistake warns before it repeats (G1) - #179

Merged
kevintseng merged 3 commits into
mainfrom
feat/g1-lesson-guards
Aug 16, 2026
Merged

feat(guards): lesson guards — a recorded mistake warns before it repeats (G1)#179
kevintseng merged 3 commits into
mainfrom
feat/g1-lesson-guards

Conversation

@kevintseng

Copy link
Copy Markdown
Contributor

What

G1 of the work-topology arc: the "stop repeating mistakes" thesis gets teeth. A failure-shaped lesson can now be promoted — through the same human review queue as every other dreamer proposal — into a guard: a regex over the tool input that warns at the exact moment the recorded mistake is about to repeat.

Propose (dreamer, new guard stage): for each active failure-lesson without a guard, one LLM call proposes {tool, pattern, message} plus its own evidence — at least two inputs that must trigger and two that must stay silent. A mechanical validator then holds the gate: the pattern must compile, must not match the empty string, must not match a benign-probe list (git status, npm test, …: a guard that nags on routine work is how safety features die), and the evidence is executed, not trusted. Failures are reported in skipped[], never staged. The stage runs after compaction (digests keep first claim on the LLM budget), caps at 3 guards/run, and never proposes twice for a lesson with a pending guard.

Accept (applyProposal, new guard kind): re-verifies the spec with no model in the loop (an acceptance months after staging must still hold), then writes metadata.guard onto the source lesson inside the same status-guarded transaction pattern the relation kind uses. Nothing is created or archived; v1 is warn-only by policy whatever the model said — the schema carries an action field so per-guard blocking can arrive once measured fire accuracy justifies it. The review card shows tool, pattern, message and the executed evidence; listProposals names guard rows instead of treating them as corrupt digests.

Evaluate (hooks, zero LLM / zero network): a new guard-check hook (PreToolUse Bash — the manifest's seventh hook) tests commands; pre-edit-recall tests Edit/Write against the path plus incoming content, deliberately outside its recall throttle — a dangerous edit is dangerous every time. A hit injects a fenced reference block carrying the lesson's [mem:id] citation handle (heeding a guard is citable — closing the loop with #178), and every fire is counted (fires, last_fired_at) so guard ROI is measurable. Guard failure of any kind — corrupt pattern, missing column, locked DB — degrades to silence: the guard system can never block the user's work.

One definition of "valid guard" and "does it fire" lives in a new zero-import leaf (src/core/guards.ts), mirrored to the hooks by generate-hook-core, so the proposer, the accept path and both evaluation points cannot drift.

Design decisions (approved before implementation)

  • Storage: metadata.guard on the source lesson (1:1 provenance, shared lifecycle) — not a new entity type.
  • v1 semantics: warn-only; schema keeps action for later per-guard block.
  • Evaluation points: Bash + Edit|Write (hook count 6 → 7; all doc claims updated in-PR).
  • Guards fire across projects: the mistakes worth guarding (pipe-eaten exit codes, git checkout -- on a dirty tree) are rarely project-local; the warning names its source lesson either way.

Verification

  • node scripts/run-tests-isolated.mjs → exit 0, 142 files / 2126 tests (net +28).
  • npm run typecheck / npm run verify:release → exit 0 (doc-claims validates the derived hook counts; audit baseline: 5 re-keys + 1 new silent-pass C1 entry for the e2e file).
  • New coverage: leaf units (validator failure modes incl. broad-pattern probes and executed evidence; matcher tool/action/corrupt-regex behavior), dreamer staging/validation-gate/NOOP/pending-dedup/apply/re-verify/archived-lesson (LLM mocked at the fetch layer), spawned-hook e2e for both evaluation points (fire + handle + fenced block + fire count; silence for non-match/disabled/wrong-tool/no-DB), and a hook-output contract case with a new patchMetadata harness step (only the accept path can write metadata.guard in production).
  • Break-tests (against the committed baseline): gutting the benign-probe check → × rejects a pattern broad enough to fire on routine work; muting guards under the recall throttle → × an Edit guard fires … even when the recall throttle would skip the file; restored → green.

Notes

  • i18n: 6 new dashboard keys (guard card + telemetry flow) in all 11 locales; key-set parity holds both directions.
  • The dreamer's guard stage consumes the same --max-llm-calls budget and reports through the same skipped[] channel as every other stage.

…epeat (G1)

The dreamer gains a guard stage: each failure-shaped lesson (Error /
Root cause / Fix) can become a {tool, pattern, message} trigger — a
regex over the Bash command or Edit/Write content about to run. Every
proposal is verified mechanically before staging (pattern compiles,
matches none of a benign-probe list, and passes its own attached
evidence: 2+ inputs that must trigger and 2+ that must stay silent,
executed rather than trusted), then joins the same human review queue as
every other proposal kind. Acceptance re-verifies with no model in the
loop and writes metadata.guard onto the source lesson — creating
nothing, archiving nothing, warn-only by v1 policy whatever the model
proposed (the schema carries an 'action' field so per-guard blocking can arrive
once measured fire accuracy justifies it).

At runtime, guards are plain regex tests — no LLM, no network: a new
guard-check hook (PreToolUse Bash, the manifest's seventh hook) covers
commands, and pre-edit-recall covers Edit/Write, deliberately OUTSIDE
its recall throttle — a dangerous edit is dangerous every time. A hit
injects the lesson's warning as a fenced reference block carrying the
lesson's [mem:id] citation handle (heeding a guard can be cited, closing
the loop with R1), and every fire is counted on the guard so one that
never fires or fires constantly surfaces for review. Guard failure of
any kind degrades to silence: the guard system can never block the
user's work.

The shared shapes live in a new zero-import leaf (src/core/guards.ts),
mirrored to the hooks by generate-hook-core, so the proposer, the accept
path and both evaluation points agree on what a valid guard is. The
review card renders tool, pattern, message and the executed evidence;
listProposals names guard rows instead of treating them as corrupt
digests; docs and the hook-count claims move in the same commit.
Coverage: leaf units, dreamer staging/validation/dedup/apply (LLM mocked
at the fetch layer), spawned-hook e2e for both evaluation points, and a
hook-output contract case (with a metadata patch step, since only the
accept path can write metadata.guard in production).
…Windows locks open files

The no-database scenario removed graph.db while this test still held it
open from beforeEach. Linux and macOS unlink open files, so the suite
was green everywhere except both Windows legs, where rmSync throws
EBUSY. closeDatabase() first; the afterEach double-close is already
guarded.
@kevintseng
kevintseng merged commit edf6e6d into main Aug 16, 2026
13 checks passed
@kevintseng
kevintseng deleted the feat/g1-lesson-guards branch August 16, 2026 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant