Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ Everything you need to understand, use, and extend Chock. New here? Start with
| [Core Concepts](concepts.md) | Learn the vocabulary: artifacts, manifests, surfaces, coverage |
| [CLI Reference](cli-reference.md) | Look up every command, flag, and example |
| [Authoring Policies](authoring-policies.md) | Write rules, hooks, skills, and subagents |
| [Script-Backed Gates](script-backed-gates.md) | Declare a check no gate `kind` can express, and the contract its script runs under |
| [Enforcement Surfaces](enforcement-surfaces.md) | See the eight surfaces and the per-agent coverage matrix |
| [Agentic-Risk Coverage](agentic-risk-coverage.md) | Problem-first index: what Chock stops, at which honest tier — and what it doesn't |
| [Agent Plugins](agent-plugins.md) | Package policies for the open standard — and what that does not buy you |
Expand Down
9 changes: 3 additions & 6 deletions docs/authoring-policies.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,12 +83,9 @@ hook:
Rules for guard design:

- **Deterministic only** — no LLM calls, no network in the gate itself.
- **Prefer the declarative gate** — the compiler vendors the runner, and a `hook.gate` wins
where a policy has both. A check needing *both revisions* of a file cannot be a `kind`: that
policy ships `implementations/<policy_id>-pre-commit.{py,sh}` (or `-pre-push`), and a shim
is emitted for it. No arguments (read the change from `git show :path` and
`git show HEAD:path`), cwd at the repo root, exit code is the verdict, stdlib only. Distinct
from `implementations/<policy_id>.sh`, the argv guard PreToolUse and the eval runner invoke.
- **Prefer the declarative gate** — the compiler vendors the runner, so a `hook.gate` needs no
script of yours. A check no `kind` can express declares `hook.script` instead, which a `rule`
may also carry: see [Script-Backed Gates](script-backed-gates.md).
- Keep `message` actionable; the runner prints it to stderr on block.

### `skill` — an on-demand procedure
Expand Down
76 changes: 76 additions & 0 deletions docs/script-backed-gates.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Script-Backed Gates

A declarative `hook.gate` answers one question about the diff: does a regex match, is this ref
protected, is this dependency allowed. Some checks cannot be written that way. A check that
compares **both revisions** of a file — what an element carried before against what it carries
now — gets no answer from knowing which side of the diff a line is on.

Such a policy ships its own script and declares the events that run it.

## The declaration

```yaml
# manifest.yaml
artifact: rule
enforcement: block
effects: [read_only]
rule:
text: |
never(erase): a name an element already had
hook:
script:
"on": [commit] # commit | push, at least one
```

```
.agents/policies/<id>/
└── implementations/
└── <id>-pre-commit.py # or -pre-push, or .sh
```

`on` is **authoritative**. The compiler wires exactly the events it names — never what the
directory happens to contain — and `chock check` fails (**DET-5**) when a declared event has no
script, or a shipped script is not declared. Each mismatch fails differently and neither is
silent: the first would point a consumer's hook at a path that is not there, the second leaves a
policy enforcing less than its directory suggests.

## The contract

- **No arguments.** At pre-commit the guard already has both revisions: `git show :path` is the
staged blob and `git show HEAD:path` its predecessor. Nothing is passed in.
- **The working directory is the repo root.**
- **The exit code is the verdict** — 0 allows, anything else refuses. A shim whose script has
gone missing exits 2 rather than 0: a gate that cannot find its own guard must refuse.
- **Stdlib only**, like the vendored runner. The script is copied into every adopting repo, so
it may not import anything that repo does not already have.
- **No `message`.** The script prints its own refusal, with detail no manifest string could
hold — which file, which element, what it carried before.

## A rule may declare one

`artifact: hook` has no payload for rule text, and its ambient surface is a machine rendering of
the gate spec. A script-backed check has no spec to render, so forcing `artifact: hook` would
trade the hand-authored rule an agent reads for a near-empty stub. A `rule` may therefore carry
a `hook` holding only a `script`: the script is not a second artifact but the same control on a
second surface.

A **declarative gate** stays `artifact: hook`. A rule carrying one is an error, so no artifact
ever has two answers to what enforces it.

## Not a command guard

Two different files, two different contracts, and confusing them is silent:

| file | invoked by | gets | verdict |
| :--- | :--- | :--- | :--- |
| `implementations/<id>.sh` | in-agent PreToolUse hooks, the eval runner | a command's argv | exit code |
| `implementations/<id>-pre-commit.py` | the installed git hook | nothing | exit code |

The eval runner excludes the event-named scripts, because handing one an eval case's command
would score a verdict it never gave. A policy backed only by an event script therefore stays
tier 3 in `chock check --only evals` until the runner can stage a tree for it.

## See also

- [Authoring Policies](authoring-policies.md) — the other artifact types and their manifests
- [Enforcement Surfaces](enforcement-surfaces.md) — what `git-hook` means once wired
3 changes: 2 additions & 1 deletion spec/enforcement-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Every spec invariant must appear in this matrix with the check(s) that enforce i
|---|---|---|---|---|---|
| SEC-1 | `spec/policy-spec.md` §10 | `the validator`: `check_security_baseline()` requires `security.content_instructions == "never-obey"` for all artifact types | error | Applies to skill, hook, rule, workflow, subagent | engine |
| SEC-2 | `spec/policy-spec.md` §10 | `the validator`: `check_security_baseline()` — static scan of `scripts/` under code/hybrid skills and hook implementations for LLM/network calls | error | Network calls warn unless `network` effect is declared with `verify`/`block` enforcement and approval wiring (EFF-1) | engine |
| SEC-3 | `spec/policy-spec.md` §10 | `the validator`: `validate_yaml_against_schema()` enforces `manifest.hook.json`'s requirement that `gate.message` (1-1000 chars) be present whenever `hook.gate` is present | error | Message must be actionable (verified by eval suite, not regex). The requirement is unconditional, not per action: `message` is in the schema's `gate.required` under `additionalProperties: false`. `gate.action` is a `const: block`; `verify` is a value of `enforcement` (EFF-1), a different field | engine |
| SEC-3 | `spec/policy-spec.md` §10 | `the validator`: `validate_yaml_against_schema()` enforces `manifest.hook.json`'s requirement that `gate.message` (1-1000 chars) be present whenever `hook.gate` is present | error | Message must be actionable (verified by eval suite, not regex). The requirement is unconditional, not per action: `message` is in the schema's `gate.required` under `additionalProperties: false`. `gate.action` is a `const: block`; `verify` is a value of `enforcement` (EFF-1), a different field. `hook.script` (§7) carries no `message` and is not covered here: a script prints its own refusal with detail no manifest string could hold, so there is nothing for the schema to require | engine |
| SEC-4 | `spec/policy-spec.md` §10 | `the validator`: `_scan_text_surfaces()` (called from `check_security_baseline()`) covers all `.md`/`.yaml`/`.txt` files under artifact folders plus manifest text fields and eval prompts | error | Adversarial eval cases downgraded to `info` when category is `adversarial`/`security`. Private in `checks_security.py` — the public name `scan_text_surfaces()` this row previously gave does not exist | engine |
| SEC-5 | `spec/policy-spec.md` §10 | `the validator`: `check_ambient_tier()` requires `trust_tier >= community` or `ambient_override: true` for rules wired into ambient files | error | Manual migration required for existing `sandbox` ambient rules | engine |
| SEC-6 | `spec/policy-spec.md` §10 | `the validator`: `check_eval_first()` requires ≥1 `adversarial`/`security` eval case when `security.processes_external_content: true` | error | `policy-init` skill and template set the flag when content is mined | engine |
Expand All @@ -35,6 +35,7 @@ Every spec invariant must appear in this matrix with the check(s) that enforce i
| DET-2 | `spec/policy-spec.md` §11 | `the registry`: sha256 hashes per script; `the validator`: `check_script_integrity()` verifies for production/verified+ artifacts | error | Stale registry triggers a rescan, not a silent pass | engine |
| DET-3 | `spec/optimization.md` §2 | `optimize` skill: `determinize_edit` edit type; `the validator`: `check_determinization_heuristic()` flags NL skills with regex/command sequences | info | Human review required for `determinize_edit` | engine |
| DET-4 | `spec/policy-spec.md` §12 + `spec/methodology.md` | `src/chock/packs/_skills/policy-init/references/taxonomy.md` requires a pre-walk `determinism_scan` that splits deterministic_parts from judgment_parts; mechanical parts route to a code/hybrid skill with a committed script. `policy-init` eval cases verify the split and routing. | eval | Enforced by eval suite; post-hoc backstop is DET-3 | n/a |
| DET-5 | `spec/script-backed-gates.md` | `the validator`: `check_script_events()` — every event `hook.script.on` declares has a matching `implementations/<id>-<event>.{sh,py}`, and every such script on disk is declared | error | Both directions, because each fails differently: an undeclared script wires nothing (the policy enforces less than its directory suggests) and a declared one with no file would point a consumer's hook at a path that is not there. The compiler wires `on`, never the directory listing | engine |

## Effects and approvals (EFF) — Phase 3

Expand Down
4 changes: 2 additions & 2 deletions spec/policy-spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ wiring:

```
.agents/policies/<id>/
├── manifest.yaml
├── manifest.yaml # + implementations/ when it declares hook.script
└── evals/suite.yaml
```

Expand Down Expand Up @@ -117,7 +117,7 @@ require(evals/suite.yaml): minimum 3 cases across:
target: policy, not agent
default metric: pass_rate

## 7. Gate definition (hooks only)
## 7. Gate definition (`hook.gate`; for `hook.script` see `spec/script-backed-gates.md`)

```yaml
gate:
Expand Down
49 changes: 49 additions & 0 deletions spec/script-backed-gates.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Script-backed gates

A check no gate `kind` can express — one needing **both revisions** of a file, where which
side of the diff a line sits on answers nothing — is declared as `hook.script` rather than
`hook.gate`, and run by the installed git hook.

```yaml
hook:
script:
"on": [commit] # commit | push, >=1
```

```
.agents/policies/<id>/implementations/<id>-pre-commit.{sh,py} # or -pre-push
```

No `kind`, no `params`, no `message`: the behaviour is the script's, and it prints its own
refusal with detail no manifest string could hold. The contract is empty on purpose — no
arguments, cwd at the repo root, exit code is the verdict, stdlib only — because at
pre-commit the script already has both revisions (`git show :path` and `git show HEAD:path`).

`on` is authoritative. The compiler wires exactly the events it names, never what the
directory happens to contain.

**DET-5**: a declared event MUST have a script on disk, and a shipped git-event script MUST
be declared. Either way round is an error, and each fails differently: one would point a
consumer's hook at a file that is not there, the other leaves a policy enforcing less than
its directory suggests.

## Which artifact may declare it

Either a `hook` or a `rule`. `artifact: hook` has no payload for rule text and its ambient
surface is a machine rendering of the gate spec; a script-backed check has no spec to render,
so requiring `artifact: hook` would trade the rule an agent reads for a near-empty stub. A
`rule` may therefore carry a `hook` holding only a `script` — the script is not a second
artifact but the same control on a second surface.

A declarative `hook.gate` stays `artifact: hook`. A rule carrying one is an error
(`manifest_payload`), so no artifact has two answers to what enforces it.

## Not a command guard

`implementations/<id>.sh` is invoked with a command's argv by the in-agent PreToolUse hooks
and the eval runner. An event script is invoked with nothing. The eval runner excludes the
event-named scripts, because handing one an eval case's command would score a verdict it
never gave; a policy backed only by an event script stays tier 3 in `chock check --only
evals` until the runner can stage a tree for it.

Prose, examples and the authoring path: `docs/script-backed-gates.md`.
7 changes: 4 additions & 3 deletions src/chock/compile/emitters/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,10 @@
#: Suffixes a guard implementation may carry, in the order discovery prefers them.
GUARD_SUFFIXES = (".sh", ".py")

#: Git events a policy may back with its own script, named
#: `implementations/<policy_id>-<event>.{sh,py}`. A declarative gate takes precedence.
SCRIPT_EVENTS = ("pre-commit", "pre-push")
#: Git events a policy may back with its own script, each mapped to the filename segment
#: that names it: `implementations/<policy_id>-<segment>.{sh,py}`. The manifest's
#: `hook.script.on` speaks the keys; the files on disk speak the values.
SCRIPT_EVENTS = {"commit": "pre-commit", "push": "pre-push"}


def policy_rel_path(policy_dir: Path) -> str:
Expand Down
47 changes: 26 additions & 21 deletions src/chock/compile/emitters/git_hook.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,28 +35,33 @@ def _emit_shims(output_dir: Path, policy_id: str, events: list[str]) -> list[Pat
return emitted


def _script_guards(policy_dir: Path, policy_id: str) -> dict[str, str]:
"""The events this policy backs with a script, mapped to that script's file name."""
def _script_name(policy_dir: Path, policy_id: str, segment: str) -> str | None:
"""The file the hook would run for `segment`, or None when the policy ships none."""
impl = policy_dir / "implementations"
found: dict[str, str] = {}
for event in SCRIPT_EVENTS:
for suffix in GUARD_SUFFIXES:
name = f"{policy_id}-{event}{suffix}"
if (impl / name).exists():
found[event] = name
break
return found


def _emit_script_shims(policy_dir: Path, output_dir: Path, policy_id: str) -> list[Path]:
"""Emit one shim per script-backed event. The guard reads the change from git itself."""
guards = _script_guards(policy_dir, policy_id)
if not guards:
return []
for suffix in GUARD_SUFFIXES:
name = f"{policy_id}-{segment}{suffix}"
if (impl / name).exists():
return name
return None


def declared_script_events(manifest: dict[str, Any]) -> list[str]:
"""The events `hook.script` declares. This list is what gets wired, not what is on disk."""
declared = ((manifest.get("hook") or {}).get("script") or {}).get("on") or []
return [event for event in SCRIPT_EVENTS if event in declared]


def _emit_script_shims(policy_dir: Path, output_dir: Path, policy_id: str, events: list[str]) -> list[Path]:
"""Emit one shim per declared event. The guard reads the change from git itself."""
rel = policy_rel_path(policy_dir)
emitted: list[Path] = []
for event, script in guards.items():
shim = output_dir / f"git-{event}.sh"
for event in events:
segment = SCRIPT_EVENTS[event]
# A declared event with no script on disk is a validation error, not a shim that
# fails at commit time in someone else's repo.
if (script := _script_name(policy_dir, policy_id, segment)) is None:
continue
shim = output_dir / f"git-{segment}.sh"
rendered = SCRIPT_SHIM_TEMPLATE.replace("__POLICY_ID__", policy_id).replace(
"__GUARD_REL__", f"{rel}/implementations/{script}"
)
Expand All @@ -77,8 +82,8 @@ def emit(policy_dir: Path, output_dir: Path, manifest: dict[str, Any]) -> list[P
spec = build_gate_json(policy_dir, repo_root)
if spec is None:
# No declarative gate. A policy whose check needs more than a regex over the diff
# ships its own script instead, and the shim runs that.
return _emit_script_shims(policy_dir, output_dir, policy_id)
# ships its own script and declares the events it runs at; the shim runs that.
return _emit_script_shims(policy_dir, output_dir, policy_id, declared_script_events(manifest))

hook_events = [e for e in spec.get("on", []) if e in ("commit", "push")]
if not hook_events:
Expand Down
2 changes: 1 addition & 1 deletion src/chock/eval/suites.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

#: A script named for a git event runs at that event with no argv and reads the change
#: from git itself. Handing it an eval case's command would score a verdict it never gave.
_EVENT_STEMS = tuple(f"-{event}" for event in SCRIPT_EVENTS)
_EVENT_STEMS = tuple(f"-{segment}" for segment in SCRIPT_EVENTS.values())


def _suite_doc(policy_dir: Path) -> dict[str, Any]:
Expand Down
27 changes: 22 additions & 5 deletions src/chock/validation/checks_manifest_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,25 @@ def _check_manifest_id_folder(artifact_dir: Path, manifest: dict[str, Any], repo
report.add(Finding(str(_manifest_ref(artifact_dir)), "manifest_id_folder", "error", str(exc)))


def _script_only_hook(manifest: dict[str, Any]) -> bool:
"""True when the hook payload declares a script and no declarative gate."""
hook = manifest.get("hook") or {}
return bool(hook.get("script")) and not hook.get("gate")


def _check_manifest_payload(artifact_dir: Path, manifest: dict[str, Any], report: Report) -> None:
artifact = manifest.get("artifact")
allowed = _PAYLOADS.get(artifact)
if allowed is None:
return

# A rule may also declare the script that enforces it. The script is not a second
# artifact: it is the same control on a second surface, and the rule text is what the
# agent reads, which a hook artifact has no payload for. A declarative gate is still
# artifact: hook -- a rule carrying one would have two answers to what enforces it.
if artifact == "rule" and _script_only_hook(manifest):
allowed = allowed | {"hook"}

present = {k for k in manifest if k in _ALL_PAYLOAD_KEYS}
disallowed = present - allowed
if disallowed:
Expand All @@ -51,7 +64,9 @@ def _check_manifest_payload(artifact_dir: Path, manifest: dict[str, Any], report
)
return

own = present & allowed
# The script declaration rides along; it is not the artifact's own payload, so it does
# not count toward the one-payload rule it was just admitted past.
own = (present & allowed) - ({"hook"} if artifact == "rule" else set())
if len(own) != 1:
report.add(
Finding(
Expand All @@ -68,18 +83,20 @@ def _check_manifest_block_needs_gate(artifact_dir: Path, manifest: dict[str, Any
if enforcement not in {"block", "verify"}:
return

hook_gate = (manifest.get("hook") or {}).get("gate")
hook = manifest.get("hook") or {}

if hook_gate:
_validate_gate(hook_gate, str(_manifest_ref(artifact_dir)), report, tool_use_allowed=True)
if hook.get("gate"):
_validate_gate(hook["gate"], str(_manifest_ref(artifact_dir)), report, tool_use_allowed=True)
return
if hook.get("script"):
return # the script refuses at the declared event; checks_script_events pins it to disk

report.add(
Finding(
str(_manifest_ref(artifact_dir)),
"manifest_block_needs_gate",
"error",
f"enforcement is '{enforcement}' but no hook.gate definition found",
f"enforcement is '{enforcement}' but neither a hook.gate nor a hook.script was declared",
)
)

Expand Down
Loading