[bench] runner+hook: fail closed on governance errors

README.md

@@ -109,9 +109,9 @@ Set `BENCH_PROVIDER=claude_code` to run the pipeline on the subscription that al
 
 ## Design Decisions
 
-### Fail-Open by Design
+### Fail-Closed by Design
 
-Bench always exits with code 0. Flow control uses JSON `permissionDecision` fields (`"allow"` or `"deny"`), never exit codes. If the governance pipeline encounters an error (API timeout, malformed response, import failure), the change is allowed through with a stderr warning. This prevents Bench from becoming a blocker that stalls Claude Code on infrastructure failures. Governance should be a gate, not a wall.
+Bench always exits with code 0. Flow control uses JSON `permissionDecision` fields (`"allow"` or `"deny"`), never exit codes. If the governance pipeline cannot adjudicate a change (API timeout, malformed response, unimportable pipeline, unreadable constitution), the change is **denied**, with a stderr warning and a `pipeline_error` VETO recorded in the ledger, rather than allowed through. A broken or exploited judge must not be able to wave changes past governance, so governance is a wall when it cannot render a verdict, not a gate that swings open on failure. Recovery from a genuinely broken pipeline is an out-of-band human action (editing files directly, outside the governed tools), never an automatic pass. The lone exception is the reentrancy guard that lets a Bench-spawned governance subprocess through, so the pipeline does not recurse into itself and deadlock.
 
 ### Diff Hardening
 
@@ -139,8 +139,9 @@ With `BENCH_PROVIDER=claude_code` (set by the repo's `.claude/settings.json`),
 the local Claude Code CLI must be recent enough to recognize these IDs: Claude
 Sonnet 5 needs Claude Code v2.1.197+ and Claude Opus 4.8 needs v2.1.154+ (per
 Claude Code's model-config docs; run `claude update` to upgrade). An older CLI
-fails the stage, and under the runner's fail-open policy that surfaces as a
-flagged `pipeline_error` rather than a hard block, so keep the CLI current.
+fails the stage, and under the runner's fail-closed policy that blocks the
+change (a flagged `pipeline_error` VETO) until the pipeline can run, so keep the
+CLI current.
 
 ## Built With
 

cli/commands.py

@@ -23,7 +23,12 @@
     ConstitutionError,
     load_constitution_snapshot,
 )
-from utils.stats import compute_ledger_stats, entry_has_pipeline_error, pct
+from utils.stats import (
+    compute_ledger_stats,
+    entry_has_pipeline_error,
+    entry_verdict,
+    pct,
+)
 from utils.viewer import generate_viewer_html
 
 _HASH_PREFIX_LEN: int = 12
@@ -80,8 +85,7 @@ def cmd_ledger(show_all: bool = False, vetoes_only: bool = False) -> int:
     if vetoes_only:
         filtered = [
             e for e in filtered
-            if isinstance(e.get("oracle"), dict)
-            and e["oracle"].get("verdict") == "VETO"
+            if entry_verdict(e) == "VETO"
         ]
 
     if not filtered:
@@ -233,8 +237,10 @@ def _print_entry_line(entry: dict) -> None:
 
     oracle: Any = entry.get("oracle")
     oracle_dict: dict = oracle if isinstance(oracle, dict) else {}
-    verdict: str = str(oracle_dict.get("verdict") or "").strip()
+    verdict: str = str(entry_verdict(entry) or "").strip()
     if not verdict:
+        # No recorded verdict: an older fail-open entry (pipeline error with
+        # no adjudicated verdict) reads as FAIL-OPEN for historical accuracy.
         if entry_has_pipeline_error(entry):
             verdict = "FAIL-OPEN"
         else:

hooks/pre-tool-use.py

@@ -10,10 +10,12 @@
   * Exit code is ALWAYS 0. Flow control is via JSON, not exit codes.
     (Exit-2 would cause Claude Code to stall.)
   * All structured output goes to stdout. All diagnostics go to stderr.
-  * The hook fails open: any internal error returns 'allow' so a broken
-    governance layer never blocks the developer. Failures are logged to
-    stderr for the developer to notice. The runner also fails open on its
-    own internal errors; this wrapper is defense in depth.
+  * The hook fails closed: if governance cannot run (pipeline import
+    failure) or the hook itself errors, the change is denied, not allowed,
+    so a broken or exploited pipeline cannot wave changes through. Failures
+    are logged to stderr. The sole exception is the reentrancy guard for a
+    Bench-spawned governance subprocess (the claude_code provider), which is
+    allowed so the pipeline does not recurse into itself and deadlock.
 """
 
 import json
@@ -321,22 +323,34 @@ def main() -> int:
         diff_info: dict[str, Any] = extract_diff_info(tool_name, tool_input)
         if run_governance_pipeline is None:
             verdict: dict[str, Any] = {
-                "verdict": "PASS",
-                "reason": "Pipeline unavailable (import failed) — failing open",
-                "remediation": None,
+                "verdict": "VETO",
+                "reason": (
+                    "Governance pipeline is unavailable (import failed); "
+                    "cannot adjudicate. Failing closed."
+                ),
+                "remediation": (
+                    "The pipeline failed to import (see stderr). Fix the import "
+                    "error, then retry. Changes are blocked until governance can "
+                    "run. Emergency recovery is a human editing files directly, "
+                    "outside Claude Code's governed tools."
+                ),
             }
         else:
             verdict = run_governance_pipeline(tool_name, tool_input, diff_info)
         response: dict[str, Any] = build_response_from_verdict(verdict)
 
-    except Exception as e:  # fail-open: governance must never block on its own bug
+    except Exception as e:  # fail-closed: an unadjudicated change must not pass
         print(
-            f"[bench hook] internal error, failing open: {type(e).__name__}: {e}",
+            f"[bench hook] internal error, failing closed: {type(e).__name__}: {e}",
             file=sys.stderr,
         )
         traceback.print_exc(file=sys.stderr)
-        response = build_allow_response(
-            "Bench governance: hook error, failing open. See stderr."
+        response = build_deny_response(
+            "BENCH VETO: governance hook error; the change could not be "
+            "adjudicated. Failing closed.",
+            "Remediation: the hook raised an unexpected error (see stderr). Fix "
+            "it, then retry. Emergency recovery is a human editing files "
+            "directly, outside Claude Code's governed tools.",
         )
 
     json.dump(response, sys.stdout)