From 83d1be89be775ad05fdf0e29e0f253771e46fd33 Mon Sep 17 00:00:00 2001 From: Sollan Systems Date: Sat, 25 Jul 2026 12:20:15 -0400 Subject: [PATCH 1/3] docs(spec): correct ST3 5.4 against ruflo's real surface The original 5.4 sketched ruflo.orchestrate() returning .visible/.holdout/ .merged_diff/.agent_trails/.converged/.rounds/.criteria_met. No such API exists: ruflo is a Node CLI with no Python package and no result object with those fields. The section also assumed a swarm-terminal hook to register the gate on; the plugin HookEvent enum carries no swarm-level terminal event (swarm:consensus-reached appears only in ruflo's docs). Rewritten as a dated correction against the verified surfaces: the blocking hive-mind spawn --claude invocation, the .swarm/ JSON files, the memory export's sparc-phases criteria vocabulary, and the supervisor-owned interrupt flag. Research dossier: review/recipes/2026-07-25-ruflo-api-research.md. --- .../2026-06-30-st3-integration-adapters.md | 67 +++++++++++++++---- 1 file changed, 53 insertions(+), 14 deletions(-) diff --git a/docs/superpowers/specs/2026-06-30-st3-integration-adapters.md b/docs/superpowers/specs/2026-06-30-st3-integration-adapters.md index c12ad36..e35eff8 100644 --- a/docs/superpowers/specs/2026-06-30-st3-integration-adapters.md +++ b/docs/superpowers/specs/2026-06-30-st3-integration-adapters.md @@ -289,30 +289,69 @@ peeked" case OpenHands can't itself catch. ### 5.4 ruflo swarm → acceptance gate *(alternate)* +> **Corrected 2026-07-25.** The original §5.4 sketched `ruflo.orchestrate()` +> returning `.visible` / `.holdout` / `.merged_diff` / `.agent_trails` / +> `.converged` / `.rounds` / `.max_rounds` / `.criteria_met`. **No such API +> exists** — ruflo is a Node CLI with no Python package, no `orchestrate()` entry +> point and no result object with those fields (verified against `ruvnet/ruflo` +> at 3.32.9). The original text also assumed a swarm-terminal hook to register +> the gate on; the plugin `HookEvent` enum has no swarm-level terminal event +> (`swarm:consensus-reached` occurs only in ruflo's documentation, never in its +> implementation). Both are replaced below by the verified surfaces. Research +> dossier: `review/recipes/2026-07-25-ruflo-api-research.md`. + **Composes:** the ORCHESTRATE tier (multi-agent swarm). A swarm's terminal is "the coordinator decided the objective is met" — pure self-report across N agents. Loop Engineer adds a single acceptance gate the swarm must pass *as a whole*. -Snippet outline: register the gate as the swarm's terminal hook (ruflo exposes -hooks / an MCP coordination server), so no individual agent can declare the swarm -done — the acceptance gate does: +Seam: `ruflo hive-mind spawn "" --claude` spawns the Claude Code CLI as +the swarm's execution body and **blocks** until that child exits, mapping `exit 0` +to success. So the gate belongs in the **host-side supervisor** that launches the +CLI and reads the run directory afterwards — no individual agent can declare the +swarm done, because nothing inside the swarm writes the terminal. Optionally a +second, independent gate runs inside the child via `hooks/stop_firewall.py` +registered as a Claude Code `Stop` hook. ```python -swarm_result = ruflo.orchestrate(objective=spec, agents=[...]) -gate = decide(visible=swarm_result.visible, holdout=swarm_result.holdout) -ac = anticheat_scan.scan(diff=swarm_result.merged_diff, - trajectory=swarm_result.agent_trails) +run = subprocess.run(["npx", "ruflo@3.32.9", "hive-mind", "spawn", objective, + "--claude", "--non-interactive"], cwd=ws) # blocks + +obs = observe(ws) # .swarm/{state.json,tasks/*,agents/*,coordination/*} + memory export +gate = decide(visible=visible_checks(ws), holdout=holdout_checks(ws)) +ac = anticheat_scan.scan(diff_text=host_git_diff(ws), # ruflo exposes no merged diff + trajectory=agent_trails(obs)) # agents' touchedPaths + consensus rows terminal = to_terminal_state( - outcome=EngineOutcome(reached_end=swarm_result.converged, external_error=None, - budget_exhausted=swarm_result.rounds >= swarm_result.max_rounds, - human_abort=False, artifacts=swarm_result.artifacts), - gate_verdict=gate, anticheat=ac, criteria_met=swarm_result.criteria_met, + outcome=EngineOutcome( + reached_end=run.returncode == 0 + and obs["state"]["status"] in {"ready", "initialized", "stopped"}, + external_error=None if run.returncode == 0 or completed_tasks(obs) else "swarm blocked", + budget_exhausted=autopilot_capped(ws), # `ruflo autopilot status --json` + human_abort=supervisor_was_interrupted(), # NEVER from the exit code — see below + artifacts=evidence_paths(ws)), + gate_verdict=gate, anticheat=ac, + criteria_met={cid: proven.get(cid) for cid in declared_criteria(obs["export"])}, ) ``` -Mapping specialization: swarm non-convergence within max rounds → `FailedBudget`; -a criterion no agent was assigned → `FailedSpecGap` (the swarm literally never -worked on it — a failure mode a self-reporting coordinator hides). +Criteria vocabulary comes from the memory export's `sparc-phases` `spec-*` entry +(`acceptanceCriteria`); their **truth** comes from the withheld holdout checks. +The `sparc-gates` namespace holds the swarm's own per-phase `pass` rows and +`truthScore` — recorded as observation, never trusted as a verdict. + +Mapping specialization: autopilot `--max-iterations` / `--timeout` reached without +a green gate → `FailedBudget`; a declared criterion no check covers → +`FailedSpecGap` (the swarm literally never worked on it — a failure mode a +self-reporting coordinator hides). + +Two traps the recipe must encode: ruflo's SIGINT path calls `process.exit(0)`, so +`AbortedByHuman` must come from the supervisor's own signal handler and never from +the exit code; and `ruflo verify` is **install-integrity** (SHA-256 + Ed25519 over +the installed artifact), not a run verdict, so it must never be wired as the gate. + +Because a live run needs Node, the `claude` binary and credentials, the shipped +example replays a committed `.swarm/` recording by default (`--live` opts in) — +stated plainly, with the gate/projection/emit/doctor/metrics path executing for +real. --- From 398d22c32cddb93a8f339bdb75d4333a6326527a Mon Sep 17 00:00:00 2001 From: Sollan Systems Date: Sat, 25 Jul 2026 12:20:27 -0400 Subject: [PATCH 2/3] feat(recipes): ruflo swarm to acceptance gate A host-side supervisor around ruflo's blocking swarm CLI, replacing the swarm's self-report with a real gate. ruflo hive-mind spawn --claude spawns the Claude Code CLI as the swarm's body and maps exit 0 to success; the sparc-gates memory namespace holds the swarm's own per-phase pass rows and truthScore. The supervisor reads only host-side surfaces, runs the withheld holdout split plus the anticheat trajectory sweep over the agent trails, projects through to_terminal_state and records via loop.emit. Zero ruflo changes, zero loop/ kernel changes. Three traps pinned by tests: ruflo exits 0 on Ctrl-C, so AbortedByHuman comes from the supervisor's own signal flag and never from the exit code; ruflo verify is install-integrity, not a run verdict, so it is never wired as the gate; the sparc-gates self-verdict stays green under sabotage while the gate refuses. Plus the false-completion invariant and a FailedSpecGap demo for a declared criterion no check covers. Deterministic and credential-free: the example replays the committed examples/ruflo-gate/fixture/ recording by default (--live opts into the real invocation), stated plainly in the README. Only the swarm is recorded - the gate, projection, emit, doctor and metrics path all execute for real. The opt-in live schema-drift alarm is skipped unless LOOP_RUFLO_LIVE=1. Verified against ruflo 3.32.9 (2026-07-25). --- docs/integrations/ruflo.md | 146 ++++++++ examples/ruflo-gate/README.md | 82 +++++ .../.claude-flow/metrics/v3-progress.json | 20 ++ .../hive-mind-prompt-swarm-2026-07-25.txt | 13 + .../fixture/.swarm/agents/agent-coder-02.json | 12 + .../fixture/.swarm/agents/agent-coder-03.json | 12 + .../fixture/.swarm/agents/agent-queen-01.json | 12 + .../.swarm/agents/agent-reviewer-04.json | 12 + .../consensus-2026-07-25T09-31-52Z.json | 17 + .../fixture/.swarm/memory-export.json | 48 +++ examples/ruflo-gate/fixture/.swarm/state.json | 9 + .../fixture/.swarm/tasks/task-001-spec.json | 9 + .../.swarm/tasks/task-002-implement.json | 9 + .../.swarm/tasks/task-003-logging.json | 9 + .../fixture/.swarm/tasks/task-004-review.json | 9 + .../ruflo-gate/fixture/dedupe-report.json | 10 + examples/ruflo-gate/fixture/dedupe.log | 16 + .../ruflo-gate/fixture/src/import_contacts.py | 24 ++ examples/ruflo-gate/swarm_example.py | 321 ++++++++++++++++++ scripts/test_ruflo_recipe.py | 199 +++++++++++ 20 files changed, 989 insertions(+) create mode 100644 docs/integrations/ruflo.md create mode 100644 examples/ruflo-gate/README.md create mode 100644 examples/ruflo-gate/fixture/.claude-flow/metrics/v3-progress.json create mode 100644 examples/ruflo-gate/fixture/.hive-mind/sessions/hive-mind-prompt-swarm-2026-07-25.txt create mode 100644 examples/ruflo-gate/fixture/.swarm/agents/agent-coder-02.json create mode 100644 examples/ruflo-gate/fixture/.swarm/agents/agent-coder-03.json create mode 100644 examples/ruflo-gate/fixture/.swarm/agents/agent-queen-01.json create mode 100644 examples/ruflo-gate/fixture/.swarm/agents/agent-reviewer-04.json create mode 100644 examples/ruflo-gate/fixture/.swarm/coordination/consensus-2026-07-25T09-31-52Z.json create mode 100644 examples/ruflo-gate/fixture/.swarm/memory-export.json create mode 100644 examples/ruflo-gate/fixture/.swarm/state.json create mode 100644 examples/ruflo-gate/fixture/.swarm/tasks/task-001-spec.json create mode 100644 examples/ruflo-gate/fixture/.swarm/tasks/task-002-implement.json create mode 100644 examples/ruflo-gate/fixture/.swarm/tasks/task-003-logging.json create mode 100644 examples/ruflo-gate/fixture/.swarm/tasks/task-004-review.json create mode 100644 examples/ruflo-gate/fixture/dedupe-report.json create mode 100644 examples/ruflo-gate/fixture/dedupe.log create mode 100644 examples/ruflo-gate/fixture/src/import_contacts.py create mode 100644 examples/ruflo-gate/swarm_example.py create mode 100644 scripts/test_ruflo_recipe.py diff --git a/docs/integrations/ruflo.md b/docs/integrations/ruflo.md new file mode 100644 index 0000000..2561c46 --- /dev/null +++ b/docs/integrations/ruflo.md @@ -0,0 +1,146 @@ +# ruflo — swarm below, acceptance gate above + +[ruflo](https://github.com/ruvnet/ruflo) owns the ORCHESTRATE tier: a multi-agent +swarm that spawns a Queen coordinator and worker agents over the Claude Code CLI, +runs the SPARC phases, and keeps its state in `.swarm/` and `.hive-mind/`. What it +cannot do is tell you whether the objective was actually met — a swarm's terminal +is "the coordinator decided", recorded as a child-process exit code plus rows the +swarm wrote about itself. Loop Engineer adds the tier *above* it: one acceptance +gate the swarm as a whole must pass. It never replaces ruflo; it certifies what +the swarm produced. + +## The seam: a host-side supervisor, not a swarm hook + +`ruflo hive-mind spawn "" --claude` **blocks** — it spawns the `claude` +binary as the swarm's execution body and awaits its exit, mapping `exit 0` to +success. So the integration point is the process you already control: the +supervisor that launches the CLI and reads the run directory afterwards. + +> There is **no swarm-terminal callback to register.** ruflo's `hooks` subcommands +> are calls *into* its learning system (`ruflo hooks post-task …`), and the plugin +> `HookEvent` enum carries no swarm-level terminal event — the `swarm:consensus-reached` +> / `task:post-complete` names in the docs do not exist in the implementation. +> Gate from the outside instead. For a second, independent gate *inside* the +> swarm's child, register [`hooks/stop_firewall.py`](../../hooks/stop_firewall.py) +> as a Claude Code `Stop` hook: it blocks a turn that ends on a `Succeeded` claim +> `loop doctor` disagrees with. + +## The pattern + +```python +from loop import emit +from loop.integrations import EngineOutcome, to_terminal_state + +run = subprocess.run(["npx", "ruflo@3.32.9", "hive-mind", "spawn", objective, + "--claude", "--non-interactive"], cwd=ws) # blocks + +obs = observe(ws) # .swarm/ JSON + memory export +gate = holdout_gate.decide(visible, holdout) # the split the swarm never saw +ac = anticheat_scan.scan(diff_text=git_diff, trajectory=agent_trails(obs)) + +criteria_met = {cid: proven.get(cid) for cid in declared_criteria(obs["export"])} +terminal = to_terminal_state( + outcome=EngineOutcome( + reached_end=run.returncode == 0 and obs["state"]["status"] in {"ready", "initialized", "stopped"}, + human_abort=supervisor_was_interrupted(), # NEVER inferred from the exit code + artifacts=[...], + ), + gate_verdict=gate, anticheat=ac, criteria_met=criteria_met, +) +emit.terminate(ws, state=terminal["state"], criteria_met=terminal["criteria_met"], + evidence=terminal["evidence"], false_completion=terminal["false_completion"], + reason=terminal["reason"], iteration_id=1) +``` + +`loop/integrations.py` needs **no ruflo-specific code** — every input is host-side +observable, and ruflo itself is unmodified. + +## What a supervisor can read (zero ruflo changes) + +| Signal | Shape | +|---|---| +| exit code of `hive-mind spawn … --claude` | `0` == the swarm's self-report of success | +| `.swarm/state.json` | `{id, topology, maxAgents, strategy, v3Mode, initializedAt, status}` | +| `.swarm/tasks/*.json` | per-task `status` ∈ `completed｜done｜in_progress｜running｜pending` | +| `.swarm/agents/*.json`, `.swarm/coordination/*.json` | the agent trails and consensus rows | +| `ruflo swarm status --format json` | live counts, progress, metrics | +| `ruflo memory export -o ` | the SPARC namespaces, incl. declared `acceptanceCriteria` | +| `ruflo autopilot status --json` | re-engagement loop state (`--max-iterations`, `--timeout`) | + +Authoritative run state lives in binary SQLite (`.swarm/memory.db`, +`.hive-mind/hive.db`). Never parse those — `memory export` is the supported +serialization. Note the flag inconsistency: `swarm status --format json` but +`autopilot status --json`. + +### Three traps + +1. **Ctrl-C exits 0.** ruflo's SIGINT path prints "Pausing session", kills the + child and calls `process.exit(0)` — an interrupted run is indistinguishable + from a successful one by exit code. `AbortedByHuman` must come from the + supervisor's own signal handler; never derive it from ruflo. +2. **`ruflo verify` is install-integrity, not a run verdict.** It checks the + SHA-256 + Ed25519 witness of the *installed artifact* against + `verification.md.json`. It says nothing about whether the objective was met — + wiring it as the gate would certify that the package downloaded correctly. +3. **`sparc-gates` is the swarm grading its own homework.** The memory export's + `sparc-gates` namespace records per-phase `pass` rows and a `truthScore`. Read + the `sparc-phases` `acceptanceCriteria` as a criteria *vocabulary*, record the + self-report as observation — and let the withheld holdout gate decide. + +## ruflo signal → typed terminal state + +| ruflo signal | Typed terminal state | +|---|---| +| exit 0, swarm settled, holdout green + anticheat clean, every declared criterion proven | `Succeeded` | +| exit 0, visible green / holdout red | `FailedUnverifiable` (`false_completion: true`) | +| the swarm declared an AC no check covers | `FailedSpecGap` | +| autopilot `--max-iterations` / `--timeout` reached without a green gate | `FailedBudget` | +| non-zero exit with no completed tasks (MCP / provider / credential failure) | `FailedBlocked` | +| operator interrupt, recorded by the supervisor | `AbortedByHuman` | +| anticheat CRITICAL (gate tampering) | `FailedSafety` | + +Precedence is `to_terminal_state`'s fixed order — safety → human → blocked → +budget → spec-gap → gate — so an interrupted or gamed run can never launder +itself into `Succeeded`. + +## Zero-install mode + +The `loop.integrations` module is convenience, not a requirement — the whole +projection is the SAME ~15 lines as the LangGraph and Temporal recipes (the +adapter is engine-neutral): + +```python +def to_terminal(gate, anticheat, criteria_met, evidence, + *, human_abort=False, blocked=None, over_budget=False): + fc = gate.get("false_completion") is True + if anticheat.get("downgrade_to") == "FailedSafety": state = "FailedSafety" + elif human_abort: state = "AbortedByHuman" + elif blocked: state = "FailedBlocked" + elif over_budget: state = "FailedBudget" + elif any(v is None for v in criteria_met.values()): state = "FailedSpecGap" + elif (not gate or not anticheat or anticheat.get("downgrade_to") + or gate.get("verdict") != "Succeeded" or fc + or not any(criteria_met.values()) or not evidence): state = "FailedUnverifiable" + else: state = "Succeeded" + return {"schema": "loop-engineer/terminal@1", "state": state, + "criteria_met": {k: v is True for k, v in criteria_met.items()}, + "evidence": list(evidence), "false_completion": fc} +``` + +## Gate it in CI + +```yaml +- run: pip install loop-engineer +- run: loop doctor run/ # -> {"ok": true}: the contract is structurally honest +- run: loop metrics run/ # -> false_completion_rate + evidence-backed scorecard +``` + +`loop metrics` scores the run from its on-disk evidence — not from the swarm's +narration, its consensus rows, or its `truthScore`. + +Verified against `ruflo` 3.32.9 (2026-07-25). ruflo moves fast (27 minor versions +in three months), so pin the version you supervise. + +Full runnable example (happy path + `--sabotage-holdout` false-completion demo + +interrupt and spec-gap demos, replaying a committed recording so it runs offline): +[`examples/ruflo-gate/`](../../examples/ruflo-gate/). diff --git a/examples/ruflo-gate/README.md b/examples/ruflo-gate/README.md new file mode 100644 index 0000000..bbff3f3 --- /dev/null +++ b/examples/ruflo-gate/README.md @@ -0,0 +1,82 @@ +# ruflo recipe — swarm below, acceptance gate above + +A runnable **host-side supervisor** around [ruflo](https://github.com/ruvnet/ruflo)'s +blocking swarm CLI. ruflo keeps its own orchestration substrate (Queen coordinator, +worker agents, SPARC phases, `.swarm/` + `.hive-mind/` state); Loop Engineer adds +the contract/proof tier above it — evidence-backed state the `loop` CLI can +independently validate and score. + +## Why a supervisor and not a hook + +`ruflo hive-mind spawn "" --claude` spawns the Claude Code CLI as the +swarm's body, **blocks** until that child exits, and maps `exit 0` to success. +There is no swarm-terminal callback to register: the `hooks` subcommands are calls +*into* ruflo, and the plugin `HookEvent` enum has no swarm-level terminal event +(`swarm:consensus-reached` appears in ruflo's docs but not in its implementation). +So the gate lives in the process that launches the CLI — plus, optionally, a +second independent gate inside the child via this repo's Claude Code `Stop`-hook +firewall (`hooks/stop_firewall.py`). + +## What it shows + +`swarm_example.py` supervises one run: + +1. **drive** the swarm (replayed by default — see below), +2. **observe** only host-side surfaces: `.swarm/state.json`, `.swarm/tasks/*.json`, + `.swarm/agents/*.json`, `.swarm/coordination/*.json`, and the + `ruflo memory export` JSON, +3. **gate** it with the withheld holdout split (`holdout_gate.decide`) plus the + trajectory sweep (`anticheat_scan.scan`) over the agent trails, +4. **project** through `to_terminal_state` and **record** via `loop.emit`, which + refuses a dishonest `Succeeded` before anything hits disk. + +The swarm's own `sparc-phases` `acceptanceCriteria` supply the criteria +*vocabulary* (`AC-1`…`AC-3`); their truth comes from the withheld checks. The +swarm's `sparc-gates` self-verdict (all phases `pass`, `truthScore: 0.97`) is +recorded as an observation in `swarm-observation.json` and never used to decide. + +## Fixture replay is the default — and it is stated, not hidden + +A live ruflo run needs Node, `npx ruflo`, the `claude` binary, credentials and +real model spend, so the shipped default **replays the committed recording in +`fixture/`** — a `.swarm/` tree in ruflo's layout plus the work product the +recorded run left behind. `--live` opts into the real invocation. + +Only the swarm is recorded. The gate, the projection, `loop.emit`, `loop doctor` +and `loop metrics` all execute for real against the replayed workspace. A recipe +that quietly faked the engine *and* the gate would be exactly the false +completion this project exists to catch. + +## Run it + +```bash +pip install loop-engineer +python swarm_example.py demo-run/ # replay: Succeeded, offline +loop doctor demo-run/ # -> {"ok": true, ...} +loop metrics demo-run/ # -> clean scorecard (FCR 0.0) +``` + +### Demos + +| Flag | What it proves | +|---|---| +| `--sabotage-holdout` | the work product still claims 41 unique rows (visible green) but the dropped-row log is truncated (holdout red) → `FailedUnverifiable`, `false_completion: true`, **never** `Succeeded` | +| `--simulate-interrupt` | ruflo exits **0** on Ctrl-C, and the gate is green — yet the supervisor's own interrupt flag yields `AbortedByHuman` | +| `--declare-unmapped-criterion` | the swarm declares `AC-4` that no check covers → `FailedSpecGap`, the failure a self-reporting coordinator hides | +| `--live` | really runs `npx ruflo@3.32.9 hive-mind spawn … --claude` (Node + `claude` + credentials + spend) | + +In live mode the supervisor installs a `SIGINT` handler *before* spawning, because +ruflo's own SIGINT path calls `process.exit(0)` — `human_abort` is never inferred +from the exit code. + +The gate tools (`holdout_gate`, `anticheat_scan`) resolve from `loop._resources`, +so a plain `pip install` is enough; a repo checkout picks them up from `scripts/`. + +## The general pattern + +The complement framing, the full host-side signal table, the three traps, and the +copy-paste (zero-install) projection live in +[`docs/integrations/ruflo.md`](../../docs/integrations/ruflo.md). + +Verified against `ruflo` 3.32.9 (2026-07-25). Fixture prose is original; only +directory names and JSON key names follow ruflo's conventions. diff --git a/examples/ruflo-gate/fixture/.claude-flow/metrics/v3-progress.json b/examples/ruflo-gate/fixture/.claude-flow/metrics/v3-progress.json new file mode 100644 index 0000000..31628ed --- /dev/null +++ b/examples/ruflo-gate/fixture/.claude-flow/metrics/v3-progress.json @@ -0,0 +1,20 @@ +{ + "version": "3.32.9", + "initialized": "2026-07-25T09:14:02Z", + "domains": { + "completed": 1, + "total": 1, + "status": "COMPLETE" + }, + "swarm": { + "activeAgents": 0, + "maxAgents": 8, + "topology": "hierarchical-mesh" + }, + "learning": { + "status": "READY", + "patternsLearned": 2, + "sessionsCompleted": 1 + }, + "_note": "Snapshot after the swarm drained back to idle. Progress, not proof." +} diff --git a/examples/ruflo-gate/fixture/.hive-mind/sessions/hive-mind-prompt-swarm-2026-07-25.txt b/examples/ruflo-gate/fixture/.hive-mind/sessions/hive-mind-prompt-swarm-2026-07-25.txt new file mode 100644 index 0000000..9b6eee3 --- /dev/null +++ b/examples/ruflo-gate/fixture/.hive-mind/sessions/hive-mind-prompt-swarm-2026-07-25.txt @@ -0,0 +1,13 @@ +Queen coordinator briefing — swarm-2026-07-25-csv-dedupe + +Objective: collapse duplicate contacts during the CSV import so that rows sharing +a normalized (email, phone) key are written once, the first-seen row is retained, +and every dropped row is recorded with its source line number. + +Coordination notes for the worker agents: + - Spawn one coder per acceptance criterion; the reviewer re-runs the import. + - Record each SPARC phase artifact into the sparc-phases memory namespace. + - Verify all subtasks are complete before reporting the objective as met. + +This prompt file is the record of what the swarm was asked to do. It is an input, +not evidence of the outcome — the outcome is decided by the supervising gate. diff --git a/examples/ruflo-gate/fixture/.swarm/agents/agent-coder-02.json b/examples/ruflo-gate/fixture/.swarm/agents/agent-coder-02.json new file mode 100644 index 0000000..dacb4eb --- /dev/null +++ b/examples/ruflo-gate/fixture/.swarm/agents/agent-coder-02.json @@ -0,0 +1,12 @@ +{ + "id": "agent-coder-02", + "swarmId": "swarm-2026-07-25-csv-dedupe", + "role": "coder", + "status": "completed", + "spawnedAt": "2026-07-25T09:16:58Z", + "tasksCompleted": 1, + "touchedPaths": [ + "src/import_contacts.py", + "dedupe-report.json" + ] +} diff --git a/examples/ruflo-gate/fixture/.swarm/agents/agent-coder-03.json b/examples/ruflo-gate/fixture/.swarm/agents/agent-coder-03.json new file mode 100644 index 0000000..b67e575 --- /dev/null +++ b/examples/ruflo-gate/fixture/.swarm/agents/agent-coder-03.json @@ -0,0 +1,12 @@ +{ + "id": "agent-coder-03", + "swarmId": "swarm-2026-07-25-csv-dedupe", + "role": "coder", + "status": "completed", + "spawnedAt": "2026-07-25T09:17:09Z", + "tasksCompleted": 1, + "touchedPaths": [ + "src/import_contacts.py", + "dedupe.log" + ] +} diff --git a/examples/ruflo-gate/fixture/.swarm/agents/agent-queen-01.json b/examples/ruflo-gate/fixture/.swarm/agents/agent-queen-01.json new file mode 100644 index 0000000..5fcc8a3 --- /dev/null +++ b/examples/ruflo-gate/fixture/.swarm/agents/agent-queen-01.json @@ -0,0 +1,12 @@ +{ + "id": "agent-queen-01", + "swarmId": "swarm-2026-07-25-csv-dedupe", + "role": "queen-coordinator", + "status": "completed", + "spawnedAt": "2026-07-25T09:14:11Z", + "tasksCompleted": 1, + "touchedPaths": [ + ".swarm/memory-export.json", + ".hive-mind/sessions/hive-mind-prompt-swarm-2026-07-25.txt" + ] +} diff --git a/examples/ruflo-gate/fixture/.swarm/agents/agent-reviewer-04.json b/examples/ruflo-gate/fixture/.swarm/agents/agent-reviewer-04.json new file mode 100644 index 0000000..185cd5a --- /dev/null +++ b/examples/ruflo-gate/fixture/.swarm/agents/agent-reviewer-04.json @@ -0,0 +1,12 @@ +{ + "id": "agent-reviewer-04", + "swarmId": "swarm-2026-07-25-csv-dedupe", + "role": "reviewer", + "status": "completed", + "spawnedAt": "2026-07-25T09:24:26Z", + "tasksCompleted": 1, + "touchedPaths": [ + "dedupe-report.json", + "dedupe.log" + ] +} diff --git a/examples/ruflo-gate/fixture/.swarm/coordination/consensus-2026-07-25T09-31-52Z.json b/examples/ruflo-gate/fixture/.swarm/coordination/consensus-2026-07-25T09-31-52Z.json new file mode 100644 index 0000000..3755883 --- /dev/null +++ b/examples/ruflo-gate/fixture/.swarm/coordination/consensus-2026-07-25T09-31-52Z.json @@ -0,0 +1,17 @@ +{ + "swarmId": "swarm-2026-07-25-csv-dedupe", + "round": 3, + "kind": "consensus", + "algorithm": "byzantine", + "recordedAt": "2026-07-25T09:31:52Z", + "messagesSent": 42, + "conflictsResolved": 1, + "participants": [ + "agent-queen-01", + "agent-coder-02", + "agent-coder-03", + "agent-reviewer-04" + ], + "outcome": "objective-met", + "_note": "The swarm's own consensus row. A supervisor reads it as a trail, never as a verdict." +} diff --git a/examples/ruflo-gate/fixture/.swarm/memory-export.json b/examples/ruflo-gate/fixture/.swarm/memory-export.json new file mode 100644 index 0000000..78d25ad --- /dev/null +++ b/examples/ruflo-gate/fixture/.swarm/memory-export.json @@ -0,0 +1,48 @@ +{ + "schema": "ruflo-memory-export/v1", + "exportedAt": "2026-07-25T09:32:10Z", + "namespace": null, + "count": 4, + "entries": [ + { + "key": "spec-csv-dedupe", + "namespace": "sparc-phases", + "value": "{\"status\":\"complete\",\"acceptanceCriteria\":[\"AC-1: Given the two seed files, when the import runs, then 41 unique contacts remain from 57 input rows\",\"AC-2: Given a completed import, when the same files import again, then zero new rows are written\",\"AC-3: Given a dropped duplicate, when it is skipped, then its source line appears in dedupe.log\"],\"constraints\":[\"case-insensitive match on email and phone only\",\"retention order is first-seen wins\"]}", + "createdAt": "2026-07-25T09:16:38Z", + "updatedAt": "2026-07-25T09:16:38Z", + "accessCount": 6, + "hasEmbedding": true, + "size": 592 + }, + { + "key": "current-phase-csv-dedupe", + "namespace": "sparc-state", + "value": "{\"feature\":\"csv-dedupe\",\"currentPhase\":\"completion\",\"phaseNumber\":5,\"status\":\"complete\"}", + "createdAt": "2026-07-25T09:16:38Z", + "updatedAt": "2026-07-25T09:31:44Z", + "accessCount": 9, + "hasEmbedding": false, + "size": 128 + }, + { + "key": "gates-csv-dedupe", + "namespace": "sparc-gates", + "value": "{\"feature\":\"csv-dedupe\",\"gates\":[{\"phase\":1,\"name\":\"specification\",\"result\":\"pass\"},{\"phase\":2,\"name\":\"pseudocode\",\"result\":\"pass\"},{\"phase\":3,\"name\":\"architecture\",\"result\":\"pass\"},{\"phase\":4,\"name\":\"refinement\",\"result\":\"pass\",\"coverage\":92},{\"phase\":5,\"name\":\"completion\",\"result\":\"pass\",\"truthScore\":0.97}]}", + "createdAt": "2026-07-25T09:17:02Z", + "updatedAt": "2026-07-25T09:31:44Z", + "accessCount": 6, + "hasEmbedding": false, + "size": 480 + }, + { + "key": "complete-csv-dedupe", + "namespace": "sparc-phases", + "value": "{\"status\":\"complete\",\"traceabilityMatrix\":[{\"ac\":\"AC-1\",\"test\":\"test_dedupe_yields_41_unique\",\"status\":\"Pass\"},{\"ac\":\"AC-2\",\"test\":\"test_second_run_is_idempotent\",\"status\":\"Pass\"},{\"ac\":\"AC-3\",\"test\":\"test_dropped_rows_are_logged\",\"status\":\"Pass\"}],\"regressionResult\":\"pass\"}", + "createdAt": "2026-07-25T09:31:40Z", + "updatedAt": "2026-07-25T09:31:40Z", + "accessCount": 2, + "hasEmbedding": true, + "size": 416 + } + ] +} diff --git a/examples/ruflo-gate/fixture/.swarm/state.json b/examples/ruflo-gate/fixture/.swarm/state.json new file mode 100644 index 0000000..ecdaedd --- /dev/null +++ b/examples/ruflo-gate/fixture/.swarm/state.json @@ -0,0 +1,9 @@ +{ + "id": "swarm-2026-07-25-csv-dedupe", + "topology": "hierarchical-mesh", + "maxAgents": 8, + "strategy": "sparc", + "v3Mode": true, + "initializedAt": "2026-07-25T09:14:02Z", + "status": "stopped" +} diff --git a/examples/ruflo-gate/fixture/.swarm/tasks/task-001-spec.json b/examples/ruflo-gate/fixture/.swarm/tasks/task-001-spec.json new file mode 100644 index 0000000..fe44f2e --- /dev/null +++ b/examples/ruflo-gate/fixture/.swarm/tasks/task-001-spec.json @@ -0,0 +1,9 @@ +{ + "id": "task-001-spec", + "swarmId": "swarm-2026-07-25-csv-dedupe", + "description": "record the dedupe acceptance criteria as the SPARC specification phase", + "assignedAgent": "agent-queen-01", + "status": "completed", + "createdAt": "2026-07-25T09:14:20Z", + "completedAt": "2026-07-25T09:16:41Z" +} diff --git a/examples/ruflo-gate/fixture/.swarm/tasks/task-002-implement.json b/examples/ruflo-gate/fixture/.swarm/tasks/task-002-implement.json new file mode 100644 index 0000000..c7494e8 --- /dev/null +++ b/examples/ruflo-gate/fixture/.swarm/tasks/task-002-implement.json @@ -0,0 +1,9 @@ +{ + "id": "task-002-implement", + "swarmId": "swarm-2026-07-25-csv-dedupe", + "description": "normalize on (lower(email), lower(phone)) and keep the first-seen row", + "assignedAgent": "agent-coder-02", + "status": "completed", + "createdAt": "2026-07-25T09:16:52Z", + "completedAt": "2026-07-25T09:24:08Z" +} diff --git a/examples/ruflo-gate/fixture/.swarm/tasks/task-003-logging.json b/examples/ruflo-gate/fixture/.swarm/tasks/task-003-logging.json new file mode 100644 index 0000000..671b7b1 --- /dev/null +++ b/examples/ruflo-gate/fixture/.swarm/tasks/task-003-logging.json @@ -0,0 +1,9 @@ +{ + "id": "task-003-logging", + "swarmId": "swarm-2026-07-25-csv-dedupe", + "description": "append every dropped duplicate and its source line to dedupe.log", + "assignedAgent": "agent-coder-03", + "status": "completed", + "createdAt": "2026-07-25T09:17:04Z", + "completedAt": "2026-07-25T09:26:55Z" +} diff --git a/examples/ruflo-gate/fixture/.swarm/tasks/task-004-review.json b/examples/ruflo-gate/fixture/.swarm/tasks/task-004-review.json new file mode 100644 index 0000000..ab2c5ad --- /dev/null +++ b/examples/ruflo-gate/fixture/.swarm/tasks/task-004-review.json @@ -0,0 +1,9 @@ +{ + "id": "task-004-review", + "swarmId": "swarm-2026-07-25-csv-dedupe", + "description": "review the retained-row ordering and re-run the import for idempotence", + "assignedAgent": "agent-reviewer-04", + "status": "completed", + "createdAt": "2026-07-25T09:24:19Z", + "completedAt": "2026-07-25T09:31:37Z" +} diff --git a/examples/ruflo-gate/fixture/dedupe-report.json b/examples/ruflo-gate/fixture/dedupe-report.json new file mode 100644 index 0000000..4c1c412 --- /dev/null +++ b/examples/ruflo-gate/fixture/dedupe-report.json @@ -0,0 +1,10 @@ +{ + "feature": "csv-dedupe", + "input_rows": 57, + "unique_rows": 41, + "dropped_rows": 16, + "second_run_inserted": 0, + "log_file": "dedupe.log", + "generated_by": "agent-coder-02", + "generated_at": "2026-07-25T09:24:08Z" +} diff --git a/examples/ruflo-gate/fixture/dedupe.log b/examples/ruflo-gate/fixture/dedupe.log new file mode 100644 index 0000000..1b6db93 --- /dev/null +++ b/examples/ruflo-gate/fixture/dedupe.log @@ -0,0 +1,16 @@ +seed-a.csv:9 dropped duplicate key (ada@example.test, 5550142) +seed-a.csv:14 dropped duplicate key (bo@example.test, 5550118) +seed-a.csv:18 dropped duplicate key (cyd@example.test, 5550196) +seed-a.csv:23 dropped duplicate key (dara@example.test, 5550107) +seed-a.csv:27 dropped duplicate key (eli@example.test, 5550133) +seed-a.csv:31 dropped duplicate key (fen@example.test, 5550171) +seed-a.csv:36 dropped duplicate key (gus@example.test, 5550159) +seed-a.csv:40 dropped duplicate key (hana@example.test, 5550124) +seed-b.csv:5 dropped duplicate key (ada@example.test, 5550142) +seed-b.csv:11 dropped duplicate key (ivo@example.test, 5550188) +seed-b.csv:16 dropped duplicate key (jun@example.test, 5550163) +seed-b.csv:20 dropped duplicate key (kai@example.test, 5550110) +seed-b.csv:24 dropped duplicate key (lia@example.test, 5550175) +seed-b.csv:29 dropped duplicate key (moe@example.test, 5550149) +seed-b.csv:33 dropped duplicate key (nia@example.test, 5550182) +seed-b.csv:38 dropped duplicate key (omar@example.test, 5550136) diff --git a/examples/ruflo-gate/fixture/src/import_contacts.py b/examples/ruflo-gate/fixture/src/import_contacts.py new file mode 100644 index 0000000..c111012 --- /dev/null +++ b/examples/ruflo-gate/fixture/src/import_contacts.py @@ -0,0 +1,24 @@ +"""The swarm's work product — the code the recorded run left behind. + +Vendored so the supervisor's anti-cheat trajectory names files that actually +exist in the replayed workspace. Nothing in this module runs during the recipe. +""" + +from __future__ import annotations + + +def normalize_key(email: str, phone: str) -> tuple[str, str]: + return email.strip().lower(), phone.strip().lower() + + +def dedupe_rows(rows): + seen: set[tuple[str, str]] = set() + kept, dropped = [], [] + for source, line_no, email, phone in rows: + key = normalize_key(email, phone) + if key in seen: + dropped.append((source, line_no, key)) + continue + seen.add(key) + kept.append((email, phone)) + return kept, dropped diff --git a/examples/ruflo-gate/swarm_example.py b/examples/ruflo-gate/swarm_example.py new file mode 100644 index 0000000..e0c2c80 --- /dev/null +++ b/examples/ruflo-gate/swarm_example.py @@ -0,0 +1,321 @@ +"""A host-side supervisor around a ruflo swarm, whose only path to a completion +claim is a real acceptance gate. + +ruflo has no swarm-terminal callback and no Python API: `ruflo hive-mind spawn +"" --claude` spawns the Claude Code CLI as the swarm's body, blocks +until that child exits, and maps `exit 0` to success. That exit code is the +swarm's SELF-report, and so is everything in the `sparc-gates` memory namespace. +This supervisor replaces both with one gate the swarm never saw: the withheld +holdout split (`holdout_gate.decide`) plus the trajectory sweep +(`anticheat_scan.scan`), projected through `to_terminal_state` and recorded by +`loop.emit` — which refuses a dishonest `Succeeded` before anything hits disk. + + python swarm_example.py [flags] + + --sabotage-holdout work product passes the visible check and + fails the withheld one -> FailedUnverifiable + with false_completion: true + --simulate-interrupt the operator-interrupt path, deterministically + --declare-unmapped-criterion the swarm declares an AC no check covers + -> FailedSpecGap + --live really invoke ruflo (needs Node, the `claude` + binary, credentials and model spend) + +By default the swarm is REPLAYED from the committed `fixture/` recording, so +this runs offline. Only the swarm is recorded: the gate, the projection, the +emit writes, `loop doctor` and `loop metrics` all execute for real. +""" + +from __future__ import annotations + +import json +import shutil +import signal +import subprocess +import sys +from dataclasses import dataclass +from pathlib import Path + +from loop import emit +from loop._resources import tools_dir +from loop.integrations import EngineOutcome, to_terminal_state + +sys.path.insert(0, str(tools_dir())) +import anticheat_scan # noqa: E402 +import holdout_gate # noqa: E402 + +FIXTURE = Path(__file__).resolve().parent / "fixture" +RUFLO_VERSION = "3.32.9" +OBJECTIVE = ( + "collapse duplicate contacts on a normalized (email, phone) key, keep the " + "first-seen row, and log every dropped row with its source line" +) +# The one blocking verb the supervisor drives. `ruflo verify` is deliberately +# absent: it checks the SHA-256/Ed25519 integrity of the INSTALLED artifact, not +# the run, so it can never stand in for an acceptance gate. +LIVE_COMMAND = ["npx", f"ruflo@{RUFLO_VERSION}", "hive-mind", "spawn", + OBJECTIVE, "--claude", "--non-interactive"] + +# The swarm's terminal states as ruflo records them in .swarm/state.json. +_SETTLED_SWARM_STATUS = {"ready", "initialized", "stopped"} + +_interrupted = False + + +def _on_sigint(_signum, _frame) -> None: + global _interrupted + _interrupted = True + + +def install_interrupt_handler() -> None: + """ruflo's own SIGINT path prints "Pausing session" and calls + `process.exit(0)` — an interrupted run is indistinguishable from a + successful one by exit code. The supervisor therefore records the interrupt + itself; `human_abort` is never inferred from ruflo.""" + signal.signal(signal.SIGINT, _on_sigint) + + +def mark_interrupted() -> None: + """Deterministic stand-in for a real Ctrl-C, for the demo and the tests.""" + global _interrupted + _interrupted = True + + +@dataclass(frozen=True) +class SwarmRun: + """What the supervisor knows about the run — never what the swarm claims.""" + + returncode: int + mode: str + human_abort: bool = False + budget_exhausted: bool = False + + +def replay_swarm(workspace: Path, *, sabotage: bool = False, + unmapped_criterion: bool = False, fixture: Path = FIXTURE) -> SwarmRun: + """Materialize the recorded run directory, then apply the requested demo.""" + shutil.copytree(fixture, workspace, dirs_exist_ok=True) + if sabotage: + # The work product still SAYS 41 unique rows (the visible claim), but the + # dropped-row log the withheld check reads is truncated. + log = workspace / "dedupe.log" + kept = log.read_text(encoding="utf-8").splitlines()[:3] + log.write_text("\n".join(kept) + "\n", encoding="utf-8") + if unmapped_criterion: + _declare_extra_criterion(workspace) + return SwarmRun(returncode=0, mode="replay", human_abort=_interrupted) + + +def run_swarm_live(workspace: Path) -> SwarmRun: + """Drive the real CLI. It blocks until the Claude Code child exits, and its + handler maps `exit 0` to success — which is exactly the claim the gate + downstream replaces.""" + workspace.mkdir(parents=True, exist_ok=True) + install_interrupt_handler() + proc = subprocess.run(LIVE_COMMAND, cwd=workspace, text=True) + subprocess.run( + ["npx", f"ruflo@{RUFLO_VERSION}", "memory", "export", "-o", ".swarm/memory-export.json"], + cwd=workspace, text=True, check=False, + ) + return SwarmRun(returncode=proc.returncode, mode="live", human_abort=_interrupted) + + +def _declare_extra_criterion(workspace: Path) -> None: + path = workspace / ".swarm" / "memory-export.json" + export = json.loads(path.read_text(encoding="utf-8")) + for entry in export["entries"]: + if entry["namespace"] == "sparc-phases" and entry["key"].startswith("spec-"): + spec = json.loads(entry["value"]) + spec["acceptanceCriteria"].append( + "AC-4: Given a contact with no phone, when it imports, then it is retained once" + ) + entry["value"] = json.dumps(spec) + break + path.write_text(json.dumps(export, indent=2) + "\n", encoding="utf-8") + + +# --- observation: everything readable WITHOUT touching ruflo ----------------- + + +def observe(workspace: Path) -> dict: + """Read the run's documented JSON surfaces. The authoritative state lives in + binary SQLite (`.swarm/memory.db`, `.hive-mind/hive.db`); those are never + parsed — `ruflo memory export` is the supported serialization.""" + swarm = Path(workspace) / ".swarm" + return { + "state": json.loads((swarm / "state.json").read_text(encoding="utf-8")), + "tasks": [json.loads(p.read_text(encoding="utf-8")) + for p in sorted((swarm / "tasks").glob("*.json"))], + "agents": [json.loads(p.read_text(encoding="utf-8")) + for p in sorted((swarm / "agents").glob("*.json"))], + "coordination": [json.loads(p.read_text(encoding="utf-8")) + for p in sorted((swarm / "coordination").glob("*.json"))], + "export": json.loads((swarm / "memory-export.json").read_text(encoding="utf-8")), + } + + +def declared_criteria(export: dict) -> list[str]: + """The acceptance-criteria ids the swarm itself declared, read as a + VOCABULARY only. Their truth is decided by the holdout gate below.""" + for entry in export["entries"]: + if entry["namespace"] == "sparc-phases" and entry["key"].startswith("spec-"): + spec = json.loads(entry["value"]) + return [ac.split(":", 1)[0].strip() for ac in spec.get("acceptanceCriteria", [])] + return [] + + +def swarm_self_report(export: dict) -> dict: + """The `sparc-gates` row — the swarm grading its own homework. Surfaced so + the contract records what was claimed, never used to decide the terminal.""" + for entry in export["entries"]: + if entry["namespace"] == "sparc-gates": + gates = json.loads(entry["value"]).get("gates", []) + completion = next((g for g in gates if g.get("name") == "completion"), {}) + return { + "all_gates_pass": bool(gates) and all(g.get("result") == "pass" for g in gates), + "truth_score": completion.get("truthScore"), + "source": f"{entry['namespace']}/{entry['key']}", + } + return {"all_gates_pass": False, "truth_score": None, "source": None} + + +def agent_trails(obs: dict) -> list[str]: + """The issue's "agent trails": every path the agents recorded touching, plus + the coordination rows. ruflo exposes no merged diff, so a live supervisor + passes its own `git diff` as `diff_text`.""" + trails: list[str] = [] + for agent in obs["agents"]: + trails.extend(agent.get("touchedPaths", [])) + trails.extend(f"consensus:{row.get('round')}" for row in obs["coordination"]) + return trails + + +# --- the gate the swarm never saw ------------------------------------------- + + +def visible_checks(workspace: Path) -> list[dict]: + """What the swarm could see while it worked — its own report claim.""" + report = Path(workspace) / "dedupe-report.json" + claim = json.loads(report.read_text(encoding="utf-8")) if report.is_file() else {} + return [ + {"id": "report-exists", "passed": report.is_file()}, + {"id": "report-claims-dedupe", "passed": claim.get("unique_rows") == 41}, + ] + + +def holdout_checks(workspace: Path) -> list[dict]: + """Withheld until terminal verification, one per declared criterion.""" + ws = Path(workspace) + report_path, log_path = ws / "dedupe-report.json", ws / "dedupe.log" + claim = json.loads(report_path.read_text(encoding="utf-8")) if report_path.is_file() else {} + log_lines = [line for line in log_path.read_text(encoding="utf-8").splitlines() + if line.strip()] if log_path.is_file() else [] + dropped = claim.get("dropped_rows") + return [ + {"id": "AC-1", "passed": claim.get("input_rows") == 57 + and claim.get("unique_rows") == 41 + and dropped == 16}, + {"id": "AC-2", "passed": claim.get("second_run_inserted") == 0}, + {"id": "AC-3", "passed": len(log_lines) == dropped + and all(":" in line.split(" ", 1)[0] for line in log_lines)}, + ] + + +def certify(workspace: Path, run: SwarmRun) -> dict: + ws = Path(workspace) + obs = observe(ws) + + gate = holdout_gate.decide(visible_checks(ws), holdout_checks(ws)) + ac = anticheat_scan.scan(diff_text="", trajectory=agent_trails(obs)) + + proven = {check["id"]: check["passed"] for check in gate["holdout"]} + criteria_met = {cid: proven.get(cid) for cid in declared_criteria(obs["export"])} + + art_dir = ws / ".loop" / "artifacts" + art_dir.mkdir(parents=True, exist_ok=True) + (art_dir / "holdout-verdict.json").write_text( + json.dumps(gate, indent=2) + "\n", encoding="utf-8") + (art_dir / "swarm-observation.json").write_text( + json.dumps({ + "mode": run.mode, + "returncode": run.returncode, + "human_abort": run.human_abort, + "swarm_status": obs["state"]["status"], + "task_status": [t["status"] for t in obs["tasks"]], + "agents": [a["id"] for a in obs["agents"]], + "trails": agent_trails(obs), + "swarm_self_report": swarm_self_report(obs["export"]), + }, indent=2) + "\n", encoding="utf-8") + bundle = { + "task": "T1", + "verify": "supervisor gate — holdout_gate.decide over visible+withheld swarm output", + "outcome": "PASS" if gate["verdict"] == "Succeeded" else "FAIL", + "iteration_id": 1, + "criteria": {cid: value is True for cid, value in criteria_met.items()}, + } + (art_dir / "verify-T1.json").write_text(json.dumps(bundle, indent=2) + "\n", encoding="utf-8") + + completed = [t for t in obs["tasks"] if t["status"] in {"completed", "done"}] + terminal = to_terminal_state( + outcome=EngineOutcome( + reached_end=run.returncode == 0 + and obs["state"]["status"] in _SETTLED_SWARM_STATUS, + external_error=None if run.returncode == 0 or completed + else f"ruflo exited {run.returncode} with no completed tasks", + budget_exhausted=run.budget_exhausted, + human_abort=run.human_abort, + artifacts=[".loop/artifacts/verify-T1.json", + ".loop/artifacts/holdout-verdict.json", + ".loop/artifacts/swarm-observation.json"], + ), + gate_verdict=gate, anticheat=ac, criteria_met=criteria_met, + ) + + passed = terminal["state"] == "Succeeded" + emit.append_iteration( + ws, iteration_id=1, outcome="task_passed" if passed else "task_failed", task_id="T1", + actions=[f"supervised ruflo hive-mind spawn ({run.mode} mode)", + "read .swarm/ state, tasks, agents, coordination and the memory export", + "ran holdout_gate.decide + anticheat_scan.scan over the swarm's output"], + verify_cmd="holdout_gate.decide(visible, holdout)", verify_outcome=gate["verdict"], + notes="verify bundle: verify-T1.json; gate verdict: holdout-verdict.json; " + "swarm self-report recorded in swarm-observation.json (not trusted)", + ) + emit.append_receipt(ws, iteration_id=1, role="orchestrate", + model="deterministic-demo", outcome="ok") + emit.terminate( + ws, state=terminal["state"], criteria_met=terminal["criteria_met"], + evidence=terminal["evidence"], false_completion=terminal["false_completion"], + reason=terminal["reason"], iteration_id=1, + ) + return terminal + + +def main(workspace: str, *, live: bool, sabotage: bool, + unmapped_criterion: bool, simulate_interrupt: bool) -> int: + ws = Path(workspace) + if simulate_interrupt: + mark_interrupted() + run = (run_swarm_live(ws) if live + else replay_swarm(ws, sabotage=sabotage, unmapped_criterion=unmapped_criterion)) + emit.open_contract(ws) + terminal = certify(ws, run) + print(f"terminal: {terminal['state']} — validate: python3 -m loop doctor {workspace}") + return 0 if terminal["state"] == "Succeeded" else 1 + + +if __name__ == "__main__": + argv = sys.argv[1:] + targets = [a for a in argv if not a.startswith("--")] + if len(targets) != 1: + print("usage: python swarm_example.py " + "[--sabotage-holdout] [--simulate-interrupt] " + "[--declare-unmapped-criterion] [--live]", file=sys.stderr) + raise SystemExit(2) + raise SystemExit(main( + targets[0], + live="--live" in argv, + sabotage="--sabotage-holdout" in argv, + unmapped_criterion="--declare-unmapped-criterion" in argv, + simulate_interrupt="--simulate-interrupt" in argv, + )) diff --git a/scripts/test_ruflo_recipe.py b/scripts/test_ruflo_recipe.py new file mode 100644 index 0000000..d614236 --- /dev/null +++ b/scripts/test_ruflo_recipe.py @@ -0,0 +1,199 @@ +"""Acceptance for the ruflo recipe (issue #38): a host-side supervisor around +ruflo's blocking swarm CLI replaces the swarm's self-report with a real gate. + +Deterministic and credential-free: every test replays the committed +``examples/ruflo-gate/fixture/`` tree, so no Node, no ``npx ruflo``, no +``claude`` binary and no model spend are involved. The gate, the projection, +``loop.emit``, ``loop doctor`` and ``loop metrics`` all execute for real — +only the swarm is recorded. + +The three traps this pins (all from the live API dossier): + +* ruflo exits **0** on Ctrl-C, so ``AbortedByHuman`` must come from the + supervisor's own signal flag and never be inferred from the exit code; +* ``ruflo verify`` is install-integrity, not a run verdict, so it is never + wired as the gate; +* ``sparc-gates`` holds the swarm's SELF-asserted verdicts, which the gate + replaces rather than trusts. +""" + +from __future__ import annotations + +import importlib.util +import json +import os +import shutil +import subprocess +import sys +from pathlib import Path + +import pytest + +REPO_ROOT = Path(__file__).resolve().parent.parent +EXAMPLE = REPO_ROOT / "examples" / "ruflo-gate" / "swarm_example.py" +FIXTURE = REPO_ROOT / "examples" / "ruflo-gate" / "fixture" +RECIPE_DOC = REPO_ROOT / "docs" / "integrations" / "ruflo.md" + + +def _load_example(): + spec = importlib.util.spec_from_file_location("ruflo_swarm_example", EXAMPLE) + module = importlib.util.module_from_spec(spec) + # dataclasses resolve annotations through sys.modules[cls.__module__] + sys.modules[spec.name] = module + spec.loader.exec_module(module) + return module + + +def _run_example(workspace: Path, *args: str) -> subprocess.CompletedProcess: + env = dict(os.environ, PYTHONPATH=str(REPO_ROOT)) + return subprocess.run( + [sys.executable, "-B", str(EXAMPLE), str(workspace), *args], + cwd=workspace.parent, env=env, capture_output=True, text=True, + ) + + +def _cli(cmd: str, workspace: Path) -> subprocess.CompletedProcess: + return subprocess.run( + [sys.executable, "-B", "-m", "loop", cmd, str(workspace)], + cwd=REPO_ROOT, capture_output=True, text=True, + ) + + +def _terminal(workspace: Path) -> dict: + return json.loads((workspace / ".loop" / "terminal_state.json").read_text(encoding="utf-8")) + + +def _gate_verdict(workspace: Path) -> dict: + return json.loads( + (workspace / ".loop" / "artifacts" / "holdout-verdict.json").read_text(encoding="utf-8") + ) + + +def test_happy_path_is_doctor_clean_and_metrics_clean(tmp_path): + ws = tmp_path / "swarm-run" + proc = _run_example(ws) + assert proc.returncode == 0, proc.stdout + proc.stderr + + doctored = _cli("doctor", ws) + assert doctored.returncode == 0, doctored.stdout + assert json.loads(doctored.stdout)["ok"] is True + + terminal = _terminal(ws) + assert terminal["state"] == "Succeeded" + assert terminal["false_completion"] is False + assert terminal["evidence"] + # criteria ids come from the swarm's OWN declared acceptanceCriteria + assert set(terminal["criteria_met"]) == {"AC-1", "AC-2", "AC-3"} + assert all(terminal["criteria_met"].values()) + + metrics = _cli("metrics", ws) + assert metrics.returncode == 0, metrics.stdout + metrics.stderr + card = json.loads(metrics.stdout) + assert card["false_completion_rate"] == 0.0 + assert card["false_completions"] == 0 + assert card["iterations_claiming_success"] >= 1 + assert card["evidence_backed"] is True + prov = card["provenance"] + assert prov["unmatched_verify"] == [] + assert prov["unrecognized_outcomes"] == [] + assert prov["fcr_methods_agree"] is True + + +def test_sabotage_holdout_is_false_completion_never_succeeded(tmp_path): + ws = tmp_path / "swarm-run-sabotaged" + proc = _run_example(ws, "--sabotage-holdout") + + terminal = _terminal(ws) + assert terminal["state"] == "FailedUnverifiable" + assert terminal["state"] != "Succeeded" + assert terminal["false_completion"] is True, (terminal, proc.stdout, proc.stderr) + + doctored = _cli("doctor", ws) + assert json.loads(doctored.stdout)["ok"] is True # an honest failure is a valid contract + + +def test_swarm_self_report_stays_green_while_the_gate_refuses(tmp_path): + """The sparc-gates namespace is the swarm's SELF-asserted verdict. Under + sabotage it still reads all-pass — and is replaced, not trusted.""" + ws = tmp_path / "swarm-run-self-report" + _run_example(ws, "--sabotage-holdout") + + example = _load_example() + export = example.observe(ws)["export"] + self_report = example.swarm_self_report(export) + assert self_report["all_gates_pass"] is True + assert self_report["truth_score"] == 0.97 + + assert _terminal(ws)["state"] == "FailedUnverifiable" + + +def test_interrupt_is_aborted_by_human_despite_exit_code_zero(tmp_path): + """ruflo's SIGINT path calls process.exit(0), so an interrupted run is + indistinguishable from success by exit code. The terminal must come from + the supervisor's own interrupt flag.""" + ws = tmp_path / "swarm-run-interrupted" + _run_example(ws, "--simulate-interrupt") + + observation = json.loads( + (ws / ".loop" / "artifacts" / "swarm-observation.json").read_text(encoding="utf-8") + ) + assert observation["returncode"] == 0 # the trap: ruflo reports success + assert _gate_verdict(ws)["verdict"] == "Succeeded" # and the gate is green + + terminal = _terminal(ws) + assert terminal["state"] == "AbortedByHuman" + assert terminal["state"] != "Succeeded" + + +def test_unmapped_declared_criterion_is_spec_gap(tmp_path): + """An acceptance criterion the swarm declared but no check covers is a + FailedSpecGap — the failure a self-reporting coordinator hides.""" + ws = tmp_path / "swarm-run-specgap" + _run_example(ws, "--declare-unmapped-criterion") + + terminal = _terminal(ws) + assert terminal["state"] == "FailedSpecGap" + assert terminal["criteria_met"]["AC-4"] is False + assert json.loads(_cli("doctor", ws).stdout)["ok"] is True + + +def test_ruflo_verify_is_never_wired_as_the_gate(): + """`ruflo verify` checks the SHA-256/Ed25519 integrity of the INSTALLED + artifact, not the run. A reader could plausibly mistake it for a verdict.""" + example = _load_example() + assert "verify" not in example.LIVE_COMMAND + assert "hive-mind" in example.LIVE_COMMAND and "spawn" in example.LIVE_COMMAND + assert "install-integrity" in RECIPE_DOC.read_text(encoding="utf-8") + + +def test_fixture_replay_touches_no_network_tooling(tmp_path): + """The shipped default replays a recording: the live command is never + executed, so the recipe needs no Node, no npx and no credentials.""" + ws = tmp_path / "swarm-run-offline" + env = dict(os.environ, PYTHONPATH=str(REPO_ROOT), PATH="") + proc = subprocess.run( + [sys.executable, "-B", str(EXAMPLE), str(ws)], + cwd=tmp_path, env=env, capture_output=True, text=True, + ) + assert proc.returncode == 0, proc.stdout + proc.stderr + assert _terminal(ws)["state"] == "Succeeded" + + +@pytest.mark.skipif( + not os.environ.get("LOOP_RUFLO_LIVE") or shutil.which("npx") is None, + reason="live schema-drift alarm: needs npx + network; set LOOP_RUFLO_LIVE=1 to run", +) +def test_live_swarm_status_still_matches_the_recorded_layout(tmp_path): + """Opt-in drift alarm. Runs the real CLI's cheapest state-writing verbs and + asserts `.swarm/state.json` still carries the keys the fixture records.""" + example = _load_example() + ws = tmp_path / "live-swarm" + ws.mkdir() + init = subprocess.run( + ["npx", f"ruflo@{example.RUFLO_VERSION}", "swarm", "init"], + cwd=ws, capture_output=True, text=True, + ) + assert init.returncode == 0, init.stdout + init.stderr + live_state = json.loads((ws / ".swarm" / "state.json").read_text(encoding="utf-8")) + recorded = json.loads((FIXTURE / ".swarm" / "state.json").read_text(encoding="utf-8")) + assert set(recorded) <= set(live_state), (recorded, live_state) From 93061accca4f9964939773866da4d455f3330497 Mon Sep 17 00:00:00 2001 From: Sollan Systems Date: Sat, 25 Jul 2026 12:20:33 -0400 Subject: [PATCH 3/3] ci: add the recipe-ruflo job Anchored directly after action-dogfood so a sibling recipe can land after recipe-temporal without a conflict. No npm, no Node, no credentials: the job installs only pyyaml/pytest/jsonschema and runs the fixture-replay recipe suite, matching the other recipe jobs' cost profile. --- .github/workflows/ci.yml | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 9618c97..56d3dea 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -209,3 +209,21 @@ jobs: run: | python -m pip install --quiet pre-commit pytest pyyaml python -B -m pytest -q -p no:cacheprovider scripts/test_precommit_hook.py + + recipe-ruflo: + name: recipe (ruflo) + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v7 + - uses: actions/setup-python@v6 + with: + python-version: "3.12" + - name: Install recipe dependencies + # No npm, no Node, no credentials: ruflo has no Python package, and a live + # swarm needs the `claude` binary plus model spend. The example replays the + # committed examples/ruflo-gate/fixture/ recording instead — the gate, + # projection, emit, doctor and metrics path all still execute for real. + # The opt-in live schema-drift alarm (LOOP_RUFLO_LIVE=1) stays skipped here. + run: python -m pip install --upgrade pip pyyaml pytest jsonschema + - name: ruflo recipe end-to-end + run: python -B -m pytest -q -p no:cacheprovider scripts/test_ruflo_recipe.py