Skip to content

CONFORMANCE.md

Latest commit

 

History

History
84 lines (66 loc) · 7.3 KB

CONFORMANCE.md

File metadata and controls

84 lines (66 loc) · 7.3 KB
Raw

machine-driver Conformance

machine-driver is a reference implementation of The Machine — the six-box harness for reliable long-running applied intelligence. The standard it conforms to is The Machine — Conformance Spec vNext (canonical: github.com/frontier-infra/the-machine). This file is the honest self-assessment; the kit is the source of truth — run it, don't trust this prose.

Current kit level: Machine-L2 Instrumented (vNext)

Run the executable kit instead of arguing a level:

cd <frontier-infra>/the-machine && python -m kit score <…>/machine-driver

On 2026-08-06 it scores machine-driver as a Machine-L2 Instrumented static structural candidate. Most L3 static rows pass: Δ1 reversibility, Δ2 operator override, Δ3 runtime health, alert-with-ACK, store-level idempotency, quarantine, governor, signed AAR, and hard-deny contract enforcement are present and locally regression-tested. The static candidate stops at L2 because Ops — Non-bypassability remains PARTIAL: source shows an apply_mutation chokepoint and denial paths, but the kit deliberately refuses to turn that into an L3 PASS without live bypass evidence.

Scope of the claim: this is not L3 conformance. The kit v0 is static-only and declares live chaos/replay obligations as NOT-RUN. Those rows are still required evidence before claiming Machine-L3 operation.

The six boxes → the code

Box Obligation In driver.py Status
0 Contracted Decomposition no move begins without a contract whose target is ratified by ≠ its proposer a commit is HARD-DENIED unless ratified_by != proposed_by — refused at the apply_mutation chokepoint, not a silent downgrade; propose runs (no durable mutation) are unaffected Enforced (Council ratifier wired manually)
1 Durable Goal + State goal/state outlive the session; resume the same move after kill -9 goal.json + atomic .tmp→replace; runs persisted; restart resumes pending tasks Pass
2 Dumb Driver control loop spends zero model tokens the while loop is pure deterministic plumbing; the only judgment is the worker_cmd shell-out Pass
3 Fresh Workers each move = a fresh, bounded context one new subprocess per task; no reused transcript Pass
4 Verify vs Reality no durable mutation without independent, ground-truth PASS; verifier ≠ subject verify_cmd exit code is ground truth; runner independent of the worker; no verifier ⇒ trust 0 ⇒ cannot pass; per-task Ed25519-signed AAR emitted via aar.mjs Enforced + Receipted (signed AAR → AAR-L2)
5 Autonomy Dial effective = min(operator, ceiling, verifier_trust); no verifier ⇒ propose-only; reversibility-aware; non-bypassable gate effective_mode(state, task) with a per-task reversibility term (irreversible actions must clear contract.irreversible_min_trust, strictest if unset); mutation routed through the single apply_mutation chokepoint; budget governor and operator override halt before mutation Enforced + Δ1 reversibility + Δ2 override

Cross-cutting obligations

Obligation Status
AAR receipts per-task Ed25519-signed AAR via aar.mjs → aar/<id>.json; self-verifies to AAR-L2 (verifier ≠ subject, ground-truth, evidence hash). Pass
Idempotency store-level idempotency-store.json rejects duplicate ACTIVE/DONE keys before a second worker side effect; tests/test_l3_controls.py proves duplicate key ⇒ one effect. Pass
Quarantine terminal blocked after max_attempts — surfaced, not re-queued. Pass
Escalation (with alert) block / quarantine / budget-halt / override / stale monitor → stdout/Telegram alert plus durable pending ACK. Budget halt ACK persistence is regression-tested for max-runs and wall-time. Pass
Cost / Resource Governor budget.max_worker_runs / max_wall_seconds → HALT + alert. Pass — the control the 131-duplicate incident lacked
Observability last_success_at heartbeat + hash-chained loop log (driver-log.jsonl) + independent --monitor-once staleness/anomaly pass; optional canonical frontier.machine.health.v1 check arrays fail closed across process, scheduler, execution, and governance with halted > blocked > propose_only > degraded > pass precedence. Pass (deterministic local mechanism); live probe evidence remains required for a deployment claim.
Operator override operator_override file/inline config checked before dispatch, after worker, and at the mutation gate; records effect latency vs override_effect_slo_seconds. Pass
Non-bypassability all durable commit mutations route through apply_mutation; negative tests cover contract hard-deny, mid-flight override before commit, and a verified worker diff blocked by an unratified contract. Static candidate only — the kit row remains PARTIAL until live bypass chaos is captured.

Two artifacts (iteration 2, 2026-06-14)

  • driver-log.jsonl — the hash-chained loop audit (dispatch / requeue / verify / halt). Telemetry.
  • aar/<task-id>.json — one canonical Ed25519-signed AAR per resolved task. The proof layer.

Configured by goal.json keys aar_tool · aar_priv · subject · principal. Absent ⇒ the driver runs keyless (skips the AAR; still writes driver-log.jsonl). The signing identity + key is the one operator-gated step: node aar.mjs keygen --did did:web:<domain>:machine-driver …, then publish did.json. The worker is replaceable; the principal is not.

Evidence (run it, don't trust it)

python3 -m unittest discover -s tests -v

Demonstrated: budget-breach → halt + alert · missing-verifier → propose-only · max-attempts → quarantine · hash-chain integrity · verified task → signed AAR that independently verifies to AAR-L2 · keyless run → AAR skipped, loop audit still written · duplicate idempotency key → one effect · operator override → no dispatch / no commit · hostile task id → no shell command execution · failed git commit → commit_failed, not commit · budget halt → durable pending ACK for max-runs and wall-time · bypass-negative verified diff → no commit without ratification · stale runtime health → ACK-required alert · canonical cross-fixtures prove green-process/provider-credit failure → blocked, missing verifier → propose-only, override → halted, optional failure → degraded, and blocker outranks propose-only.

What's left for L3, L4, and L5 (vNext)

  • Static L3 blocker: attach or ingest a live bypass chaos probe so the non-bypassability row can move from PARTIAL to PASS without relying on source-text heuristics.
  • True L3 blocker: capture a dated live chaos/replay packet for kill-driver resume, lying-worker-caught, override-bypass refusal, duplicate idempotency single effect, and budget/thermal breach timing. Kit v0 declares those NOT-RUN rather than faking them.
  • L4 blocker: add stable deployment_id / scope_id into signed receipts and capture independent signed-chain verification.
  • L5: barred until verifier-earned trust can safely raise the autonomy dial from executed, receipted history.