feat: add typed experiment ledger

CONTEXT.md

@@ -0,0 +1,17 @@
+# OpenFlywheel
+
+OpenFlywheel governs evidence-backed changes to an agent harness.
+
+## Language
+
+**Experiment attempt**:
+One terminal evaluation of a focused harness-change hypothesis against a parent revision.
+_Avoid_: Trial, experiment run
+
+**Verifier receipt**:
+An opaque identifier proving that an authoritative outcome was recorded for one evaluated task.
+_Avoid_: Score, verifier result
+
+**Gate decision**:
+The terminal decision to admit or reject an experiment attempt after applying configured checks.
+_Avoid_: Status, verdict

plugins/openflywheel/.codex-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "openflywheel",
   "version": "0.6.0",
-  "description": "Initialize ITSM-bench harness workspaces, query Langfuse trajectories, and mine compact failure diagnoses and exact patterns.",
+  "description": "Initialize ITSM-bench harness workspaces, query Langfuse trajectories, and record outcomes, failure diagnoses, experiment attempts, and exact patterns.",
   "author": {
     "name": "OpenFlyWheel"
   },
@@ -10,8 +10,8 @@
   "skills": "./skills/",
   "interface": {
     "displayName": "OpenFlyWheel",
-    "shortDescription": "Diagnose and mine ITSM failure patterns",
-    "longDescription": "Initialize an ITSM-bench agent-harness optimization workspace, inspect bounded Langfuse evidence, record authoritative outcomes and compact diagnoses, and mine exact recurring patterns without copying trace payloads.",
+    "shortDescription": "Ledger and mine ITSM failures",
+    "longDescription": "Initialize an ITSM-bench agent-harness optimization workspace, inspect bounded Langfuse evidence, record authoritative outcomes, compact diagnoses, and terminal experiment attempts, and mine exact recurring patterns without copying trace payloads.",
     "developerName": "OpenFlyWheel",
     "category": "Productivity",
     "capabilities": ["Read", "Write"],

plugins/openflywheel/.mcp.json

@@ -4,7 +4,7 @@
       "command": "uvx",
       "args": [
         "--from",
-        "git+https://github.com/divo12/OpenFlyWheel.git@ab0ef62cbe1e6cddf0bfd8ec61374d10120c61aa",
+        "git+https://github.com/divo12/OpenFlyWheel.git@7076900824da68b3ba62690985f893c273d5748a",
         "--with",
         "mcp>=1.13,<2",
         "openflywheel-mcp"

plugins/openflywheel/program_templates/base.md

@@ -54,7 +54,19 @@ Run only the prepared experiment command and gates declared by the workspace. Co
 task-level verifier outcomes and report quality, cost, and latency separately. Missing or
 errored trials remain visible and cannot disappear from the denominator.
 
-### 6. Keep or revert
+### 6. Record the experiment attempt
+
+After every candidate reaches a terminal gate decision, call `record_experiment` exactly once.
+Record the current accepted parent revision, focused hypothesis, all available authoritative
+verifier receipt IDs, gate decision, total Langfuse cost, latency, UTC decision time, and a
+rejection reason when the gate rejects the candidate. Cost, latency, and verifier receipts may
+be absent only when a rejected candidate never produced that evidence.
+
+Retain the returned `.workspace/experiments/` artifact path. If recording fails, stop rather
+than committing an unledgered change. The ledger contains references and aggregate measurements,
+never Langfuse trace payloads.
+
+### 7. Keep or revert
 
 Keep the change only when the configured gate admits it. Otherwise revert only the current
 iteration's harness edit, retain the evidence, and try a different hypothesis. Never weaken
@@ -66,7 +78,7 @@ trailers. Do not commit failed candidates, generated run artifacts, credentials,
 outside the editable surface. Do not push or open a pull request without explicit user
 authorization.
 
-### 7. Repeat
+### 8. Repeat
 
 Return to step 2 with the newly recorded run. Stop when the configured goal is met, the
 budget or iteration limit is exhausted, the no-improvement condition is reached, or required

src/ofw/__init__.py

@@ -27,6 +27,14 @@
     WorkspaceFile,
 )
 from ofw.evaluation import (
+    ExperimentAttempt,
+    ExperimentDecision,
+    ExperimentId,
+    ExperimentLedgerErrorCode,
+    ExperimentLedgerFailure,
+    ExperimentRecordObservation,
+    ExperimentRecordStatus,
+    ExperimentRunId,
     FailureDiagnosis,
     FailureDiagnosisError,
     FailureErrorCode,
@@ -46,6 +54,7 @@
     OutcomeScoreSubmission,
     OutcomeStoreObservation,
     OutcomeStoreStatus,
+    RecordExperimentInput,
     TaskId,
     VerifierId,
 )
@@ -112,6 +121,14 @@ def editable(self, path: Path) -> EditableFile:
     "E2BSandbox",
     "EditableFile",
     "EvidenceReference",
+    "ExperimentAttempt",
+    "ExperimentDecision",
+    "ExperimentId",
+    "ExperimentLedgerErrorCode",
+    "ExperimentLedgerFailure",
+    "ExperimentRecordObservation",
+    "ExperimentRecordStatus",
+    "ExperimentRunId",
     "FailureDiagnosis",
     "FailureDiagnosisError",
     "FailureErrorCode",
@@ -151,6 +168,7 @@ def editable(self, path: Path) -> EditableFile:
     "RepositorySnapshot",
     "ProcessCommand",
     "ProcessLimits",
+    "RecordExperimentInput",
     "RunErrorCode",
     "RunResult",
     "RunStatus",

src/ofw/evaluation/__init__.py

@@ -1,5 +1,16 @@
 """Provider-agnostic evaluation contracts."""
 
+from ofw.evaluation.experiment_ledger import (
+    ExperimentAttempt,
+    ExperimentDecision,
+    ExperimentId,
+    ExperimentLedgerErrorCode,
+    ExperimentLedgerFailure,
+    ExperimentRecordObservation,
+    ExperimentRecordStatus,
+    ExperimentRunId,
+    RecordExperimentInput,
+)
 from ofw.evaluation.failure import (
     FailureDiagnosis,
     FailureDiagnosisError,
@@ -31,6 +42,14 @@
 )
 
 __all__ = [
+    "ExperimentAttempt",
+    "ExperimentDecision",
+    "ExperimentId",
+    "ExperimentLedgerErrorCode",
+    "ExperimentLedgerFailure",
+    "ExperimentRecordObservation",
+    "ExperimentRecordStatus",
+    "ExperimentRunId",
     "FailureDiagnosis",
     "FailureDiagnosisError",
     "FailureErrorCode",
@@ -50,6 +69,7 @@
     "OutcomeScoreSubmission",
     "OutcomeStoreObservation",
     "OutcomeStoreStatus",
+    "RecordExperimentInput",
     "TaskId",
     "VerifierId",
 ]