Skip to content

feat: execute isolated candidates - #35

Merged
divo12 merged 14 commits into
stack/v2-01-experimentfrom
stack/v2-02-candidate
Sep 3, 2026
Merged

feat: execute isolated candidates#35
divo12 merged 14 commits into
stack/v2-01-experimentfrom
stack/v2-02-candidate

Conversation

@divo12

@divo12 divo12 commented Sep 2, 2026

Copy link
Copy Markdown
Owner

Summary

  • add strict candidate identity, execution observations, and typed failure/blocker receipts over the PR1 policy and hypothesis authority
  • create detached candidate worktrees from the accepted experiment commit, freeze every non-target path, seal the Git tree, and commit exact OFW-Experiment and OFW-Run trailers
  • generalize Harbor behind the existing baseline adapter, launch and poll re-entrantly with frozen tasks/model/verifier/config/environment/concurrency/retries, then map exact Langfuse traces into existing authoritative outcome score receipts
  • expose execute_candidate and advance the closed-loop program through candidate execution while stopping before admission

Evidence and trust flow

MCP execute_candidate -> CandidateExecutionService -> PR1 policy/hypothesis repository -> CandidateGitGateway -> HarborExperimentRunner -> bounded Langfuse root-trace locator -> existing LangfuseOutcomeStore.

Caller paths identify local runtime locations only. The accepted commit, exact editable targets, model, ordered task IDs, verifier, config digest, environment, concurrency, and retries come from the immutable PR1 snapshot and are revalidated before launch and poll. CandidateId binds the policy digest, hypothesis ID, accepted source commit, candidate Git tree, and controls digest. Local state retains only IDs, digests, paths, counts, blockers, and outcome receipts; Langfuse trace payloads remain remote.

Verification

  • uv sync --extra dev --extra plugin
  • uv run ruff check src tests plugins/openflywheel/scripts/mcp_server.py
  • uv run mypy src tests plugins/openflywheel/scripts/mcp_server.py
  • uv run pytest --cov=ofw --cov-report=term-missing --cov-fail-under=90 -q: 271 passed, 2 skipped, 95.02%
  • uvx --from radon radon cc -s -a src tests plugins/openflywheel/scripts/mcp_server.py: every changed production function CC <= 5
  • all seven packaged skill quick validators passed
  • plugin validation passed

Risks

  • candidate execution intentionally stops before admission or accepted-branch mutation; gating belongs to the next stacked change
  • Harbor and Langfuse live compatibility remains covered by the existing opt-in live tests, which were skipped locally without credentials

Stacked on PR #34. Do not merge before its base is accepted.


Summary by cubic

The closed-loop flow now executes recorded hypotheses in isolated candidate worktrees and records durable evaluated-run receipts instead of stopping after hypothesis creation. It still stops before admission or mutation of the accepted branch.

Candidate execution

  • Adds the execute_candidate MCP tool with re-entrant launch, polling, recovery, and timeout cancellation.
  • Creates a detached worktree from the accepted commit, edits only policy-approved targets, and preserves frozen controls across retries and restarts.
  • Binds candidates to policy, hypothesis, source commit, Git tree, and controls digests; stale or tampered inputs fail safely.
  • Runs generalized Harbor experiments, accepts Harbor task id objects, and maps bounded Langfuse traces to strict receipts with defaults for omitted metrics.

Verification

  • Covers candidate isolation, lifecycle recovery, Harbor execution, receipt validation, MCP behavior, and template routing.

Written for commit b9df048. Summary will update on new commits.

Review in cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 19 files

Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.

Re-trigger cubic

Comment thread src/ofw/evolution/candidate_service.py Outdated
Comment thread src/ofw/evolution/candidate_git.py
Comment thread src/ofw/evolution/hypothesis_repository.py Outdated
Comment thread src/ofw/evolution/candidate_service.py
Comment thread src/ofw/preparation/harbor.py Outdated
Comment thread src/ofw/evolution/candidate.py
Comment thread src/ofw/preparation/harbor.py Outdated
Comment thread src/ofw/preparation/contracts.py
Comment thread src/ofw/evolution/candidate_service.py Outdated
Comment thread tests/test_harbor_experiment.py Outdated
@divo12
divo12 force-pushed the stack/v2-01-experiment branch from dd67464 to d6d19a7 Compare September 3, 2026 07:49

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 12 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread src/ofw/preparation/harbor.py Outdated
Comment thread src/ofw/evolution/candidate_service.py

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 8 files (changes from recent commits).

Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic

Comment thread src/ofw/preparation/harbor.py

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 10 files (changes from recent commits).

Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic

Comment thread src/ofw/evaluation/outcome.py Outdated
Comment thread src/ofw/evaluation/outcome.py Outdated
Comment thread tests/test_evaluated_run_receipt.py

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 7 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/ofw/evolution/candidate_service.py">

<violation number="1" location="src/ofw/evolution/candidate_service.py:81">
P2: When an existing candidate state was written by the previous service format, this state migration rejects its `outcome_receipts` and `blockers` fields and returns `INVALID_RESULT` instead of resuming or replaying the candidate. Add a versioned compatibility migration before replacing the persisted fields.</violation>
</file>

Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic


@dataclass(frozen=True, slots=True)
class _OutcomeReduction:
receipts: tuple[EvaluatedTaskReceipt, ...]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When an existing candidate state was written by the previous service format, this state migration rejects its outcome_receipts and blockers fields and returns INVALID_RESULT instead of resuming or replaying the candidate. Add a versioned compatibility migration before replacing the persisted fields.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/ofw/evolution/candidate_service.py, line 81:

<comment>When an existing candidate state was written by the previous service format, this state migration rejects its `outcome_receipts` and `blockers` fields and returns `INVALID_RESULT` instead of resuming or replaying the candidate. Add a versioned compatibility migration before replacing the persisted fields.</comment>

<file context>
@@ -72,24 +72,22 @@ class _CandidateState(StrictModel):
 class _OutcomeReduction:
-    receipts: tuple[CandidateOutcomeReceipt, ...]
-    blockers: tuple[CandidateBlocker, ...]
+    receipts: tuple[EvaluatedTaskReceipt, ...]
+    blockers: tuple[EvaluatedRunBlocker, ...]
 
</file context>

@divo12
divo12 force-pushed the stack/v2-02-candidate branch from 6de892f to 1e56f29 Compare September 3, 2026 12:25
@divo12
divo12 merged commit 70a47e0 into stack/v2-01-experiment Sep 3, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant