Skip to content

Feed deterministic suggestions to orchestrator planner#377

Draft
sepo-agent-app[bot] wants to merge 4 commits into
mainfrom
agent/implement-issue-367/codex-26699976832
Draft

Feed deterministic suggestions to orchestrator planner#377
sepo-agent-app[bot] wants to merge 4 commits into
mainfrom
agent/implement-issue-367/codex-26699976832

Conversation

@sepo-agent-app

Copy link
Copy Markdown

Summary

  • Rename the internal handoff policy helper to computeDeterministicTransition and add planner-facing deterministic suggestion generation.
  • Emit ORCHESTRATOR_SUGGESTED_* fields from orchestrator preflight into the planner prompt while keeping runtime validation mandatory.
  • Preserve delegated child issue execution as the deterministic worker chain and update docs/workflow descriptions to treat heuristics as a compatibility alias.

Tests

  • npm run build
  • node --test dist/__tests__/handoff.test.js
  • node --test dist/__tests__/orchestrator-preflight-cli.test.js
  • node --test dist/__tests__/envelope.test.js
  • node --test dist/__tests__/orchestrate-handoff-cli.test.js
  • git diff --check

Closes #367

@sepo-agent-app sepo-agent-app Bot added the agent Handled by the agent label May 31, 2026
@sepo-agent-app

This comment has been minimized.

@sepo-agent-app

This comment has been minimized.

@sepo-agent-app

This comment has been minimized.

@sepo-agent-app

This comment has been minimized.

@sepo-agent-app

This comment has been minimized.

@sepo-agent-app

This comment has been minimized.

@sepo-agent-app

This comment has been minimized.

@sepo-agent-app

This comment has been minimized.

@lolipopshock lolipopshock left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sepo-agent check comments

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is a first high-level comment: the updates to the documentation is not in the right tone.

For example, after this update, there is no point talking about AGENT_AUTOMATION_MODE -- it's completely gone. Also it's very weird to write in the tone of

Planner-backed orchestration is the recommended public mode. The deterministic
transition table remains internal runtime policy: it is used as a planner
suggestion and as validation after the planner responds. 

Since for the reader there is no reason for them to know this context. This could be a helpful technical detail but should not be surfaced to the audience.

ORCHESTRATOR_SUGGESTED_DECISION: ${{ steps.preflight.outputs.suggested_decision }}
ORCHESTRATOR_SUGGESTED_NEXT_ACTION: ${{ steps.preflight.outputs.suggested_next_action }}
ORCHESTRATOR_SUGGESTED_REASON: ${{ steps.preflight.outputs.suggested_reason }}
ORCHESTRATOR_SUGGESTED_HANDOFF_CONTEXT: ${{ steps.preflight.outputs.suggested_handoff_context }}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don wonder if there's a better way to pass variables between action steps. This seems so ugly ...

@sepo-agent-app

Copy link
Copy Markdown
Author

Checked. There are two unresolved, non-outdated threads, and both are valid.

  • Docs tone: agree. The architecture doc currently surfaces internal implementation language (AGENT_AUTOMATION_MODE, deterministic transition policy, heuristics compatibility) too prominently. I’d rewrite that section reader-first and move/remove compatibility details from the public narrative. Keep only a terse config/reference note if the workflow still accepts the value internally.
  • Workflow env vars: the current env mapping works and matches how run-agent-task interpolates prompt variables, but it is clunky. A cleaner fix is to have preflight emit one structured suggestion output, e.g. suggested_transition_json, then pass one env var like ORCHESTRATOR_SUGGESTED_TRANSITION. The prompt can render that JSON, while runtime validation remains unchanged.

Recommended next step: run a focused fix-pr pass for those two comments before merge.


codex | gpt-5.5/medium | high | Macmini-runner-3

@lolipopshock

Copy link
Copy Markdown
Contributor

Checked. There are two unresolved, non-outdated threads, and both are valid.

  • Docs tone: agree. The architecture doc currently surfaces internal implementation language (AGENT_AUTOMATION_MODE, deterministic transition policy, heuristics compatibility) too prominently. I’d rewrite that section reader-first and move/remove compatibility details from the public narrative. Keep only a terse config/reference note if the workflow still accepts the value internally.
  • Workflow env vars: the current env mapping works and matches how run-agent-task interpolates prompt variables, but it is clunky. A cleaner fix is to have preflight emit one structured suggestion output, e.g. suggested_transition_json, then pass one env var like ORCHESTRATOR_SUGGESTED_TRANSITION. The prompt can render that JSON, while runtime validation remains unchanged.

Recommended next step: run a focused fix-pr pass for those two comments before merge.

codex | gpt-5.5/medium | high | Macmini-runner-3

@sepo-agent /orchestrate can you fix the first documentation issue? And the second point is about a more general issue?

@sepo-agent-app

This comment has been minimized.

@sepo-agent-app

This comment has been minimized.

@sepo-agent-app

This comment has been minimized.

@sepo-agent-app

This comment has been minimized.

@sepo-agent-app

This comment has been minimized.

@lolipopshock

Copy link
Copy Markdown
Contributor

@sepo-agent /answer why does

 automation_mode:
        description: "Post-action orchestration mode (agent public default; heuristics is a compatibility alias)"

this variable should be completely removed from the corresponding files right?

@sepo-agent-app

Copy link
Copy Markdown
Author

Yes, for the public/user-facing surface, that variable should go away.

The reason is that after this change, AGENT_AUTOMATION_MODE is no longer a meaningful user choice. Describing:

automation_mode:
  description: "Post-action orchestration mode (agent public default; heuristics is a compatibility alias)"

still teaches users that there are modes they should think about. That is exactly the wrong tone and product surface. heuristics should be an internal compatibility/execution detail, not a documented option.

I would handle it as:

  • Remove AGENT_AUTOMATION_MODE from public docs/config docs.
  • Stop wiring public entry workflows through ${{ vars.AGENT_AUTOMATION_MODE || 'agent' }}; just use agent.
  • Remove or hide automation_mode from user-facing workflow_dispatch inputs where possible.
  • Keep lower-level internal compatibility only where runtime still needs it, especially for existing installed repos and delegated child worker-chain behavior.
  • Rename/refactor the remaining internal use later so it no longer looks like a public “mode” selector.

So: yes for docs and public workflow surfaces; not necessarily a blind delete from every internal file in the same PR unless we also replace the internal compatibility path.


codex | default model | high | Macmini-runner-1

@lolipopshock

Copy link
Copy Markdown
Contributor

Yes, for the public/user-facing surface, that variable should go away.

The reason is that after this change, AGENT_AUTOMATION_MODE is no longer a meaningful user choice. Describing:

automation_mode:
  description: "Post-action orchestration mode (agent public default; heuristics is a compatibility alias)"

still teaches users that there are modes they should think about. That is exactly the wrong tone and product surface. heuristics should be an internal compatibility/execution detail, not a documented option.

I would handle it as:

  • Remove AGENT_AUTOMATION_MODE from public docs/config docs.
  • Stop wiring public entry workflows through ${{ vars.AGENT_AUTOMATION_MODE || 'agent' }}; just use agent.
  • Remove or hide automation_mode from user-facing workflow_dispatch inputs where possible.
  • Keep lower-level internal compatibility only where runtime still needs it, especially for existing installed repos and delegated child worker-chain behavior.
  • Rename/refactor the remaining internal use later so it no longer looks like a public “mode” selector.

So: yes for docs and public workflow surfaces; not necessarily a blind delete from every internal file in the same PR unless we also replace the internal compatibility path.

codex | default model | high | Macmini-runner-1

@sepo-agent /orchestrate

@sepo-agent-app

This comment has been minimized.

@sepo-agent-app

Copy link
Copy Markdown
Author

Sepo pushed fixes for this PR. Branch: agent/implement-issue-367/codex-26699976832. Requested by @sepo-agent.

  • Removed AGENT_AUTOMATION_MODE from public config/docs and reworded orchestration docs so users are not taught a mode selector.
  • Public entry paths now pass agent directly; lower-level workflow inputs remain only as internal Sepo handoff state for compatibility and child worker chains.
  • Updated workflow/docs contract coverage for the public-surface change.
  • Checks: npm run build; node --test dist/__tests__/envelope.test.js dist/__tests__/dispatch-agent-orchestrator-cli.test.js dist/__tests__/orchestrate-handoff-cli.test.js; git diff --check.

codex | default model | xhigh | Macmini-runner-4

@sepo-agent-app

sepo-agent-app Bot commented May 31, 2026

Copy link
Copy Markdown
Author

Sepo is dispatching follow-up automation.

Source Next Target Round Status
fix-pr review PR #377 3 / 40 Dispatched

Reason: agent planner selected review: fix-pr completed successfully on the pull request, so one follow-up review is warranted to verify the public workflow/docs surface changes and ensure no regressions.

@sepo-agent-app

Copy link
Copy Markdown
Author

Rubrics Review

Total Score Verdict Rubrics Scored
100 PASS 13
Dimension Rubric Result Score Evidence
coding_workflow Make surgical changes pass 8/8 Scope stays on orchestration mode public surface, planner suggestion wiring, docs, and focused tests.
coding_workflow Reuse existing code pass 8/8 computeDeterministicSuggestion() wraps the existing transition logic rather than duplicating policy.
export function computeDeterministicSuggestion(input: HandoffInput): DeterministicTransitionSuggestion {
coding_style Prefer simple implementations pass 7/7 The change uses a small preflight output path plus prompt env vars, with no new policy engine.
const suggestion = computeDeterministicSuggestion({
coding_workflow Prefer explicit structured inputs pass 7/7 Suggested transition data is separate typed fields: suggested_decision, suggested_next_action, suggested_reason, and suggested_handoff_context.
setOutput("suggested_decision", suggestion.suggestedDecision);
coding_workflow Prefer agent-derived decisions pass 7/7 The planner receives deterministic guidance as a default, while retaining judgment and mandatory runtime validation.
## Deterministic Suggestion
coding_workflow Separate decision gates from action authorization pass 8/8 Self-approval and self-merge remain explicit gated transitions in runtime policy.
if (sourceAction === "review") {
coding_workflow Validate delegated route authorization pass 8/8 Preflight still checks delegated-route capability before planner dispatch.
const authorizationStopReason = initialOrchestrateCapabilityStopReason({
coding_workflow Default follow-up orchestration to one-shot pass 8/8 Prompt now defaults later user-launched follow-up rounds to stop unless continued automation is explicit.
- For second-or-later user-launched follow-up rounds, default to `stop`: if the
coding_workflow Keep docs in sync pass 7/7 Docs now describe planner-backed orchestration, internal deterministic worker chains, and max-round configuration without exposing a public mode selector.
The packaged workflows use planner-backed orchestration by default and keep
communication Write audience-first documentation pass 7/7 The architecture doc opens with user task framing and no longer front-loads AGENT_AUTOMATION_MODE or heuristics.
The orchestrator is the explicit route for bounded follow-up automation. Start
communication Keep temporary migration notes out of durable docs pass 6/6 AGENT_AUTOMATION_MODE is removed from the public configuration list rather than preserved as a compatibility note.
| `AGENT_AUTOMATION_MAX_ROUNDS` | Maximum number of explicit orchestration handoff rounds. Defaults to `12`. |
coding_workflow Name config by capability pass 6/6 Public entry workflows now pass automation_mode: agent directly instead of wiring a repository variable as a user choice.
automation_mode: agent
coding_workflow Minimal defaults, native escape hatches pass 8/8 Lower-level workflow inputs are retained only as internal handoff state, while public entry defaults stay minimal.

Notes

  • The latest PR head matches the linked Consider removing heuristic orchestrate mode #367 plan and the later maintainer clarification: remove the public/user-facing mode selector, keep internal compatibility for existing handoff behavior and deterministic child worker chains.
  • Focused contract tests now guard against reintroducing AGENT_AUTOMATION_MODE into config docs and public entry workflow wiring.
  • No rubric-specific follow-up needed.

Findings

  • INFO: No rubric-specific findings.

Final Rubric Verdict

PASS


codex | gpt-5.5/medium | high | Macmini-runner-4

@sepo-agent-app

Copy link
Copy Markdown
Author

AI Review Synthesis

Dual-agent review by Claude and Codex.
Requested by @lolipopshock.

Summary of PR/Issue

PR #377 implements phase 1 of #367 by reframing the old public heuristics mode as internal deterministic transition policy. It adds planner-facing ORCHESTRATOR_SUGGESTED_* outputs from orchestrator preflight while keeping runtime validation mandatory after planner output. Follow-up fixes remove AGENT_AUTOMATION_MODE from public docs/config and public entry workflow wiring, while preserving internal handoff compatibility for Sepo-dispatched chains. The broader goal is planner-backed orchestration as the public default with deterministic policy kept as a guardrail.

Review

The current reviewer artifacts converge on ship-ready status. Codex and the rubrics review report no blocking or warning findings; Claude’s remaining notes are optional cleanup or awareness items rather than required branch work.

Issue Severity Description
No actionable issues INFO No BLOCKING or WARNING findings remain. The remaining env-var fan-out and type-hardening notes are optional future cleanup, not required branch changes for this PR.

Progress

  • The earlier one-shot follow-up prompt warning is fixed and covered by prompt contract assertions.
  • The latest public-surface request is addressed: public docs no longer list AGENT_AUTOMATION_MODE, public entry workflows pass agent directly, and lower-level workflow inputs are framed as internal handoff state.
  • The docs-tone human thread appears addressed in the current head; the env-var thread remains a broader cleanup already covered by existing human inline feedback.
  • No inline comments were posted, replied to, minimized, or resolved; existing review threads are human-authored and should not be mutated by this synthesis.
  • Current PR check rollup is green for Test Scripts / check on head eaadc460933ebb24757c8dff9021b68e88b48595.

Recommended Next Step

NO_AUTOMATED_ACTION: no unresolved required branch-change work remains.

Final Verdict

SHIP

Action Items

  • No required branch-change work.

codex | gpt-5.5/medium | xhigh | Macmini-runner-2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent Handled by the agent

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Consider removing heuristic orchestrate mode

2 participants