Skip to content

fix(runtime): preserve Orka tool approval outcomes - #27

Merged
sozercan merged 1 commit into
mainfrom
human-approval-v2
Sep 20, 2026
Merged

sozercan merged 1 commit into
mainfrom
human-approval-v2

Conversation

@sozercan

@sozercan sozercan commented Sep 14, 2026 •

Copy link
Copy Markdown
Owner

Orka-managed tool calls can remain open while a person reviews the action. Preserve the final approval or execution outcome for that original call, including when Microsoft Agent Framework hides ordinary tool exception messages.

  • Normalize the six explicit Orka error codes to fixed messages in direct MAF tool results and hosted continuations. Keep declined, expired, cancelled, stale, failed, and unknown outcomes distinct without forwarding tool-controlled error text.
  • Stop direct runs and finish hosted continuations on tool_outcome_unknown without another model request. Preserve the original call identity and cached continuation behavior.
  • Document and verify the qualified Orka child’s AGENTKIT_MCP_TIMEOUT=900 override for the existing MCP request and HTTP timeouts. The default remains 120 seconds elsewhere. Hosted approval setups require a pending-state TTL and session idle timeout of at least 1800 seconds.
  • Cover counted waits, cancellation with late decisions, independent conversations, process restart with persisted pending state, lost or expired state, and duplicate or conflicting continuation delivery.

Validation passed:

  • Common Python suite: 930 tests.
  • Microsoft Agent Framework suite: 142 tests.
  • Production HTTP MCP client and SDK with AGENTKIT_TEST_APPROVAL_WAIT_SECONDS=121: one test passed in 121.53 seconds.
  • go test ./..., make lint, and git diff --check.
  • Local code review completed with no actionable findings.

Companion implementations: Orka PR 589 and Foundry PR 5, for Orka issue 582. These checks establish local runtime and hosted continuation compatibility; they do not claim Azure-hosted end-to-end validation. Orka continues to own review policy, persisted decisions, cancellation, and execution authority.

Signed-off-by: Sertac Ozercan <sozercan@gmail.com>
Copilot AI balanced review requested due to automatic review settings September 14, 2026 21:48
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 14, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-14T21:51:24.178671Z 490bb6d PR opened
🔒 Security Review ✅ Completed 2026-09-14T21:53:49.378553Z 490bb6d PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Cross-repository execution semantics and the inaccessible SAML-protected companion Orka changes require final human validation.

Review tier: Balanced (auto)
Findings: None

Note

Copilot is running an experiment and ran this review at Balanced.

What changed in this PR

Preserves final Orka approval outcomes across direct MAF and hosted Foundry continuations without exposing tool-controlled diagnostics or retrying uncertain actions.

Changes:

  • Adds fixed normalization for six Orka error outcomes.
  • Stops model continuation for tool_outcome_unknown.
  • Documents timeout, persistence, and deployment requirements with expanded tests.

Review found no actionable issues. The PR-reported tests were not rerun, and the structured review helper was unavailable without command execution.

File Description
runtimes/​microsoft-agent-framework/​tests/​test_orka_approvals.py Tests direct approval outcomes, cancellation, isolation, and timeout behavior.
runtimes/​microsoft-agent-framework/​agentkit_serve/​agent_factory.py Preserves normalized Orka errors and terminates unknown outcomes.
runtimes/​common/​tests/​test_foundry_brokered_protocol.py Updates expected normalized broker responses.
runtimes/​common/​tests/​test_foundry_approvals.py Tests hosted approval recovery and continuation semantics.
runtimes/​common/​agentkit_serve_common/​tool_errors.py Defines the fixed Orka outcome allowlist.
runtimes/​common/​agentkit_serve_common/​foundry.py Normalizes deterministic hosted responses.
runtimes/​common/​agentkit_serve_common/​foundry_model_loop.py Sanitizes outcomes and stops uncertain continuations.
docs/​runtime-capabilities.md Documents qualified approval capability support.
docs/​runtime-adapters.md Clarifies MAF timeout behavior.
docs/​orka.md Documents Orka registration requirements.
docs/​orka-human-approval.md Adds approval setup, outcome, and acceptance guidance.
docs/​foundry-hosted-brokered.md Documents hosted approval and persistence requirements.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@sozercan
sozercan merged commit 9165dd1 into main Sep 20, 2026
23 checks passed
@sozercan
sozercan deleted the human-approval-v2 branch September 20, 2026 22:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants