fix(runtime): preserve Orka tool approval outcomes - #27
Conversation
Signed-off-by: Sertac Ozercan <sozercan@gmail.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
Cross-repository execution semantics and the inaccessible SAML-protected companion Orka changes require final human validation.
Review tier: Balanced (auto)
Findings: None
Note
Copilot is running an experiment and ran this review at Balanced.
What changed in this PR
Preserves final Orka approval outcomes across direct MAF and hosted Foundry continuations without exposing tool-controlled diagnostics or retrying uncertain actions.
Changes:
- Adds fixed normalization for six Orka error outcomes.
- Stops model continuation for
tool_outcome_unknown. - Documents timeout, persistence, and deployment requirements with expanded tests.
Review found no actionable issues. The PR-reported tests were not rerun, and the structured review helper was unavailable without command execution.
| File | Description |
|---|---|
runtimes/microsoft-agent-framework/tests/test_orka_approvals.py |
Tests direct approval outcomes, cancellation, isolation, and timeout behavior. |
runtimes/microsoft-agent-framework/agentkit_serve/agent_factory.py |
Preserves normalized Orka errors and terminates unknown outcomes. |
runtimes/common/tests/test_foundry_brokered_protocol.py |
Updates expected normalized broker responses. |
runtimes/common/tests/test_foundry_approvals.py |
Tests hosted approval recovery and continuation semantics. |
runtimes/common/agentkit_serve_common/tool_errors.py |
Defines the fixed Orka outcome allowlist. |
runtimes/common/agentkit_serve_common/foundry.py |
Normalizes deterministic hosted responses. |
runtimes/common/agentkit_serve_common/foundry_model_loop.py |
Sanitizes outcomes and stops uncertain continuations. |
docs/runtime-capabilities.md |
Documents qualified approval capability support. |
docs/runtime-adapters.md |
Clarifies MAF timeout behavior. |
docs/orka.md |
Documents Orka registration requirements. |
docs/orka-human-approval.md |
Adds approval setup, outcome, and acceptance guidance. |
docs/foundry-hosted-brokered.md |
Documents hosted approval and persistence requirements. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Orka-managed tool calls can remain open while a person reviews the action. Preserve the final approval or execution outcome for that original call, including when Microsoft Agent Framework hides ordinary tool exception messages.
tool_outcome_unknownwithout another model request. Preserve the original call identity and cached continuation behavior.AGENTKIT_MCP_TIMEOUT=900override for the existing MCP request and HTTP timeouts. The default remains 120 seconds elsewhere. Hosted approval setups require a pending-state TTL and session idle timeout of at least 1800 seconds.Validation passed:
AGENTKIT_TEST_APPROVAL_WAIT_SECONDS=121: one test passed in 121.53 seconds.go test ./...,make lint, andgit diff --check.Companion implementations: Orka PR 589 and Foundry PR 5, for Orka issue 582. These checks establish local runtime and hosted continuation compatibility; they do not claim Azure-hosted end-to-end validation. Orka continues to own review policy, persisted decisions, cancellation, and execution authority.