Skip to content

Let Foundry-hosted agents wait for Orka tool approvals #4

Description

Problem

An agent running in Microsoft Foundry may ask to run an operation, such as creating a work order, that needs a person's approval. Orka should collect that decision and control whether the operation runs. The Foundry runtime is the bridge that must keep the request waiting and then return the outcome to the same agent conversation.

This requires the Orka implementation tracked in orka-agents/orka#582. The bridge currently documents approval-required tools as unsupported.

There is also a concrete timing limit: the bridge gives each tool HTTP request a fixed 120-second total timeout. If Orka allows a longer review, simply enabling approvals would still let the bridge abandon a valid request before the person decides.

This issue targets the deployment used by Fibey: the runtime bridge runs in Kubernetes, and the agent runs in Foundry.

Why this matters

A person may need several minutes to review an action. That waiting period must not cause a false failure, another execution of the tool, or a new model request pretending the tool has finished.

The bridge already waits for tool results and remembers which tool call and previous response belong together. Extend that existing flow. The bridge's broker keeps track of which task owns the remote work. Orka remains responsible for approval policy, authorized reviewers, saved decisions, and tool execution.

What should happen

  1. The hosted agent asks for a tool that Orka has configured to require approval.
  2. Orka presents the review. The bridge waits within the agreed time limit; neither the tool nor the next model step runs.
  3. Approval allows Orka to execute the tool. The bridge returns the actual result to the original pending call.
  4. Decline returns Orka's defined denial outcome without running the tool.
  5. Cancellation or expiry stops the wait, and a later decision cannot restart the old work.

A tool failing after approval, a person declining a review, and a caller lacking permission are different outcomes. Preserve those distinctions.

How to approach it

  • Follow the waiting and final-result behavior defined by Add human approval for tools used by Orka v2 agents orka#582. The current tool client expects one final JSON response. Use that path if it is sufficient; implement additional messages only if Orka's design requires them.
  • Align the tool wait with Orka's allowed review duration and task cancellation. Separate a long approval wait from the shared 120-second timeout if needed. Keep connection, model-request, and cleanup timeouts bounded.
  • Preserve the original pending call and the broker's record of which task owns it. Keep ownership valid during an authorized wait and reject continuation after that ownership ends.
  • Use the existing continuation checks when a final result arrives. Never interpret an ordinary authorization error as a request to wait or retry.
  • Reuse the current cancellation and cleanup paths. A closed connection alone does not prove that remote work stopped or that an action did not run.
  • Extend the existing integration tests with a harmless tool that counts executions and a real held approval. The existing simulated authorization-denial and held-model tests do not establish this behavior.
  • Document the supported versions, review time limits, and expected outcomes. Remove the unsupported warning only after the complete flow passes.

The AgentKit compatibility work in #3 uses an approved field in its tool-result format. In that format, true accompanies a successful tool result and false accompanies an error. Neither value means "waiting for a person," and true is not proof of human approval. Keep the human decision in Orka's records; return only the agreed, safe result to the agent.

Acceptance checks

  • A review can remain pending for the documented duration, including longer than two minutes if Orka supports that duration. There is no tool execution or model continuation while waiting.
  • Approval produces one counted tool execution and one continuation of the original call. A later approval-required action needs its own decision.
  • Decline executes nothing and returns the agreed outcome. Authorization failures still stop the request.
  • Cancellation, task timeout, approval expiry, and a decision arriving afterward cannot start or resume work that has lost permission to run.
  • Approval racing cancellation, duplicate decisions, and stale decisions cannot cause extra tool executions or repeated model submissions.
  • Losing the tool result after execution does not trigger automatic repetition. If the outcome of a remote request is unknown, the existing ownership evidence is preserved.
  • Broker restart while review is pending does not silently resume an old request or accept a late decision using its old ownership. Preserve the current supported recovery limits.
  • Existing result validation and checks that a continuation belongs to the correct task, session, and call still pass.
  • Reviewer credentials, private decision metadata, and continuation secrets do not reach the model or logs.
  • Existing tools that do not require human approval continue to work.
  • A live test uses Orka's real approval API and the configured Foundry gateway, with a harmless counted tool. Record the decision, execution count, conversation continuation, and cleanup. Local tests alone do not mark this live check complete.

Scope and dependencies

This issue does not add another approval database or review interface. It also does not promise recovery from every runtime crash: the existing limitation on resuming work after supervisor loss remains a separate problem. Work with an unknown outcome must stay blocked without automatic repetition.

Code starting points

The current limitation and timeout are present on main at 4e83f883:

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions