Skip to content

feat(runtime): support skills and governed tool workflows - #25

Merged
sozercan merged 9 commits into
mainfrom
feat/governed-tool-workflows
Sep 14, 2026
Merged

sozercan merged 9 commits into
mainfrom
feat/governed-tool-workflows

Conversation

@sozercan

@sozercan sozercan commented Sep 11, 2026 •

Copy link
Copy Markdown
Owner

Agents can load packaged instruction skills and complete workflows that need several Orka-controlled tools. Microsoft Agent Framework exposes load_skill for SKILL.md files packaged in the image. The hosted Foundry model loop uses the same catalog and supports up to 16 sequential tool calls per user turn.

Each operational call returns to Orka. Hosted continuations validate response/call pairs, cache retries, preserve completed rounds across restarts, and retain bounded dialogue for session follow-ups. Hosted /responses acknowledges streaming requests before model work, both with and without brokered tools, and keeps the same response ID and creation time through completion or failure. A client disconnect cancels the active model request, allowing Orka to settle an acknowledged response through authenticated session stop and idle checks.

Brokered model failures preserve known error codes and the upstream HTTP status across continuations. Model and tool-validation messages come from a fixed list; malformed tool names, argument keys, and argument paths are omitted. Provider messages, headers, URLs, identifiers, and credentials never enter the diagnostic payload.

Explicit model HTTP 429 rejections can retry twice within the same hosted response. Retry delays honor server headers up to 60 seconds each; disconnecting the stream cancels the wait. Transport failures and other HTTP statuses are not retried. Streaming events carry sequence numbers for the acknowledgement and terminal result.

Validation:

  • Common Python suite: 868 passed, including early acknowledgement with and without tools, native model HTTP socket closure on disconnect, initial and continuation cancellation, stable response IDs and creation times, persisted replay after restart, exact message-size boundaries, safe model and tool-validation diagnostics, and bounded 429 retries. Real HTTP/ASGI tests verify disconnect during a 60-second backoff. Six retry regressions and twenty validation-error regressions fail against their preceding implementations.
  • Microsoft Agent Framework suite: 128 passed for the skill integration.
  • go test ./... and make lint passed.
  • Cross-repository tests with the real hosted AgentKit server and Foundry broker passed sequential tools, tool-error recovery, authorization denial, stripped-proof rejection, cancellation, and lease expiry. Dropping the acknowledgement keeps the request uncertain and prevents replay or deletion.

The cross-repository tests use model and Azure lifecycle fixtures. Orka external v2 human tool approvals remain unsupported.

Companion broker change: orka-agents/agent-runtime-foundry#3

Signed-off-by: Sertac Ozercan <sozercan@gmail.com>
Copilot AI balanced review requested due to automatic review settings September 11, 2026 20:08
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 11, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-12T00:03:44.051385Z cc6745a New commits
🔒 Security Review ✅ Completed 2026-09-11T20:14:45.116257Z 5e2b9b8 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Expired session-bound responses can be treated as stateless when the session identity is omitted.

Get a fresh assessment by requesting another Copilot review.

Review tier: Balanced
Findings: 1 Medium severity

New issues introduced by this change (1)
Severity Finding
Medium severity runtimes/​common/​agentkit_serve_common/​foundry.py — Reject expired session state when the session ID is omitted View comment
What changed in this PR

Adds packaged instruction skills and multi-step governed Foundry tool workflows with durable replay and session history.

Changes:

  • Adds immutable, validated SKILL.md catalogs and MAF load_skill support.
  • Enables bounded sequential brokered tool calls with restart-safe replay.
  • Documents packaging, hosting, and Orka integration.
File Description
runtimes/​microsoft-agent-framework/​tests/​test_instruction_skills.py Tests governed MAF skill loading.
runtimes/​microsoft-agent-framework/​agentkit_serve/​agent_factory.py Exposes packaged skills through load_skill.
runtimes/​common/​tests/​test_instruction_skills.py Tests catalog validation and filesystem safety.
runtimes/​common/​tests/​test_foundry_tool_workflows.py Tests sequential workflows, replay, and history.
runtimes/​common/​tests/​test_foundry_brokered_protocol.py Updates brokered-loop expectations.
runtimes/​common/​tests/​test_acp_protocol.py Tests ACP skill binding and capability checks.
runtimes/​common/​README.md Documents ACP skill support.
runtimes/​common/​agentkit_serve_common/​skills.py Implements immutable packaged-skill catalogs.
runtimes/​common/​agentkit_serve_common/​foundry.py Adds workflow state, replay, and follow-ups.
runtimes/​common/​agentkit_serve_common/​foundry_model_loop.py Implements bounded sequential tool execution.
runtimes/​common/​agentkit_serve_common/​config.py Stores verified skill snapshots privately.
runtimes/​common/​agentkit_serve_common/​acp.py Permits verified packaged skills in ACP.
docs/​runtime-adapters.md Updates adapter capabilities.
docs/​orka.md Documents governed skill constraints.
docs/​instruction-skills.md Adds skill packaging guidance.
docs/​foundry-hosted-brokered.md Documents workflows and follow-ups.
docs/​agentkitfile.md Documents instruction-only skill configuration.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread runtimes/common/agentkit_serve_common/foundry.py Outdated
Signed-off-by: Sertac Ozercan <sozercan@gmail.com>
Copilot AI review requested due to automatic review settings September 11, 2026 20:18

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The persisted multi-round workflow and security boundary are high-risk, while public gateway and deployed Fibey validation remain outstanding.

Review tier: Balanced
Findings: None

Issues resolved since last review (1)
Severity Finding
Medium severity runtimes/​common/​agentkit_serve_common/​foundry.py — Reject expired session state when the session ID is omitted View resolved comment

Signed-off-by: Sertac Ozercan <sozercan@gmail.com>
Copilot AI review requested due to automatic review settings September 11, 2026 21:51

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The model-message limit uses a different JSON encoding than persistence, causing valid near-limit requests to be rejected.

Get a fresh assessment by requesting another Copilot review.

Review tier: Balanced
Findings: 1 Medium severity

Open findings (1)

Comment thread runtimes/common/agentkit_serve_common/foundry_model_loop.py Outdated
Signed-off-by: Sertac Ozercan <sozercan@gmail.com>
Copilot AI review requested due to automatic review settings September 11, 2026 22:03

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

It changes security-sensitive loading and durable streaming state, while live cancellation validation remains pending.

Review tier: Balanced
Findings: None

Resolved findings (1)

Signed-off-by: Sertac Ozercan <sozercan@gmail.com>
Copilot AI review requested due to automatic review settings September 11, 2026 22:18

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

It changes persisted workflow and cancellation semantics, while deployed streaming validation remains pending.

Review tier: Balanced
Findings: None

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 604cd83118

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread runtimes/common/agentkit_serve_common/foundry_streaming.py
Signed-off-by: Sertac Ozercan <sozercan@gmail.com>
Copilot AI review requested due to automatic review settings September 11, 2026 22:33

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Intermediate sequential-tool responses underreport cumulative model usage.

Review tier: Balanced
Findings: None

Previously missed findings (1)

In code that hasn't changed since last review

Medium severity Preserve cumulative usage on intermediate tool rounds

runtimes/​common/​agentkit_serve_common/​foundry.py:2272

result.usage only covers model calls since the latest resume, while state.initial_usage contains all earlier rounds and the final response reports their cumulative total. From the second operational tool onward, this intermediate function_call therefore underreports usage. Compute the combined usage once and use it for both the payload and the next state.

Signed-off-by: Sertac Ozercan <sozercan@gmail.com>
Copilot AI review requested due to automatic review settings September 11, 2026 23:11

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Intermediate tool responses underreport cumulative token usage.

Review tier: Balanced
Findings: None

Previously missed findings (1)

In code that hasn't changed since last review

Medium severity Carry cumulative usage into intermediate tool responses

runtimes/​common/​agentkit_serve_common/​foundry.py:2284

A resumed round that requests another tool reports only that resume's token usage, even though following.initial_usage and the final response accumulate all prior rounds. This undercounts the intermediate function_call response and makes usage jump inconsistently at completion. Pass the combined usage here as well.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 710e874020

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread runtimes/common/agentkit_serve_common/foundry_streaming.py Outdated
Comment thread runtimes/common/agentkit_serve_common/foundry_model_loop.py
Signed-off-by: Sertac Ozercan <sozercan@gmail.com>
Copilot AI review requested due to automatic review settings September 11, 2026 23:40

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Untrusted model validation details can still be exposed through public error messages.

Get a fresh assessment by requesting another Copilot review.

Review tier: Balanced
Findings: 1 Medium severity

Open findings (1)

Comment thread runtimes/common/agentkit_serve_common/foundry.py Outdated
Signed-off-by: Sertac Ozercan <sozercan@gmail.com>
Copilot AI review requested due to automatic review settings September 11, 2026 23:57

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The broad protocol, persistence, cancellation, and cross-repository changes warrant final human validation.

Review tier: Balanced
Findings: None

Resolved findings (1)

@sozercan
sozercan merged commit d2ad852 into main Sep 14, 2026
15 checks passed
@sozercan
sozercan deleted the feat/governed-tool-workflows branch September 14, 2026 19:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants