You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Restores the Fibey example for harness v2. Users send the same fictional pump alert to AgentKit in Kubernetes and to AgentKit hosted in Azure AI Foundry, then compare the saved answers with orka task result.
The walkthrough explains Orka terms as they appear and keeps deployment and credentials in the setup guide. Each submission verifies its selected runtime. If a read fails after submission, the script directs users to inspect the existing Task before submitting again. Controller recovery reuses the saved terminal record so a lost status update cannot cause a conflicting rewrite.
The supervisor also fixes a tool-capacity failure found during live validation. When two tool calls are running, an additional call now receives a request-specific capacity error without consuming approval or reaching the tool broker. Tool listing and ping remain available. In the controlled three-call comparison, callbacks failed in 10–22 ms instead of waiting 120 seconds. The two admitted tool effects still completed; the error does not roll them back.
The live runtime tests record bounded failure metadata and recognized error markers from authenticated event responses before checking for successful tool execution. Diagnostics omit tool arguments, output, and arbitrary error text. The Codex read check asks for the file contents without supplying the expected answer in its prompt.
Validation:
make lint-fix, make test, focused MCP proxy tests, six submission regressions, and code review passed.
All 29 CI checks passed on 1a1da5d0. Both Copilot proxy runs passed 12 selected cases without a retry, and both general E2E runs passed 81 selected cases, including the metrics check.
All nine core images were rebuilt for Linux amd64 and Orka was deployed in harness v2 mode on AKS. The four primary demo runtimes then received the capacity fix with verified images that changed only the supervisor executable. Existing hosted agent versions and previous ledgers were preserved.
After replacement, both pump demos, a hosted search, and a direct work-order lookup passed. The latter recorded skill loading, discovery, and reading the requested record. Fixture counts matched the governed calls, CLI and API answers matched, and all runtimes returned to idle with cleanup proof.
Seven hosted fault Tasks covered HTTP 429 recovery and exhaustion, cancellation during backoff, malformed JSON, empty model results, and invalid tool arguments. The model upstream was a deterministic fixture behind the real Foundry and Orka path. All expected failure assertions passed, with no repeated tool effects. Late cancellation checks found no delayed retries.
Both backends handled four active prompts and 31 successful turns in 11 conversations. Cancellation left peers unaffected. Ten resident conversations were accepted, the eleventh was rejected, and admission resumed after deletion. Concurrent governed tool calls matched independent fixture counts. Hosted cleanup matched 44 Tasks to 24 retired remote sessions.
Eight handoff/crash cases and two supervisor-alive lost-result cases each executed one tool effect. Exact successful retries returned the same receipt. Six interrupted Tasks retained OutcomeUnknown; both supervisor-alive cases retained an unknown tool-effect record. Evidence and the two earlier uncertain executions remain preserved.
Known limits from the new tests:
Answer grounding is unresolved. Across 36 frozen samples, 13 passed, 22 failed, and one returned no assessable answer. Missing and contradictory records were handled in the final samples, but undocumented instrument attributes and misleading retrieved text still produced failures. Three instruction revisions did not establish a fix and are excluded from this change. These sample counts are not reliability estimates, and the two backends used different models.
A separate direct smoke claimed it had searched despite having no recorded tool call. Its prompt conflicted with the configured skill workflow. A follow-up using that workflow passed, but did not establish the cause of the first failure; both outcomes are retained.
The public Task history does not clearly expose an uncertain external tool effect when the runtime itself fails and cleans up. The durable effect record remains OutcomeUnknown, while public events report a generic tool/Task failure.
The controller has four dispatch workers. Runtime rejection of a fifth active prompt, eight simultaneously active prompts across both backends, and the global 16-request HTTP guard were not independently exercised. The sustained test included an expired client-token collection pause, so it is not an uninterrupted load benchmark.
The companion changes are in AgentKit #25 and agent-runtime-foundry #3. No additional source changes were needed in either repository during this validation round.
The demo uses simulated, read-only operational data. Orka governs the outer tool-search and call-tool calls; the fixture constrains their nested operations. No uncertain invocation was replayed or force-cleared.
The reason will be displayed to describe this comment to others. Learn more.
Copilot review overview
🔵 Needs a closer look
The cross-repository runtime integrations and image contracts were not validated through a live end-to-end execution.
Review tier: Balanced Findings: None
Issues resolved since last review (2)
Severity
Finding
scripts/tests/fibey-v2-demo-test.sh — This handwritten fixture includes the same unsupported task-level allowBash field as the… View resolved comment
examples/fibey-custom-agent-demo/task.yaml — runtimeRef Tasks reject any task-level agentRuntime.allowBash value, including false… View resolved comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot review overview
🟢 Approval recommended
The restored example follows external harness-v2 contracts and includes focused validation for manifests, compatibility, bindings, and failure handling.
The reason will be displayed to describe this comment to others. Learn more.
Copilot review overview
🟢 Approval recommended
The example, safety checks, tests, and documentation align with the external harness v2 contracts.
Review tier: Balanced Findings: None
Issues resolved since last review (2)
Severity
Finding
examples/fibey-custom-agent-demo/build-images.md — .status.result is not a Task field, so this verification output is always null, even after a… View resolved comment
examples/fibey-custom-agent-demo/README.md — TaskStatus has no top-level .status.result; successful runs only set… View resolved comment
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
Restores the Fibey example for harness v2. Users send the same fictional pump alert to AgentKit in Kubernetes and to AgentKit hosted in Azure AI Foundry, then compare the saved answers with
orka task result.The walkthrough explains Orka terms as they appear and keeps deployment and credentials in the setup guide. Each submission verifies its selected runtime. If a read fails after submission, the script directs users to inspect the existing Task before submitting again. Controller recovery reuses the saved terminal record so a lost status update cannot cause a conflicting rewrite.
The supervisor also fixes a tool-capacity failure found during live validation. When two tool calls are running, an additional call now receives a request-specific capacity error without consuming approval or reaching the tool broker. Tool listing and ping remain available. In the controlled three-call comparison, callbacks failed in 10–22 ms instead of waiting 120 seconds. The two admitted tool effects still completed; the error does not roll them back.
The live runtime tests record bounded failure metadata and recognized error markers from authenticated event responses before checking for successful tool execution. Diagnostics omit tool arguments, output, and arbitrary error text. The Codex read check asks for the file contents without supplying the expected answer in its prompt.
Validation:
make lint-fix,make test, focused MCP proxy tests, six submission regressions, and code review passed.1a1da5d0. Both Copilot proxy runs passed 12 selected cases without a retry, and both general E2E runs passed 81 selected cases, including the metrics check.OutcomeUnknown; both supervisor-alive cases retained an unknown tool-effect record. Evidence and the two earlier uncertain executions remain preserved.Known limits from the new tests:
OutcomeUnknown, while public events report a generic tool/Task failure.The companion changes are in AgentKit #25 and agent-runtime-foundry #3. No additional source changes were needed in either repository during this validation round.
The demo uses simulated, read-only operational data. Orka governs the outer
tool-searchandcall-toolcalls; the fixture constrains their nested operations. No uncertain invocation was replayed or force-cleared.