Skip to content

docs(test): maintain AgentKit image parity matrix for Foundry samples #13

Description

@sozercan

This was generated by AI during triage.

Agent Brief

Category: enhancement
Summary: Build a maintained native-sample parity matrix for AgentKit-built images.

Current behavior:
Parity validation has been capability-driven rather than an exhaustive conversion of every native Microsoft Agent Framework hosted-agent sample into AgentKit-built images. The current validation proves several AgentKit equivalents, but there is no maintained matrix that records each native sample, whether AgentKit has an equivalent image, and whether that image passes standalone and Foundry hosted validation.

Desired behavior:
Create a reproducible parity matrix that tracks native Foundry hosted-agent samples against AgentKit-built image equivalents. Each row should record repository, commit, sample name, required resources, AgentKit equivalent status, standalone result, hosted result, and any follow-up issue.

The matrix should keep the two upstream sample trees separate:

  • microsoft/agent-framework hosted-agent samples with underscore names.
  • microsoft-foundry/foundry-samples hosted-agent samples with hyphenated names and newer scenarios.

Key interfaces:

  • AgentKit build frontend and runtime adapter selection.
  • Foundry hosted-agent wrapper/protocol adapter.
  • Generic deployment profiles and env/resource wiring.
  • Validation result format for standalone vs hosted parity.

Investigation notes:

  • Current public sample gallery investigated at microsoft-foundry/foundry-samples@5d9dc1eb93e134b19939c3f517ce3786c1648186 includes samples such as basic, tools, MCP, Foundry Toolbox, workflows, files, skills, observability, downstream Azure, Azure Search RAG, Foundry Skills, Foundry Memory, browser automation, optimization/travel approver, content safety guardrail, Foundry IQ toolbox, and A2A delegation.
  • Current old sample tree investigated at microsoft/agent-framework@daac8c15f3505502370cb429630fae4d34947132 includes responses 01_basic through 12_foundry_toolbox_mcp_skills, plus invocations samples.
  • Known AgentKit-equivalent scenarios that have passed include basic, remote MCP/toolbox, code interpreter arithmetic, skills via toolbox MCP, filesystem skills, and Azure Search RAG.
  • Known not-proven scenarios should link to dedicated follow-ups: durable memory, workflow/multi-agent, file/artifact lifecycle, and any future browser/guardrail/optimization/A2A work.

Acceptance criteria:

  • Add a versioned parity matrix document or machine-readable report format.
  • Matrix records upstream repository and commit for every result.
  • Matrix separates old microsoft/agent-framework samples from current microsoft-foundry/foundry-samples samples.
  • Each sample is classified as pass, fail, blocked, unsupported, or not-yet-converted for AgentKit-built images.
  • Each supported sample has commands to build the AgentKit image, run standalone validation, deploy as Foundry hosted agent, and compare outputs.
  • Blocked/unsupported rows link to follow-up issues or state the missing resource/capability.
  • The matrix is reproducible without relying on previous local terminal history.
  • Documentation warns not to mark native MAF sample success as AgentKit image parity unless an AgentKit-built image was actually tested.

Out of scope:

  • Implementing every missing capability in this issue.
  • Merging the two upstream sample trees into one ambiguous list.
  • Treating native MAF local sample success as AgentKit parity.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions