Add experimental standalone Copilot SDK onboarding

.github/copilot-instructions.md

@@ -2,6 +2,18 @@
 
 Skills for instrumenting and registering Microsoft Agent 365 agents. When a user asks for any of the trigger phrases below, follow the corresponding SKILL.md exactly.
 
+**Standalone GitHub Copilot SDK spike:** TypeScript + a direct `@github/copilot-sdk`
+dependency uses [the standalone route](../plugins/agent365/shared/copilot-sdk-standalone.md)
+through existing setup/registration/observability skills. Re-check before cache
+reuse; GitHub Copilot is not a Microsoft 365 Copilot/CEA signal. This exception
+overrides generic hosting/latest-version rules. Confirm standalone scope/S2S,
+blueprint reuse/preview approval, verified sample contract and diff approval before
+instrumentation. Pin published releases; separate local checks from live evidence.
+No teammate/Teams/WorkIQ/agentic-user/mailbox/licensing, runtime rewrite, model-based
+telemetry, or cloud operations during this local unpublished spike.
+The shared route records explicit scope/S2S and operation approvals; report-only
+hook completion never authorizes registration or instrumentation.
+
 ---
 
 ## Quick reference

.gitignore

@@ -9,8 +9,12 @@ BUGBASH.md
 
 # Environment variables
 .env
-.env.local
-.env.*.local
+.env.*
+!.env.example
+
+# Local Copilot runtime state and trace output (never source fixtures)
+.copilot-local/
+.copilot-traces/
 
 # ASP.NET Core app settings with secrets
 appsettings.Production.json
@@ -38,4 +42,3 @@ Thumbs.db
 
 # Claude Code session state — local only, not repo config
 .claude/
-

AGENTS.md

@@ -3,6 +3,18 @@
 This file documents conventions for contributors working on the `agent365` plugin skills.
 Read this before making any changes to skill files.
 
+**Standalone GitHub Copilot SDK spike:** TypeScript projects declaring
+`@github/copilot-sdk` use [the standalone route](plugins/agent365/shared/copilot-sdk-standalone.md)
+through existing setup/registration/observability skills. Re-check the dependency
+before cached routing; never infer Microsoft 365 Copilot/AI Teammate intent.
+This route overrides generic hosting, latest-version installs, and completion rules:
+published pins, scope/S2S confirmation, blueprint reuse/preview approval, verified
+sample contract and diff approval before instrumentation, and explicit local/live
+evidence separation. No teammate/Teams/WorkIQ/agentic-user/licensing, runtime rewrite,
+model-based telemetry, or cloud operations during the local spike.
+The shared route records explicit scope/S2S and operation approvals; report-only
+hook completion never authorizes registration or instrumentation.
+
 ---
 
 ## Plugin Purpose

CLAUDE.md

@@ -3,6 +3,17 @@
 This repository is a **Claude Code / GitHub Copilot CLI plugin marketplace** containing
 skills for the Microsoft Agent 365 platform. Read this file before making any changes.
 
+**Standalone GitHub Copilot SDK spike:** A direct `@github/copilot-sdk` dependency
+plus TypeScript source uses [the standalone route](plugins/agent365/shared/copilot-sdk-standalone.md)
+through existing setup/registration/observability skills. Re-check before cache
+reuse. This exception overrides generic hosting and latest-version installs: confirm
+standalone scope/S2S, blueprint reuse/preview approval, and verified sample/diff
+approval before edits. Pin published releases and separate offline from live
+evidence. Never infer Microsoft 365 Copilot or add teammate/Teams/WorkIQ/agentic-user,
+licensing, runtime rewrites, model-based telemetry, or local-spike cloud operations.
+The shared route records explicit scope/S2S and operation approvals; report-only
+hook completion never authorizes registration or instrumentation.
+
 ---
 
 ## What's in this repo

README.md

@@ -50,6 +50,21 @@ Or install via the marketplace first (inside a Claude Code session), then the CL
 
 ### GitHub Copilot CLI — `gh skill` (recommended)
 
+For **session-local plugin testing**, use the full Copilot CLI development host
+and the absolute path to this checkout's `plugins\agent365` directory, not its
+`.claude-plugin` subdirectory. The root `plugin.json` declares the same skills as
+the Claude manifest without its automatic version-check hooks. The full CLI
+[manifest reference](https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-plugin-reference)
+also supports legacy Claude manifests; adding a root manifest is not proof of
+runtime discovery.
+
+**Keep development-time onboarding separate from the standalone agent runtime.**
+Plugin discovery depends on the full CLI's capabilities and configuration; do not
+assume the bundled agent runtime supports its launch arguments. Keep the app's
+runtime configuration unchanged and use an isolated development host. Verify both
+actual skill invocation and successful fixture/reference reads before counting
+the model response as a content-based plugin test.
+
 The fastest way to install for GitHub Copilot CLI and VS Code agent mode:
 
 ```bash
@@ -87,6 +102,19 @@ gh copilot suggest "Instrument observability for this agent"
 
 ## Recommended Workflow
 
+**Experimental Copilot SDK support:** TypeScript projects with a direct
+`@github/copilot-sdk` dependency can use `a365-setup` for the guarded
+[standalone registration/basic-observability route](plugins/agent365/shared/copilot-sdk-standalone.md).
+It preserves the existing runtime and hosting; it does not add AI Teammate, Teams,
+Digital Worker, Agent Template, agentic users/mailboxes, WorkIQ, or notifications.
+Published versions are pinned, provisioning requires preview/approval, and
+instrumentation is gated on the companion `microsoft/Agent365-Samples`
+`nodejs/copilot-sdk` helper, which is pending/unpublished (no public immutable
+revision linked). Tenant registration,
+grants, and ingestion are not proven by offline checks. No marketplace publication
+or global installation is required for local evaluation. Strict report-only
+response adherence is not guaranteed; this is not autonomous end-to-end onboarding.
+
 **Start with `a365-setup`** — it verifies CLI and Azure prerequisites, asks which capabilities you want, then delegates to the right skill:
 
 ```

evals/agent365/a365-setup/evals.json

@@ -2,6 +2,28 @@
   "skill_name": "a365-setup",
   "eval_instructions": "Test against real agent projects in clean state. Skill MUST output a mandatory intro message before doing anything else — describing the 4-step flow (detect, confirm, capabilities, then auth mode conditionally for non-AI Teammate). Phase 1A detects agentStack, programmingLanguage, usesTeamsOrCopilot, hasBlueprintConfig, AND the three primary state flags (has_aiteammate_structure, has_obs, has_workiq) — these three drive the 8-row matrix in make-ai-teammate Phase 0C. The legacy hasAITeammateChanges is DERIVED inline (has_aiteammate_structure && has_obs) — it is no longer stored in the cache. Phase 1B shows all detections; asks blueprint question when hasBlueprintConfig=1 (reuse vs fresh — never assumes). CEA rule: if usesTeamsOrCopilot=1, authMode auto-set to 'agentic-user' and capabilities auto-set to [Register, Observability, WorkIQ, AI Teammate] — no questions asked. Derived 'already an AI Teammate' rule: if (has_aiteammate_structure && has_obs)=true, authMode auto-set to 'agentic-user' and capabilities menu shows only Register and WorkIQ (further filtered: WorkIQ hidden if has_workiq=true). For non-CEA agents: capabilities question asked first — capability rows are auto-hidden when their flag is true (Observability hidden if has_obs=true; WorkIQ hidden if has_workiq=true); authMode question (obo or s2s only — agentic-user is not user-selectable here) asked AFTER capabilities if AI Teammate was not selected; if AI Teammate was selected, authMode question is skipped (auto-set to agentic-user); if s2s selected and user had also picked WorkIQ, WorkIQ is dropped with a warning; selecting AI Teammate auto-includes Register and Observability (WorkIQ is optional and offered later in make-ai-teammate Phase 9.6). Step 1 runs a parallel quick scan of all tools and shows a ✅/❌ summary; only ❌ sections are processed — ✅ tools are skipped entirely (no reinstall, no re-prompt), with one explicit exception: the a365 CLI is always updated to latest via `dotnet tool update` regardless of ✅/❌ status. Step 2 covers Azure login and Entra ID roles only. Step 3 delegates. Azure login MUST use 'az login --allow-no-subscriptions'. For the AI Teammate path (isAITeammate=true), Step 3 reads make-ai-teammate/SKILL.md. For all other paths, Step 3 reads make-a365-agent/SKILL.md. a365-setup does NOT run a365 setup all, does NOT create a365.config.json, and does NOT run a365 publish. Phase 1C derives registrationType from usesTeamsOrCopilot. registrationType is derived, never asked. The cache writer in Phase 1C writes has_aiteammate_structure, has_obs, has_workiq individually; it does NOT write hasAITeammateChanges (derived).",
   "evals": [
+    {
+      "id": 1001,
+      "prompt": "Register this TypeScript GitHub Copilot SDK agent with Agent 365, standalone registration and basic observability only. No tenant credentials or cloud/auth approval are available. This is a read-only fixture eval: detect and report blockers, no edits or commands that install, authenticate, provision, or start a runtime.",
+      "description": "Local unpublished Copilot SDK spike: use tests/fixtures/copilot-sdk; this case overrides generic setup eval instructions",
+      "expected_output": "Recognizes exact @github/copilot-sdk 1.0.14 plus TypeScript before cache/CEA routing and exclusively follows Route A. Uses only the three-paragraph positive template: Detected / preserved, Local evidence, Blocked / not verified, then ends. No next gate, admin handoff, command names/examples (even parenthetical or negated), or secret requests. Stops on absent approval without edits, prerequisite scans, or login. Missing useMicrosoftOpenTelemetry alone is not evidence of missing observability. Repeating gives the same routing and unchanged runtime.",
+      "files": ["tests/fixtures/copilot-sdk/package.json", "tests/fixtures/copilot-sdk/src/index.ts"],
+      "expectations": [
+        "Reads shared/copilot-sdk-standalone.md before generic capability or install phases",
+        "GitHub Copilot SDK is not Microsoft 365 Copilot; no AI Teammate, Digital Worker, Agent Template, Teams, agentic user, mailbox, license, WorkIQ, or notifications",
+        "Direct dependency overrides a stale LangChain or AI Teammate cache; conflicting actual hosting markers block rather than silently changing scope",
+        "No a365 setup all, login, dry-run, scopes, resources, global installs, latest-version upgrades, or source/config edits in this read-only eval",
+        "Final answer must not recommend setup all, az login as a setup-all prerequisite, generic Steps 1-3/.NET quick scan, auto-installs, or generic todos, even when no tool executed them",
+        "Final answer uses exactly the three-paragraph positive report and ends; no next gate, generic outro, admin handoff, command names/examples, or request for secrets/tokens",
+        "The literal a365 setup all is absent from the final answer, even in parenthetical recommendations for admins or negated cannot-run explanations; recommendations are governed as strictly as execution",
+        "Missing useMicrosoftOpenTelemetry alone is not a missing-observability finding; inspect the explicit provider/exporter/token-resolver/scopes contract or report not evaluated",
+        "Correct SDK detection, successful shared-reference reads, invoked a365-setup, and unchanged fixture are insufficient for a safe-routing pass if final-answer guardrails fail",
+        "Records released SDK 1.0.14/bundled runtime 1.0.85 and CLI 1.1.221, without claiming live registration or ingestion",
+        "Missing approved sign-in blocks even blueprint dry-run because CLI 1.1.221 may launch WAM",
+        "Repeating the eval does not add providers, wrappers, hosting, credentials, or a detection cache",
+        "Setup --report-only hook returns a non-authorizing report with pending prerequisites; absent cache is allowed only for that report, stale/conflicting cache still blocks, and action validators remain fail closed"
+      ]
+    },
     {
       "id": 1,
       "prompt": "Run a365 setup for this agent",

evals/agent365/instrument-observability/evals.json

@@ -2,6 +2,40 @@
   "skill_name": "instrument-observability",
   "eval_instructions": "When running these evals, test against real .NET AgentFramework, Node.js LangChain, and Python agent projects. Verify that all instrumented code includes the marker comment appropriate for the language: '// A365 Observability — best-effort instrumentation (verify against official sample)' for .NET and Node.js, or '# A365 Observability — best-effort instrumentation (verify against official sample)' for Python.",
   "evals": [
+    {
+      "id": 1002,
+      "prompt": "Use the inspected local Copilot SDK sample helpers to add basic observability to this standalone agent. I approve the minimal local diff only; keep export disabled and preserve my current model, tools, session behavior, and hosting.",
+      "description": "Companion explicit-provider template adaptation after source verification, no first-class adapter or hosted fallback",
+      "expected_output": "Reviews the local source/pinned contract, preserves the existing runtime, adapts only config/auth/telemetry bootstrap and invocation/custom-tool wrapping, and checks offline build/tests. Uses NodeTracerProvider and direct Agent365Exporter only behind the false-by-default export gate, not useMicrosoftOpenTelemetry or hosted baggage middleware.",
+      "files": [],
+      "expectations": [
+        "Checks the actual local sample source is supplied and matches the build-verified contract; no invented release URL",
+        "Records explicit scope/S2S and observability source-contract/diff approval; report-only setup completion never authorizes instrumentation",
+        "Uses SDK 1.0.14/bundled runtime 1.0.85, distro 1.4.0, MSAL 7.0.0 and exact OTel dependency pins",
+        "Uses InvokeAgentScope/ExecuteToolScope with explicit AgentDetails and tool parentContext",
+        "Preserves existing runtime callbacks and custom tools; does not copy the arithmetic demo agent/model/system prompt over the app",
+        "Creates one provider and shutdown path; stops for verified integration if a provider already exists",
+        "Exporter remains opt-in false, actual usage stays an SDK event, and no InferenceScope is fabricated",
+        "Runs local build/tests/offline smoke only; no tenant token acquisition, registration or live prompt is implied",
+        "Separates static hook success, actual offline tests, real Copilot events, and unverified tenant ingestion"
+      ]
+    },
+    {
+      "id": 1001,
+      "prompt": "Add basic Agent 365 observability to my standalone TypeScript @github/copilot-sdk app without changing hosting. The local reference helper has not yet been verified and I have no tenant credentials.",
+      "description": "Copilot SDK instrumentation contract gate; overrides generic Node.js snippets",
+      "expected_output": "Reads the standalone reference and reports observability wiring pending verified sample contract. Leaves the existing runtime unchanged rather than generating a Copilot adapter, generic TurnContext hosting, or token recipe. Local deterministic evidence and future live evidence are separated.",
+      "files": [],
+      "expectations": [
+        "Uses existing instrument-observability skill, not a competing onboarding skill",
+        "No runtime helper edits until source revision/files, exact released pins, auth/env contract, lifecycle and tests are verified",
+        "No TurnContext, AgentApplication, configureA365Hosting, Teams, agentic user/mailbox, or WorkIQ",
+        "No model-generated telemetry, invented token counts, sendAndWait-as-inference span, or claim of built-in tool/all model coverage",
+        "Export is opt-in, disabled without identity/grants; no silent live smoke test",
+        "Reruns preserve SDK model/tools/session and avoid duplicate providers/listeners/wrappers",
+        "Offline build/tests never count as registration, S2S grants, ingestion, or MAC/Defender visibility"
+      ]
+    },
     {
       "id": 1,
       "prompt": "Instrument observability for this agent",

evals/agent365/make-a365-agent/evals.json

@@ -2,6 +2,22 @@
   "skill_name": "make-a365-agent",
   "eval_instructions": "Test against real agent projects. The skill receives capabilities + language context from a365-setup (or asks directly if invoked standalone). Phase 1.0 checks for existing a365.config.json / a365.generated.config.json BEFORE collecting inputs — if found, asks the developer whether to reuse (skips Phase 2 entirely) or create fresh. Phase 1 collects agent name and project directory — agent name rules: letters/numbers/hyphens, start with letter, 3–20 chars, preserve case as typed (do NOT normalize), and 'default' maps to 'developer'. Phase 1.1 asks if agent is cloud-hosted or local/dev-tunnel; if local, guides through devtunnel install, login, create, and host to produce the messagingEndpoint. Phase 2.2 handles Windows Account Manager (WAM) prompts — if 'Authenticating via Windows Account Manager...' appears in CLI output, tell user to complete the dialog without killing the process. It runs a365 setup all (Phase 2) when reuseBlueprint=false; skips Phase 2 when reuseBlueprint=true. CEA agents (usesTeamsOrCopilot=1) run a365 setup all --m365 and then a365 setup permissions bot. It always offers instrument-observability (Phase 3, optional) and add-workiq-tools (Phase 4, optional) regardless of the capability path. Blueprint creation is confirmed by a365.generated.config.json existing. The skill does NOT generate code, does NOT create a365.config.json, and does NOT run a365 publish.",
   "evals": [
+    {
+      "id": 1001,
+      "prompt": "Preview standalone Agent 365 registration for my TypeScript @github/copilot-sdk app. Keep the runtime and hosting unchanged. I have not approved tenant login or resource creation.",
+      "description": "Copilot SDK blueprint-only procedure: no generic setup-all or endpoint; overrides generic eval instructions",
+      "expected_output": "Reads the standalone reference and permitted local blueprint files, then reports missing approvals and unverified identity/grants without running a dry-run or showing next-step commands. The setup report is not permission to register; the independently invoked registration validator remains blocked until approvals and consistent config are present.",
+      "files": [],
+      "expectations": [
+        "No setup all, --aiteammate, --m365, bot permissions, hosting question, dev tunnel, or WorkIQ offer",
+        "Recipe uses setup blueprint --agent-name <confirmed-name> --tenant-id <confirmed-tenant-id> --no-endpoint --dry-run only after approved authentication",
+        "Preview is checked for scope and needs explicit approval before apply; no cleanup or replacement on repeat runs",
+        "Requires recorded scope/S2S and registration/reuse approval plus consistent local config for action completion; no report-only validator bypass",
+        "Blueprint, runtime identity, S2S application roles, and telemetry evidence are separate; custom --scopes is not an S2S app-role grant",
+        "No missing Web App managed identity warning for standalone registration",
+        "Does not suppress credential prerequisites, expose secrets, or claim catalog/portal/E2E success"
+      ]
+    },
     {
       "id": 1,
       "prompt": "Run a365 setup for this agent",