Add data-plane support in the typespec authoring skill

.github/skills/azure-typespec-author/SKILL.md

@@ -3,7 +3,7 @@ name: azure-typespec-author
 license: MIT
 metadata:
   version: "1.0.0"
-description: "Authors and modifies Azure TypeSpec (.tsp) API specifications. USE FOR: any TypeSpec/tsp change — api versions (add, bump, preview, stable, promote), resources, operations, models, properties, decorators, visibility, constraints, breaking changes, LRO, suppressions, operationId, spread model. Covers ARM resource-manager and data-plane services. DO NOT USE FOR: SDK generation, releasing SDK packages, or single MCP tool calls. INVOKES: azure-sdk-mcp:azsdk_typespec_generate_authoring_plan, azure-sdk-mcp:azsdk_run_typespec_validation."
+description: "Authors and modifies Azure TypeSpec (.tsp) API specifications. MUST BE USED FOR ALL TypeSpec changes regardless of complexity — even adding a single property or enum value requires this skill's validation workflow. USE FOR: any TypeSpec/tsp change — api versions (add, bump, preview, stable, promote), resources, operations, models, properties, decorators, visibility, constraints, breaking changes, LRO, suppressions, operationId, spread model. Covers both ARM resource-manager (Azure.ResourceManager) and data-plane (Azure.Core) services. DO NOT USE FOR: SDK generation, releasing SDK packages, or single MCP tool calls. INVOKES: azure-sdk-mcp:azsdk_typespec_generate_authoring_plan, azure-sdk-mcp:azsdk_run_typespec_validation."
 compatibility: "azure-sdk-mcp server with azsdk_typespec_generate_authoring_plan and azsdk_run_typespec_validation tools"
 ---
 
@@ -20,16 +20,19 @@ compatibility: "azure-sdk-mcp server with azsdk_typespec_generate_authoring_plan
 
 # When to invoke the azure-typespec-author skill
 
-The `azure-typespec-author` skill **must** be invoked immediately in all modes (including plan mode) for any task that involves creating and modifying TypeSpec (`.tsp`) files except for `client.tsp` under the specification directory in this repository. This includes but is not limited to:
+The `azure-typespec-author` skill **must** be invoked immediately in all modes (including plan mode) for any task that involves creating and modifying TypeSpec (`.tsp`) files except for `client.tsp` under the specification directory in this repository. **This skill MUST be used regardless of how simple the task appears** — there are no "simple" TypeSpec edits. Even trivial-seeming changes (adding a single enum value, one property, one operation) require the full workflow because versioning decorators, validation, and compliance checks are mandatory.
 
-- Adding, bumping, or promoting API versions (preview, stable)
+This includes but is not limited to:
+
+- Adding, bumping, or promoting API versions (preview, stable) for ARM or data-plane services
 - Adding or modifying resources, operations, models, properties, or decorators
 - Changing visibility, constraints, breaking changes, LRO patterns, or suppressions
 - Defining or updating operationId, spread models, or extension resources
 - Converting Swagger to TypeSpec (post-conversion edits)
 
 ## Constraints
 
+- **Do NOT skip this skill for "simple" tasks** — there are no simple TypeSpec edits. A single property addition can require `@added` decorators, version gating, and validation. Always invoke this skill.
 - **Always follow the full workflow** — even seemingly simple changes (e.g. adding a default value) can require complex versioning decorator changes. Never skip steps.
 - **Mandatory for ALL `.tsp` edits** — even a single `?` change can be breaking.
 - **Minimal, scoped edits** — only change what the request requires.
@@ -88,3 +91,6 @@ Output all referenced document URLs from Step 3. This gives the user direct link
 - "Add a new preview API version 2026-01-01-preview for widget resource manager"
 - "Add an ARM resource named Asset with CRUD operations"
 - "Add a new property to the Widget model"
+- "Add a list operation for the WidgetSuite resource using Azure.Core templates"
+- "Add a new preview API version to a data-plane service"
+- "Create a data-plane resource interface with full CRUD and list operations"

.github/skills/azure-typespec-author/evaluate/.vally.yaml

@@ -43,5 +43,7 @@ suites:
     evals: ["evals/004001.eval.yaml","evals/004002.eval.yaml","evals/004003.eval.yaml"]
   warning:
     evals: ["evals/005001.eval.yaml"]
+  dataplane:
+    evals: ["evals/006001.eval.yaml","evals/006002.eval.yaml","evals/006003.eval.yaml","evals/006004.eval.yaml","evals/006005.eval.yaml","evals/006006.eval.yaml","evals/006007.eval.yaml","evals/006008.eval.yaml","evals/006009.eval.yaml"]
   all:
     evals: ["evals/*.eval.yaml"]

.github/skills/azure-typespec-author/evaluate/README.md

@@ -23,6 +23,7 @@ eval $(node scripts/setup-environment.js)
 ```
 
 This script:
+
 1. Clones `package.json` and `package-lock.json` from [azure-rest-api-specs](https://github.com/Azure/azure-rest-api-specs) into `fixtures/Microsoft.Widget/Widget/`.
 2. Runs `npm ci` in that directory.
 3. Outputs the shell command to set `FIXTURE_NODE_MODULES` for symlink usage.
@@ -65,6 +66,7 @@ Test suites are defined in `.vally.yaml` under the `suites` key. Available suite
 | `longrunningoperation` | Long-running operation cases (003xxx) |
 | `decorators`           | Decorator cases (004xxx)              |
 | `warning`              | Warning cases (005xxx)                |
+| `dataplane`            | Data-plane cases (006xxx)             |
 | `all`                  | Every eval case                       |
 
 Run a suite by name:
@@ -75,11 +77,11 @@ vally eval --suite versioning --output-dir versioning
 
 ### Useful flags
 
-| Flag | Purpose |
-|---|---|
+| Flag                           | Purpose                                                        |
+| ------------------------------ | -------------------------------------------------------------- |
 | `--keep-executor-session-logs` | Preserve agent session logs under `--output-dir` for debugging |
-| `--verbose` | Show full agent output during the run |
-| `--workers <n>` | Run multiple stimuli in parallel (default: 5) |
+| `--verbose`                    | Show full agent output during the run                          |
+| `--workers <n>`                | Run multiple stimuli in parallel (default: 5)                  |
 
 ### Parallel run the environment
 

.github/skills/azure-typespec-author/evaluate/evals/006001.eval.yaml

@@ -0,0 +1,63 @@
+name: azure-typespec-author-eval
+description: Evaluation suite for azure-typespec-author.
+version: "1.0"
+type: capability
+
+environment: azsdk-mcp
+
+config:
+  runs: 1
+  timeout: "1800s"
+  model: claude-opus-4.6
+  executor: copilot-sdk
+
+stimuli:
+- name: 006001-dp-add-create-operation
+  prompt: |
+    Add a create or update operation for the WidgetSuite resource to the Widgets interface. The operation should allow creating or updating a Widget asynchronously.
+  constraints:
+    max_turns: 10
+    max_tokens: 5000
+  environment:
+    files:
+    - src: ../fixtures/006001-dp-add-create-operation\main.tsp
+      dest: main.tsp
+    - src: ../fixtures/006001-dp-add-create-operation\shared.tsp
+      dest: shared.tsp
+    - src: ../fixtures/Microsoft.Widget/Widget/package.json
+      dest: package.json
+    - src: ../fixtures/Microsoft.Widget/Widget/package-lock.json
+      dest: package-lock.json
+    - src: ../fixtures/006001-dp-add-create-operation\tspconfig.yaml
+      dest: tspconfig.yaml
+  graders:
+  - type: tool-calls
+    config:
+      required:
+        - edit
+        - azure-sdk-mcp-azsdk_run_typespec_validation
+  - type: skill-invocation
+    config:
+      required:
+        - azure-typespec-author
+  - type: file-matches
+    config:
+      path: main.tsp
+      pattern: 'LongRunningResourceCreateOrUpdate<WidgetSuite>|ResourceCreateOrReplace<WidgetSuite>|ResourceCreateOrUpdate<WidgetSuite>'
+  - type: file-matches
+    config:
+      path: main.tsp
+      pattern: '@doc\('
+  - type: prompt
+    config:
+      prompt: The Widgets interface should include a create or update operation for WidgetSuite using one of the Azure.Core resource operation templates. The operation should have a descriptive @doc decorator.
+      model: claude-opus-4.6
+      scoring: scale_1_5
+      threshold: 1.0
+scoring:
+  weights:
+    tool-calls: 1
+    skill-invocation: 1
+    file-matches: 3
+    prompt: 2
+  threshold: 0.95

.github/skills/azure-typespec-author/evaluate/evals/006002.eval.yaml

@@ -0,0 +1,63 @@
+name: azure-typespec-author-eval
+description: Evaluation suite for azure-typespec-author.
+version: "1.0"
+type: capability
+
+environment: azsdk-mcp
+
+config:
+  runs: 1
+  timeout: "1800s"
+  model: claude-opus-4.6
+  executor: copilot-sdk
+
+stimuli:
+- name: 006002-dp-add-get-operation
+  prompt: |
+    Add a get operation for the WidgetSuite resource to the Widgets interface. The operation should allow fetching a Widget by name.
+  constraints:
+    max_turns: 10
+    max_tokens: 5000
+  environment:
+    files:
+    - src: ../fixtures/006002-dp-add-get-operation\main.tsp
+      dest: main.tsp
+    - src: ../fixtures/006002-dp-add-get-operation\shared.tsp
+      dest: shared.tsp
+    - src: ../fixtures/Microsoft.Widget/Widget/package.json
+      dest: package.json
+    - src: ../fixtures/Microsoft.Widget/Widget/package-lock.json
+      dest: package-lock.json
+    - src: ../fixtures/006002-dp-add-get-operation\tspconfig.yaml
+      dest: tspconfig.yaml
+  graders:
+  - type: tool-calls
+    config:
+      required:
+        - edit
+        - azure-sdk-mcp-azsdk_run_typespec_validation
+  - type: skill-invocation
+    config:
+      required:
+        - azure-typespec-author
+  - type: file-matches
+    config:
+      path: main.tsp
+      pattern: 'ResourceRead<WidgetSuite>'
+  - type: file-matches
+    config:
+      path: main.tsp
+      pattern: '@doc\('
+  - type: prompt
+    config:
+      prompt: The Widgets interface should include a get/read operation for WidgetSuite using the ResourceRead<WidgetSuite> Azure.Core operation template. The operation should have a descriptive @doc decorator.
+      model: claude-opus-4.6
+      scoring: scale_1_5
+      threshold: 1.0
+scoring:
+  weights:
+    tool-calls: 1
+    skill-invocation: 1
+    file-matches: 3
+    prompt: 2
+  threshold: 1.0

.github/skills/azure-typespec-author/evaluate/evals/006003.eval.yaml

@@ -0,0 +1,63 @@
+name: azure-typespec-author-eval
+description: Evaluation suite for azure-typespec-author.
+version: "1.0"
+type: capability
+
+environment: azsdk-mcp
+
+config:
+  runs: 1
+  timeout: "1800s"
+  model: claude-opus-4.6
+  executor: copilot-sdk
+
+stimuli:
+- name: 006003-dp-add-list-operation
+  prompt: |
+    Add a list operation for the WidgetSuite resource to the Widgets interface. The operation should allow listing Widget resources with standard list query parameters.
+  constraints:
+    max_turns: 10
+    max_tokens: 5000
+  environment:
+    files:
+    - src: ../fixtures/006003-dp-add-list-operation\main.tsp
+      dest: main.tsp
+    - src: ../fixtures/006003-dp-add-list-operation\shared.tsp
+      dest: shared.tsp
+    - src: ../fixtures/Microsoft.Widget/Widget/package.json
+      dest: package.json
+    - src: ../fixtures/Microsoft.Widget/Widget/package-lock.json
+      dest: package-lock.json
+    - src: ../fixtures/006003-dp-add-list-operation\tspconfig.yaml
+      dest: tspconfig.yaml
+  graders:
+  - type: tool-calls
+    config:
+      required:
+        - edit
+        - azure-sdk-mcp-azsdk_run_typespec_validation
+  - type: skill-invocation
+    config:
+      required:
+        - azure-typespec-author
+  - type: file-matches
+    config:
+      path: main.tsp
+      pattern: 'ResourceList<WidgetSuite'
+  - type: file-matches
+    config:
+      path: main.tsp
+      pattern: '@doc\('
+  - type: prompt
+    config:
+      prompt: The Widgets interface should include a list operation for WidgetSuite using the ResourceList<WidgetSuite> Azure.Core operation template. The operation should have a descriptive @doc decorator.
+      model: claude-opus-4.6
+      scoring: scale_1_5
+      threshold: 1.0
+scoring:
+  weights:
+    tool-calls: 1
+    skill-invocation: 1
+    file-matches: 3
+    prompt: 2
+  threshold: 1.0

.github/skills/azure-typespec-author/evaluate/evals/006004.eval.yaml

@@ -0,0 +1,76 @@
+name: azure-typespec-author-eval
+description: Evaluation suite for azure-typespec-author.
+version: "1.0"
+type: capability
+
+environment: azsdk-mcp
+
+config:
+  runs: 1
+  timeout: "1800s"
+  model: claude-opus-4.6
+  executor: copilot-sdk
+
+stimuli:
+- name: 006004-dp-create-resource-with-crud
+  prompt: |
+    Add a Widgets interface with full CRUD and list operations for the WidgetSuite resource. The interface should include operations to create or update, get, list, and delete widgets. Use the appropriate Azure.Core resource operation templates.
+  constraints:
+    max_turns: 10
+    max_tokens: 5000
+  environment:
+    files:
+    - src: ../fixtures/006004-dp-create-resource-with-crud\main.tsp
+      dest: main.tsp
+    - src: ../fixtures/006004-dp-create-resource-with-crud\shared.tsp
+      dest: shared.tsp
+    - src: ../fixtures/Microsoft.Widget/Widget/package.json
+      dest: package.json
+    - src: ../fixtures/Microsoft.Widget/Widget/package-lock.json
+      dest: package-lock.json
+    - src: ../fixtures/006004-dp-create-resource-with-crud\tspconfig.yaml
+      dest: tspconfig.yaml
+  graders:
+  - type: tool-calls
+    config:
+      required:
+        - edit
+        - azure-sdk-mcp-azsdk_run_typespec_validation
+  - type: skill-invocation
+    config:
+      required:
+        - azure-typespec-author
+  - type: file-matches
+    config:
+      path: main.tsp
+      pattern: 'ResourceRead<WidgetSuite>'
+  - type: file-matches
+    config:
+      path: main.tsp
+      pattern: 'LongRunningResourceCreateOrUpdate<WidgetSuite>|ResourceCreateOrReplace<WidgetSuite>|ResourceCreateOrUpdate<WidgetSuite>'
+  - type: file-matches
+    config:
+      path: main.tsp
+      pattern: 'ResourceList<WidgetSuite'
+  - type: file-matches
+    config:
+      path: main.tsp
+      pattern: 'ResourceDelete<WidgetSuite>|LongRunningResourceDelete<WidgetSuite>'
+  - type: prompt
+    config:
+      prompt: |
+        A Widgets interface should be defined with operations for the WidgetSuite resource including:
+        1. A get/read operation using ResourceRead<WidgetSuite>
+        2. A create or update operation using an Azure.Core create template
+        3. A list operation using ResourceList<WidgetSuite>
+        4. A delete operation using ResourceDelete or LongRunningResourceDelete
+      model: claude-opus-4.6
+      scoring: scale_1_5
+      threshold: 1.0
+scoring:
+  weights:
+    tool-calls: 1
+    skill-invocation: 1
+    file-matches: 3
+    prompt: 2
+  threshold: 1.0