Question: setup all --authmode s2s --> observations on agent identity app role assignment (fallback strategy and inheritance overlap)

[Jose-BV]

Summary

While running a365 setup all --authmode s2s --skip-sp-provisioning as a Global Administrator, the blueprint side of the flow completed successfully, but the agent identity app role assignment step (NonDwBlueprintSetupOrchestrator.GrantOrInstructAgentIdentityAppPermissionsAsync) returned 403 and the CLI printed PowerShell instructions to manually run New-MgServicePrincipalAppRoleAssignment against the agent identity SP.

After tracing the flow through the code, two things stood out, both questions about expected behavior rather than confirmed bugs:

1. The agent identity S2S path does not have the same az rest fallback that BatchPermissionsOrchestrator already uses for the blueprint SP.
2. The role flagged in the PowerShell instructions (Agent365.Observability.OtelWrite on the Observability API SP : ConfigConstants.ObservabilityApiAppId = 9b975845-388f-4429-889e-eab1ef63949c) is already inherited by the agent identity SP via the allAllowed inheritable permissions configured earlier in the flow, making the direct grant on the agent identity SP appear redundant.

Command execution logs:

project-directoy % a365 setup all --agent-name <agent-name> --tenant-id <tenant-id> --authmode s2s --skip-sp-provisioning         
Running "a365 setup all --agent-name <agent-name> --tenant-id <tenant-id> --authmode s2s --skip-sp-provisioning"...

Resolving client app by display name "Agent 365 CLI"...
Opening browser for authentication...
Browser authentication is not supported on this platform: macOS
Falling back to device code authentication...
Please sign in with your Microsoft account

==========================================================================
To sign in, use a web browser to open the page:
    https://login.microsoft.com/device

And enter the code: <code>
==========================================================================

Bootstrap config resolved:
    TenantId:             <tenant-id>
    ClientAppId:          <client-app-id>
    BlueprintDisplayName: <agent-name>
    IdentityDisplayName:  <agent-name> Identity

Checking requirements...
    Pass: Azure Authentication
    Warn: Frontier Preview Program - Tenant enrollment cannot be verified automatically - Ensure your tenant is enrolled before proceeding. See: https://adoption.microsoft.com/copilot/frontier-program/
    Pass: PowerShell Modules (Microsoft.Graph.Authentication, Microsoft.Graph.Applications)
    Requirements: 2 passed, 1 warnings, 0 failed
Opening browser for authentication...
Browser authentication is not supported on this platform: macOS
Falling back to device code authentication...
Please sign in with your Microsoft account

==========================================================================
To sign in, use a web browser to open the page:
    https://login.microsoft.com/device

And enter the code: <code>
==========================================================================

Creating agent blueprint...
    Verifying consent for agent blueprint operations...
    Successfully ensured delegated application consent
    Found existing blueprint by display name
        Blueprint ID: <blueprint-id>

    Creating blueprint client secret...
        DPAPI encryption not available on this platform. Secret will be stored in plaintext.
        Client secret created successfully!
        Blueprint client secret: <blueprint-client-secret>
        Copy this value now — it will not be shown again automatically.
        To retrieve it later, run 'a365 setup blueprint --show-secret' from the same folder.

        Keep your credentials secure and do not commit them to source control!

Configuring inheritable permissions...
    Microsoft Graph: inheritable permissions already configured (kind=allAllowed on scopes and roles)
    Agent 365 Tools: inheritable permissions configured (kind=allAllowed on scopes and roles)
    Observability API: inheritable permissions configured (kind=allAllowed on scopes and roles)
    Power Platform API: inheritable permissions configured (kind=allAllowed on scopes and roles)

Configuring application permissions...
    The following application permission will be granted to the agent blueprint:
        Observability API: Agent365.Observability.OtelWrite

    Assign this application permission now? [y/N]: y
    Assigning S2S app roles...
      'Observability API': app role 'Agent365.Observability.OtelWrite' assigned.
    Application permissions granted.

Configuring delegated permissions...
    Opening browser for admin consent...
    Sign in and Accept the permission(s). If the tab shows an error after Accept, consent likely succeeded — the CLI will still detect it (timeout: 180s).
    Consent granted (All permissions).

Creating agent identity...
    Agent identity created (ID: <agent-identity-id>)

S2S app role assignments require Application Administrator or Global Administrator. Run the following PowerShell:

  # Connect to Microsoft Graph
  Connect-MgGraph -TenantId '<tenant-id>' -Scopes 'AppRoleAssignment.ReadWrite.All','Application.Read.All' -UseDeviceCode

  $agentSpId = '<agent-identity-id>'

  # Observability API
  $resourceSp = Get-MgServicePrincipal -Filter "appId eq '9b975845-388f-4429-889e-eab1ef63949c'"
  $roleId = ($resourceSp.AppRoles | Where-Object { $_.Value -eq 'Agent365.Observability.OtelWrite' }).Id
  New-MgServicePrincipalAppRoleAssignment -ServicePrincipalId $agentSpId -PrincipalId $agentSpId -ResourceId $resourceSp.Id -AppRoleId $roleId

Registering agent...
    Agent registered (ID: <agent-registered-id>)

Updating project settings...
    No .NET, Node.js, or Python project detected in <project-directory>; skipping project settings update.

Setup Summary

  1. Prerequisites                validated
  2. Blueprint                    reused   '<agent-name> Blueprint' (ID: <blueprint-id>)
  3. Inheritable Permissions      configured
  4. Blueprint Permission Grants  granted  tenant-wide delegated
  5. Agent identity               created   '<agent-name> Identity' (ID: <agent-identity-id>)
  6. Agent Registration           registered   '<agent-name> Agent' (ID: <agent-registration-id>)
  7. Messaging endpoint           skipped (non-M365 agent)

Action Required:

  1. Observability API S2S app role (PowerShell):
     Required role: Application Administrator or Global Administrator
       Connect-MgGraph -TenantId '<tenant-id>' -Scopes 'AppRoleAssignment.ReadWrite.All','Application.Read.All'
       $agentSpId = '<agent-identity-id>'
       $obs = Get-MgServicePrincipal -Filter "appId eq '9b975845-388f-4429-889e-eab1ef63949c'"
       $rid = ($obs.AppRoles | Where-Object { $_.Value -eq 'Agent365.Observability.OtelWrite' }).Id
       New-MgServicePrincipalAppRoleAssignment -ServicePrincipalId $agentSpId -PrincipalId $agentSpId -ResourceId $obs.Id -AppRoleId $rid

     Tenant        : <tenant-id>
     Agent Identity: <agent-identity-id>

Warnings:
  S2S app role assignments require Application Administrator or Global Administrator. PowerShell instructions have been printed above.

Setup completed — action required before proceeding

Output Analysis

After tracing the flow through the code, I noticed two things I'd like the repo owners to weigh in on, since I'm not sure whether either is expected behavior.

Observation 1: Asymmetric fallback strategy between blueprint SP and agent identity SP

BatchPermissionsOrchestrator (the orchestrator that runs against the blueprint SP) implements a two-step strategy for S2S app role assignment:

1. BlueprintService.GrantAppRoleAssignmentAsync: the programmatic Graph API path.
2. AzRestS2SRunner.TryRunAsync: on 403/Forbidden, falls back to issuing the same writes via az rest against the operator's existing az session.

The rationale, documented in BatchPermissionsOrchestrator.cs (lines 229–235) and the AzRestS2SRunner class header (lines 11–19), is that a Global Admin's az token implicitly carries every Graph application permission via the directory role, so az rest succeeds where the CLI's MSAL token (which deliberately does not request AppRoleAssignment.ReadWrite.All per AuthenticationConstants.RequiredS2SGrantScopes = []) does not.

The agent identity equivalent in NonDwBlueprintSetupOrchestrator.GrantOrInstructAgentIdentityAppPermissionsAsync (lines 663–740) calls the same BlueprintService.GrantAppRoleAssignmentAsync but on failure jumps directly to PowerShell instructions, AzRestS2SRunner.TryRunAsync is not invoked for the agent identity SP target.

AzRestS2SRunner.TryRunAsync is generic and accepts an arbitrary SP object ID as its principalId parameter (AzRestS2SRunner.cs:178 and the POST URL at line 185 are not blueprint-specific), so structurally the same runner could be used against the agent identity SP.

Question: Is the asymmetry intentional (e.g., a deliberate decision that agent-identity grants must be operator-driven via PowerShell)? Or could GrantOrInstructAgentIdentityAppPermissionsAsync be extended with the same AzRestS2SRunner.TryRunAsync fallback that BatchPermissionsOrchestrator already uses?

Observation 2: Direct grant on agent identity SP overlaps with what allAllowed inheritance already provides

Before the agent identity is created, BatchPermissionsOrchestrator.ConfigureInheritedPermissionsAsync (lines 131–183) configures the blueprint application's inheritablePermissions with the wildcard allAllowed form for both inheritableScopes and inheritableRoles. This is sent via AgentBlueprintService.SetInheritablePermissionsAsync (lines 382–484), using the wire types in Models/InheritablePermissionTypes.cs (AllAllowedScopes, AllAllowedRoles).

Then BatchPermissionsOrchestrator.PerformS2SGrantsAsync (lines 185–255) grants the configured AppRoleScopes on the blueprint SP.

Later, NonDwBlueprintSetupOrchestrator.CreateAgentIdentityAsync (lines 436–512) provisions the agent identity SP under the blueprint. Because inheritablePermissions is set to allAllowed and the blueprint SP already holds the role, every app role granted to the blueprint SP is automatically inherited by the agent identity SP.

After that, NonDwBlueprintSetupOrchestrator.GrantOrInstructAgentIdentityAppPermissionsAsync (lines 663–740) calls BlueprintService.GrantAppRoleAssignmentAsync again, this time targeting the agent identity SP, to grant the same AppRoleScopes directly.

After manually running the suggested PowerShell command, the Azure Portal shows the role twice on the agent identity SP:

• One row labeled "Inherited from parent" (Application)
• One row labeled "Directly granted" (Application)

The "Directly granted" row corresponds to the manual PowerShell step; the "Inherited from parent" row was already present because of the allAllowed configuration on the blueprint.

Execution order, with the function names from the code:

Order: 1
Function: ConfigureInheritedPermissionsAsync → SetInheritablePermissionsAsync
File: BatchPermissionsOrchestrator.cs / AgentBlueprintService.cs
Lines: 131–183 / 382–484
Effect: Sets inheritablePermissions = allAllowed on the blueprint app
────────────────────────────────────────
Order: 2
Function: PerformS2SGrantsAsync (+ AzRestS2SRunner.TryRunAsync fallback)
File: BatchPermissionsOrchestrator.cs
Lines: 185–255
Effect: Grants AppRoleScopes to the blueprint SP
────────────────────────────────────────
Order: 3
Function: CreateAgentIdentityAsync
File: NonDwBlueprintSetupOrchestrator.cs
Lines: 436–512
Effect: Creates the agent identity SP under the blueprint; it inherits all blueprint SP app roles via allAllowed
────────────────────────────────────────
Order: 4
Function: GrantOrInstructAgentIdentityAppPermissionsAsync → GrantAppRoleAssignmentAsync
File: NonDwBlueprintSetupOrchestrator.cs / BlueprintService.cs
Lines: 663–740 / 689–695
Effect: Attempts to directly grant the same AppRoleScopes to the agent identity SP

By the time step 4 runs, the inherited row is already in place from step 1 + step 2 + step 3.

Question: Is the direct grant in GrantOrInstructAgentIdentityAppPermissionsAsync intentionally redundant? Or could it be skipped, and the PowerShell prompt suppressed, when ConfigureInheritedPermissionsAsync has already succeeded with allAllowed, since the role is already inherited? In particular, when GrantAppRoleAssignmentAsync returns OperatorActionRequired for the agent identity target, is the PowerShell prompt actually necessary in this case, given that the "Inherited from parent" row already grants the same effective permission?

[github-actions]

[AUTO-TRIAGE] Team Assistant Triage

This issue has been automatically analyzed and triaged.

AI Decision Reasoning: Type: Classified as 'question' because the user is seeking clarification on expected behavior and implementation details rather than reporting a confirmed issue or requesting a new feature.
Priority: Elevated to P1 due to security concern: The issue describes a potential security concern related to app role assignments and permissions for service principals, which could lead to improper access control or privilege escalation if not configured correctly. Additionally, the fallback and inheritance overlap could introduce unintended behavior or vulnerabilities.
Copilot: Security issues require human review and should not be auto-fixed
Assignment: Security issue assigned to security lead. The issue describes a potential security concern related to app role assignments and permissions for service principals, which could lead to improper access control or privilege escalation if not configured correctly. Additionally, the fallback and inheritance overlap could introduce unintended behavior or vulnerabilities.
Labels: Applied labels question, P1, security based on issue type and priority

Labels and assignee have been applied based on this analysis.

[github-actions]

[AUTO-TRIAGE] Results

Classification: question
Priority: P1
Confidence: 95.0%

Recommended Assignee: Sellakumaran Kanagarathnam (@sellakumaran)

Analysis:
type rationale: Classified as 'question' because the user is seeking clarification on expected behavior and implementation details rather than reporting a confirmed issue or requesting a new feature.

priority rationale: Elevated to P1 due to security concern: The issue describes a potential security concern related to app role assignments and permissions for service principals, which could lead to improper access control or privilege escalation if not configured correctly. Additionally, the fallback and inheritance overlap could introduce unintended behavior or vulnerabilities.

copilot rationale: Security issues require human review and should not be auto-fixed

assignment rationale: Security issue assigned to security lead. The issue describes a potential security concern related to app role assignments and permissions for service principals, which could lead to improper access control or privilege escalation if not configured correctly. Additionally, the fallback and inheritance overlap could introduce unintended behavior or vulnerabilities.

labels rationale: Applied labels question, P1, security based on issue type and priority

[SUGGESTION] Suggested Approach

1. Review the implementation of `NonDwBlueprintSetupOrchestrator.GrantOrInstructAgentIdentityAppPermissionsAsync` in the `src` directory to confirm why the `az rest` fallback strategy used in `BatchPermissionsOrchestrator` is not applied here. If the fallback is missing, refactor the method to include it for consistency and robustness.
2. Investigate the logic for inheritable permissions in `ConfigConstants.ObservabilityApiAppId` and confirm if the `allAllowed` permissions already cover the `Agent365.Observability.OtelWrite` role. If redundancy is confirmed, update the flow to skip redundant role grants and ensure this behavior is documented in the code comments.
3. Add a validation step in the CLI flow to detect and warn about potential permission redundancies before attempting role assignments. This could be implemented in the `a365 setup all` command handler to improve user feedback and avoid unnecessary operations.
4. Enhance the error handling for the 403 response in `GrantOrInstructAgentIdentityAppPermissionsAsync` to provide clearer guidance on whether the fallback strategy should be manually triggered or if specific prerequisites (e.g., admin consent) are missing. Update the PowerShell instructions accordingly.
5. Update the documentation in `docs/SECRETS.md` or create a new section in the CLI usage guide to explain the fallback mechanism, role inheritance, and how to troubleshoot permission-related issues. Include examples of both automated and manual resolution steps for clarity.

---

Generated by TeamAssistant Auto-Triage powered by GitHub Models

[sellakumaran]

Thanks Jose-BV for the detailed trace — both observations were spot on.

Confirmed the root cause: when the blueprint holds the app roles and inheritable permissions are allAllowed, the agent identity inherits them, so the direct grant — and the "Action Required" PowerShell block — is redundant. PR up for review: it skips the direct grant when inheritance is in force, with an az rest fallback for the non-inherited path.