diff --git a/Deployment/how-to-configure-apim-for-service-fabric-managed-cluster.md b/Deployment/how-to-configure-apim-for-service-fabric-managed-cluster.md new file mode 100644 index 00000000..8ce62d4b --- /dev/null +++ b/Deployment/how-to-configure-apim-for-service-fabric-managed-cluster.md @@ -0,0 +1,1279 @@ +# How to configure APIM for a Service Fabric managed cluster Service Connection + +This document describes the steps to configure [Azure API Management](https://learn.microsoft.com/azure/api-management/) (APIM) to route traffic to a back-end service in a Service Fabric managed cluster (SFMC) using PowerShell. For unmanaged clusters, refer to [Integrate API Management with Service Fabric in Azure](https://learn.microsoft.com/azure/service-fabric/service-fabric-tutorial-deploy-api-management). + +> [!IMPORTANT] +> Service Fabric managed clusters now have the capability that allows the use of a static cluster / server certificate configuration. The new FQDN format for the cluster is: **`...sfmc.io`**. Prior to this option, connecting to a managed cluster required knowing the current cluster certificate thumbprint. By default, this certificate is auto-generated and rotated periodically. For APIM and managed clusters, it is highly recommended to use this new configuration to prevent connectivity issues from APIM when the cluster certificate is rotated. This update requires: + +> - **ARM Template**: Minimum `api-version` of `2024-06-01-preview` (tested with `2024-09-01-preview`) +> - **PowerShell**: Az.ServiceFabric module + +```diff +--- a/sfmc-template.json ++++ b/sfmc-template.json +@@ -81,7 +81,7 @@ + }, + "resources": [ + { +- "apiVersion": "2022-06-01-preview", ++ "apiVersion": "2024-09-01-preview", + "type": "Microsoft.ServiceFabric/managedclusters", + "name": "[parameters('clusterName')]", + "location": "[resourcegroup().location]", +@@ -89,6 +89,7 @@ + "name": "[parameters('clusterSku')]" + }, + "properties": { ++ "autoGeneratedDomainNameLabelScope": "ResourceGroupReuse", + "subnetId": "[parameters('subnetId')]", + "dnsName": "[toLower(parameters('clusterName'))]", + "adminUserName": "[parameters('adminUserName')]", +@@ -126,7 +127,7 @@ + } + }, + { +- "apiVersion": "2022-06-01-preview", ++ "apiVersion": "2024-09-01-preview", + "type": "Microsoft.ServiceFabric/managedclusters/nodetypes", + "name": "[concat(parameters('clusterName'), '/', parameters('nodeType1Name'))]", + "location": "[resourcegroup().location]", +``` + +## SFMC Architecture Note + +> [!NOTE] +> Service Fabric managed clusters create infrastructure resources in a **separate auto-managed resource group** named `SFC_`. This resource group contains the actual infrastructure components (VMSS, storage accounts, load balancers, network security groups, virtual networks, etc.), while the primary resource group contains only the cluster resource itself and optionally a managed identity. When troubleshooting or monitoring deployments, check both resource groups. + +## Requirements + +- **Service Fabric managed cluster with static FQDN** (`autoGeneratedDomainNameLabelScope` set to `ResourceGroupReuse`). This provides a persistent FQDN that survives cluster certificate rotation (auto-rotates every 90 days). See [How to Configure Service Fabric Managed Cluster with Static FQDN](./how-to-configure-service-fabric-managed-cluster-static-fqdn.md) for detailed configuration instructions. +- Service Fabric managed cluster deployed using an existing external virtual network. + This configuration is required for management of network settings that is not available with a default managed cluster deployment. See [Bring your own virtual network](https://learn.microsoft.com/azure/service-fabric/how-to-managed-cluster-networking#bring-your-own-virtual-network) in [Configure network settings for Service Fabric managed clusters](https://learn.microsoft.com/azure/service-fabric/how-to-managed-cluster-networking) for additional information and requirements. + +- **Service Fabric managed cluster with a client certificate** (for admin access and APIM authentication). Certificate **MUST include Client Authentication EKU** (1.3.6.1.5.5.7.3.2). See [Client Certificate Configuration](#client-certificate-configuration) section below. +- Azure API Management service. See [Azure API Management](https://learn.microsoft.com/azure/api-management/) for additional information. +- Azure Key vault with certificate (optional - only if using Key Vault approach). + +> [!NOTE] +> **Static FQDN Configuration**: The `autoGeneratedDomainNameLabelScope` property can be configured during cluster creation or added to existing clusters. For complete setup instructions, troubleshooting, and timing considerations, see [How to Configure Service Fabric Managed Cluster with Static FQDN](./how-to-configure-service-fabric-managed-cluster-static-fqdn.md). + +## Validated Configuration + +Testing has confirmed the following configuration successfully deploys a Service Fabric managed cluster with the static FQDN format: + +**FQDN Format Generated**: `...sfmc.io` + +**Example**: `sfmctest1nt31.duhcd6fxcrbbffb0.centralus.sfmc.io` + +**Key Benefits**: + +- ✅ Static FQDN that persists across certificate rotations and resource group redeployments +- ✅ Automatic certificate rotation (every 90 days) without FQDN change +- ✅ No dependency on cluster certificate thumbprint for connectivity +- ✅ Compatible with Azure API Management backend configuration + +For complete configuration details, ARM template examples, and troubleshooting guidance, see [How to Configure Service Fabric Managed Cluster with Static FQDN](./how-to-configure-service-fabric-managed-cluster-static-fqdn.md). + +## Client Certificate Configuration + +### Overview + +Service Fabric managed clusters require a client certificate for authentication. This certificate is **separate from the cluster certificate** (which auto-rotates). For APIM integration, the client certificate serves two purposes: + +1. **Admin access** for PowerShell management connections +2. **APIM authentication** to the Service Fabric cluster + +> [!NOTE] +> **Client certificates are NOT installed on the cluster**. They are only needed on client machines (such as workstations, build servers, or APIM instances) that connect to the cluster. The cluster only stores the client certificate thumbprint or common name for validation. + +### CRITICAL: Extended Key Usage (EKU) Requirement + +**Client certificates MUST include the Client Authentication EKU** (OID: 1.3.6.1.5.5.7.3.2) or Service Fabric will reject the connection. This is the most common cause of authentication failures. + +### Configuration Options + +Service Fabric supports **two validation methods** for client certificates: + +#### Option 1: Thumbprint-Based (RECOMMENDED for Most Organizations) + +**Best for**: Organizations without private CA infrastructure, dev/test environments, or when using self-signed certificates. + +**Advantages**: + +- ✅ Works with self-signed certificates +- ✅ Simple setup (no CA required) +- ✅ Full control over rotation schedule +- ✅ Certificates valid for 1 year per current best practices + +**Configuration**: + +```json +{ + "clients": [ + { + "isAdmin": true, + "thumbprint": "0123456789ABCDEF0123456789ABCDEF01234567" + } + ] +} +``` + +**Generate self-signed certificate** (PowerShell): + +```powershell +$cert = New-SelfSignedCertificate ` + -Subject "CN=apim-sf-client" ` + -CertStoreLocation "Cert:\CurrentUser\My" ` + -KeyUsage DigitalSignature, KeyEncipherment ` + -Type Custom ` + -KeySpec Signature ` + -KeyLength 2048 ` + -KeyExportPolicy Exportable ` + -NotAfter (Get-Date).AddYears(5) ` + -TextExtension @("2.5.29.37={text}1.3.6.1.5.5.7.3.2,1.3.6.1.5.5.7.3.1") + # Client Auth EKU ^^^ (REQUIRED) Server Auth EKU ^^^ (optional) + +# Verify EKUs (CRITICAL) +$cert.EnhancedKeyUsageList | Format-Table +# Must show: Client Authentication (1.3.6.1.5.5.7.3.2) + +# Export for APIM +$pw = ConvertTo-SecureString -String "YourPassword" -Force -AsPlainText +Export-PfxCertificate -Cert $cert -FilePath "apim-client.pfx" -Password $pw +``` + +**Alternative: Generate via Azure Key Vault** + +Azure Key Vault can generate self-signed certificates with the required Client Authentication EKU: + +1. Navigate to Azure Key Vault → Certificates → Generate/Import +2. Select "Generate" method +3. Configure: + - Certificate Name: `apim-sf-client` + - Type: Self-signed certificate + - Subject: `CN=apim-sf-client` + - Validity Period: 12 months + - **Advanced Policy Configuration**: + - Extended Key Usages (EKUs): Add `1.3.6.1.5.5.7.3.2` (Client Authentication) + - Key Usage Flags: Digital Signature, Key Encipherment +4. Download certificate as PFX for distribution to client machines + +> [!NOTE] +> Key Vault serves as **centralized storage and distribution** for client certificates. Certificates must still be downloaded and installed on client machines (workstations, APIM instances, etc.) that need to connect to the cluster. + +**Trade-off**: Requires manual APIM backend update when certificate expires. + +#### Option 2: Common Name-Based (ONLY for Enterprises with Private CA) + +**Best for**: Enterprises with private CA infrastructure (ADCS, OpenSSL CA, etc.) that can issue Client Authentication EKU certificates. + +> [!IMPORTANT] +> **Common name-based authentication REQUIRES CA-issued certificates** - self-signed certificates are NOT supported. Service Fabric validates the certificate against the `issuerThumbprint` of the CA that issued it. Public CAs typically REFUSE to issue Client Authentication EKU due to CA/Browser Forum baseline requirements, so this approach realistically requires a private/enterprise CA infrastructure. + +**Advantages**: + +- ✅ Fully automated rotation (new certificate from same CA issuer automatically trusted) +- ✅ Zero manual updates when certificate rotates +- ✅ Aligned with cluster certificate rotation approach + +**Prerequisites**: + +- Private CA infrastructure (ADCS, OpenSSL CA, etc.) +- CA configured to issue certificates with Client Authentication EKU + +**Configuration**: + +```json +{ + "clients": [ + { + "isAdmin": true, + "commonName": "apim-client.yourcompany.com", + "issuerThumbprint": "5F3660C715EBBDA31DB1FFDCF508302348DE8E7A" + } + ] +} +``` + +> [!NOTE] +> **Common Name Format**: The `commonName` value must be the certificate's SubjectName **without the 'CN=' prefix**. For example, if the certificate subject is `CN=apim-client.yourcompany.com`, use `apim-client.yourcompany.com` as the commonName value. + +**PowerShell**: + +```powershell +Add-AzServiceFabricManagedClusterClientCertificate ` + -ResourceGroupName "rg" ` + -ClusterName "cluster" ` + -CommonName "apim-client.yourcompany.com" ` + -IssuerThumbprint "5F3660C715EBBDA31DB1FFDCF508302348DE8E7A" ` + -Admin +``` + +**Trade-off**: Requires private CA infrastructure (not available to most organizations). + +### Recommendation Summary + +| Scenario | Recommended Approach | +|----------|---------------------| +| **Most organizations** | **Thumbprint-based with long-lived self-signed** | +| Production without private CA | Thumbprint-based (1 year validity) | +| Enterprises with private CA | Common name-based (fully automated rotation) | +| Dev/Test | Thumbprint-based (simplest setup) | + +## Process + +1. Create an Azure Resource Group to host the virtual network. + + ```powershell + $resourceGroupName = 'TestRG' + $location = 'EastUS' + New-AzResourceGroup -Name $resourceGroupName -location $location + ``` + +2. Create an Azure Network Security Group (NSG) for APIM. + + ```powershell + $networkSecurityGroupName = 'vnet-apim-nsg' + $networkSecurityGroup = New-AzNetworkSecurityGroup -Name $networkSecurityGroupName ` + -ResourceGroupName $resourceGroupName ` + -Location $location + ``` + +3. Configure NSG rules for APIM using PowerShell or in the [Azure portal](https://portal.azure.com). + + ```powershell + Add-AzNetworkSecurityRuleConfig -Name 'AllowManagementEndpoint' ` + -NetworkSecurityGroup $networkSecurityGroup ` + -Description 'Management endpoint for Azure portal and PowerShell' ` + -Access Allow ` + -Protocol Tcp ` + -Direction Inbound ` + -Priority 300 ` + -SourceAddressPrefix ApiManagement ` + -SourcePortRange * ` + -DestinationAddressPrefix VirtualNetwork ` + -DestinationPortRange 3443 + + # Update the network security group + Set-AzNetworkSecurityGroup -NetworkSecurityGroup $networkSecurityGroup + ``` + + > [!NOTE] + > Create additional rules as needed according to [Virtual network configuration requirements](https://learn.microsoft.com/azure/api-management/virtual-network-reference?tabs=stv2#required-ports). + +4. Create an Azure Virtual Network (VNET) named VNet with IP address prefix 10.0.0.0/16. + + ```powershell + $vnet = @{ + Name = 'VNet' + ResourceGroupName = $resourceGroupName + Location = $location + AddressPrefix = '10.0.0.0/16' + } + + $virtualNetwork = New-AzVirtualNetwork @vnet + ``` + +5. Create subnet configurations for the Service Fabric managed cluster and APIM. + + ```powershell + $sfmcSubnet = @{ + Name = 'sfmc' + VirtualNetwork = $virtualNetwork + AddressPrefix = '10.0.0.0/24' + } + + $apimSubnet = @{ + Name = 'apim' + VirtualNetwork = $virtualNetwork + AddressPrefix = '10.0.1.0/24' + NetworkSecurityGroup = $networkSecurityGroup + } + + Add-AzVirtualNetworkSubnetConfig @sfmcSubnet + Add-AzVirtualNetworkSubnetConfig @apimSubnet + ``` + +6. Associate the subnet configurations to the virtual network. + + ```powershell + $virtualNetwork | Set-AzVirtualNetwork + ``` + +7. Configure the Service Fabric Resource Provider permissions for Bring Your Own Virtual Network (BYOVNET). Refer to [Requirements](#requirements) for additional information. + + - Enumerate the Service Fabric Resource Provider (SFRP) principals from the current subscription: + + ```powershell + $sfrpPrincipal = @(Get-AzADServicePrincipal -DisplayName 'Azure Service Fabric Resource Provider') + ``` + + - Obtain the subnet resource ID from the existing VNet for the managed cluster: + + ```powershell + $virtualNetwork = Get-AzVirtualNetwork -Name $vnet.name -ResourceGroupName $resourceGroupName + $sfmcSubnetID = $virtualNetwork.Subnets | Where-Object Name -eq $sfmcSubnet.Name | Select-Object -ExpandProperty Id + ``` + + - Assign Network Contributor role to the Service Fabric Resource Provider principals: + + ```powershell + foreach($sfrpPrincipal in $sfrpPrincipal) { + New-AzRoleAssignment -PrincipalId $sfrpPrincipal.Id -RoleDefinitionName 'Network Contributor' -Scope $sfmcSubnetId + } + ``` + +8. Create a Public IP Address for APIM. + + ```powershell + $domainNameLabel = 'apimip' + $ip = @{ + Name = 'apimip' + ResourceGroupName = $resourceGroupName + Location = $location + Sku = 'Standard' + AllocationMethod = 'Static' + IpAddressVersion = 'IPv4' + DomainNameLabel = $domainNameLabel + } + + New-AzPublicIpAddress @ip + ``` + +9. Create the API Management Service with external VNET integration. This operation can take approximately 1 hour to complete. + + ```powershell + $virtualNetwork = Get-AzVirtualNetwork -Name $vnet.name -ResourceGroupName $resourceGroupName + $apimSubnetId = $virtualNetwork.Subnets | Where-Object Name -eq $apimSubnet.Name | Select-Object -ExpandProperty Id + $apimNetwork = New-AzApiManagementVirtualNetwork -SubnetResourceId $apimSubnetId + $publicIpAddressId = Get-AzPublicIpAddress -Name $ip.name -ResourceGroupName $resourceGroupName | Select-Object -ExpandProperty Id + $apimName = 'myApimCloud' + $adminEmail = 'admin@contoso.com' + $organization = 'contoso' + + New-AzApiManagement -ResourceGroupName $resourceGroupName ` + -Location $location ` + -Name $apimName ` + -Organization $organization ` + -AdminEmail $adminEmail ` + -VirtualNetwork $apimNetwork ` + -VpnType 'External' ` + -Sku 'Developer' ` + -PublicIpAddressId $publicIpAddressId + ``` + +10. Create the Service Fabric managed cluster within the existing VNET. + + > [!NOTE] + > An example Service Fabric Managed Cluster ARM template is provided in the [SFMC ARM deployment template](#sfmc-arm-deployment-template) section. + + > Additional Service Fabric managed cluster ARM templates with domainNameLabel configuration are available: + > - [SF-Managed-Standard-SKU-1-NT-DomainNameLabel](https://github.com/Azure-Samples/service-fabric-cluster-templates/tree/master/SF-Managed-Standard-SKU-1-NT-DomainNameLabel) + > - [SF-Managed-Standard-SKU-1-NT-DomainNameLabel-KVVM-MI](https://github.com/Azure-Samples/service-fabric-cluster-templates/tree/master/SF-Managed-Standard-SKU-1-NT-DomainNameLabel-KVVM-MI) + > - [Standard SKU Service Fabric managed cluster, 2 node types, deployed in to existing subnet](https://github.com/Azure-Samples/service-fabric-cluster-templates/tree/master/SF-Managed-Standard-SKU-2-NT-BYOVNET) + + ```powershell + $templateFile = "$pwd\sfmc-template.json" + $adminPassword = '' + $clientCertificateThumbprint = '' + + $sfmc = @{ + clusterName = 'sfmcapim' + clusterSku = 'Standard' + adminUserName = 'cloudadmin' + adminPassword = $adminPassword + clientCertificateThumbprint = $clientCertificateThumbprint + nodeType1name = 'nodetype1' + nodeType1vmSize = 'Standard_D2s_v3' + nodeType1vmInstanceCount = 5 + nodeType1dataDiskSizeGB = 256 + nodeType1vmImagePublisher = 'MicrosoftWindowsServer' + nodeType1vmImageOffer = 'WindowsServer' + nodeType1vmImageSku = '2022-Datacenter' + nodeType1vmImageVersion = 'latest' + subnetId = $sfmcSubnetId + } + + New-AzResourceGroupDeployment -Name 'sfmcDeployment' ` + -ResourceGroupName $resourceGroupName ` + -TemplateFile $templateFile ` + -TemplateParameterObject $sfmc + ``` + +11. Deploy an ASP.NET Web API service to Service Fabric. Refer to the [WeatherForecast Service Fabric API Example](#weatherforecast-service-fabric-api-example) section for guidance. + +12. Upload the client certificate to APIM. + + **Option A: Direct Upload (RECOMMENDED - Simpler)** + + Upload the PFX file directly to APIM (no Key Vault or managed identity required): + + ```powershell + $apiMgmtContext = New-AzApiManagementContext -ResourceGroupName $resourceGroupName -ServiceName $apimName + $certPath = "C:\path\to\apim-client.pfx" + $certPassword = "YourPassword" + + New-AzApiManagementCertificate -Context $apiMgmtContext ` + -CertificateId "sf-client-cert" ` + -PfxFilePath $certPath ` + -PfxPassword $certPassword + + $certificate = Get-AzApiManagementCertificate -Context $apiMgmtContext -CertificateId "sf-client-cert" + $clientCertThumbprint = $certificate.Thumbprint + ``` + + Or upload via Azure Portal: **APIM → Certificates → Add → Upload PFX** + + **Option B: Key Vault Integration (Optional - More Complex)** + + Use this approach only if you have existing Key Vault infrastructure or require centralized certificate management. + + a. Create a system-assigned managed identity for APIM: + + ```powershell + $apimService = Get-AzApiManagement -ResourceGroupName $resourceGroupName -Name $apimName + Set-AzApiManagement -InputObject $apimService -SystemAssignedIdentity + ``` + + b. Configure Key Vault access policy: + + For Key Vaults using **Access policies**: + + ```powershell + $keyVaultName = 'apimKV' + $managedIdentityId = (Get-AzADServicePrincipal -SearchString $apimName).Id + + Set-AzKeyVaultAccessPolicy -VaultName $keyVaultName ` + -ObjectId $managedIdentityId ` + -PermissionsToSecrets get,list ` + -PermissionsToCertificates get,list ` + -PermissionsToKeys get,list + ``` + + For Key Vaults using **RBAC**: + + ```powershell + $keyVaultName = 'apimKV' + $managedIdentityId = (Get-AzADServicePrincipal -SearchString $apimName).Id + $keyVaultId = (Get-AzKeyVault -VaultName $keyVaultName).Id + + New-AzRoleAssignment -ObjectId $managedIdentityId ` + -RoleDefinitionName 'Key Vault Secrets User' ` + -Scope $keyVaultId + New-AzRoleAssignment -ObjectId $managedIdentityId ` + -RoleDefinitionName 'Key Vault Reader' ` + -Scope $keyVaultId + ``` + + c. Create Key Vault certificate reference in APIM: + + ```powershell + $kvcertId = 'apimcloud-com' + $secretIdentifier = 'https://apimKV.vault.azure.net/secrets/apimcloud-com/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' + $apiMgmtContext = New-AzApiManagementContext -ResourceGroupName $resourceGroupName -ServiceName $apimName + + $keyvault = New-AzApiManagementKeyVaultObject -SecretIdentifier $secretIdentifier + $certificate = New-AzApiManagementCertificate -Context $apiMgmtContext ` + -CertificateId $kvcertId ` + -KeyVault $keyvault + + $clientCertThumbprint = $certificate.Thumbprint + ``` + +13. Create a Service Fabric backend in APIM. + + **Key Configuration Points**: + - **clientCertificateThumbprint**: The client certificate thumbprint from step 12 (authenticates APIM to cluster) + - **serverX509Names**: The cluster FQDN (validates cluster certificate by common name) + - **validateCertificateChain/Name**: Set to `true` when using a publicly trusted CA certificate (recommended). Set to `false` only if using self-signed or internal CA certificates that are not in the Azure trusted root store. + + > [!NOTE] + > APIM uses the standard Windows/Azure trusted root store, which includes most major public CAs (DigiCert, Let's Encrypt, etc.). If your cluster certificate is issued by a public CA, `validateCertificateChain: true` and `validateCertificateName: true` will work without additional configuration. No issuer pinning (`issuerCertificateThumbprint`) is required in `serverX509Names` when using public CA certificates. + + The cluster certificate is **separate** from the client certificate: + - **Cluster cert**: Authenticates cluster to APIM (auto-rotates every 90 days, validated by common name) + - **Client cert**: Authenticates APIM to cluster (your control, validated by thumbprint or common name) + + > [!NOTE] + > The APIM ARM deployment template is provided in the [APIM ARM deployment template](#apim-arm-deployment-template) section. + + ```powershell + $serviceFabricAppUrl = 'fabric:/sfWeatherApiCore/WeatherApi' # Format: 'fabric://' + $clusterName = 'sfmcapim' + $clusterResource = Get-AzResource -Name $clusterName -ResourceType 'Microsoft.ServiceFabric/managedclusters' + $cluster = Get-AzServiceFabricManagedCluster -Name $clustername -ResourceGroupName $clusterResource.ResourceGroupName + + $backend = @{ + apimName = $apimName + backendName = 'ServiceFabricBackend' + description = 'Service Fabric backend' + clientCertificateThumbprint = $clientCertThumbprint + managementEndpoints = @("https://$($cluster.Fqdn):$($cluster.HttpGatewayConnectionPort)") + maxPartitionResolutionRetries = 5 + serviceFabricManagedClusterFqdn = $cluster.Fqdn + protocol = 'http' + url = $serviceFabricAppUrl + validateCertificateChain = $true + validateCertificateName = $true + } + + $backend | ConvertTo-Json + + New-AzResourceGroupDeployment -Name 'apimBackendDeployment' ` + -ResourceGroupName $resourceGroupName ` + -TemplateFile "$pwd\apim-backend.json" ` + -TemplateParameterObject $backend + ``` + +14. Create an API in APIM. + + ```powershell + $apiId = 'service-fabric-weatherforecast-app' + $apiName = 'Service Fabric WeatherForecast App' + $serviceUrl = 'http://servicefabric' # This value is not used for Service Fabric and can be any value + + New-AzApiManagementApi -Context $apiMgmtContext ` + -ApiId $apiId ` + -Name $apiName ` + -ServiceUrl $serviceUrl ` + -Protocols @('http', 'https') ` + -Path 'api' + ``` + +15. Create an Operation. + + ```powershell + $operationId = 'service-fabric-weatherforecast-app-operation' + $operationName = 'Service Fabric WeatherForecast App Operation' + + New-AzApiManagementOperation -Context $apiMgmtContext ` + -ApiId $apiId ` + -OperationId $operationId ` + -Name $operationName ` + -Method 'GET' ` + -UrlTemplate '' ` + -Description '' + ``` + +16. Create a Policy. + + ```powershell + $sfResolveCondition = '@((int)context.Response.StatusCode != 200)' + $policyString = " + + + + + + + + + + + + + + + " + + Set-AzApiManagementPolicy -Context $apiMgmtContext ` + -ApiId $apiId ` + -Policy $policyString ` + -Format 'application/vnd.ms-azure-apim.policy.raw+xml' + ``` + +## Testing + +Test the APIM connection by navigating to the APIM service API in the [Azure Portal](https://ms.portal.azure.com/#view/HubsExtension/BrowseResource/resourceType/Microsoft.ApiManagement%2Fservice). + +1. Create a new API Operation. In this example, the only available API is 'WeatherForecast'. + + ![Create API Operation](../media/how-to-configure-apim-for-service-fabric-managed-cluster/apim-api-operation-create.png) + +2. Test the operation. + + ![Test API Operation](../media/how-to-configure-apim-for-service-fabric-managed-cluster/apim-api-operation-test.png) + +3. If needed, trace the operation for debugging purposes. + + ![Trace API Operation](../media/how-to-configure-apim-for-service-fabric-managed-cluster/apim-api-operation-trace.png) + +## Troubleshooting + +### Client Certificate Authentication Issues + +**Symptom**: "Certificate authentication failed" or "Access denied" errors when connecting to cluster + +**Most Common Cause**: Certificate missing Client Authentication EKU + +**Verification**: + +```powershell +# Check certificate EKUs +$thumbprint = "0123456789ABCDEF0123456789ABCDEF01234567" +$cert = Get-ChildItem Cert:\CurrentUser\My\$thumbprint + +Write-Host "Certificate Subject: $($cert.Subject)" -ForegroundColor Yellow +Write-Host "Certificate EKUs:" -ForegroundColor Yellow +$cert.EnhancedKeyUsageList | Format-Table FriendlyName, ObjectId + +# Verify Client Authentication EKU is present +if ($cert.EnhancedKeyUsageList.ObjectId -contains "1.3.6.1.5.5.7.3.2") { + Write-Host "✅ Certificate has Client Authentication EKU" -ForegroundColor Green +} else { + Write-Host "❌ Certificate MISSING Client Authentication EKU - will not work!" -ForegroundColor Red + Write-Host "You must regenerate the certificate with correct EKU" -ForegroundColor Yellow +} + +# Check validity period +Write-Host "`nValidity:" -ForegroundColor Yellow +Write-Host " Valid From: $($cert.NotBefore)" +Write-Host " Valid To: $($cert.NotAfter)" +Write-Host " Current: $(Get-Date)" + +if ((Get-Date) -lt $cert.NotBefore -or (Get-Date) -gt $cert.NotAfter) { + Write-Host "❌ Certificate is expired or not yet valid" -ForegroundColor Red +} +``` + +**Solution**: If EKU is missing, regenerate certificate with correct EKU (see [Client Certificate Configuration](#client-certificate-configuration) section). + +### Verify cluster connectivity + +Verify the ability to connect successfully to the cluster using PowerShell. The 'servicefabric' module is required and is installed as part of the Service Fabric SDK. + +```powershell +import-module servicefabric +import-module az.resources + +$location = 'EastUS' +$clusterName = 'sfmcapim' +$clusterResource = Get-AzResource -ResourceGroupName $resourceGroupName ` + -Name $clusterName ` + -ResourceType 'Microsoft.ServiceFabric/managedclusters' ` + -ExpandProperties +$serverCertThumbprint = $clusterResource.Properties.clusterCertificateThumbprints +$clusterFqdn = $clusterResource.Properties.fqdn +$clusterEndpoint = "$($clusterFqdn):19000" + +# Connect using client certificate thumbprint +Connect-ServiceFabricCluster -ConnectionEndpoint $clusterEndpoint ` + -ServerCommonName $clusterFqdn ` + -StoreLocation CurrentUser ` + -StoreName My ` + -X509Credential ` + -FindType FindByThumbprint ` + -FindValue '' ` + -Verbose + +# Connect using client certificate common name +Connect-ServiceFabricCluster -ConnectionEndpoint $clusterEndpoint ` + -ServerCommonName $clusterFqdn ` + -StoreLocation CurrentUser ` + -StoreName My ` + -X509Credential ` + -FindType FindBySubjectName ` + -FindValue '' ` + -Verbose + +# Connect using Entra ID authentication +Connect-ServiceFabricCluster -ConnectionEndpoint $clusterEndpoint ` + -AzureActiveDirectory ` + -ServerCertThumbprint $serverCertThumbprint ` + -Verbose + +# If above methods fail but below works, the issue might be with domain name label scope configuration +Connect-ServiceFabricCluster -ConnectionEndpoint $clusterEndpoint ` + -ServerThumbprint $serverCertThumbprint ` + -StoreLocation 'CurrentUser' ` + -StoreName 'My' ` + -X509Credential ` + -FindType FindByThumbprint ` + -FindValue '' ` + -Verbose + +# Connect using server thumbprint and client common name +Connect-ServiceFabricCluster -ConnectionEndpoint $clusterEndpoint ` + -ServerThumbprint $serverCertThumbprint ` + -StoreLocation 'CurrentUser' ` + -StoreName 'My' ` + -X509Credential ` + -FindType FindBySubjectName ` + -FindValue '' ` + -Verbose +``` + +#### Alternative: sfmc-connect Script (Recommended for SFMC) + +For Service Fabric Managed Clusters, the `sfmc-connect` helper script provides a simpler connection method that automatically queries Azure for cluster server certificate thumbprints, eliminating the need to have the server certificate in your local store. + +**Requirements**: + +- PowerShell with Az module authenticated +- Client certificate with Client Authentication EKU in CurrentUser\My +- Script from [Azure/Service-Fabric-Troubleshooting-Guides](https://github.com/Azure/Service-Fabric-Troubleshooting-Guides/blob/master/Scripts/sfmc-connect.ps1) + +**Download and Usage**: + +```powershell +# Download script if not present +$scriptUrl = "https://raw.githubusercontent.com/Azure/Service-Fabric-Troubleshooting-Guides/master/Scripts/sfmc-connect.ps1" +$scriptPath = "$env:TEMP\sfmc-connect.ps1" +Invoke-WebRequest -Uri $scriptUrl -OutFile $scriptPath + +# Connect with thumbprint (always use named parameters for reliability) +& $scriptPath ` + -clusterEndpoint "sfmcapim.centralus.cloudapp.azure.com" ` + -thumbprint "0123456789ABCDEF0123456789ABCDEF01234567" + +# OR connect with common name +& $scriptPath ` + -clusterEndpoint "sfmcapim.centralus.cloudapp.azure.com" ` + -commonName "apim-client.yourcompany.com" + +# Connect with domainNameLabelScope switch (RECOMMENDED for clusters with static FQDN) +# Use this for clusters configured with autoGeneratedDomainNameLabelScope property +& $scriptPath ` + -clusterEndpoint "sfmcapim.duhcd6fxcrbbffb0.centralus.sfmc.io" ` + -thumbprint "0123456789ABCDEF0123456789ABCDEF01234567" ` + -domainNameLabelScope + +# OR with common name and domainNameLabelScope +& $scriptPath ` + -clusterEndpoint "sfmcapim.duhcd6fxcrbbffb0.centralus.sfmc.io" ` + -commonName "apim-client.yourcompany.com" ` + -domainNameLabelScope +``` + +**Key Features**: + +- Automatically queries Azure for cluster certificate thumbprints via `Get-AzServiceFabricManagedCluster` +- Uses `-ServerCertThumbprint` parameter (no need for cluster cert in local certificate store) +- Supports both client certificate thumbprint and common name authentication +- **`-domainNameLabelScope` switch** for clusters with static FQDN configuration + - When specified, validates server certificate by common name (FQDN) instead of thumbprint + - **Recommended for clusters with `autoGeneratedDomainNameLabelScope` configured** + - Aligns with the static FQDN format (`...sfmc.io`) + - Automatically handles certificate rotation without connection updates +- Works from any machine with Azure PowerShell access + +**Verification After Connection**: + +```powershell +# Verify connection +Get-ServiceFabricNode | Format-Table NodeName, NodeStatus, HealthState + +# Check cluster health +Get-ServiceFabricClusterHealth + +# View cluster configuration +Get-ServiceFabricClusterConfiguration | ConvertFrom-Json | Format-List +``` + +### Test network connectivity + +Test network connectivity to the cluster management port using PowerShell. + +```powershell +$clusterEndpoint = 'sfmcapim.eastus.cloudapp.azure.com' +$clusterHttpPort = 19080 +Test-NetConnection -ComputerName $clusterEndpoint -Port $clusterHttpPort +``` + +### Test API response directly from cluster + +Test the API response directly from the cluster to verify service functionality. + +```powershell +$clusterEndpoint = 'sfmcapim.eastus.cloudapp.azure.com' +$apiPort = 8080 +Test-NetConnection -ComputerName $clusterEndpoint -Port $apiPort +Invoke-RestMethod "http://$($clusterEndpoint):$($apiPort)/WeatherForecast" + +# Expected output format: +# date temperatureC temperatureF summary +# ---- ------------ ------------ ------- +# 5/11/2023 10:11:03 AM 54 129 Mild +# 5/12/2023 10:11:03 AM 9 48 Sweltering +# 5/13/2023 10:11:03 AM 21 69 Sweltering +# 5/14/2023 10:11:03 AM 54 129 Scorching +# 5/15/2023 10:11:03 AM 42 107 Scorching +``` + +### Test APIM connectivity + +Test connectivity to APIM to verify end-to-end functionality. + +```powershell +$apimEndpoint = 'sfmcapimcloud.azure-api.net' +$apimPort = 443 +Test-NetConnection -ComputerName $apimEndpoint -Port $apimPort +Invoke-RestMethod "https://$($apimEndpoint)/api/WeatherForecast" +``` + +### DNS and Network Connectivity Issues + +**Symptom**: APIM returns 500/503 errors when calling Service Fabric backend, or backend health shows "unhealthy". + +**Architecture Context**: APIM connects to the Service Fabric managed cluster through two paths: + +1. **Management plane** (service discovery): APIM resolves the SFMC FQDN (e.g., `sfmcapim...sfmc.io`) to the Service Fabric managed cluster load balancer's **public IP** on port 19080. This is used for SF service instance resolution. +2. **Data plane** (API calls): After resolving service instances, APIM connects directly to cluster nodes via their **private IPs** (e.g., `10.0.0.x:8080`) through VNet routing. + +> [!IMPORTANT] +> This guide uses **External** VNet mode for APIM (`-VpnType 'External'`). External mode provides both public internet access and VNet connectivity, allowing APIM to reach the SFMC load balancer's public IP for management endpoint resolution while also routing data plane traffic directly to cluster nodes via VNet. + +**Common DNS Issues**: + +| Issue | Cause | Solution | +|-------|-------|----------| +| SFMC FQDN does not resolve | Custom DNS servers on VNet cannot resolve `*.sfmc.io` | Add Azure DNS (`168.63.129.16`) as a forwarder in your custom DNS, or create an Azure Private DNS zone with an A record for the SFMC FQDN pointing to the SFC_ load balancer public IP | +| APIM cannot reach management endpoint (19080) | APIM in Internal VNet mode cannot reach public IPs | Use External VNet mode (recommended), or configure UDR/NAT to allow outbound internet from APIM subnet | +| NSG blocking outbound traffic | NSG on APIM subnet blocking outbound to Internet | Ensure outbound rules allow APIM to reach the SFMC public IP on port 19080 | + +**Verification**: + +```powershell +# Verify SFMC FQDN resolves correctly +$clusterFqdn = 'sfmcapim...sfmc.io' +Resolve-DnsName $clusterFqdn + +# Verify the resolved IP matches the SFC_ managed resource group load balancer public IP +$sfcResourceGroup = Get-AzResourceGroup | Where-Object { $_.ResourceGroupName -like 'SFC_*' } +Get-AzPublicIpAddress -ResourceGroupName $sfcResourceGroup.ResourceGroupName | Format-Table Name, IpAddress + +# Verify APIM can reach the management endpoint +Test-NetConnection -ComputerName $clusterFqdn -Port 19080 +``` + +> [!NOTE] +> If your VNet uses custom DNS servers (e.g., on-premises DNS or Azure DNS Private Resolver) that cannot resolve `*.sfmc.io`, you have two options: +> 1. **Recommended**: Add Azure DNS (`168.63.129.16`) as a conditional forwarder for the `sfmc.io` zone in your custom DNS server. +> 2. **Alternative**: Create an Azure Private DNS zone linked to your VNet with an A record mapping the SFMC FQDN to the cluster load balancer's public IP address. + +### Deploying Certificates to Cluster Nodes (vmSecrets) + +If your Service Fabric application requires a certificate on the cluster nodes (e.g., for HTTPS endpoints), you need to deploy the certificate using `vmSecrets` in the SFMC node type configuration. This is **separate from the client certificate** used for APIM/admin authentication. + +> [!IMPORTANT] +> The client certificate configured in the `clients` array is NOT installed on cluster nodes. It is only used for authentication validation. If your SF application needs a certificate on the nodes (e.g., for HTTPS bindings), you must deploy it separately via `vmSecrets`. + +**Prerequisites**: +- Azure Key Vault with the certificate imported +- Key Vault access policy or RBAC granting the SFMC identity access to get secrets + +**ARM Template Configuration** (in node type properties): + +```json +"vmSecrets": [ + { + "sourceVault": { + "id": "/subscriptions//resourceGroups//providers/Microsoft.KeyVault/vaults/" + }, + "vaultCertificates": [ + { + "certificateUrl": "https://.vault.azure.net/secrets//", + "certificateStore": "My" + } + ] + } +] +``` + +**PowerShell approach** (update existing node type): + +```powershell +# Get the Key Vault certificate URL +$kvCert = Get-AzKeyVaultCertificate -VaultName $keyVaultName -Name $certName +$certUrl = $kvCert.SecretId +$kvId = (Get-AzKeyVault -VaultName $keyVaultName).ResourceId + +# Redeploy the SFMC template with vmSecrets parameters +$sfmcParams = @{ + # ... existing parameters ... + keyVaultResourceId = $kvId + keyVaultCertificateUrl = $certUrl +} + +New-AzResourceGroupDeployment -Name 'sfmcVmSecrets' ` + -ResourceGroupName $resourceGroupName ` + -TemplateFile $clusterTemplateFile ` + -TemplateParameterObject $sfmcParams +``` + +> [!WARNING] +> **Known Issue**: vmSecrets deployment can fail mid-rollout if a VMSS upgrade domain encounters an error during reimaging. This leaves some nodes with the certificate and others without. +> +> **Workaround**: +> 1. If a Service Fabric application is deployed, delete the application/service first +> 2. Resubmit the vmSecrets deployment (same ARM template PUT) +> 3. Wait for all upgrade domains to complete +> 4. Verify all VMSS instances show `latestModelApplied = True` +> 5. Redeploy the Service Fabric application +> +> **Verification**: Check VMSS instances in the SFC_ managed resource group: +> ```powershell +> $sfcRg = (Get-AzResourceGroup | Where-Object { $_.ResourceGroupName -match '^SFC_' }).ResourceGroupName +> $vmss = Get-AzVmss -ResourceGroupName $sfcRg +> Get-AzVmssVM -ResourceGroupName $sfcRg -VMScaleSetName $vmss.Name | ForEach-Object { +> $instance = Get-AzVmssVM -ResourceGroupName $sfcRg -VMScaleSetName $vmss.Name -InstanceId $_.InstanceId +> [PSCustomObject]@{ +> Name = $_.Name +> LatestModelApplied = $instance.LatestModelApplied +> } +> } +> ``` + +## Certificate Rotation + +### Understanding Certificate Rotation + +Service Fabric managed clusters use **two independent certificates** for APIM integration: + +#### Cluster Certificate + +- **Purpose**: Authenticates cluster to APIM +- **Managed by**: Service Fabric Resource Provider (automatic) +- **Rotation frequency**: Every 90 days (automatic) +- **APIM validation**: By common name (`serverX509Names`) - NOT thumbprint +- **Impact on APIM**: **Zero manual intervention** when using `autoGeneratedDomainNameLabelScope` +- **Installation**: Auto-installed on cluster nodes only (NOT on client machines) + +With `autoGeneratedDomainNameLabelScope: "ResourceGroupReuse"`, the cluster FQDN remains stable (e.g., `clustername.hash.region.sfmc.io`), and the server certificate continues to match this common name through rotation cycles. + +#### Client Certificate (APIM Authentication Certificate) + +- **Purpose**: Authenticates APIM to cluster +- **Managed by**: You (manual or via private CA) +- **Rotation frequency**: Your choice (recommend yearly for thumbprint-based) +- **Cluster validation**: By thumbprint OR common name + issuer +- **Impact on APIM**: Depends on validation method + +**Thumbprint-based** (most common): + +- Manual update required when certificate expires +- Update process: + 1. Generate new certificate with Client Auth EKU + 2. Add new certificate to cluster `clients` array (keep old one during transition) + 3. Upload new certificate to APIM + 4. Update APIM backend with new thumbprint + 5. Remove old certificate from cluster after validation +- Frequency: Yearly (manageable operational overhead) + +**Common name-based** (requires private CA): + +- Automatic rotation when new certificate issued from same CA +- No APIM backend updates required +- Requires private CA infrastructure + +### Best Practice + +**For most organizations**: Use thumbprint-based client certificate with 1 year validity. The annual manual rotation is an acceptable trade-off for simpler infrastructure requirements compared to maintaining private CA infrastructure. + +**For enterprises with private CA**: Use common name-based client certificate for fully automated rotation aligned with cluster certificate rotation. + +## Reference + +### Service Fabric managed cluster templates + +- Service Fabric managed cluster with DomainNameLabel configuration: [SF-Managed-Standard-SKU-1-NT-DomainNameLabel](https://github.com/Azure-Samples/service-fabric-cluster-templates/tree/master/SF-Managed-Standard-SKU-1-NT-DomainNameLabel) +- Service Fabric managed cluster with DomainNameLabel configuration, Key Vault VM manager, and user-assigned Managed Identity: [SF-Managed-Standard-SKU-1-NT-DomainNameLabel-KVVM-MI](https://github.com/Azure-Samples/service-fabric-cluster-templates/tree/master/SF-Managed-Standard-SKU-1-NT-DomainNameLabel-KVVM-MI) + +### SFMC ARM deployment template + +sfmc-template.json + +```json +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "parameters": { + "clusterName": { + "type": "string", + "minLength": 4, + "maxLength": 23, + "metadata": { + "description": "Name of your cluster - Between 3 and 23 characters. Letters and numbers only" + } + }, + "clusterSku": { + "type": "string", + "allowedValues": ["Basic", "Standard"], + "defaultValue": "Standard" + }, + "adminUserName": { + "type": "string", + "defaultValue": "vmadmin" + }, + "adminPassword": { + "type": "securestring" + }, + "clientCertificateThumbprint": { + "type": "string" + }, + "nodeType1Name": { + "type": "string", + "maxLength": 9, + "defaultValue": "NT1" + }, + "nodeType1VmSize": { + "type": "string", + "defaultValue": "Standard_D2s_v3" + }, + "nodeType1VmInstanceCount": { + "type": "int", + "defaultValue": 5 + }, + "nodeType1DataDiskSizeGB": { + "type": "int", + "defaultValue": 256 + }, + "nodeType1managedDataDiskType": { + "type": "string", + "allowedValues": ["Standard_LRS", "StandardSSD_LRS", "Premium_LRS"], + "defaultValue": "StandardSSD_LRS" + }, + "nodeType1vmImagePublisher": { + "type": "string", + "defaultValue": "MicrosoftWindowsServer" + }, + "nodeType1vmImageOffer": { + "type": "string", + "defaultValue": "WindowsServer" + }, + "nodeType1vmImageSku": { + "type": "string", + "defaultValue": "2022-Datacenter" + }, + "nodeType1vmImageVersion": { + "type": "string", + "defaultValue": "latest" + }, + "subnetId": { + "type": "string", + "metadata": { + "description": "/subscriptions//resourceGroups//providers/Microsoft.Network/virtualNetworks//subnets/" + } + } + }, + "variables": {}, + "resources": [ + { + "apiVersion": "2024-09-01-preview", + "type": "Microsoft.ServiceFabric/managedclusters", + "name": "[parameters('clusterName')]", + "location": "[resourcegroup().location]", + "sku": { + "name": "[parameters('clusterSku')]" + }, + "properties": { + "autoGeneratedDomainNameLabelScope": "ResourceGroupReuse", + "subnetId": "[parameters('subnetId')]", + "dnsName": "[toLower(parameters('clusterName'))]", + "adminUserName": "[parameters('adminUserName')]", + "adminPassword": "[parameters('adminPassword')]", + "clientConnectionPort": 19000, + "httpGatewayConnectionPort": 19080, + "allowRdpAccess": false, + "enableIpv6": false, + "addonFeatures": ["DnsService"], + "clients": [ + { + "isAdmin": true, + "thumbprint": "[parameters('clientCertificateThumbprint')]" + } + ], + "loadBalancingRules": [ + { + "frontendPort": 443, + "backendPort": 443, + "protocol": "tcp", + "probeProtocol": "tcp" + }, + { + "frontendPort": 8080, + "backendPort": 8080, + "protocol": "tcp", + "probeProtocol": "tcp" + } + ] + }, + "tags": { + "Environment": "APIM-SF" + } + }, + { + "apiVersion": "2024-09-01-preview", + "type": "Microsoft.ServiceFabric/managedclusters/nodetypes", + "name": "[concat(parameters('clusterName'), '/', parameters('nodeType1Name'))]", + "location": "[resourcegroup().location]", + "dependsOn": [ + "[concat('Microsoft.ServiceFabric/managedclusters/', parameters('clusterName'))]" + ], + "properties": { + "isPrimary": true, + "vmImagePublisher": "[parameters('nodeType1vmImagePublisher')]", + "vmImageOffer": "[parameters('nodeType1vmImageOffer')]", + "vmImageSku": "[parameters('nodeType1vmImageSku')]", + "vmImageVersion": "[parameters('nodeType1vmImageVersion')]", + "vmSize": "[parameters('nodeType1VmSize')]", + "vmInstanceCount": "[parameters('nodeType1VmInstanceCount')]", + "dataDiskSizeGB": "[parameters('nodeType1DataDiskSizeGB')]", + "dataDiskType": "[parameters('nodeType1managedDataDiskType')]", + "applicationPorts": { + "startPort": 20000, + "endPort": 30000 + }, + "ephemeralPorts": { + "startPort": 49152, + "endPort": 65534 + } + } + } + ], + "outputs": { + "serviceFabricExplorer": { + "value": "[concat('https://', reference(parameters('clusterName')).fqdn, ':', reference(parameters('clusterName')).httpGatewayConnectionPort)]", + "type": "string" + }, + "clientConnectionEndpoint": { + "value": "[concat(reference(parameters('clusterName')).fqdn, ':', reference(parameters('clusterName')).clientConnectionPort)]", + "type": "string" + }, + "clusterProperties": { + "value": "[reference(parameters('clusterName'))]", + "type": "object" + } + } +} +``` + +### APIM ARM deployment template + +apim-backend.json + +```json +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "parameters": { + "apimName": { + "type": "string" + }, + "backendName": { + "type": "string" + }, + "description": { + "type": "string" + }, + "clientCertificatethumbprint": { + "type": "string" + }, + "managementEndpoints": { + "type": "array" + }, + "maxPartitionResolutionRetries": { + "type": "int" + }, + "serviceFabricManagedClusterFqdn": { + "type": "string" + }, + "protocol": { + "type": "string" + }, + "url": { + "type": "string" + }, + "validateCertificateChain": { + "type": "bool" + }, + "validateCertificateName": { + "type": "bool" + } + }, + "variables": {}, + "resources": [ + { + "type": "Microsoft.ApiManagement/service/backends", + "apiVersion": "2022-08-01", + "name": "[concat(parameters('apimName'), '/', parameters('backendName'))]", + "properties": { + "description": "[parameters('description')]", + "properties": { + "serviceFabricCluster": { + "clientCertificatethumbprint": "[parameters('clientCertificatethumbprint')]", + "managementEndpoints": "[parameters('managementEndpoints')]", + "maxPartitionResolutionRetries": "[parameters('maxPartitionResolutionRetries')]", + "serverX509Names": [ + { + "name": "[parameters('serviceFabricManagedClusterFqdn')]" + } + ] + } + }, + "protocol": "[parameters('protocol')]", + "tls": { + "validateCertificateChain": "[parameters('validateCertificateChain')]", + "validateCertificateName": "[parameters('validateCertificateName')]" + }, + "url": "[parameters('url')]" + } + } + ] +} +``` + +### WeatherForecast Service Fabric Api Example + +To create an example .NET Core API Service Fabric application project to test the APIM connection setup, the built-in .NET Core Stateless Web API template can be used as shown in the figures below: + +#### **Create Service Fabric Application Project** + +![create service fabric application project](../media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-1.png) + +#### **Configure project** + +![configure project](../media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-2.png) + +#### **Create a new service** + +![create a new service](../media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-3.png) + +#### **Create a new ASP.NET Core web application** + +![create a new asp.net core web application](../media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-4.png) + +#### **Modify ApplicationManifest.xml** + +To use the Swagger functionality configured in the template, set the ASPNETCORE_ENVIRONMENT variable to 'Development' in 'ApplicationManifest.xml'. + +![modify application manifest](../media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-5.png) + +#### **Modify ServiceManifest.xml** + +To configure the web application TCP listening port, modify the 'ServiceManifest.xml' file and set 'Port' to '8080'. This step is not required but can help with testing and troubleshooting. + +![modify service manifest](../media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-6.png) + +#### **Publish application and verify** + +After completing the modifications above, publish the application to the Service Fabric cluster. + +To verify deployment, use Service Fabric Explorer (SFX) to check application configuration and status. + +![verify deployment](../media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-sfx.png) + +To verify API functionality, use PowerShell, Postman, or a browser. + +> [!NOTE] +> NSG and Load Balancer rules must be configured for APIM port 443 and API test port 8080 before a connection can be established. + +![verify functionality](../media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-ps.png) + +![verify swagger](../media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-swagger.png) + +### Additional Resources + +- [Add a managed identity to a Service Fabric managed cluster node type](https://learn.microsoft.com/azure/service-fabric/how-to-managed-identity-managed-cluster-virtual-machine-scale-sets) + +- [Azure built-in roles](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles) diff --git a/Deployment/how-to-configure-service-fabric-managed-cluster-static-fqdn.md b/Deployment/how-to-configure-service-fabric-managed-cluster-static-fqdn.md new file mode 100644 index 00000000..5a029454 --- /dev/null +++ b/Deployment/how-to-configure-service-fabric-managed-cluster-static-fqdn.md @@ -0,0 +1,295 @@ +# How to Configure Service Fabric Managed Cluster with Static FQDN (autoGeneratedDomainNameLabelScope) + +This document describes how to configure Service Fabric managed clusters with a static FQDN using the `autoGeneratedDomainNameLabelScope` property. This configuration provides a persistent FQDN that survives cluster certificate rotation, enabling stable connectivity for API Management (APIM), CI/CD pipelines, and other external integrations. + +## Overview + +Service Fabric managed clusters (SFMC) automatically rotate the cluster certificate every 90 days. By default, this rotation changes the cluster's FQDN, breaking external connections that rely on certificate thumbprint validation. The `autoGeneratedDomainNameLabelScope` property solves this by providing a static FQDN that persists across certificate rotations. + +**Traditional FQDN format**: `..cloudapp.azure.com` +**New static FQDN format**: `...sfmc.io` + +**Example**: `sfmctest1nt31.duhcd6fxcrbbffb0.centralus.sfmc.io` + +## Key Benefits + +- ✅ **Static FQDN** that persists across resource group redeployments +- ✅ **Automatic certificate rotation** (every 90 days) without FQDN change +- ✅ **No thumbprint dependency** - validate by common name instead +- ✅ **Compatible with APIM, CI/CD pipelines, and monitoring tools** +- ✅ **No manual intervention** required during certificate rotation + +## Requirements + +- ARM template API version: **2024-06-01-preview** or later (tested with **2024-09-01-preview**) +- PowerShell: **Az.ServiceFabric** module version **3.7.0** or later + +> [!NOTE] +> To check your current Az.ServiceFabric version: `Get-Module -Name Az.ServiceFabric -ListAvailable | Select-Object Version` +> To update: `Update-Module -Name Az.ServiceFabric` + +## Configuration Methods + +### Option 1: New Cluster Deployment (Recommended) + +Configure `autoGeneratedDomainNameLabelScope` during initial cluster creation for immediate availability. + +**ARM Template**: + +```json +{ + "apiVersion": "2024-09-01-preview", + "type": "Microsoft.ServiceFabric/managedclusters", + "name": "[parameters('clusterName')]", + "location": "[resourcegroup().location]", + "sku": { + "name": "[parameters('clusterSku')]" + }, + "properties": { + "autoGeneratedDomainNameLabelScope": "ResourceGroupReuse", + "dnsName": "[toLower(parameters('clusterName'))]", + "adminUserName": "[parameters('adminUserName')]", + "adminPassword": "[parameters('adminPassword')]", + // ... other properties + } +} +``` + +**PowerShell**: + +```powershell +$parameters = @{ + ResourceGroupName = 'TestRG' + Name = 'sfmccluster' + Location = 'centralus' + ClusterSku = 'Standard' + AdminUserName = 'cloudadmin' + AdminPassword = $adminPassword + AutoGeneratedDomainNameLabelScope = 'ResourceGroupReuse' + # ... other parameters +} + +New-AzServiceFabricManagedCluster @parameters +``` + +### Option 2: Update Existing Cluster + +For existing clusters without `autoGeneratedDomainNameLabelScope`, you can add this property post-deployment. **Important**: The backend provisioning process can take up to 4 hours to complete, though in practice it may complete much faster. + +#### Understanding the Update Process + +When updating an existing cluster, `Set-AzServiceFabricManagedCluster` performs the following: + +1. **3 Upgrade Domain (UD) Walks**: The operation walks through all 3 upgrade domains to apply the configuration change +2. **Long-Running Operation**: Due to the UD walks and backend periodic job, the operation can exceed standard ARM timeout limits +3. **Automatic Revert on Failure**: If the operation fails or times out, the cluster **automatically reverts** the change to maintain stability +4. **Safe to Retry**: The revert behavior makes it safe to retry immediately after timeout or failure + +**Expected Behavior**: +- ✅ ARM operation may timeout before backend job completes (normal) +- ✅ Cluster remains accessible during the entire process +- ✅ Configuration reverts automatically if operation fails +- ⏳ Backend periodic job can take up to 4 hours after successful ARM operation + +**Using PowerShell**: + +```powershell +# Update existing cluster with autoGeneratedDomainNameLabelScope +$resourceGroupName = 'TestRG' +$clusterName = 'sfmccluster' + +Set-AzServiceFabricManagedCluster -ResourceGroupName $resourceGroupName ` + -Name $clusterName ` + -AutoGeneratedDomainNameLabelScope 'ResourceGroupReuse' + +# Note: The command may timeout due to the 4-hour backend job window. +# This is expected behavior - the cluster will revert the change if the operation fails. +# Safe to retry immediately if timeout occurs. +``` + +**Handling Timeouts**: + +```powershell +try { + Set-AzServiceFabricManagedCluster -ResourceGroupName $resourceGroupName ` + -Name $clusterName ` + -AutoGeneratedDomainNameLabelScope 'ResourceGroupReuse' ` + -ErrorAction Stop + Write-Host "✅ Operation completed successfully" -ForegroundColor Green +} +catch { + if ($_.Exception.Message -match "timeout|timed out") { + Write-Host "⚠️ Operation timed out (expected). Retrying..." -ForegroundColor Yellow + Write-Host " Note: Cluster reverted change automatically." -ForegroundColor Cyan + + # Retry immediately - safe because cluster reverted + Set-AzServiceFabricManagedCluster -ResourceGroupName $resourceGroupName ` + -Name $clusterName ` + -AutoGeneratedDomainNameLabelScope 'ResourceGroupReuse' + } + else { + Write-Error $_.Exception.Message + } +} +``` + +**Using ARM Template**: + +```json +{ + "apiVersion": "2024-09-01-preview", + "type": "Microsoft.ServiceFabric/managedclusters", + "properties": { + "autoGeneratedDomainNameLabelScope": "ResourceGroupReuse", + // ... other existing properties + } +} +``` + +> [!NOTE] +> ARM template deployments are subject to the same timeout and UD walk behavior as PowerShell commands. Consider using PowerShell with retry logic for better timeout handling. + +## Monitoring FQDN Provisioning + +After applying the configuration, monitor the FQDN to determine when the backend job completes: + +```powershell +# Check current FQDN +$cluster = Get-AzServiceFabricManagedCluster -ResourceGroupName $resourceGroupName -Name $clusterName +Write-Host "Current FQDN: $($cluster.Fqdn)" + +# Check if updated to *.sfmc.io format +if ($cluster.Fqdn -like "*.sfmc.io") { + Write-Host "✅ FQDN successfully updated to static format" -ForegroundColor Green +} else { + Write-Host "⏳ FQDN not yet updated. Backend job still in progress." -ForegroundColor Yellow + Write-Host " Current: $($cluster.Fqdn)" + Write-Host " Wait and check again (can take up to 4 hours)" +} +``` + +## Verification After Backend Job Completes + +```powershell +# The new FQDN format will be: ...sfmc.io +# Example: sfmctest1nt31.duhcd6fxcrbbffb0.centralus.sfmc.io + +# Test connectivity to new endpoint +$cluster = Get-AzServiceFabricManagedCluster -ResourceGroupName $resourceGroupName -Name $clusterName +Test-NetConnection -ComputerName $cluster.Fqdn -Port 19000 + +# Verify FQDN format +if ($cluster.Fqdn -match "^[^.]+\.[^.]+\.[^.]+\.sfmc\.io$") { + Write-Host "✅ FQDN format correct: $($cluster.Fqdn)" -ForegroundColor Green +} else { + Write-Host "⚠️ FQDN format unexpected: $($cluster.Fqdn)" -ForegroundColor Yellow +} +``` + +## Troubleshooting Slow Provisioning + +The `Set-AzServiceFabricManagedCluster` command may timeout due to the 4-hour backend job window and 3 UD walks. **Important**: If the operation fails or times out, the cluster automatically reverts the change. You can retry immediately - don't wait the full 4 hours. + +```powershell +# Check if FQDN has changed to *.sfmc.io format +$cluster = Get-AzServiceFabricManagedCluster -ResourceGroupName $resourceGroupName -Name $clusterName +if ($cluster.Fqdn -notlike "*.sfmc.io") { + Write-Host "FQDN not yet updated. Retrying Set-AzServiceFabricManagedCluster..." -ForegroundColor Yellow + + # Retry the operation - safe to retry even if less than 4 hours + # If previous operation timed out, cluster already reverted the change + Set-AzServiceFabricManagedCluster -ResourceGroupName $resourceGroupName ` + -Name $clusterName ` + -AutoGeneratedDomainNameLabelScope 'ResourceGroupReuse' + + Write-Host "Retry initiated. Backend periodic job can take up to 4 hours." -ForegroundColor Cyan + Write-Host "Check again periodically or if command times out." -ForegroundColor Cyan +} else { + Write-Host "✅ FQDN successfully updated: $($cluster.Fqdn)" -ForegroundColor Green +} +``` + +### Common Issues + +| Issue | Description | Resolution | +|-------|-------------|------------| +| **"ARM operation times out"** | Set-AzServiceFabricManagedCluster returns timeout after 3 UD walks | **Retry immediately** - cluster automatically reverted change, safe to retry | +| **"Operation appears stuck"** | Command running longer than expected | Normal - 3 UD walks take time. Wait for timeout or completion | +| **"Set-AzServiceFabricManagedCluster fails"** | Command returns error | Cluster reverted automatically. Retry the operation | +| **"FQDN still shows old format"** | Backend job hasn't completed after successful command | Retry Set-AzServiceFabricManagedCluster to re-trigger backend job | +| **"ARM shows property but FQDN unchanged"** | Azure Resource Manager updated but backend pending | Normal - ARM updates immediately, backend job follows (up to 4 hours) | +| **"Multiple retries needed"** | Several attempts before FQDN updates | Normal - backend periodic job timing varies. Keep retrying | + +**Key Points**: +- ✅ **Automatic revert on failure** - cluster rolls back change if operation fails +- ✅ **Safe to retry immediately** - revert behavior ensures cluster stability +- ✅ **No need to wait 4 hours** - the 4-hour window is for backend job, not retry wait time +- ✅ **3 UD walks required** - operation touches all upgrade domains (takes time) +- ✅ **Retrying re-triggers the backend job** - helps if periodic job missed first request +- ⏳ **Backend job can take up to 4 hours** after successful ARM operation + +## Validated Test Results + +**Test Configuration** (January 18, 2026): +- Cluster: `sfmctest1nt31` +- Resource Group: `sfmctest1nt31` +- Location: `centralus` +- Initial FQDN: `sfmctest1nt31.centralus.cloudapp.azure.com` + +**Command Executed**: +```powershell +Set-AzServiceFabricManagedCluster -ResourceGroupName "sfmctest1nt31" ` + -Name "sfmctest1nt31" ` + -AutoGeneratedDomainNameLabelScope 'ResourceGroupReuse' +``` + +**Results**: +- ✅ ARM operation completed: ~6 minutes +- ✅ FQDN immediately updated: `sfmctest1nt31.duhcd6fxcrbbffb0.centralus.sfmc.io` +- ✅ Connectivity verified: Port 19000 accessible +- ✅ Format validated: `*.sfmc.io` pattern correct + +**Key Finding**: While Microsoft documentation states the backend job **can take up to 4 hours**, in this test the FQDN was updated immediately upon command completion. Actual provisioning time may vary by cluster configuration and backend load. + +## Important Notes + +- **Deployment-time configuration recommended**: For new clusters, configure during initial deployment for immediate availability +- **Property update is immediate**: Azure Resource Manager reflects the change immediately +- **Backend job timing varies**: FQDN provisioning may complete immediately or take up to 4 hours +- **Cluster remains accessible**: Old endpoint continues working during transition period +- **No downtime**: Certificate rotation happens seamlessly with static FQDN + +## SFMC Architecture Note + +Service Fabric managed clusters create infrastructure resources in a **separate auto-managed resource group** named `SFC_`. This resource group contains: +- Virtual Machine Scale Sets (VMSS) +- Storage accounts +- Load balancers +- Network security groups +- Virtual networks (if not BYO VNET) + +The primary resource group contains only the cluster resource itself and optionally a managed identity. When troubleshooting or monitoring deployments, check both resource groups. + +## Use Cases + +This configuration is essential for: + +1. **Azure API Management (APIM)**: Backend configuration using common name validation instead of thumbprint +2. **CI/CD Pipelines**: Stable endpoint for deployment automation +3. **Monitoring Tools**: Consistent FQDN for metrics collection and alerting +4. **External Applications**: Any system connecting to the cluster that cannot handle FQDN changes +5. **Certificate Management**: Eliminates manual updates when cluster certificate rotates + +## Related Documentation + +- [How to configure APIM for a Service Fabric managed cluster](./how-to-configure-apim-for-service-fabric-managed-cluster.md) +- [How to Export Service Fabric Managed Cluster Configuration](./how-to-export-service-fabric-managed-cluster-configuration.md) +- [Bring your own virtual network (Microsoft Learn)](https://learn.microsoft.com/azure/service-fabric/how-to-managed-cluster-networking#bring-your-own-virtual-network) +- [Managed TLS Solution (Internal - requires Microsoft authentication)](https://eng.ms/docs/products/azure/service-fabric/how-to/publiccacluster/managed-tls-solution) + +## Reference Templates + +Example Service Fabric managed cluster ARM templates with domainNameLabel configuration: +- [SF-Managed-Standard-SKU-1-NT-DomainNameLabel](https://github.com/jagilber/service-fabric-cluster-templates/blob/sfmcapim/SF-Managed-Standard-SKU-1-NT-DomainNameLabel/azuredeploy.json) +- [SF-Managed-Standard-SKU-1-NT-DomainNameLabel-KVVM-MI](https://github.com/jagilber/service-fabric-cluster-templates/blob/sfmcapim/SF-Managed-Standard-SKU-1-NT-DomainNameLabel-KVVM-MI/azuredeploy.json) +- [Standard SKU Service Fabric managed cluster, 2 node types, deployed in to existing subnet](https://github.com/Azure-Samples/service-fabric-cluster-templates/tree/master/SF-Managed-Standard-SKU-2-NT-BYOVNET) diff --git a/Scripts/sfmc-connect.ps1 b/Scripts/sfmc-connect.ps1 index 4009a35e..d0c93209 100644 --- a/Scripts/sfmc-connect.ps1 +++ b/Scripts/sfmc-connect.ps1 @@ -32,6 +32,9 @@ .EXAMPLE .\sfmc-connect.ps1 -clusterEndpoint mycluster.eastus.cloudapp.azure.com -commonName *.mycluster.com +.EXAMPLE + .\sfmc-connect.ps1 -clusterEndpoint mycluster.eastus.cloudapp.azure.com -thumbprint ABCD... -domainNameLabelScope + .LINK invoke-webRequest "https://raw.githubusercontent.com/Azure/Service-Fabric-Troubleshooting-Guides/master/Scripts/sfmc-connect.ps1" -outFile "$pwd\sfmc-connect.ps1"; @@ -58,7 +61,11 @@ param( [Parameter(ParameterSetName = 'thumbprint')] [Parameter(ParameterSetName = 'commonName')] - $clusterendpointPort = 19000 + $clusterendpointPort = 19000, + + [Parameter(ParameterSetName = 'thumbprint')] + [Parameter(ParameterSetName = 'commonName')] + [switch]$domainNameLabelScope ) function main() { @@ -130,23 +137,47 @@ function main() { write-host "using server thumbprint:$serverCertThumbprint" -ForegroundColor Cyan } - write-host "Connect-ServiceFabricCluster -ConnectionEndpoint $clusterEndpoint`:$clusterendpointPort `` - -ServerCertThumbprint $serverCertThumbprint `` - -StoreLocation $storeLocation `` - -StoreName $storeName `` - -X509Credential `` - -FindType $findType `` - -FindValue $findValue `` - -Verbose" -ForegroundColor Green - - Connect-ServiceFabricCluster -ConnectionEndpoint "$clusterEndpoint`:$clusterendpointPort" ` - -ServerCertThumbprint $serverCertThumbprint ` - -StoreLocation $storeLocation ` - -StoreName $storeName ` - -X509Credential ` - -FindType $findType ` - -FindValue $findValue ` - -verbose + # Extract FQDN for ServerCommonName if using domainNameLabelScope + $clusterFqdn = $clusterEndpoint -replace ':\d+$', '' + + if ($domainNameLabelScope) { + write-host "Connect-ServiceFabricCluster -ConnectionEndpoint $clusterEndpoint`:$clusterendpointPort `` + -ServerCommonName $clusterFqdn `` + -StoreLocation $storeLocation `` + -StoreName $storeName `` + -X509Credential `` + -FindType $findType `` + -FindValue $findValue `` + -Verbose" -ForegroundColor Green + + Connect-ServiceFabricCluster -ConnectionEndpoint "$clusterEndpoint`:$clusterendpointPort" ` + -ServerCommonName $clusterFqdn ` + -StoreLocation $storeLocation ` + -StoreName $storeName ` + -X509Credential ` + -FindType $findType ` + -FindValue $findValue ` + -verbose + } + else { + write-host "Connect-ServiceFabricCluster -ConnectionEndpoint $clusterEndpoint`:$clusterendpointPort `` + -ServerCertThumbprint $serverCertThumbprint `` + -StoreLocation $storeLocation `` + -StoreName $storeName `` + -X509Credential `` + -FindType $findType `` + -FindValue $findValue `` + -Verbose" -ForegroundColor Green + + Connect-ServiceFabricCluster -ConnectionEndpoint "$clusterEndpoint`:$clusterendpointPort" ` + -ServerCertThumbprint $serverCertThumbprint ` + -StoreLocation $storeLocation ` + -StoreName $storeName ` + -X509Credential ` + -FindType $findType ` + -FindValue $findValue ` + -verbose + } } function get-clientCert($storeLocation, $storeName, $thumbprint = $null, $commonName = $null) { diff --git a/media/how-to-configure-apim-for-service-fabric-managed-cluster/apim-api-operation-create.png b/media/how-to-configure-apim-for-service-fabric-managed-cluster/apim-api-operation-create.png new file mode 100644 index 00000000..42fe3f35 Binary files /dev/null and b/media/how-to-configure-apim-for-service-fabric-managed-cluster/apim-api-operation-create.png differ diff --git a/media/how-to-configure-apim-for-service-fabric-managed-cluster/apim-api-operation-test.png b/media/how-to-configure-apim-for-service-fabric-managed-cluster/apim-api-operation-test.png new file mode 100644 index 00000000..c342cc68 Binary files /dev/null and b/media/how-to-configure-apim-for-service-fabric-managed-cluster/apim-api-operation-test.png differ diff --git a/media/how-to-configure-apim-for-service-fabric-managed-cluster/apim-api-operation-trace.png b/media/how-to-configure-apim-for-service-fabric-managed-cluster/apim-api-operation-trace.png new file mode 100644 index 00000000..32dc5378 Binary files /dev/null and b/media/how-to-configure-apim-for-service-fabric-managed-cluster/apim-api-operation-trace.png differ diff --git a/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-1.png b/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-1.png new file mode 100644 index 00000000..00d9c5f8 Binary files /dev/null and b/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-1.png differ diff --git a/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-2.png b/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-2.png new file mode 100644 index 00000000..88e31185 Binary files /dev/null and b/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-2.png differ diff --git a/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-3.png b/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-3.png new file mode 100644 index 00000000..c001a814 Binary files /dev/null and b/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-3.png differ diff --git a/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-4.png b/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-4.png new file mode 100644 index 00000000..22d836df Binary files /dev/null and b/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-4.png differ diff --git a/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-5.png b/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-5.png new file mode 100644 index 00000000..ae79d75c Binary files /dev/null and b/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-5.png differ diff --git a/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-6.png b/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-6.png new file mode 100644 index 00000000..29443eba Binary files /dev/null and b/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-6.png differ diff --git a/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-ps.png b/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-ps.png new file mode 100644 index 00000000..86c87ea3 Binary files /dev/null and b/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-ps.png differ diff --git a/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-sfx.png b/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-sfx.png new file mode 100644 index 00000000..74e0661b Binary files /dev/null and b/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-sfx.png differ diff --git a/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-swagger.png b/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-swagger.png new file mode 100644 index 00000000..e7b9408e Binary files /dev/null and b/media/how-to-configure-apim-for-service-fabric-managed-cluster/vs22-weatherforecast-swagger.png differ