Data Protection: Azure modes, certificate rotation, and comprehensive tests

Directory.Packages.props

@@ -14,8 +14,8 @@
     <PackageVersion Include="AspNetCore.HealthChecks.MongoDb" Version="9.0.0" />
     <PackageVersion Include="AspNetCore.HealthChecks.UI.Client" Version="9.0.0" />
     <PackageVersion Include="AspNetCore.Identity.MongoDbCore" Version="7.0.0" />
-    <PackageVersion Include="Azure.Extensions.AspNetCore.DataProtection.Blobs" Version="1.5.1" />
-    <PackageVersion Include="Azure.Extensions.AspNetCore.DataProtection.Keys" Version="1.6.1" />
+    <PackageVersion Include="Azure.Extensions.AspNetCore.DataProtection.Blobs" Version="1.5.2" />
+    <PackageVersion Include="Azure.Extensions.AspNetCore.DataProtection.Keys" Version="1.6.2" />
     <PackageVersion Include="Azure.Identity" Version="1.21.0" />
     <PackageVersion Include="Azure.Storage.Blobs" Version="12.27.0" />
     <PackageVersion Include="Microsoft.AspNetCore.DataProtection.StackExchangeRedis" Version="10.0.7" />
@@ -47,7 +47,9 @@
     <PackageVersion Include="OpenTelemetry.Instrumentation.AspNetCore" Version="1.15.2" />
     <PackageVersion Include="OpenTelemetry.Instrumentation.Http" Version="1.15.1" />
     <PackageVersion Include="OpenTelemetry.Instrumentation.Runtime" Version="1.15.1" />
-    <PackageVersion Include="Scalar.AspNetCore" Version="2.14.4" />
+    <PackageVersion Include="Scalar.AspNetCore" Version="2.14.11" />
+    <!-- Pin transitive Snappier (pulled in by MongoDB.Driver) to 1.3.1+; 1.0.0 has GHSA-pggp-6c3x-2xmx. -->
+    <PackageVersion Include="Snappier" Version="1.3.1" />
     <PackageVersion Include="System.CommandLine" Version="2.0.7" />
     <PackageVersion Include="Spectre.Console" Version="0.55.2" />
     <PackageVersion Include="Terminal.Gui" Version="2.0.0-develop.5213" />
@@ -58,6 +60,7 @@
     <PackageVersion Include="NSubstitute" Version="5.3.0" />
     <PackageVersion Include="Shouldly" Version="4.3.0" />
     <PackageVersion Include="Testcontainers.MongoDb" Version="4.11.0" />
+    <PackageVersion Include="Testcontainers.Redis" Version="4.11.0" />
     <PackageVersion Include="Verify.SourceGenerators" Version="2.5.0" />
     <PackageVersion Include="Verify.XunitV3" Version="31.16.2" />
     <PackageVersion Include="xunit.v3" Version="3.2.2" />

docs/design-docs/Deployment-Architecture.md

@@ -180,9 +180,11 @@ environment:
   - DataProtection__Mode=Azure
   - DataProtection__Azure__BlobStorageUri=https://<account>.blob.core.windows.net/data-protection/keys.xml
   - DataProtection__Azure__KeyVaultKeyUri=https://<vault>.vault.azure.net/keys/groundcontrol-dp
+  - DataProtection__AzureCredential__Mode=ManagedIdentity
+  - DataProtection__AzureCredential__ClientId=<user-assigned-mi-client-id>  # omit for system-assigned MI
 ```
 
-Azure Key Vault handles key ring protection natively — no manual certificate management is needed. Use `DefaultAzureCredential` (Managed Identity in AKS) for authentication to both Blob Storage and Key Vault. All regions share the same blob container and Key Vault key, ensuring a consistent key ring.
+Azure Key Vault handles key ring protection natively — no manual certificate management is needed. The credential type is configured via `DataProtection:AzureCredential:Mode` and applies to both Blob Storage and Key Vault. Supported modes: `Default` (chained `DefaultAzureCredential`, the recommended starting point), `ManagedIdentity` (system- or user-assigned), `WorkloadIdentity` (AKS workload identity), `ClientSecret`, `AzureCli` (local development), and `Environment`. All regions share the same blob container and Key Vault key, ensuring a consistent key ring.
 
 ---
 
@@ -330,18 +332,18 @@ Application configuration for GroundControl server:
 | `DataProtection:Mode` | `FileSystem` | Key ring configurator: `FileSystem`, `Certificate`, `Redis`, `Azure` |
 | `DataProtection:KeyStorePath` | `./keys` | File system path for key ring storage (`FileSystem`, `Certificate`) |
 | `DataProtection:UseDpapi` | `false` | Use DPAPI for key protection (`FileSystem` on Windows only) |
-| `DataProtection:CertificateProvider` | `FileSystem` | Certificate provider: `FileSystem`, `AzureBlob` |
-| `DataProtection:Certificate:FileSystem:CurrentPath` | — | Current X.509 certificate path (`.pfx`) |
-| `DataProtection:Certificate:FileSystem:Password` | — | Certificate password (prefer environment variable or secrets manager) |
-| `DataProtection:Certificate:FileSystem:PreviousPaths` | `[]` | Previous certificate paths for decrypting old keys during rotation |
-| `DataProtection:Certificate:AzureBlob:ContainerUri` | — | Azure Blob Storage container URI for certificates |
-| `DataProtection:Certificate:AzureBlob:CurrentBlobName` | — | Blob name of the current certificate |
-| `DataProtection:Certificate:AzureBlob:PreviousBlobNames` | `[]` | Blob names of previous certificates |
-| `DataProtection:Certificate:AzureBlob:Password` | — | Certificate password (shared across blobs) |
-| `DataProtection:Redis:ConnectionString` | — | Redis connection string (`Redis` configurator) |
-| `DataProtection:Redis:KeyName` | `GroundControl-DP-Keys` | Redis key name for key ring storage |
-| `DataProtection:Azure:BlobStorageUri` | — | Azure Blob Storage URI for key ring storage |
-| `DataProtection:Azure:KeyVaultKeyUri` | — | Azure Key Vault key URI for key ring protection |
+| `DataProtection:CertificateProvider` | — | Certificate provider: `FileSystem` or `AzureBlob` (required for `Certificate` and `Redis` modes) |
+| `DataProtection:FileSystemCertificate:Path` | — | Path to the current `.pfx` certificate (`FileSystem` provider) |
+| `DataProtection:FileSystemCertificate:Password` | — | Password for the current and previous file-system certificates (prefer environment variable or secrets manager) |
+| `DataProtection:FileSystemCertificate:PreviousPaths` | `[]` | Optional array of `.pfx` paths for previously-used certificates, kept available for decrypting key XML written under the prior cert until rotation completes |
+| `DataProtection:AzureBlobCertificate:BlobUri` | — | Azure Blob URI for the current certificate (`AzureBlob` provider) |
+| `DataProtection:AzureBlobCertificate:Password` | — | Password for the current and previous Azure Blob certificates (prefer environment variable or secrets manager) |
+| `DataProtection:AzureBlobCertificate:PreviousBlobUris` | `[]` | Optional array of blob URIs for previously-used certificates (`AzureBlob` provider). Same semantics as `FileSystemCertificate:PreviousPaths` |
+| `DataProtection:Redis:ConnectionString` | — | Redis connection string (`Redis` mode) |
+| `DataProtection:Redis:KeyName` | `groundcontrol-data-protection` | Redis key name for key ring storage |
+| `DataProtection:Redis:ConnectTimeoutMs` | `5000` | Redis connection timeout |
+| `DataProtection:Azure:BlobUri` | — | Azure Blob Storage URI for key ring storage (`Azure` mode) |
+| `DataProtection:Azure:KeyVaultKeyId` | — | Azure Key Vault key URI for key ring protection (`Azure` mode) |
 | `DataProtection:KeyRotation:Enabled` | `false` | Enable automatic key rotation |
 | `DataProtection:KeyRotation:KeyLifetime` | `90` | Days before a new key is generated (when rotation is enabled) |
 | `Cache:PrewarmOnStartup` | `false` | Load all active snapshots into cache at startup (trades memory for guaranteed first-request cache hits) |

docs/design-docs/Security-Model.md

@@ -205,7 +205,7 @@ IDataProtectionCertificateProvider
 └── GetPreviousCertificatesAsync() → IReadOnlyList<X509Certificate2>
 ```
 
-`GetCurrentCertificateAsync()` returns the active certificate used by `ProtectKeysWithCertificate()` to encrypt new key ring entries. `GetPreviousCertificatesAsync()` returns any retired certificates passed to `UnprotectKeysWithAnyCertificate()` for decrypting old key ring entries during rotation. Selected via `DataProtection:Certificate:Provider`.
+`GetCurrentCertificateAsync()` returns the active certificate used by `CertificateXmlEncryptor` to encrypt new key ring entries. `GetPreviousCertificatesAsync()` returns any retired certificates that, together with the current certificate, are passed to `UnprotectKeysWithAnyCertificate()` so old key ring entries remain decryptable during and after rotation. Selected via `DataProtection:CertificateProvider`.
 
 **Implementations:**
 
@@ -222,13 +222,12 @@ IDataProtectionCertificateProvider
 ```json
 {
   "DataProtection": {
-    "Certificate": {
-      "Provider": "FileSystem",
-      "FileSystem": {
-        "CurrentPath": "/certs/dp-2026.pfx",
-        "Password": "...",
-        "PreviousPaths": ["/certs/dp-2024.pfx"]
-      }
+    "Mode": "Certificate",
+    "CertificateProvider": "FileSystem",
+    "FileSystemCertificate": {
+      "Path": "/certs/dp-2026.pfx",
+      "Password": "...",
+      "PreviousPaths": [ "/certs/dp-2024.pfx" ]
     }
   }
 }
@@ -239,20 +238,21 @@ IDataProtectionCertificateProvider
 ```json
 {
   "DataProtection": {
-    "Certificate": {
-      "Provider": "AzureBlob",
-      "AzureBlob": {
-        "ContainerUri": "https://account.blob.core.windows.net/certificates",
-        "CurrentBlobName": "dp-2026.pfx",
-        "PreviousBlobNames": ["dp-2024.pfx"],
-        "Password": "..."
-      }
+    "Mode": "Certificate",
+    "CertificateProvider": "AzureBlob",
+    "AzureBlobCertificate": {
+      "BlobUri": "https://account.blob.core.windows.net/certificates/dp-2026.pfx",
+      "Password": "...",
+      "PreviousBlobUris": [ "https://account.blob.core.windows.net/certificates/dp-2024.pfx" ]
+    },
+    "AzureCredential": {
+      "Mode": "ManagedIdentity"
     }
   }
 }
 ```
 
-The `AzureBlobCertificateProvider` uses `DefaultAzureCredential` for authentication to the storage account.
+The `AzureBlobCertificateProvider` authenticates to the storage account using the credential configured under `DataProtection:AzureCredential` (the same credential used by `DataProtection:Mode = Azure`). Supported modes are `Default`, `ManagedIdentity`, `WorkloadIdentity`, `ClientSecret`, `AzureCli`, and `Environment`; see `AzureCredentialOptions` for per-mode required fields. `AzureBlobCertificate:PreviousBlobUris` is the AzureBlob equivalent of `FileSystemCertificate:PreviousPaths`; certificates downloaded from those URIs are added to the decryption pipeline so key XML written under a previous certificate remains decryptable during and after rotation.
 
 ### Certificate Lifecycle (Key Ring Protection)
 
@@ -277,26 +277,24 @@ However, there is a practical caveat: if configured via **thumbprint** (`Protect
 
 1. Generate a new X.509 certificate.
 2. Make the new certificate available to the `IDataProtectionCertificateProvider` (e.g., deploy the file, upload to blob storage).
-3. Update configuration: register the new certificate as current and move the old certificate to the previous certificates list (provider-specific config).
-4. Perform a rolling restart. The key ring configurator calls `IDataProtectionCertificateProvider` to obtain certificates, then:
-   - `ProtectKeysWithCertificate(currentCert)` — new data protection keys are encrypted with the new certificate.
-   - `UnprotectKeysWithAnyCertificate(previousCerts)` — existing keys encrypted with old certificates remain decryptable.
-5. After all old data protection keys have expired (90+ days) or been re-encrypted, remove the old certificate from the provider's previous certificates configuration.
+3. Update configuration: set the new certificate as `DataProtection:FileSystemCertificate:Path` (or `DataProtection:AzureBlobCertificate:BlobUri`) and add the old certificate to `DataProtection:FileSystemCertificate:PreviousPaths` (or `DataProtection:AzureBlobCertificate:PreviousBlobUris`).
+4. Perform a rolling restart. At startup `DataProtectionModule` resolves the provider, loads the current and previous certificates, and:
+   - Sets `KeyManagementOptions.XmlEncryptor = new CertificateXmlEncryptor(current)` — new data protection keys are encrypted with the new certificate.
+   - Calls `dataProtectionBuilder.UnprotectKeysWithAnyCertificate(current, ...previous)` — every existing key encrypted under the current or any previous certificate remains decryptable.
+5. After all data protection keys protected by the old certificate have expired (90+ days) or been re-encrypted, remove that path from `PreviousPaths`/`PreviousBlobUris`.
 
 **Configuration example (certificate rotation in progress, FileSystem provider):**
 
 ```json
 {
   "DataProtection": {
-    "KeyRing": "Certificate",
-    "KeyStorePath": "./keys",
-    "Certificate": {
-      "Provider": "FileSystem",
-      "FileSystem": {
-        "CurrentPath": "/certs/dp-2026.pfx",
-        "Password": "...",
-        "PreviousPaths": ["/certs/dp-2024.pfx"]
-      }
+    "Mode": "Certificate",
+    "KeyStorePath": "/keys",
+    "CertificateProvider": "FileSystem",
+    "FileSystemCertificate": {
+      "Path": "/certs/dp-2026.pfx",
+      "Password": "...",
+      "PreviousPaths": [ "/certs/dp-2024.pfx" ]
     },
     "KeyRotation": {
       "Enabled": true,

docs/guide/server/configuration.md

@@ -62,14 +62,34 @@ Keys are stored as XML files in `KeyStorePath`. Suitable for single-instance dep
 
 ### Certificate mode
 
-Keys are stored on the file system and protected with an X.509 certificate.
+Keys are stored on the file system and the key XML is encrypted at rest with an X.509 certificate. The same configuration applies under Redis mode (which uses Redis storage but the same certificate-based key protection).
 
 | Setting | Description |
 |---|---|
-| `DataProtection:CertificateProvider` | `FileSystem` or `AzureBlob` |
-| `DataProtection:CertificatePath` | Path to the .pfx certificate file (FileSystem provider) |
-| `DataProtection:CertificatePassword` | Certificate password (FileSystem provider) |
-| `DataProtection:CertificateAzureBlobUrl` | Blob URL for certificate download (AzureBlob provider) |
+| `DataProtection:CertificateProvider` | `FileSystem` or `AzureBlob`. |
+| `DataProtection:FileSystemCertificate:Path` | Path to the current `.pfx` certificate file (`FileSystem` provider). |
+| `DataProtection:FileSystemCertificate:Password` | Certificate password used for the current and previous file-system certificates. |
+| `DataProtection:FileSystemCertificate:PreviousPaths` | Optional array of `.pfx` paths for certificates that previously protected the key ring (`FileSystem` provider). Required during certificate rotation so existing key XML remains decryptable until rotated out. |
+| `DataProtection:AzureBlobCertificate:BlobUri` | Blob URI for the current certificate (`AzureBlob` provider). |
+| `DataProtection:AzureBlobCertificate:Password` | Certificate password used for the current and previous Azure Blob certificates. |
+| `DataProtection:AzureBlobCertificate:PreviousBlobUris` | Optional array of blob URIs for previously-used certificates (`AzureBlob` provider). Same semantics as `FileSystemCertificate:PreviousPaths`. |
+
+```json
+{
+  "DataProtection": {
+    "Mode": "Certificate",
+    "KeyStorePath": "/keys",
+    "CertificateProvider": "FileSystem",
+    "FileSystemCertificate": {
+      "Path": "/certs/dp-current.pfx",
+      "Password": "<from-secret-store>",
+      "PreviousPaths": [ "/certs/dp-previous.pfx" ]
+    }
+  }
+}
+```
+
+> **Certificate rotation:** generate the new cert, deploy it as `FileSystemCertificate:Path`, move the old cert into `FileSystemCertificate:PreviousPaths`, and perform a rolling restart. New key ring entries are encrypted with the new cert; entries written under the previous cert remain decryptable as long as that cert stays in the previous list. Remove a cert from `PreviousPaths` only after every key encrypted with it has expired (90+ days by default) or been re-encrypted — otherwise the data those keys protect becomes permanently unreadable.
 
 ### Redis mode
 
@@ -81,7 +101,7 @@ Keys are stored in Redis and protected with an X.509 certificate. Suitable for m
 | `DataProtection:Redis:KeyName` | `groundcontrol-data-protection` | Redis key name for the key ring. |
 | `DataProtection:Redis:ConnectTimeoutMs` | `5000` | Connection timeout in milliseconds. |
 
-Also requires a certificate provider (`CertificateProvider`, `CertificatePath`/`CertificateAzureBlobUrl`).
+Also requires the same certificate settings as Certificate mode (`CertificateProvider`, `FileSystemCertificate:Path` / `AzureBlobCertificate:BlobUri`, optional previous-certificate paths/URIs).
 
 ### Azure mode
 
@@ -158,8 +178,8 @@ export Authentication__Seed__AdminPassword="YourSecurePassword123!"
 export DataProtection__Mode="Redis"
 export DataProtection__Redis__ConnectionString="redis:6379"
 export DataProtection__CertificateProvider="FileSystem"
-export DataProtection__CertificatePath="/certs/dp.pfx"
-export DataProtection__CertificatePassword="certpass"
+export DataProtection__FileSystemCertificate__Path="/certs/dp.pfx"
+export DataProtection__FileSystemCertificate__Password="certpass"
 
 # Change notification
 export ChangeNotifier__Mode="MongoChangeStream"