[Bug] Can't remove custom CA certs in Azure Kubernetes Service (AKS) in cluster level.

[wayden88]

Is your feature request related to a problem?
According to https://learn.microsoft.com/en-us/azure/aks/custom-certificate-authority, it's possible to enable a custom certificate authority (CA) at the cluster level in AKS like below:

=====
az aks create --resource-group --name --node-count 2 --custom-ca-trust-certificates FileWithCAs --generate-ssh-keys

=====
az aks update --resource-group --name --custom-ca-trust-certificates

However, there is currently no supported method to disable the custom CA certificate once it has been applied at the cluster level.

While disabling is possible at the node pool level using the command az aks nodepool update --disable-custom-ca-trust, this does not apply to cluster-level configurations.

Describe the solution you'd like
The solution we are looking for would be a command similar to:
az aks update --disable-custom-ca-trust-certificates
This would allow users to remove the custom CA trust from the cluster level in a straightforward and supported way.

[sjwaight]

@wayden88 have you tried passing an empty file to the --custom-ca-trust-certificates parameter? Looking at the REST API for AKS I can see the ARM definition below.

    "securityProfile": {
      "customCATrustCertificates": [
        "ZHVtbXlFeGFtcGxlVGVzdFZhbHVlRm9yQ2VydGlmaWNhdGVUb0JlQWRkZWQ="
      ]
    }

I think passing in an empty file (or a file with a blank line?) should work as it should clear the array. I haven't tested though as I don't have a cluster handy with this configuration. Granted if this works it's not a great UX. If it doesn't, we'll need to address.

You can also patch the cluster ARM object as well by using the REST API, including via the Azure CLI (see docs).

Please try and let us know.

[wayden88]

@sjwaight Thanks for looking into this. Below are the test result.

---

az aks create --resource-group test --name test --node-count 2 --custom-ca-trust-certificates /home/waydenlee/testcert/fabrikam.crt --generate-ssh-keys

az aks show -g test -n test | grep securityProfile -A 4

  "securityProfile": {
    "enableSecureBoot": false,
    "enableVtpm": false,
    "sshAccess": "LocalUser"
  },

--
"securityProfile": {
"azureKeyVaultKms": null,
"customCaTrustCertificates": [
"-----BEGIN CERTIFICATE-----\nMIIBxTCCAWsCFDgsYcGQE1kR5oaaDkfh+QfXZkC9MAoGCCqGSM49BAMCMIGEMQsw\nCQYDVQQGEwJVUzELMAkGA1UECAwCT0gxEzARBgNVBAcMCllvdW5nc3Rvd24xCzAJ\nBgNVBAoMAk1TMQswCQYDVQQLDAJEQzEYMBYGA1UEAwwPd3d3LmNvbnRvc28uY29t\nMR8wHQYJKoZIhvcNAQkBFhB1c2VyQGNvbnRvc28uY29tMB4XDTI1MDgxMzEzMTEx\nN1oXDTI2MDgxMzEzMTExN1owRTELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUt\nU3RhdGUxITAfBgNVBAoMGEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDBZMBMGByqG\nSM49AgEGCCqGSM49AwEHA0IABBdVO6rhGD5bKR07QrryKWFt4p6/HbQ8SGiwXW+a\nfFrHhlBSnoV21hVrZCy5z1twekQKPyMqmN6kbJnc+QGWEbUwCgYIKoZIzj0EAwID\nSAAwRQIgKImGUwIpfD+bjhZm50b5KIXcFmV4tSrOUG+0j6DAkr0CIQDFNe/oba+a\nSKEQoYIl9yVyDjSVoyOKY6ci0FJ4713LGA==\n-----END CERTIFICATE-----\n"
],

---

az aks update --resource-group test --name test --custom-ca-trust-certificates ""
-> Nothing has been changed and still custom CA cert existed.

---

touch testca.crt
echo "" > testca.crt
az aks update --resource-group test --name test --custom-ca-trust-certificates /home/waydenlee/testcert/testca.crt
-> Nothing has been changed and still custom CA(fabrikam.crt from the above) cert existed.

az aks show -g test -n test | grep securityProfile -A 4
WARNING: The behavior of this command has been altered by the following extension: aks-preview
"securityProfile": {
"enableSecureBoot": false,
"enableVtpm": false,
"sshAccess": "LocalUser"
},

"securityProfile": {
"azureKeyVaultKms": null,
"customCaTrustCertificates": [
"-----BEGIN CERTIFICATE-----\nMIIBxTCCAWsCFDgsYcGQE1kR5oaaDkfh+QfXZkC9MAoGCCqGSM49BAMCMIGEMQsw\nCQYDVQQGEwJVUzELMAkGA1UECAwCT0gxEzARBgNVBAcMCllvdW5nc3Rvd24xCzAJ\nBgNVBAoMAk1TMQswCQYDVQQLDAJEQzEYMBYGA1UEAwwPd3d3LmNvbnRvc28uY29t\nMR8wHQYJKoZIhvcNAQkBFhB1c2VyQGNvbnRvc28uY29tMB4XDTI1MDgxMzEzMTEx\nN1oXDTI2MDgxMzEzMTExN1owRTELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUt\nU3RhdGUxITAfBgNVBAoMGEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDBZMBMGByqG\nSM49AgEGCCqGSM49AwEHA0IABBdVO6rhGD5bKR07QrryKWFt4p6/HbQ8SGiwXW+a\nfFrHhlBSnoV21hVrZCy5z1twekQKPyMqmN6kbJnc+QGWEbUwCgYIKoZIzj0EAwID\nSAAwRQIgKImGUwIpfD+bjhZm50b5KIXcFmV4tSrOUG+0j6DAkr0CIQDFNe/oba+a\nSKEQoYIl9yVyDjSVoyOKY6ci0FJ4713LGA==\n-----END CERTIFICATE-----\n"

---

az rest --method patch
--url "https://management.azure.com/subscriptions/4f1fac03-9e19-44c4-b121-982f444061bc/resourceGroups/test/providers/Microsoft.ContainerService/managedClusters/test?api-version=2023-11-01"
--body '{"properties": {"securityProfile": {"customCATrustCertificates": []}}}'

-> Nothing has been changed and still custom CA(fabrikam.crt from the above) cert existed.

---

**cat testca.crt
-----BEGIN CERTIFICATE-----

-----END CERTIFICATE-----**

az aks update --resource-group test --name test --custom-ca-trust-certificates /home/waydenlee/testcert/testca.crt
Argument '--custom-ca-trust-certificates' is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
The behavior of this command has been altered by the following extension: aks-preview
(CATrustPreviewCAPEMDecodeFailure) failed to decode one of SecurityProfile.CustomCATrustCertificates to PEM after base64 decoding.
Code: CATrustPreviewCAPEMDecodeFailure
Message: failed to decode one of SecurityProfile.CustomCATrustCertificates to PEM after base64 decoding.

[sjwaight]

@wayden88 OK, this looks like an issue specific to this command. Let's convert this to a bug.

[sjwaight]

Hi @allyford - looking at #2259 it looks like this feature is GA since May, but there doesn't seem to be a way to remove / unset the custom certs you load using the --custom-ca-trust-certificates. Additionally, the Azure CLI shows this argument as still in preview.

Are you able to take a look please?