Help to clarify the ARM token cache for managed identity login

[northtyphoon]

Related command
az login --identity
any az command

Describe the bug
I run az commands in Azure VM with managed identity enabled. I log into cli using "az login --identity".

After login, I noticed all az commands always tried to connect "http://169.254.169.254/metadata/identity/oauth2/token" to get ARM token when the corresponding code tried to query ARM resource. I'd expect CLI could cache the ARM token and tried to reuse it if possible. I shared one debug output as below. As you can also see, after the token request, it mentioned "MSI: token is found in cache". If the token could be found, why the code didn't check it at first place?

cli.azure.cli.core.commands.client_factory: Getting management service client client_type=ComputeManagementClient
urllib3.connectionpool: Starting new HTTP connection (1): 169.254.169.254:80
urllib3.connectionpool: http://169.254.169.254:80 "GET /metadata/identity/oauth2/token?resource=https%3A%2F%2Fmanagement.core.windows.net%2F&api-version=2018-02-01 HTTP/1.1" 200 1739
msrestazure.azure_active_directory: MSI: Retrieving a token from http://169.254.169.254/metadata/identity/oauth2/token, with payload {'resource': 'https://management.core.windows.net/', 'api-version': '2018-02-01'}
msrestazure.azure_active_directory: MSI: Token retrieved
cli.azure.cli.core.auth.adal_authentication: MSIAuthenticationWrapper.get_token: scopes=('https://management.core.windows.net//.default',), kwargs={}
msrestazure.azure_active_directory: MSI: token is found in cache.
cli.azure.cli.core.auth.adal_authentication: Normalize expires_on: '1663545077' -> 1663545077

To Reproduce

1. set up an Azure VM with manage identity enabled
2. install the latest azure-cli
3. az login --identity
4. run any az command to get the resources a few times like

az vmss list --debug
az storage list --debug
az acr list --debug

Expected behavior
ARM token is cached and reused if not expired.

Environment summary
CLI: 2.40.0
Azure VM with manage identity enabled

[jiasli]

msrestazure.azure_active_directory: MSI: token is found in cache.

This line is actually from https://github.com/Azure/msrestazure-for-python/blob/8849f398b6ebd4607de63c2f5d1318f44ec1d822/msrestazure/azure_active_directory.py#L644

    def get_token(self, resource):
        ...
                _LOGGER.debug("MSI: token is found in cache.")
                return token_entry

_ImdsTokenProvider.get_token was called by MSIAuthentication.set_token.

https://github.com/Azure/msrestazure-for-python/blob/8849f398b6ebd4607de63c2f5d1318f44ec1d822/msrestazure/azure_active_directory.py#L600

    def set_token(self):
        ...
            token_entry = self._vm_msi.get_token(self.resource)

MSIAuthentication.set_token was first called by

https://github.com/Azure/msrestazure-for-python/blob/8849f398b6ebd4607de63c2f5d1318f44ec1d822/msrestazure/azure_active_directory.py#L592

    def __init__(self, port=50342, **kwargs):
        ...
        self.set_token()

After MSIAuthentication is instantiated, Azure CLI also calls MSIAuthentication.set_token at

azure-cli/src/azure-cli-core/azure/cli/core/auth/adal_authentication.py

Line 46 in 0fb75af

self.set_token()

leading to the token is found in cache message.

We can easily see that this cache is only an in-memory cache:

https://github.com/Azure/msrestazure-for-python/blob/8849f398b6ebd4607de63c2f5d1318f44ec1d822/msrestazure/azure_active_directory.py#L632

    def __init__(self, msi_conf=None, timeout=None):
        ...
        self.cache = {}

Each Azure CLI command is run in a separate process, so each Azure CLI command can't share the same in-memory cache. The only way to share cache between processes is to persist the cache to disk.

http://169.254.169.254/metadata/identity/oauth2/token is a special endpoint which is very quick. No outbound internet connection is made.

Also, Managed identity endpoint http://169.254.169.254/metadata/identity/oauth2/token has its own caching mechanism, so developing another cache to persist the access token on disk is neither necessary nor secure.