compatibility issues between azure-core and azure-cli: cannot import name 'case_insensitive_dict' from 'azure.core.utils'

[harneetvirk]

az feedback auto-generates most of the information requested below, as of CLI version 2.0.62

Related command

Describe the bug
We are noticing compatibility issues between azure-core and azure-cli: cannot import name 'case_insensitive_dict' from 'azure.core.utils'.
Until cli 2.37.0, AZ CLI had the dependency azure-core==1.21.1
Starting 2.38.0, AZ CLI is having dependency azure-core==1.24.0

In case, we install a package that has a direct or indirect dependency on the lower azure-core==1.21.1, the AZ CLI encounters the compatibility issues between azure-core and azure-cli and emits the error: cannot import name 'case_insensitive_dict' from 'azure.core.utils'

To Reproduce
Two ways of reproducing the issue:

1. Install latest az cli
2. Install any package that depends on lower older version of azure-core

or

1. Install az cli (2.37.0 or lower)
2. Install cli ML extension 2.5.0. (ML Extension brings in the latest azure-storage-blob package that depends on latest azure-core=1.24.0).
   az extension add --name ml --version 2.5.0

Expected behavior
CLI should not fail.

[yonzhan]

@jiasli for awareness

[Ikaberry202]

Good job

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @azureml-github.

Issue Details

---

az feedback auto-generates most of the information requested below, as of CLI version 2.0.62

Related command

Describe the bug
We are noticing compatibility issues between azure-core and azure-cli: cannot import name 'case_insensitive_dict' from 'azure.core.utils'.
Until cli 2.37.0, AZ CLI had the dependency azure-core==1.21.1
Starting 2.38.0, AZ CLI is having dependency azure-core==1.24.0

In case, we install a package that has a direct or indirect dependency on the lower azure-core==1.21.1, the AZ CLI encounters the compatibility issues between azure-core and azure-cli and emits the error: cannot import name 'case_insensitive_dict' from 'azure.core.utils'

To Reproduce
Two ways of reproducing the issue:

1. Install latest az cli
2. Install any package that depends on lower older version of azure-core

or

1. Install az cli (2.37.0 or lower)
2. Install cli ML extension 2.5.0. (ML Extension brings in the latest azure-storage-blob package that depends on latest azure-core=1.24.0).
   az extension add --name ml --version 2.5.0

Expected behavior
CLI should not fail.

Author:	harneetvirk
Assignees:	jiasli, jsntcy
Labels:	Service Attention, Machine Learning, Installation, Auto-Assign
Milestone:	Backlog

[jiasli]

The dependency on azure-core was bumped by #22676.

It's ml's responsibility to use latest azure-core.

Latest azure-storage-blob-12.13.0.zip downloaded from https://pypi.org/project/azure-storage-blob/#files has "azure-core<2.0.0,>=1.23.1" in setup.py.

    install_requires=[
        "azure-core<2.0.0,>=1.23.1",
        "msrest>=0.6.21",
        "cryptography>=2.1.4"
    ],

[harneetvirk]

ML has already shipped a mitigation for this issue in the following release 2.6.1 by pinning the azure-storage-blob. But this is one common issue that is being encountered outside the scope of the ML extension.

The failure may be encountered in 2 ways.

1. With the new az cli (2.38.0) wherever the user packages have direct or indirect dependencies on the older version of azure-core.
2. With older az cli (2.37.0 or less), if the user packages have direct or indirect dependencies on the latest azure-core. For e.g., using azure-storage-blob 12.13.0 with az cli 2.37.0 or less.

[sozercan]

We are also seeing this on az cli 2.37.0 and the latest version (0.5.92) of aks-preview extension.

https://dev.azure.com/AzureContainerUpstream/Azure%20Workload%20Identity/_build/results?buildId=52370&view=logs&j=3a3ff92c-2ab0-5cd5-ecad-e283f5e1d802&t=6d5f3379-e684-5896-70a5-701deb97a928

Pinning aks-preview to 0.5.87 resolved it.