-
Notifications
You must be signed in to change notification settings - Fork 721
Description
Describe the bug
The azdls backend does not load Azure credentials from environment variables, unlike the azblob backend. Specifically, it's not calling AzureStorageConfig::default().from_env(), which means Azure Workload Identity environment variables are never loaded.
This causes authentication to fail in environments like Azure Kubernetes Service (AKS) with Workload Identity enabled, where credentials are provided via environment variables rather than explicit configuration.
Steps to Reproduce
- Deploy an application using the
azdlsbackend in AKS with Workload Identity enabled - Set up the following environment variables (automatically injected by Azure Workload Identity):
AZURE_FEDERATED_TOKEN_FILEAZURE_CLIENT_IDAZURE_TENANT_IDAZURE_AUTHORITY_HOST
- Try to access Azure Data Lake Storage Gen2 without explicitly configuring credentials in code
- Authentication fails and falls back to IMDS, which doesn't work in AKS
Expected Behavior
The azdls backend should automatically load Azure credentials from environment variables, just like the azblob backend does (as implemented in PR #4705).
Additional Context
PR #4705 fixed this issue for the azblob backend to close #4704, but the same fix was not applied to the azdls backend. The two backends should have consistent behavior for credential loading.
Are you willing to submit a PR to fix this bug?
- Yes, I would like to submit a PR.