Skip to content

fix: skip IMDS credential discovery in _make_cognito_client#11

Merged
eman merged 1 commit into
mainfrom
bugfix/imds-hang-on-cognito-refresh
Jul 1, 2026
Merged

fix: skip IMDS credential discovery in _make_cognito_client#11
eman merged 1 commit into
mainfrom
bugfix/imds-hang-on-cognito-refresh

Conversation

@eman

@eman eman commented Jul 1, 2026

Copy link
Copy Markdown
Owner

Problem

On non-EC2 hosts (notably Home Assistant Yellow running HAOS), the EC2 instance metadata endpoint at 169.254.169.254 accepts TCP connections but never responds. When the cached Cognito IdToken expires, the auth refresh path calls _make_cognito_client() in a thread via run_in_executor. Boto3 runs its full credential-provider chain — including an IMDS lookup — which hangs for the OS-level TCP timeout, far exceeding the integration's 20-second asyncio_setup budget.

Result: integration enters setup_retry with "Timed out fetching initial Quilt snapshot" on every HA restart after token expiry, even though the Quilt cloud is reachable.

Fix

  • Set metadata_service_num_attempts=0 on the botocore session to skip IMDS entirely.
  • Add connect_timeout=5, read_timeout=15 to bound the Cognito API calls.

Cognito user-pool flows (CUSTOM_AUTH, REFRESH_TOKEN_AUTH) don't require AWS IAM credentials for public clients, so disabling IMDS is safe.

Deployment

After merge, push tag v0.5.2 to trigger the release workflow and PyPI publish. Then update quilt-hp-python>=0.5.2 in the HA integration manifest.

On non-EC2 hosts (e.g., Home Assistant Yellow), the EC2 instance
metadata endpoint at 169.254.169.254 can accept TCP connections but
never send a response. When the Cognito IdToken expires and a refresh
is needed, boto3 runs the full credential-provider chain — including
an IMDS lookup — inside run_in_executor. The hanging thread blocks for
the full OS-level TCP timeout, which exceeds the integration's 20-second
async_setup budget and causes a spurious ConfigEntryNotReady.

Cognito user-pool auth flows (CUSTOM_AUTH, REFRESH_TOKEN_AUTH) don't
require AWS IAM credentials for public clients, so disabling IMDS via
metadata_service_num_attempts=0 is safe. Also adds explicit connect and
read timeouts to bound the Cognito API calls themselves.
@eman
eman merged commit 86708b5 into main Jul 1, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant