Skip to content

Consume MQTT ack futures on abandonment to prevent asyncio warnings#98

Merged
eman merged 2 commits into
mainfrom
bugfix/issue-97-consume-mqtt-ack-futures
Jul 6, 2026
Merged

Consume MQTT ack futures on abandonment to prevent asyncio warnings#98
eman merged 2 commits into
mainfrom
bugfix/issue-97-consume-mqtt-ack-futures

Conversation

@eman

@eman eman commented Jul 6, 2026

Copy link
Copy Markdown
Owner

Fixes #97

Problem

_await_ack() in mqtt/connection.py (used by connect/disconnect/close/subscribe/unsubscribe/publish) and the inline acknowledgement waits in mqtt/subscriptions.py's subscribe()/unsubscribe() shield the AWS CRT future so a timeout or task cancellation never propagates into the SDK future — the underlying operation keeps running in the background.

If that shielded future later completes with an exception (notably AwsCrtError / AWS_ERROR_MQTT_CANCELLED_FOR_CLEAN_SESSION during a reconnect race) after the awaiting task has already given up, nobody ever retrieves the exception. asyncio then logs an unhandled Future exception was never retrieved / exception in shielded future warning at garbage-collection time, which is exactly the noise that the Home Assistant integration had to suppress with a global loop exception handler.

Fix

Attach a done_callback to the shielded future whenever a wait is abandoned (cancelled or timed out) so its eventual result/exception is always retrieved. Benign errors are logged at debug level instead of leaking as an unhandled asyncio warning.

  • mqtt/connection.py: new _consume_abandoned_future() helper, wired into both the CancelledError and TimeoutError branches of _await_ack().
  • mqtt/subscriptions.py: same pattern applied to the inline subscribe()/unsubscribe() waits (also added the previously-missing TimeoutError handling/logging there).

Testing

  • Added TestAbandonedAckFutureConsumed regression tests in tests/test_mqtt_reliability.py covering both the timeout and cancellation paths. Verified these tests fail (asyncio logs "exception in shielded future") without the fix and pass with it.
  • ruff check/ruff format --check: pass
  • mypy src/nwp500: no issues
  • pytest: 611/611 passed

Fixes #97. _await_ack() (mqtt/connection.py) and the inline
subscribe/unsubscribe waits (mqtt/subscriptions.py) shield the AWS
CRT acknowledgement future so a timeout/cancellation never propagates
into the SDK future. If that shielded future later completed with an
exception (e.g. AwsCrtError from clean-session cancellation during
reconnect) after the caller had already given up, nobody retrieved the
exception and asyncio logged an unhandled 'Future exception was never
retrieved' / 'exception in shielded future' warning at
garbage-collection time.

Attach a done callback whenever a wait is abandoned (cancelled or
timed out) so the eventual result/exception is retrieved and logged
at debug level, instead of leaking as an asyncio-level warning that
integrations had to suppress globally.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR prevents noisy asyncio warnings caused by MQTT acknowledgement futures that outlive their awaiting task (timeout/cancellation), by ensuring the eventual result/exception is still retrieved and logged instead of being left for GC to report as “Future exception was never retrieved”.

Changes:

  • Add _consume_abandoned_future() and wire it into _await_ack() cancellation/timeout paths to consume late exceptions.
  • Apply the same “consume abandoned ack future” pattern to subscribe()/unsubscribe() acknowledgement waits (and add missing timeout handling there).
  • Add regression tests covering both timeout and cancellation abandonment scenarios, and document the fix in the changelog.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File Description
src/nwp500/mqtt/connection.py Adds consumption of late ack future exceptions when _await_ack() is abandoned.
src/nwp500/mqtt/subscriptions.py Extends the abandonment-consumption pattern to subscribe/unsubscribe ack waits and adds timeout handling.
tests/test_mqtt_reliability.py Adds regression tests to ensure abandoned ack futures don’t leak asyncio warnings.
CHANGELOG.rst Documents the fix under Unreleased.

Comment thread src/nwp500/mqtt/subscriptions.py
Comment thread src/nwp500/mqtt/subscriptions.py
Comment thread src/nwp500/mqtt/subscriptions.py
Comment thread src/nwp500/mqtt/subscriptions.py
Use functools.partial with the static _consume_abandoned_future
helper referenced via the class, rather than a lambda closing over
self, so an abandoned subscribe/unsubscribe ack future doesn't keep
the MqttSubscriptionManager instance alive until it eventually
completes.
@eman
eman merged commit 52af247 into main Jul 6, 2026
7 checks passed
@eman
eman deleted the bugfix/issue-97-consume-mqtt-ack-futures branch July 6, 2026 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Consume MQTT ack futures so clean-session cancellations don't leak asyncio warnings

2 participants