fix(events): correct the listener invocation, removal, and context release - #2100
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #2100 +/- ##
==========================================
- Coverage 93.62% 93.55% -0.08%
==========================================
Files 181 181
Lines 12644 12656 +12
==========================================
+ Hits 11838 11840 +2
- Misses 806 816 +10
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Base automatically changed from
fix/event-manager-listener-self-await
to
master
August 5, 2026 10:04
vdusek
force-pushed
the
refactor/event-manager-modernization
branch
from
August 5, 2026 10:25
d432297 to
072df83
Compare
vdusek
marked this pull request as ready for review
August 5, 2026 12:03
Mantisus
approved these changes
Aug 5, 2026
Mantisus
left a comment
Collaborator
There was a problem hiding this comment.
let's also update __aexit__ in LocalEventManager. Otherwise LGTM.
B4nan
approved these changes
Aug 6, 2026
vdusek
added a commit
to apify/apify-sdk-python
that referenced
this pull request
Aug 6, 2026
`ApifyEventManager` ignored the re-entrancy contract of Crawlee's `EventManager`, which tracks active contexts with `_active_ref_count`. `BasicCrawler._run_crawler` always enters the *global* event manager, and `Actor.init` registers the `ApifyEventManager` as exactly that and has already entered it — so on the platform every crawler run re-entered it and opened a second websocket. Measured on master, with a local events server and a `BasicCrawler.run()` inside an entered `ApifyEventManager`: - Two connections stay open for as long as the run lasts, so every platform event arrives twice. One `Migrating` became two `Migrating` listener calls and two `PersistState(is_migrating=True)` — state persisted twice per migration. The extra connection also counts against the platform limit of 10 per run. - The crawler's exit tears down that newer connection, which leaves `__aexit__` bookkeeping pointing at an already-closed one. The Actor's own connection and its message-processing task therefore survive `Actor.exit()`, re-breaking the iterator shutdown from #1077. The websocket is now owned by the outermost context only, mirroring `LocalEventManager`. On top of that: - The shutdown moved into `_teardown_platform_websocket()` and the parent context is left in a `finally`, so a failed shutdown can no longer keep the manager active for good — which would mean never emitting `PersistState` again. - Task, connection and future are reset on exit, so the context can be entered again. - A cancelled entry cleans up as well. Without it, a later entry mistook itself for a nested context and silently returned a manager that received no platform events at all. - The error that prevented the first connection is raised as the `__cause__` of the `RuntimeError`, not only logged. - `Actor.reboot()` reads `_listeners_to_wrappers` with `get`, so the lookup no longer inserts entries for events nobody listens to. Adopts the patterns from apify/crawlee-python#2100 (Crawlee 1.9.1). *✍️ Drafted by Claude Code*
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Get rid of
pyeeas we really don't need it and modernize theEventManagerandLocalEventManager. No public API changes.Fixes
__call__is async (a class instance, not a function) was treated as sync and handed toasyncio.to_thread, which returned its coroutine without ever awaiting it - the listener silently never ran.off()decided between "remove this listener" and "remove all listeners of the event" by the truthiness oflistener, so a listener object that is falsy in a boolean context wiped all listeners of the event.off()left an empty entry behind in_listeners_to_wrappers, keeping a reference to a listener that is no longer registered, and created entries for events and listeners that were never registered at all.__aexit__released the context only on its happy path. A cancellation or a failing emission left the manager active for good, and since re-entering an active manager is a no-op,PersistStatewas never emitted again.pyee is gone
emitcreates the listener task itself instead of going throughpyee.asyncio.AsyncIOEventEmitter, which:emit, sowait_for_all_listeners_to_complete()can no longer miss the listeners of a just-emitted event - the one-tick defer that fix(events): prevent deadlock when closing or waiting for listeners from within a listener #2088 had to make explicit is not needed anymore,pyeeis still pulled in byplaywrightfor the browser extras).The
_listeners_to_wrapperslayout is deliberately kept as it is - apify-sdk-python reaches into it inActor.reboot(). Its full unit test suite passes against this branch.The rest
on()into_wrap_listener.__aexit__branches were collapsed into a singlelast_exitflag,__aenter__mirrors it, and the manual "not active" check was replaced by theensure_contextdecorator, so the message matches the one every other method raises.EventManager.on.listener_wrapper(): ...DEBUG lines are gone - they reported that a task is awaited, that it completed, and that it was discarded.LocalEventManagerreads the CPU and the memory info concurrently -get_cpu_infoalone blocks its thread for 100 ms while sampling the CPU.__aenter__ -> Selfandset[asyncio.Task[None]].wait_for_all_tasks_for_finishrenamed towait_for_all_tasks_to_finish.EventManager, and the attributes of both managers are documented with docstrings instead of comments.New tests cover every fix above, the synchronous task registration in
emit, the nested context teardown, and the concurrent system info readings.✍️ Drafted by Claude Code