AQE: handle executor failure on the adaptive execution path #1986

Activity

1. andygrove

   added

   enhancementNew feature or requestNew feature or request

   on Jul 10, 2026
2. andygrove commented on Oct 14, 2026

   

   andygrove

   on Oct 14, 2026

   MemberAuthor

   More actions

   Some specifics from reading AdaptiveExecutionGraph::update_task_status side by side with the static graph, at commit ea1126c, that are relevant here:

   • The reset-running-stages path is dead code on the AQE graph. reset_running_stages is declared as an immutable empty map (aqe/mod.rs#L817), so the reset loop at L1106-L1130 never runs and the !reset_running_stages.contains_key(&stage_id) check at L1013 is always true. In the static graph the same map is mut and is populated by the late fetch-failure handling.
   • The late fetch-failure path for UnResolved stages is missing. StaticExecutionGraph::update_task_status handles fetch failures reported for stages that are still UnResolved (execution_graph.rs#L1016-L1088); the AQE copy has no equivalent, which is the main source of the dead code above.
   • The max-task-failures message is less useful on AQE. The AQE copy reports only the internal task slot id (aqe/mod.rs#L942-L948), while the static copy reports the partitions that exceeded the limit (execution_graph.rs#L910-L937).

   These are all consequences of the two update_task_status implementations being separate copies. #2595 tracks sharing the execution-graph code between the static and adaptive graphs, which would let the recovery logic here be implemented once. A related AQE-only bug where a stage is dropped from the graph on a failed conversion is #2594.