Problem
runPlansConcurrent executes the cross-deploy graph level by level with a
barrier between levels (runLevel calls g.Wait() before the next level
starts). A deploy in level N+1 therefore waits for the entire level N to
finish, even when it depends on only one node in that level.
deploy "vm" promises lxc:100 (fast)
deploy "dns" independent (slow)
deploy "app" target inputs lxc:100 (depends only on vm)
Level 0: [vm, dns] ── g.Wait() ──┐
Level 1: [app] ◄── blocked on the whole level, including dns
app only needs lxc:100 from vm, but the level barrier makes it wait
behind dns too. The levels are always correct (no dependency is violated),
just pessimistic on latency: a slow independent deploy in an early tier stalls
dependents that never touch it.
Fix
Convert the deploy runner from level-barrier to dependency-driven: fire each
deploy the moment its own upstream deps settle, not when its whole level does.
app starts when vm finishes; dns runs off on its own and never gates
app.
The graph already carries what's needed: deployNode.deps holds the real
per-node edges (app.deps == [vm]), and kahnLevels is only one consumer of
them. This is the exact pattern the engine already uses one layer down: the op
scheduler (opNode.deps + a pending counter, scheduled on completion) and the
step scheduler. Lift the same shape to the deploy layer.
Must preserve: failure propagation (#275)
The level model also does failure propagation: "an upstream level produced
failures, skip downstream nodes rather than racing them into ops that depend on
resources the failed producer was supposed to create." A DAG runner has to keep
that per edge: if vm fails, app must be skipped/aborted (its producer
never created lxc:100), while dns and anything independent still run. Get
the skip-on-failed-producer semantics right before merging.
Notes
Suggested labels
kind/optimization, impact/medium, priority/low
Problem
runPlansConcurrentexecutes the cross-deploy graph level by level with abarrier between levels (
runLevelcallsg.Wait()before the next levelstarts). A deploy in level N+1 therefore waits for the entire level N to
finish, even when it depends on only one node in that level.
apponly needslxc:100fromvm, but the level barrier makes it waitbehind
dnstoo. The levels are always correct (no dependency is violated),just pessimistic on latency: a slow independent deploy in an early tier stalls
dependents that never touch it.
Fix
Convert the deploy runner from level-barrier to dependency-driven: fire each
deploy the moment its own upstream
depssettle, not when its whole level does.appstarts whenvmfinishes;dnsruns off on its own and never gatesapp.The graph already carries what's needed:
deployNode.depsholds the realper-node edges (
app.deps == [vm]), andkahnLevelsis only one consumer ofthem. This is the exact pattern the engine already uses one layer down: the op
scheduler (
opNode.deps+ apendingcounter, scheduled on completion) and thestep scheduler. Lift the same shape to the deploy layer.
Must preserve: failure propagation (#275)
The level model also does failure propagation: "an upstream level produced
failures, skip downstream nodes rather than racing them into ops that depend on
resources the failed producer was supposed to create." A DAG runner has to keep
that per edge: if
vmfails,appmust be skipped/aborted (its producernever created
lxc:100), whilednsand anything independent still run. Getthe skip-on-failed-producer semantics right before merging.
Notes
longest-path depth (a pure graph property, independent of execution timing),
so the level-major lane ordinals the renderer uses remain stable and correct
after this change.
Suggested labels
kind/optimization,impact/medium,priority/low