Skip to content

feat(delegate): plan-then-dispatch guidance — batch independent delegations in one turn - #320

Merged
Brian Krabach (bkrabach) merged 1 commit into
mainfrom
feat/parallel-delegation-guidance
Aug 27, 2026
Merged

feat(delegate): plan-then-dispatch guidance — batch independent delegations in one turn#320
Brian Krabach (bkrabach) merged 1 commit into
mainfrom
feat/parallel-delegation-guidance

Conversation

@bkrabach

Copy link
Copy Markdown
Collaborator

Summary

Stage 1 ("Option A", guidance-only) of the approved pg1-parallel-delegation spec. Text-only guidance edits — no code, schema, or behavior changes.

Measured evidence

A measured eval showed root sessions delegating in 3–4 sequential waves, idle 91–94% of wall time, with ~57% of wall structurally recoverable by batching independent delegations into a single turn. One eval run's wave 2 was [explorer, explorer, git-ops] — none of which consumed wave 1's output. The root had all the information needed to batch those into wave 1 and did not.

Why guidance, not machinery

The parallel-dispatch machinery already works end-to-end and needed no changes:

  • The orchestrator mints one parallel_group_id per assistant turn and runs every tool call from that turn concurrently via asyncio.gather (amplifier_module_loop_streaming).
  • The OpenAI provider already sets parallel_tool_calls=True by default; Anthropic never disables it.
  • Concurrent delegates don't race on dispatch context (task-keyed, not session-keyed).

There is no mechanical serialization to remove. A root that emits three delegate calls in one turn already gets three concurrent sub-sessions. The measured idle time is the machinery faithfully executing a one-delegation-per-turn plan — a planning-policy defect, not an execution-machinery defect. Per KERNEL_PHILOSOPHY.md §1, policy belongs at the edges (instructions, tool descriptions), not in new kernel-adjacent code.

Every existing mention of parallelism in this repo argued quality ("different agents bring different tools") — none argued wall-clock, and none told the model to enumerate its full delegation set before dispatching. This PR fixes that gap directly at its source.

A companion approach (a delegations[] array with explicit dependency edges) was evaluated and rejected in the spec: it asks the model for exactly the judgment whose absence is the defect, re-implements asyncio.gather one layer below where it already exists correctly, and its marginal wall-clock gain over a correctly-batched guidance fix is under 1%.

Changes (3 verbatim text replacements, +175 tokens/request budgeted)

  • modules/tool-delegate/amplifier_module_tool_delegate/__init__.py — replaced the delegate tool's "Agent usage notes" block (re-rendered into every request's tool spec, last thing read before a delegate call) with explicit batching guidance and an independence heuristic.
  • context/agents/multi-agent-patterns.md — replaced "## Parallel Agent Dispatch" (framed only as "richer results") with "## Plan the Whole Set, Then Dispatch", adding the wall-clock framing and an explicit independence test.
  • context/agents/delegation-instructions.md — replaced "## Scaling with Multiple Instances" (same-agent only) with "## Wave Discipline: Batch Everything Independent", covering both same-agent and cross-agent batching.

Each edit replaces its section rather than appending, per ISSUE_HANDLING.md's guidance on not growing files without cause. No file grows a section count.

Verification

  • uv run pytest tests/ -q --tb=short (the exact CI command): 1634 passed, 1 skipped
  • modules/tool-delegate/tests/ (62 tests, editable-installed locally to resolve the package): 62 passed — the existing description/schema tests are unaffected by the wording change
  • python_check on the touched .py file: clean except 8 pre-existing warnings elsewhere in the file (confirmed present before this change via git stash), none introduced by this edit
  • Confirmed no return + { substring was introduced — amplifier_foundation.bundle_docs.tool_schema._extract_input_schema still successfully ast.literal_eval-extracts the tool's static input schema after the edit
  • Grepped tests/ and modules/*/tests/ repo-wide for the replaced phrases ("Launch multiple agents concurrently", "Parallel Agent Dispatch", "Scaling with Multiple Instances", etc.) — no test asserts the old wording, so no test updates were needed

Deferred (not in this PR)

Stage 2 (hooks-delegation-batching, a nudge hook that reinforces this guidance at the actual decision point — after wave N's results land, before wave N+1 is planned) is fully specified in the spec but not implemented here. Per the spec's rollout plan, it ships only if Stage 1's DTU measurement (median wall-time reduction) misses the ≥40% target. If Stage 1 alone hits target, Stage 2 never ships.

B′ (a flat delegations[] array, no dependency edges) is kept on the shelf as a contingency, gated on a specific signature (a backend whose tool-call parser structurally emits at most one call per turn) that has not been observed. Not scheduled.

Post-merge note

After merge, amplifier reset --remove cache -y is required before the new guidance is loaded by a running Amplifier process — foundation context files are read from the bundle cache, and the guidance is prose read at composition time, not code.

🤖 Generated with Amplifier

…ations in one turn

Measured eval runs showed roots delegating in 3-4 sequential waves, idle
91-94% of wall time, with ~57% of wall structurally recoverable by
batching independent delegations into a single turn. The machinery
already parallelizes multiple delegate calls emitted in one assistant
turn (asyncio.gather, one parallel_group_id per turn) -- the defect is
the root model's one-delegation-wave-per-turn planning policy, not
missing machinery.

This is Stage 1 ("Option A", guidance-only) of the approved
pg1-parallel-delegation spec. Every existing mention of parallelism
argued quality ("different agents bring different tools") but never
argued wall-clock, and never told the model to enumerate its full
delegation set before dispatching. This change replaces the weak/buried
guidance with instructions that state the wall-clock cost directly and
give the model an explicit independence test.

Three verbatim text replacements, no code/schema/behavior changes:

- modules/tool-delegate/amplifier_module_tool_delegate/__init__.py:
  replace the delegate tool's "Agent usage notes" block (rendered into
  every request's tool spec) with explicit batching guidance.
- context/agents/multi-agent-patterns.md: replace "## Parallel Agent
  Dispatch" with "## Plan the Whole Set, Then Dispatch", adding the
  wall-clock framing and an explicit independence test.
- context/agents/delegation-instructions.md: replace "## Scaling with
  Multiple Instances" with "## Wave Discipline: Batch Everything
  Independent", covering both same-agent and cross-agent batching.

Token cost: +175 tokens/request (budgeted in the spec against a ~12,000
token foundation context load). Each edit replaces its section rather
than appending, per ISSUE_HANDLING.md's guidance on not growing files
without cause.

Stage 2 (a `hooks-delegation-batching` nudge hook that reinforces this
guidance at the actual decision point -- after wave N's results land,
before wave N+1 is planned) is fully specified but deferred. It ships
only if Stage 1's DTU measurement misses the >=40% wall-reduction
target.

Verified: full suite green (uv run pytest tests/ -q --tb=short: 1634
passed, 1 skipped), modules/tool-delegate/tests/ green (62 passed),
python_check clean on the touched .py file (only pre-existing warnings
elsewhere in the file, none introduced), and the tool-schema token
estimator's `ast.literal_eval` extraction confirmed still succeeds
(no `return{` substring introduced).

Spec: pg1-parallel-delegation-spec.md (openai_improvement-pg1)

Note for reviewers: after merge, `amplifier reset --remove cache -y` is
required before the new guidance is loaded by a running Amplifier
process -- foundation context files are read from the bundle cache.

🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
@bkrabach

Copy link
Copy Markdown
Collaborator Author

Merge basis (documented admin-merge — precedent: #317/#318/#319, gemini#39, merged same-day under this pattern)

  • Test suite: full suite green — 1634 passed.
  • DTU A/B measurement: Live A/B in a Digital Twin Universe measured the tool-description edit directly:
    • First-delegation-wave width rose 1 → 3, with zero dependency-breaking batches observed.
    • Token cost held within ±6% of baseline.
    • Wave-count and wall-clock targets were not met in this harness — the harness loads only 1 of the 3 edited files, which understates the effect of the full guidance change. A clean re-measure (all 3 files loaded) is in flight.
  • Net assessment: directionally positive (more parallel delegation, no correctness regressions, negligible token overhead), zero-risk guidance-only change (no behavioral/code-path changes, docs/prompt-guidance only), with an outstanding clean re-measure to fully confirm the wave/wall targets.
  • Review gate: PR is self-authored; GitHub blocks self-approval, so reviewDecision is stuck at REVIEW_REQUIRED with all required checks green (mergeStateStatus: BLOCKED on review only, not checks).
  • Action: merging via --admin at explicit user direction, consistent with the established maintainer admin-merge pattern used for fix(agents): idempotency discipline for git-ops (context-truncation thrash) #317, fix(spawn): sub-agent provider override must outrank a priority-0 primary #318, fix(delegate): carry agent identity through resume completions + emit agent_resumed #319, and gemini#39 earlier today.

@bkrabach
Brian Krabach (bkrabach) merged commit eb324ad into main Aug 27, 2026
7 checks passed
Brian Krabach (bkrabach) added a commit that referenced this pull request Aug 28, 2026
…idempotency section (evidence refuted) (#328)

A payload-level investigation found that three recent changes were built on
a measurement artifact: analyses counted the same events 2-4x across
duplicate snapshot copies of session files (proven by identical
tool_call_id + nanosecond timestamps across "duplicates"). Deduped, the
phenomena each change was built to address vanish. This reverts all three
on that evidence.

1. Remove hooks-dedupe module entirely (PR #323 as hooks-tool-dedupe,
   renamed in PR #326). Deleted modules/hooks-dedupe/ and its `hooks:`
   entry in behaviors/agents.yaml. Deduped, the "same-batch duplicate
   reads" it coalesces do not exist: 0 across 314 sessions / 10,320 reads.
   Measured real value was ~19k tokens (~$0.003) across 5 runs against a
   claimed 16.4% savings -- a ~4,800x overstatement.

   Kept: tests/test_hook_module_classification.py (added by #326). It
   guards a real amplifier-core loader fragility (name-based module-type
   guessing misclassifying a `hooks-*` module as `tool`) that is unrelated
   to whether hooks-dedupe itself exists. Removed only the hooks-dedupe
   specific references: the hardcoded
   `assert "hooks-dedupe" in HOOK_MODULE_IDS` and the
   `test_hooks_dedupe_passes_real_validation` end-to-end test (and its
   now-unused `ModuleValidationError` import). The general, dynamically
   parametrized `test_hook_module_classifies_as_hook` test is untouched
   and still covers every remaining `hooks-*` module.

2. Remove PR #320's "BATCH YOUR DELEGATIONS" guidance block from the
   delegate tool's description in
   modules/tool-delegate/amplifier_module_tool_delegate/__init__.py,
   restoring the original "- Launch multiple agents concurrently when
   tasks are independent" line it replaced. A clean same-commit 5v5 A/B
   measured the guidance as inert (treatment waves median 4 vs control
   median 3) -- the earlier apparent win was a version confound -- while
   it cost ~175 tokens on every single request.

   PR #320's two context-file edits (context/agents/multi-agent-patterns.md,
   context/agents/delegation-instructions.md) are untouched; only the
   scope named above is in this revert. PR #327's own deletions in this
   same string (the CRITICAL/ALWAYS/NEVER preamble and the "DEFAULT TO
   DELEGATION" line) are also untouched -- they stay removed.

3. Remove PR #317's "## Idempotency Discipline" section from
   agents/git-ops.md. Its evidence -- "git-ops re-ran identical clone x8 /
   ls-remote x7 under context truncation" -- is refuted: deduped session
   data shows 8 clones of 8 *different* repos, each cloned once, with
   per-repo ls-remote calls. The thrash it was written to prevent never
   happened.

Verified: full suite green (uv run pytest tests/ -q: 1640 passed, 1
skipped -- down from 1642 by exactly the two hooks-dedupe-specific test
instances removed in (1), both accounted for). python_check clean on all
touched files (only pre-existing, unrelated ruff warnings remain in
tool-delegate's __init__.py, none introduced by this change). Repo-wide
grep confirms modules/hooks-dedupe/ is gone with no dangling functional
references (three remaining mentions of hooks-tool-dedupe/hooks-dedupe are
historical narrative inside the kept regression test's docstring/assert
message, explaining why that general test exists).

🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-authored-by: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
@bkrabach

Copy link
Copy Markdown
Collaborator Author

CORRECTION/OUTCOME: a clean same-foundation-commit 5v5 DTU A/B measured this guidance INERT — treatment waves median 4 vs control 3, wall/token deltas within noise; the earlier apparent front-loading win was a foundation-version confound. The tool-description block is being removed in #328 (~175 tokens/request with no measured benefit). The two context-file sections remain (net-zero replacements). Lesson recorded: prose guidance does not change delegation behavior on these models; mechanical levers do.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants