feat(delegate): flag-gated per-leg call budget via orchestrator max_iterations (Layer 1) by bkrabach · Pull Request #325 · microsoft/amplifier-foundation

Brian Krabach (bkrabach) · 2026-08-27T14:21:46Z

Summary

Layer 1 of the "Layered Bounding for Delegated Sessions" spec (298-replacement — replaces the wall-clock-only default in #298, which is being revised separately rather than merged as-is). Gives tool-delegate the ability to inject a per-session-leg LLM-call budget into a child session, using the delegate's existing orchestrator_config channel — zero new kernel surface, zero app-cli changes.

Enforcement lives in the child's own orchestrator loop (max_iterations), not in this module — this module only resolves what value to inject and reports what came back. See companion PR microsoft/amplifier-module-loop-streaming#43 for the orchestrator-side work this depends on for the budget to actually do anything.

Ships DARK. settings.max_llm_calls defaults to None. With no config change, orchestrator_config is byte-for-byte what it was before this PR — nothing is injected, nothing changes.

Precedence chain

Rank	Source	Status
1	Per-call tool input (`max_llm_calls`)	✅ implemented
2	Per-agent frontmatter (`agents[name]["budget"]["max_llm_calls"]`)	❌ not implemented — see below
3	`settings.max_llm_calls` (default `None`)	✅ implemented
4	Inherited parent `orchestrator_config.max_iterations`	✅ implemented (pre-existing path, untouched when no budget applies)

An explicit 0 at rank 1 means "no Layer 1 budget for this call" (the wall-clock settings.timeout backstop still applies) and correctly overrides rank 3's default — this required care: a naive "0 → None" collapse done too early loses the ability to distinguish "explicitly opted out" from "not specified at all," which would have silently ignored a caller's opt-out. Covered by TestOptOut in the new test file.

Per-agent frontmatter override did NOT make the cut — verified, not assumed

The spec flagged this as an open item requiring verification before shipping: does a top-level budget: block in an agent .md file's frontmatter actually reach coordinator.config["agents"][name]["budget"]?

Verified empirically (wrote and ran a test against the real loader, not just read the source): it does not round-trip. amplifier_foundation.bundle._dataclass._load_agent_file_metadata only forwards a fixed allowlist of top-level frontmatter keys (tools, providers, hooks, session, provider_preferences, model_role, agents) — budget is silently dropped. See test_agent_frontmatter_budget_key_is_dropped in the new test file (uses model_role, an allowlisted key, as a control to prove this isn't a wholesale frontmatter-loading failure).

Per the owner's decision, rank 2 is left out of this PR rather than faked. Ranks 1/3/4 ship and are fully tested. Closing this gap requires a change to amplifier_foundation.bundle._dataclass._load_agent_file_metadata (add budget to the allowlist) — filed as a follow-up rather than bundled here, since it's a foundation-loader change, not a tool-delegate change. See the module README's "Known gaps" section.

Also added

Eager validation (_check_call_budget_type / _validate_call_budget): rejects bool, non-int, and negative values at the point supplied — module construction for the settings default, execute() for the per-call override — never silently coerced, never deferred to spawn time.
Negotiated-feature warning (spec §4.4): if a budget was requested but the child's orchestrator reports no llm_call_budget telemetry (e.g. a third-party orchestrator without max_iterations support), logs a warning naming the agent and sets metadata.budget_enforced = false on the returned ToolResult — silence is exactly the failure mode this design exists to eliminate.
max_llm_calls entry in the tool's input schema (kept as a pure literal for the static token-cost estimator, per this file's existing convention).

Files changed

modules/tool-delegate/amplifier_module_tool_delegate/__init__.py:
- _check_call_budget_type / _validate_call_budget (module-level validation helpers)
- __init__: settings.max_llm_calls / budget_warn_ratio
- execute(): per-call max_llm_calls parsing + validation
- _resolve_call_budget (new method): 2-rank precedence (1 and 3; rank 4 falls through untouched)
- _spawn_new_session: orchestrator_config build now copies (never mutates) the parent's config before injecting the budget; negotiated-feature warning + budget_enforced metadata
- _static_input_schema(): max_llm_calls parameter
modules/tool-delegate/README.md: "Layer 1 call budget" section + "Known gaps"
modules/tool-delegate/tests/test_delegate_call_budget.py (new): 14 tests covering the precedence chain, opt-out, validation, status/metadata passthrough, the negotiated-feature warning, resume behavior, and the frontmatter verification

Testing

New tests: uv run pytest modules/tool-delegate/tests/test_delegate_call_budget.py -q → 14 passed
Full tool-delegate module suite: uv run pytest modules/tool-delegate/tests/ -q → 80 passed (zero regressions)
Full foundation repo suite: uv run pytest tests/ -q → 1634 passed, 1 skipped — identical to the pre-change baseline
python_check (ruff lint/format, pyright): no new issues vs. baseline (all remaining warnings/errors are pre-existing, same lines shifted, confirmed via before/after diff)

Relationship to other PRs

Companion PR: feat(delegate): flag-gated per-leg call budget via orchestrator max_iterations (Layer 1) amplifier-module-loop-streaming#43 (orchestrator-side enforcement — required for this PR's config to have any effect)
fix(delegate): bound delegated sessions by default #298 is being revised separately (not closed) to raise its wall-clock default 1800s → 14400s and reframe as the Layer 3 backstop that sits behind this budget

Do not merge — up for review alongside the companion PR and the #298 revision.

🤖 Generated with Amplifier

Reframes this PR's timeout as Layer 3 of the "Layered Bounding for Delegated Sessions" design (spec: 298-replacement) -- the orchestrator- independent wall-clock backstop that sits behind a per-leg LLM-call budget (Layer 1, see microsoft#325 and companion PR microsoft/amplifier-module-loop-streaming#43), not the primary bound. Net functional change: `1800` -> `14400` for `settings.timeout`'s default. Everything else in this PR is kept verbatim: `_DelegateTimeoutExpired`, `_validate_timeout`, `_await_child_with_deadline`, the hard parent-release semantics, the honest `resumable: false` / `resume_status: pending_child_cleanup` reporting, and all 30 focused timeout tests (only the default-value assertions are retargeted). Why 14400s: ~12x the measured healthy sub-session upper bound (996-1168s), and ~2x below the worst observed runaway (17h34m) -- generous enough that a working Layer 1 budget should make this backstop fire zero times in practice. If it ever fires with Layer 1 active, that's a Layer 1 bug report, not evidence this default is wrong. Docs updated to frame this as the backstop: module docstring, README's "Delegate Timeout" section (retitled "Layered bounding: call budget (Layer 1) + wall-clock backstop (Layer 3)"), and the settings.timeout config comment. Files: - modules/tool-delegate/amplifier_module_tool_delegate/__init__.py: module docstring `settings.timeout` description; default 1800 -> 14400 - modules/tool-delegate/README.md: "Delegate Timeout" section rewritten as "Layered bounding"; config example comment - modules/tool-delegate/tests/test_delegate_timeout.py: default-value assertions retargeted to 14400 (T3.1) Testing: - modules/tool-delegate/tests/test_delegate_timeout.py: 30 passed - Full tool-delegate module suite: 90 passed (zero regressions) - Full foundation repo suite (tests/, this branch's own base): 1549 passed - python_check: no new issues vs this branch's own baseline (pre-existing I001 import-sort warning unchanged, confirmed via stash diff) 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>

… (Layer 1) (#43) feat(orchestrator): flag-gated per-leg call budget via max_iterations (Layer 1) Ships dark: max_iterations call-budget behavior defaults to unlimited/off — zero behavior change until an operator opts in. Includes one real correctness fix: budget wrap-up call now sends tools=None so the final call must produce a summary. 185/185 tests green. Companion to microsoft/amplifier-foundation#325. Self-authored; admin-merged at explicit user/operator direction (self-approval not possible; see PR comment for full basis).

…terations (Layer 1) Layered Bounding for Delegated Sessions (spec: 298-replacement, replacing the wall-clock-only default in #298). Adds a per-session-leg LLM-call budget as the first line of defense in front of the delegate's existing settings.timeout wall-clock backstop, delivered with zero new kernel surface: tool-delegate writes max_iterations (and a new budget_warn_ratio) into the orchestrator_config dict it already passes to spawn_fn, and amplifier-app-cli's session_spawner already does a caller-wins .update() into the child's config -- zero app-cli changes needed. Enforcement itself lives in the orchestrator loop (see companion PR microsoft/amplifier-module-loop-streaming#43), which already counts LLM calls via max_iterations and already exits exhaustion via a normal return (graceful wrap-up), so the resulting transcript is complete and resumable -- unlike a cancellation-based timeout. Ships DARK: settings.max_llm_calls defaults to None, so no budget is injected into any child session and orchestrator_config is byte-for-byte what it was before this change. Nothing here changes behavior until an operator explicitly sets settings.max_llm_calls. Precedence chain (highest first): 1. Per-call tool input (`max_llm_calls`) -- implemented 2. Per-agent frontmatter (`agents[name]["budget"]["max_llm_calls"]`) -- NOT implemented, see below 3. This module's settings.max_llm_calls (default None) -- implemented 4. Inherited parent orchestrator_config's max_iterations -- implemented (the pre-existing inheritance path, left untouched when no budget applies) Per-agent frontmatter override (rank 2) does not ship: verified empirically (not just read from source) that a top-level `budget:` block in an agent .md's frontmatter is dropped by amplifier_foundation.bundle._dataclass._load_agent_file_metadata, which only forwards a fixed allowlist of top-level keys (tools, providers, hooks, session, provider_preferences, model_role, agents) -- budget is not among them. Reproduced in tests/test_delegate_call_budget.py::test_agent_frontmatter_budget_key_is_dropped. Ranks 1, 3, and 4 ship; rank 2 is a follow-up requiring a change to the frontmatter loader itself, documented in this module's README "Known gaps" section. Also adds: - Eager validation (_validate_call_budget / _check_call_budget_type): reject bool, non-int, and negative values at the point supplied (module construction for the settings default, execute() for the per-call override) -- never at spawn time. - Negotiated-feature warning (spec §4.4): if a budget was requested but the child's orchestrator reports no llm_call_budget telemetry (e.g. a third-party orchestrator with no max_iterations support), logs a warning and sets metadata.budget_enforced = false on the returned ToolResult, so the gap is loud rather than silent. - max_llm_calls entry in the tool's input schema (kept a pure literal for the static token-cost estimator). Files: - modules/tool-delegate/amplifier_module_tool_delegate/__init__.py: _check_call_budget_type / _validate_call_budget module functions; settings.max_llm_calls / budget_warn_ratio in __init__; per-call max_llm_calls parsing + validation in execute(); _resolve_call_budget method; orchestrator_config build (copy-not-mutate + budget injection) and negotiated-feature warning in _spawn_new_session; max_llm_calls schema entry - modules/tool-delegate/README.md: "Layer 1 call budget" section + "Known gaps" - modules/tool-delegate/tests/test_delegate_call_budget.py (new): T2.1, T2.2, T2.4, T2.5, T2.6, T2.7, T2.8, T2.9, T2.10, T2.11 + the frontmatter round-trip verification test (14 tests) Testing: - New tests: 14 passed - Full tool-delegate module suite: 80 passed (was ~66; zero regressions) - Full foundation repo suite (tests/): 1634 passed, 1 skipped -- matches pre-change baseline exactly - ruff/pyright: no new issues vs baseline Part of the 298-replacement design (Layer 1 of 3). Companion PR: microsoft/amplifier-module-loop-streaming#43 (orchestrator-side enforcement). #298 is being revised separately to reframe its wall-clock default as the Layer 3 backstop behind this budget. 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>

Brian Krabach (bkrabach) · 2026-08-27T15:11:42Z

Rebased feat/delegate-call-budget onto current main (which now includes #322, #323, and #324) to resolve the CONFLICTING/DIRTY state reported after #324 merged (squash 1439cf5) ahead of this PR.

Conflict resolution — both additive features kept in full, nothing dropped:

modules/tool-delegate/amplifier_module_tool_delegate/__init__.py:
- Config-parsing / module-header region: kept feat(delegate): flag-gated structured return contract (findings/not_covered/artifacts) #324's RETURN_CONTRACT_SCHEMA / RETURN_CONTRACT_INSTRUCTION / _return_contract_event_fields / _JSON_FENCE_PATTERN block, followed by this PR's _check_call_budget_type / _validate_call_budget helpers, ahead of mount().
- Return-path region: merged feat(delegate): flag-gated structured return contract (findings/not_covered/artifacts) #324's annotated-output comment with this PR's output_metadata construction (budget_enforced merged into the metadata bag), then combined both additive output keys — "metadata": output_metadata (this PR) and "contract": contract (feat(delegate): flag-gated structured return contract (findings/not_covered/artifacts) #324) — into the same ToolResult.
modules/tool-delegate/README.md: kept both new sections — feat(delegate): flag-gated structured return contract (findings/not_covered/artifacts) #324's "Structured Return Contract" followed by this PR's "Layer 1 call budget" — no content removed from either.

Testing (post-rebase, both features exercised together):

modules/tool-delegate/tests/ (module suite): 120 passed — includes feat(delegate): flag-gated structured return contract (findings/not_covered/artifacts) #324's return-contract tests (40, across 4 files) and this PR's call-budget tests (14, test_delegate_call_budget.py) running side by side with no interaction regressions; both parse cleanly from the same config/settings dict.
Full repo suite uv run pytest tests/: 1634 passed, 1 skipped.
python_check (ruff lint/format, pyright) on the touched module file: 0 errors; only pre-existing warnings inherited from main (verified against main's copy of the file) remain — a trivial pre-existing ruff format line-length nit from this PR's own diff was fixed in the process.

Ships dark: settings.max_llm_calls defaults to None — no behavior change for existing users until an operator opts in.

Self-authored / merging with admin at user direction, per this week's precedent for author-approved same-repo branches.

Reframes this PR's timeout as Layer 3 of the "Layered Bounding for Delegated Sessions" design (spec: 298-replacement) -- the orchestrator- independent wall-clock backstop that sits behind a per-leg LLM-call budget (Layer 1, see microsoft#325 and companion PR microsoft/amplifier-module-loop-streaming#43), not the primary bound. Net functional change: `1800` -> `14400` for `settings.timeout`'s default. Everything else in this PR is kept verbatim: `_DelegateTimeoutExpired`, `_validate_timeout`, `_await_child_with_deadline`, the hard parent-release semantics, the honest `resumable: false` / `resume_status: pending_child_cleanup` reporting, and all 30 focused timeout tests (only the default-value assertions are retargeted). Why 14400s: ~12x the measured healthy sub-session upper bound (996-1168s), and ~2x below the worst observed runaway (17h34m) -- generous enough that a working Layer 1 budget should make this backstop fire zero times in practice. If it ever fires with Layer 1 active, that's a Layer 1 bug report, not evidence this default is wrong. Docs updated to frame this as the backstop: module docstring, README's "Delegate Timeout" section (retitled "Layered bounding: call budget (Layer 1) + wall-clock backstop (Layer 3)"), and the settings.timeout config comment. Files: - modules/tool-delegate/amplifier_module_tool_delegate/__init__.py: module docstring `settings.timeout` description; default 1800 -> 14400 - modules/tool-delegate/README.md: "Delegate Timeout" section rewritten as "Layered bounding"; config example comment - modules/tool-delegate/tests/test_delegate_timeout.py: default-value assertions retargeted to 14400 (T3.1) Testing: - modules/tool-delegate/tests/test_delegate_timeout.py: 30 passed - Full tool-delegate module suite: 90 passed (zero regressions) - Full foundation repo suite (tests/, this branch's own base): 1549 passed - python_check: no new issues vs this branch's own baseline (pre-existing I001 import-sort warning unchanged, confirmed via stash diff) 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>

…k backstop, 14400s) * fix(delegate): bound delegated sessions by default * fix(delegate): raise the wall-clock backstop default to 14400s (Layer 3) Reframes this PR's timeout as Layer 3 of the "Layered Bounding for Delegated Sessions" design (spec: 298-replacement) -- the orchestrator- independent wall-clock backstop that sits behind a per-leg LLM-call budget (Layer 1, see #325 and companion PR microsoft/amplifier-module-loop-streaming#43), not the primary bound. Net functional change: `1800` -> `14400` for `settings.timeout`'s default. Everything else in this PR is kept verbatim: `_DelegateTimeoutExpired`, `_validate_timeout`, `_await_child_with_deadline`, the hard parent-release semantics, the honest `resumable: false` / `resume_status: pending_child_cleanup` reporting, and all 30 focused timeout tests (only the default-value assertions are retargeted). Why 14400s: ~12x the measured healthy sub-session upper bound (996-1168s), and ~2x below the worst observed runaway (17h34m) -- generous enough that a working Layer 1 budget should make this backstop fire zero times in practice. If it ever fires with Layer 1 active, that's a Layer 1 bug report, not evidence this default is wrong. Docs updated to frame this as the backstop: module docstring, README's "Delegate Timeout" section (retitled "Layered bounding: call budget (Layer 1) + wall-clock backstop (Layer 3)"), and the settings.timeout config comment. Files: - modules/tool-delegate/amplifier_module_tool_delegate/__init__.py: module docstring `settings.timeout` description; default 1800 -> 14400 - modules/tool-delegate/README.md: "Delegate Timeout" section rewritten as "Layered bounding"; config example comment - modules/tool-delegate/tests/test_delegate_timeout.py: default-value assertions retargeted to 14400 (T3.1) Testing: - modules/tool-delegate/tests/test_delegate_timeout.py: 30 passed - Full tool-delegate module suite: 90 passed (zero regressions) - Full foundation repo suite (tests/, this branch's own base): 1549 passed - python_check: no new issues vs this branch's own baseline (pre-existing I001 import-sort warning unchanged, confirmed via stash diff) 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> --------- Co-authored-by: Sam Schillace <ramparte@users.noreply.github.com> Co-authored-by: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>

Brian Krabach (bkrabach) force-pushed the feat/delegate-call-budget branch from b96829b to 3050860 Compare August 27, 2026 15:08

Brian Krabach (bkrabach) merged commit 4b66fea into main Aug 27, 2026
7 checks passed

Brian Krabach (bkrabach) mentioned this pull request Aug 27, 2026

fix(hooks): hooks-tool-dedupe fails protocol_compliance via top-level hooks block #326

Merged

Brian Krabach (bkrabach) mentioned this pull request Aug 28, 2026

fix(delegate): bound delegated sessions by default #298

Merged

Navigation Menu

Uh oh!

feat(delegate): flag-gated per-leg call budget via orchestrator max_iterations (Layer 1) - #325

feat(delegate): flag-gated per-leg call budget via orchestrator max_iterations (Layer 1)#325
Brian Krabach (bkrabach) merged 1 commit into
mainfrom
feat/delegate-call-budget

Brian Krabach (bkrabach) commented Aug 27, 2026

Uh oh!

Brian Krabach (bkrabach) commented Aug 27, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

Brian Krabach (bkrabach) commented Aug 27, 2026

Summary

Precedence chain

Per-agent frontmatter override did NOT make the cut — verified, not assumed

Also added

Files changed

Testing

Relationship to other PRs

Uh oh!

Brian Krabach (bkrabach) commented Aug 27, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants