feat(tool-delegate): return partial results on per-delegate timeout (k64 prereq 1 of 2, CONSUMER half) - #353
Conversation
A delegate that exceeds settings.timeout already RETURNED rather than raised (14d5a52), so completed siblings in the same asyncio.gather batch already survived. What it still discarded was the straggler's own work: there was no channel carrying what the sub-session produced before the deadline, and no flag saying whether any such work existed. - add an optional app-layer `session.partial` capability (sub_session_id) -> {"text", "segments", "source"}, sync or async. Absent, empty, malformed or raising degrades to partial_available: false -- never to an error, because raising out of the timeout handler is precisely what would discard the completed siblings this path protects. - carry partial_available / partial_response / partial_segments / partial_source / partial_truncated / partial_chars_total / completed / guidance on both timeout paths (spawn and resume). Preserved text NEVER lands on `response`, the success-only key. - settings.partial_max_chars (default 20000) caps preserved text, keeping the most recent tail. - emit elapsed_s on delegate:error and in metadata, so leg durations are measured rather than inferred. - BREAKING (small): output/event `status` is now "timeout", was "timed_out", via the single constant TIMEOUT_STATUS. Grep shows no consumer outside this module (not elsewhere in foundation, not in amplifier-app-cli). This is the literal string the downstream eval's gate observes. Normal completions are byte-identical: same script run against the parent tree and this one produces the same serialized output, sha256 18dfd629... on both sides. The source diff removes six lines, all of them "timed_out" literals or timeout f-strings; nothing on the success path is touched. This is HALF of the prerequisite. partial_available is false for every timeout until the app-cli producer half registers session.partial. That is expected and documented, not a defect. 11 of 12 new tests fail on parent 5ebf1da; the one that passes is the byte-identity guard, which must pass on both. Full suite: 1905 passed, 1 skipped (parent: 1893 passed, 1 skipped -- delta is exactly the 12 new tests). Co-authored-by: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
Manager verification (independent, scratch clone)Head Full suite: Fail-before, reproduced independently. The test file alone fails at collection on the parent ( That is the defect and its repair, on the real code path, with no new symbols required to observe it. Default behaviour unchanged — shown, not asserted. I diffed the probe's section D (a normal completion, including the exact serialized string the model receives) between parent and head: byte-identical. That is the deliverable most easily satisfied by assertion, and this PR earns it. RT-2, the one breaking change, verified rather than trusted. Scope: 11 files — the tool-delegate module, its tests, its README, and the lane's own artifact root. Nothing else. Two findings that matter more than the diffF-1 — half the stated harm no longer existed. Gate G-D1 (a straggler discarding completed siblings) was already satisfied on the parent: F-3 — this does not unblock k64, exactly as designed. 37n's patch did not apply at all — all 10 source hunks failed, and the "verified base" Squash + |
This is HALF of a two-patch prerequisite
This is patch (1) of 2, the CONSUMER half, for
model_performance-k64.k64 remains BLOCKED until the app-cli producer half (
model_performance-9w0)also lands. On its own this change makes
partial_availableobservable butalways
false, because nothing in amplifier-foundation registers asession.partialcapability. k64's gate G-D4 requires at least one timeout tocarry
partial_available: true; run with only this half merged, it willcorrectly record
PARTIAL-PATH-NOT-EXERCISEDand stop. That is the designed,documented behaviour — it has not been faked.
Safe to land alone. Do not enable
settings.timeoutsweeps until theproducer half is in.
What this fixes
When a delegate exceeds
settings.timeout, its own work was discarded. Therewas no channel carrying what the sub-session produced before the deadline, and
no flag stating whether any such work existed.
session.partialcapability:(sub_session_id) -> {"text", "segments", "source"} | None, sync or async.Absent / empty / malformed / raising all degrade to
partial_available: false— never to an error. Raising out of thetimeout handler is exactly what would propagate through
asyncio.gatheranddiscard completed siblings.
partial_available,partial_response,partial_segments,partial_source,partial_truncated,partial_chars_total,completed,guidance.Preserved text never lands on
response, the success-only key.settings.partial_max_chars(default20000) caps preserved text, keepingthe most recent tail.
elapsed_semitted ondelegate:errorand inmetadata— leg durationsmeasured, not inferred.
statuson the timeout result and ondelegate:erroris now"timeout",was
"timed_out". Single constantTIMEOUT_STATUS; reverting is one line.Rationale: it is the literal string the downstream eval's gate observes
(
'every observed timeout result carries status "timeout"'), and"timed_out"is not that string nor a substring of it. Blast radius measured before
changing:
Only this module's own code, README and tests. Both channels changed together
so they cannot disagree.
Finding: gate G-D1 was already satisfied — reported, not staged
The item describes two harms. Only one still existed.
14d5a52alreadymade the timeout handler return rather than raise, so completed siblings were
already surviving. Measured on the parent commit, unmodified:
There is therefore no fail-before test showing siblings being discarded,
because they are not. What this PR adds for G-D1 is a regression pin. The
previously measured 1.2% (4/533) harm rate predates
14d5a52and should bere-derived before being quoted as current.
Fail-before evidence
Parent
5ebf1da,git apply --checkon the pre-existing patch —all 10 source hunks failed, zero applied, nothing force-applied:
Every hunk re-targeted by hand; the per-hunk record is in the DONE-NOTE §2.
New tests on the parent commit: 11 failed, 1 passed. The one that passes is
test_success_result_carries_no_partial_keys— the byte-identity guard, whichmust pass on both sides.
Normal completions are byte-identical — shown, not asserted
Same probe script, parent tree vs this one,
diffed verbatim including theexact serialized string the model receives:
Structurally: the source diff removes 6 lines, every one a
"timed_out"literal or a timeout f-string. Zero lines removed from the success path.
Tests
Delta is exactly +12, the new tests. The 1 warning is pre-existing in
tests/test_subprocess_runner.py, on both sides.Reconciliation: the source patch's notes predicted "59 passed (48
pre-existing + 11 new)". Both numbers are stale — the module suite has 177
pre-existing tests at this base, so 177 + 12 = 189.
Note (pre-existing, not fixed here): CI runs
pytest tests/, whichoverrides
testpathsand excludesmodules/tool-delegate/testsentirely.Every test in this PR, and all 177 pre-existing module tests, are invisible to
CI. Verified locally with the commands above.
Full record — per-hunk re-targeting, the eight re-targeting decisions and their
reasons, findings, and spend ($0.00 of $0.00 authorized) — is in
docs/lanes/bp0-delegate-timeout-partial-consumer/DONE-NOTE.md.