executor: resolve InputRequiredResult elicitations from static or REST answer sources#74
Merged
Merged
Conversation
…T answer sources (WOR-1383)
Part 3 (final) of the 2026-07-28 runtime wiring. A server can answer a tools/call
with an InputRequiredResult (SEP-2322); the recognizer, retry builder, and static
fixture existed but nothing outside tests constructed them. Wire the live runner
to satisfy an elicitation and retry, with a configurable, layered answer source.
- suite config (suite/types.rs, suite/elicitation_parse.rs): a tool test declares
`input_responses` (a static `id: value` map) or `input_responder`
(`rest: { url, headers?, timeout_ms? }`); a suite-level `input_responder` sets a
default. The two test-level keys are mutually exclusive. Both the root and the
embedded schema (+ schema_validation) gain the keys and $defs.
- runtime responder (executor/elicitation.rs): a `Responder` (Static or Rest) built
at plan time. Static delegates to the existing collect_responses; Rest POSTs the
elicitation (the full inputRequests, including descriptions) to a proxy-aware
reqwest client and validates the reply (non-2xx, invalid JSON,
missing/duplicate/unknown ids, missing required, and kind). Uniform
string/number/boolean kind validation for both sources.
- live retry loop (executor/dispatch.rs::call_server): after a tools/call (and any
task-handle poll), if the result is an InputRequiredResult and a responder is
configured, resolve answers, build the retry off the transformed params, re-poll
any task handle, and loop to the round cap; return the final request id so
response-header assertions inspect the final response. Metamorphic follow-ups do
not elicit.
- plan build (run_live_plans.rs): resolve the effective responder (test override
beats the suite default) with the run-wide proxy; a bad REST url or proxy fails
plan building, not the first round. A REST responder marks the test cache-exempt.
- observability: every round emits a redacted `mcptest_core::elicitation` trace
event (ids, counts, responder kind, request id, a stable requestState hash;
never the raw state, the answer values, or the headers).
- docs (elicitation, yaml-reference, reference/config) and example: async-tasks.yml
book_flight now resolves from static input_responses to the final result.
Cassette-recorded elicitation answers remain a follow-up (WOR-943).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The `mcptest schema` command emits crates/mcptest/schemas/v1.json via include_str!, kept byte-identical to the root schema by check-vendored-assets.sh. The Part 3 schema additions updated the root and the mcptest-config copy but missed this third vendored copy, so the cli_schema byte-for-byte tests failed in CI. Sync it. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Part 3 (final) of the 2026-07-28 runtime wiring. A server can answer a
tools/callwith anInputRequiredResult(SEP-2322); the recognizer, retry builder, and static fixture existed but nothing outside tests constructed them. This wires the live runner to satisfy an elicitation and retry, with a configurable, layered answer source (static map, REST endpoint), per the approved design.Changes
input_responses(staticid: valuemap) orinput_responder(rest: { url, headers?, timeout_ms? }); a suite-levelinput_responderis the default. Test-level keys are mutually exclusive. Parsed config (InputResponderSpec/RestInputResponderSpec/InputResponseSpec) is kept separate from the runtime responder. Both root + embedded schemas (+schema_validation) gain the keys and$defs.Responder(Static | Rest) built at plan time. Static delegates to the existingcollect_responses; Rest POSTs the elicitation (fullinputRequests, incl descriptions) to a proxy-aware reqwest client and validates the reply (non-2xx, invalid JSON, missing/duplicate/unknown ids, missing required, andkind). Uniformstring/number/booleankind validation for both sources.executor/dispatch.rs::call_server): after atools/call(and any task-handle poll), anInputRequiredResultwith a configured responder is resolved, the retry is built off the transformed params, any task handle is re-polled, and it loops to the round cap. The final request id is returned for response-header assertions. Metamorphic follow-ups do not elicit.mcptest_core::elicitationtrace event (round_started/answer_resolved/retry_dispatched/completed/failed) with ids, counts, responder kind, request id, and a stablerequestStatehash — never the raw state, answer values, or headers.elicitation.md,yaml-reference.md,reference/config.md;examples/async-tasks.ymlbook_flightnow resolves from staticinput_responsesto the final result (its own server, WOR-1473 guard).Testing
schema_validationtests.examples/async-tasks.yml(examples gate); 30/30 under induced cargo-build load.-D warnings,cargo doc -D warnings, em-dash, module-size green locally; whole workspace compiles all-targets.Cassette-recorded elicitation answers remain a follow-up (WOR-943). This completes WOR-1383 (Parts 1 #72, 2 #73, 3 here).