Skip to content

executor: resolve InputRequiredResult elicitations from static or REST answer sources#74

Merged
rickcrawford merged 2 commits into
mainfrom
rickcrawford/wor-1383-input-required-retry
Jun 18, 2026
Merged

executor: resolve InputRequiredResult elicitations from static or REST answer sources#74
rickcrawford merged 2 commits into
mainfrom
rickcrawford/wor-1383-input-required-retry

Conversation

@rickcrawford

Copy link
Copy Markdown
Contributor

What

Part 3 (final) of the 2026-07-28 runtime wiring. A server can answer a tools/call with an InputRequiredResult (SEP-2322); the recognizer, retry builder, and static fixture existed but nothing outside tests constructed them. This wires the live runner to satisfy an elicitation and retry, with a configurable, layered answer source (static map, REST endpoint), per the approved design.

Changes

  • Suite config: a tool test declares input_responses (static id: value map) or input_responder (rest: { url, headers?, timeout_ms? }); a suite-level input_responder is the default. Test-level keys are mutually exclusive. Parsed config (InputResponderSpec/RestInputResponderSpec/InputResponseSpec) is kept separate from the runtime responder. Both root + embedded schemas (+ schema_validation) gain the keys and $defs.
  • Runtime responder: a Responder (Static | Rest) built at plan time. Static delegates to the existing collect_responses; Rest POSTs the elicitation (full inputRequests, incl descriptions) to a proxy-aware reqwest client and validates the reply (non-2xx, invalid JSON, missing/duplicate/unknown ids, missing required, and kind). Uniform string/number/boolean kind validation for both sources.
  • Live retry loop (executor/dispatch.rs::call_server): after a tools/call (and any task-handle poll), an InputRequiredResult with a configured responder is resolved, the retry is built off the transformed params, any task handle is re-polled, and it loops to the round cap. The final request id is returned for response-header assertions. Metamorphic follow-ups do not elicit.
  • Plan build: resolves the effective responder (test override > suite default) with the run-wide proxy; a bad REST url/proxy fails plan building, not the first round. A REST responder marks the test cache-exempt (live IO).
  • Observability: every round emits a redacted mcptest_core::elicitation trace event (round_started/answer_resolved/retry_dispatched/completed/failed) with ids, counts, responder kind, request id, and a stable requestState hash — never the raw state, answer values, or headers.
  • Docs + example: elicitation.md, yaml-reference.md, reference/config.md; examples/async-tasks.yml book_flight now resolves from static input_responses to the final result (its own server, WOR-1473 guard).

Testing

  • 5 parser tests, 12 elicitation tests (incl static + REST wiremock: happy path, HTTP error, unknown id, wrong-kind), 99 schema_validation tests.
  • Offline end-to-end via examples/async-tasks.yml (examples gate); 30/30 under induced cargo-build load.
  • fmt, clippy -D warnings, cargo doc -D warnings, em-dash, module-size green locally; whole workspace compiles all-targets.

Cassette-recorded elicitation answers remain a follow-up (WOR-943). This completes WOR-1383 (Parts 1 #72, 2 #73, 3 here).

rickcrawford and others added 2 commits June 18, 2026 08:38
…T answer sources (WOR-1383)

Part 3 (final) of the 2026-07-28 runtime wiring. A server can answer a tools/call
with an InputRequiredResult (SEP-2322); the recognizer, retry builder, and static
fixture existed but nothing outside tests constructed them. Wire the live runner
to satisfy an elicitation and retry, with a configurable, layered answer source.

- suite config (suite/types.rs, suite/elicitation_parse.rs): a tool test declares
  `input_responses` (a static `id: value` map) or `input_responder`
  (`rest: { url, headers?, timeout_ms? }`); a suite-level `input_responder` sets a
  default. The two test-level keys are mutually exclusive. Both the root and the
  embedded schema (+ schema_validation) gain the keys and $defs.
- runtime responder (executor/elicitation.rs): a `Responder` (Static or Rest) built
  at plan time. Static delegates to the existing collect_responses; Rest POSTs the
  elicitation (the full inputRequests, including descriptions) to a proxy-aware
  reqwest client and validates the reply (non-2xx, invalid JSON,
  missing/duplicate/unknown ids, missing required, and kind). Uniform
  string/number/boolean kind validation for both sources.
- live retry loop (executor/dispatch.rs::call_server): after a tools/call (and any
  task-handle poll), if the result is an InputRequiredResult and a responder is
  configured, resolve answers, build the retry off the transformed params, re-poll
  any task handle, and loop to the round cap; return the final request id so
  response-header assertions inspect the final response. Metamorphic follow-ups do
  not elicit.
- plan build (run_live_plans.rs): resolve the effective responder (test override
  beats the suite default) with the run-wide proxy; a bad REST url or proxy fails
  plan building, not the first round. A REST responder marks the test cache-exempt.
- observability: every round emits a redacted `mcptest_core::elicitation` trace
  event (ids, counts, responder kind, request id, a stable requestState hash;
  never the raw state, the answer values, or the headers).
- docs (elicitation, yaml-reference, reference/config) and example: async-tasks.yml
  book_flight now resolves from static input_responses to the final result.

Cassette-recorded elicitation answers remain a follow-up (WOR-943).

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The `mcptest schema` command emits crates/mcptest/schemas/v1.json via
include_str!, kept byte-identical to the root schema by
check-vendored-assets.sh. The Part 3 schema additions updated the root and the
mcptest-config copy but missed this third vendored copy, so the cli_schema
byte-for-byte tests failed in CI. Sync it.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@rickcrawford rickcrawford merged commit e71e3f2 into main Jun 18, 2026
19 checks passed
@rickcrawford rickcrawford deleted the rickcrawford/wor-1383-input-required-retry branch June 18, 2026 16:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant