Derived output schemas and structured content (ADR 0005)

[drshade]

Implements ADR 0005: the highest-leverage Batch 1 roadmap item, completing the library's thesis — typed in, typed out.

What it adds

data WeatherReport = WeatherReport
    { temperature :: Int, sky :: Sky, alerts :: [Text], humidity :: Maybe Int }

handleTool :: ClientContext -> MyTool -> IO (ToolOutput WeatherReport)
handleTool _ (GetWeather city) = pure $ ToolOutput (lookupWeather city)

tools = Just $(deriveToolHandlerWithOutput ''MyTool 'handleTool ''WeatherReport)

• The result record derives the tools' outputSchema via the existing Schema machinery (primitives, Maybe→optional, lists, all-nullary enums, nested records) — shared with input derivation per the ADR, so future schema-keyword work benefits both directions.
• The serializer is generated by the same TH walk as the schema, so the structured value provably matches what the schema promises (snake_cased enums, Nothing fields omitted). No reliance on user ToJSON instances that could drift.
• ToolOutput carries the spec-recommended behavior: a plain ToolOutput also returns the serialized JSON as a text content block for pre-structured-output clients; ToolOutputWith overrides the content; ToolOutputError maps to isError; ToolOutputRaw is the escape hatch.
• Output types must resolve to a record (outputSchema is an object schema per spec) — anything else is rejected at TH time with a clear message.
• Plain ToToolResult handlers are untouched; deriveToolHandler(WithDescription) now share a generalized implementation with the output-typed variants rather than duplicating the dispatch generation.

Verification

• 161 test examples (was 151): a DerivedOutput spec covers the schema shape (enum values, nested object, array, optional-Maybe required-list, description wiring), structured/text-block equality, Nothing omission, all four ToolOutput forms.
• Conformance corpus: the extended reference server gains echo_structured — derived through this machinery so the corpus pins the exact TH wire shape — with tools-call-structured cases in both eras and tools-list-extended showing the serialized outputSchema. Corpus README updated; v0.2.0-anchored fixtures untouched.
• Built and tested on GHC 9.10.3 and 9.14.1.

Versioning

Purely additive (new exports, no changed signatures) → joins the pending 0.2.1.0 line per the release policy. ADR_0005 marked Landed (0.2.1.0).

[drshade]

Review verdict: one fix requested (canonical text-block encoding), one nit. The design is exactly ADR 0005 — and generating the serializer from the same TH walk as the schema is the right way to make the schema/value agreement structural rather than hoped-for. The ToolOutput four-form API is clean (plain/with-content/error/raw covers the space without a typeclass), the spec-recommended text block appears exactly when the handler didn't supply content, TH-time rejection of non-record output types matches the ADR, and unifying deriveToolHandler* through one generic implementation pays down duplication rather than adding to it. Test coverage hits every form plus the schema shape details, and the corpus deliberately pins the TH wire shape with a derived tool. 161/161 locally; CI pending as I write this — merge will wait for green as usual.

[drshade]

The requested fix: make the text block's JSON canonical (sorted keys). encode's object key order depends on the aeson/hashable pair in the build plan — which is exactly why GoldenWire compares parsed Values. But this string is JSON inside a string: the corpus fixture tools-call-structured.response.json embeds "{\"echoedLength\":6,\"echoedText\":\"golden\"}" literally, and parsed-Value comparison can't see through it. If a future build plan resolves a different hashable, the text block's key order changes and the fixture breaks — worse, for external corpus consumers the expected bytes would be aeson-implementation-specific, undermining the corpus's neutrality.

Fix is small: encode the text block from a canonically-ordered structure, e.g. a recursive Value -> Value that rebuilds every object via sorted KeyMap.toList before encode (aeson encodes KeyMap.fromList of sorted pairs in that order... safer still, convert objects to Data.Map Text Value recursively and encode that — Map's ToJSON is ordered by key). Deterministic output has a side benefit the spec explicitly cares about: stable tool results improve client prompt-cache hit rates.

Apply it only to the text block (structuredContent itself is compared structurally everywhere and can stay as-is), regenerate the two tools-call-structured fixtures, and ideally note in the corpus README that embedded-JSON strings are canonical (sorted keys).

[drshade]

Nit, fix or defer: the record serializer applies field accessors ($(varE fieldName) $(varE rVar)), which breaks for users with DuplicateRecordFields enabled (bare accessor application is ambiguous there, and duplicate field names across records are increasingly common in the codebases that would want typed outputs). The rest of the TH machinery never relies on accessors — input decoding constructs values, and your own enum/wrapper cases here pattern-match. Building this the same way — one conP binding every field to fresh names — makes the serializer immune:

-- \(Con f0 f1 ...) -> object (concat [...pairs from f0, f1...])

Same generated semantics, no accessor lookup.

[drshade]

Both addressed in 304f069 (162 examples):

• Requested fix: the text block is now canonical — compact JSON with sorted object keys, produced by an encoding-only newtype whose toEncoding emits objects through a sorted Map (so aeson's escaping is reused and no KeyMap is rebuilt). Applied only to the embedded-JSON text block as suggested; structuredContent stays as-is since it's compared structurally everywhere. The two tools-call-structured fixtures are regenerated, the corpus README documents that embedded-JSON strings are canonical, and a new unit test pins the exact text-block bytes.
• Nit: the record serializer now binds fields via a single conP pattern with fresh names — no accessor application, so DuplicateRecordFields users are safe, consistent with how the rest of the TH machinery avoids accessors.

[drshade]

Approved. Both points verified on 304f069: the Canonical encoding-only newtype makes the embedded text-block JSON byte-deterministic across build plans (sorted keys via Data.Map's ordered toEncoding, aeson escaping reused), with the guarantee documented in the corpus README and pinned by an exact-bytes unit test; the record serializer now pattern-binds fields so DuplicateRecordFields users are safe. 162/162 tests locally, CI green. The typed-in/typed-out story is complete — merging. ADR_0005 status flip to Landed confirmed in the diff.