feat(modalities): add bounded capability contracts (#154)

CHANGELOG.md

@@ -8,6 +8,12 @@ details, release history over commit history.
 
 ### Added
 
+- **Bounded modality capability contracts** (WF-ADR-0067, issue #154). The shared Rust routing
+  contract now carries explicit embeddings, image-generation, audio, and batch capabilities and
+  applies them as hard pre-score eligibility filters. The current gateway remains text-only and
+  rejects known modality payloads before provider delivery; each non-text surface will be enabled
+  independently only after its adapter, bounds, accounting, and parity fixtures pass.
+
 - **Bounded OpenAI Responses API compatibility** (WF-ADR-0066, issue #153). The Rust gateway now
   exposes authenticated `/v1/responses` and `/responses` aliases with strict text/multi-turn input,
   normalized usage, bounded SSE streaming, one terminal event, and explicit unsupported-field

README.md

@@ -80,6 +80,8 @@ The gateway exposes:
 
 - OpenAI-compatible: `http://127.0.0.1:8088/v1`
 - OpenAI Responses compatibility: `POST /v1/responses` (bounded text and multi-turn contract)
+- Modality compatibility: explicit embeddings, image, audio, and batch capability contracts;
+  non-text surfaces remain fail-closed until their reviewed adapters are enabled
 - Anthropic-compatible: `http://127.0.0.1:8088`
 - Health: `http://127.0.0.1:8088/healthz`
 

apple/Packages/WayfinderRoutingBridge/Tests/WayfinderRoutingBridgeTests/WayfinderRoutingBridgeTests.swift

@@ -31,7 +31,12 @@ final class WayfinderRoutingBridgeTests: XCTestCase {
                 text: true,
                 streaming: true,
                 imageInput: false,
-                tools: false
+                tools: false,
+                embeddings: false,
+                imageGeneration: false,
+                audioInput: false,
+                audioOutput: false,
+                batch: false
             ),
             automaticEligible: true
         )
@@ -69,7 +74,8 @@ final class WayfinderRoutingBridgeTests: XCTestCase {
                     contextTokens: 1_024,
                     imageInput: false,
                     tools: false,
-                    streaming: true
+                    streaming: true,
+                    surface: .text
                 )
             ),
             candidates: [
@@ -129,7 +135,8 @@ final class WayfinderRoutingBridgeTests: XCTestCase {
                         contextTokens: nil,
                         imageInput: false,
                         tools: false,
-                        streaming: false
+                        streaming: false,
+                        surface: .text
                     )
                 ),
                 candidates: []

decisions/WF-ADR-0056-hard-destination-eligibility.md

@@ -50,8 +50,9 @@ control contract when entering the shared chat path.
 Request requirements are derived from the bounded OpenAI-compatible body:
 estimated prompt plus requested output context, image content, tool/function
 declarations, and streaming. Provider defaults are conservative for the two
-text-only native adapters; OpenAI-compatible delivery retains its existing
-pass-through multimodal/tool contract.
+text-only native adapters; non-text surfaces are additionally governed by the
+explicit capability contract in WF-ADR-0067 and fail closed until a reviewed
+adapter is enabled. Tool declarations remain an explicit provider capability.
 
 Explicit pins and named `@route/<name>` presets fail closed with
 `422 wayfinder_router_destination_ineligible` when no eligible destination
@@ -69,7 +70,8 @@ remain unchanged.
 - A provider with unknown native readiness is not guessed ready; missing
   credentials are excluded before delivery.
 - Generic OpenAI-compatible endpoints remain backward-compatible for their
-  existing text, image, tool, and streaming pass-through requests.
+  existing text, tool, and streaming requests; non-text payloads are rejected
+  until their modality adapters are independently enabled.
 - Explicit pins may now return a specific eligibility error instead of a
   provider-side malformed-request or privacy leak.
 

decisions/WF-ADR-0067-bounded-modality-contracts.md

@@ -0,0 +1,50 @@
+# WF-ADR-0067: Bounded modality capability contracts
+
+- Status: accepted for incremental implementation
+- Date: 2026-08-02
+- Roadmap: `WF-ROADMAP-0010`
+- Issue: #154
+
+## Decision
+
+Represent non-text execution as an explicit `InferenceSurface` in the shared
+runtime contract. Destination snapshots carry independent, secret-free
+capabilities for embeddings, image generation, audio input/output, and batch
+execution. The routing core applies those capabilities as hard exclusions
+before deterministic scoring; a score cannot make an incompatible destination
+eligible.
+
+The current Rust gateway advertises and executes text surfaces only. The new
+capability flags default to `false`, and the existing provider adapters do not
+opt into them. A known embeddings, image, audio, or batch-shaped payload sent
+to the text endpoint fails with `wayfinder_router_unsupported_modality` before
+provider delivery. No surface is advertised merely because a provider happens
+to accept an OpenAI-shaped request.
+
+## Incremental enablement
+
+Each surface gets its own provider adapter and parity fixtures before being
+enabled:
+
+1. embeddings — bounded input arrays, response vectors, dimensions, usage, and
+   provider-specific cost accounting;
+2. image — strict binary/input and generated-output bounds;
+3. audio — input/output format, duration, streaming, and retention bounds;
+4. batch — durable state, cancellation, expiry, reconciliation, and batch
+   accounting.
+
+The surfaces remain independently shippable. Image, audio, and batch are not
+implicitly enabled by the embeddings work, and no batch persistence exists in
+this slice.
+
+## Consequences
+
+- Swift/FFI hosts and the gateway share one capability vocabulary and stable
+  exclusion reason names.
+- Unsupported modality content cannot be silently forwarded to a text-only
+  provider or counted as a successful text turn.
+- Existing text routing remains wire-compatible; the default contract is
+  unchanged for ordinary Chat Completions and Responses requests.
+- Provider adapters must explicitly opt in only after request/response bounds,
+  concurrency/deadline policy, usage normalization, cost classification, and
+  gateway-versus-adapter parity tests are complete.

docs/gateway-config.md

@@ -269,8 +269,8 @@ may select a privacy boundary with `x-wayfinder-privacy-posture`:
 The body also contributes hard requirements: estimated prompt plus requested
 output context, image content, tool/function declarations, and streaming.
 Apple Foundation Models and the bounded ChatGPT adapter are text-only and are
-excluded for image/tool requests; OpenAI-compatible adapters retain their
-existing pass-through contract. Missing credentials, declared windows that are
+excluded for image/tool requests; the gateway's explicit modality gate also
+rejects non-text payloads before any OpenAI-compatible delivery. Missing credentials, declared windows that are
 too small, unsupported capabilities, and denied privacy boundaries are
 excluded before reliability retries or failover. Models that omit
 `context_window` retain the legacy prompt-precheck behavior. Every concrete
@@ -280,6 +280,12 @@ preset with no eligible member returns
 `422 wayfinder_router_destination_ineligible` with stable reason names rather
 than silently switching privacy boundary (WF-ADR-0056).
 
+Non-text execution surfaces are separately capability-gated. The shared
+contract names embeddings, image generation, audio, and batch support, but the
+current gateway defaults every one to unsupported. Known non-text payloads fail
+before provider delivery rather than being forwarded through a text adapter;
+see WF-ADR-0067 for the independent enablement sequence.
+
 OpenAI- and Anthropic-compatible chat endpoints preserve the same bounded
 Wayfinder control-header allowlist. In particular, privacy and offline controls
 cannot disappear when an Anthropic-shaped request enters the shared router.

docs/integrations.md

@@ -168,8 +168,14 @@ thinking are not translated yet (WF-DESIGN-0011).
   `model`, `input`, `instructions`, `stream`, `max_output_tokens`, and `temperature` contract.
   Unsupported fields fail specifically instead of being dropped; see
   [WF-ADR-0066](../decisions/WF-ADR-0066-bounded-responses-api.md).
-- **Tool calling / vision** depend on the *upstream* model you route to, not on Wayfinder —
-  the gateway forwards your request body unchanged (plus the resolved model id).
+- **Non-text surfaces** are currently fail-closed. Embeddings, image, audio, and batch payloads
+  are not advertised or forwarded until their provider adapter, request/response bounds, usage
+  accounting, and parity fixtures are enabled; known modality fields sent to the text endpoint
+  return `wayfinder_router_unsupported_modality` before provider delivery. See
+  [WF-ADR-0067](../decisions/WF-ADR-0067-bounded-modality-contracts.md).
+- **Tool calling** depends on the *upstream* model you route to, while vision/audio/image
+  surfaces remain disabled until their reviewed adapters are enabled; known non-text payloads
+  are rejected by the modality gate rather than forwarded unchanged.
 - **Per-request overrides** travel as headers (e.g. `X-Wayfinder-Threshold`), so you can
   tune routing without changing client config.
 

docs/rust-migration-capability-matrix.md

@@ -147,6 +147,7 @@ verified subset; the proposed owner is an architectural mapping, not permission
 | Config preview and feedback endpoints | `POST /router/config`, `/v1/feedback` | gateway/UI/feedback tests | `routes::{config_preview,feedback}` | Config preview is constrained/read-only; feedback may require an environment token but is open by default and persists raw prompt labels. Critical auth/storage risk. |
 | OpenAI chat aliases | `POST /v1/chat/completions`, `/chat/completions` | `test_gateway.py` | `routes::chat` | Real upstream success is relayed, routing headers added, and upstream headers mostly discarded. Bare alias is a compatibility contract. Critical wire parity. |
 | OpenAI Responses aliases | `POST /v1/responses`, `/responses` | gateway Responses contract tests, WF-ADR-0066 | `wayfinder-gateway::responses` | Strict bounded text/multi-turn adapter over the authenticated Chat Completions path; normalized usage and SSE events; unsupported fields fail specifically. Critical compatibility/bounds contract. |
+| Modality capability contracts | embeddings, image, audio, batch capability flags and hard exclusions | routing-core/FFI/gateway modality tests, WF-ADR-0067 | `wayfinder-runtime-contracts`, `wayfinder-routing-core`, `wayfinder-apple-ffi`, `wayfinder-gateway::modalities` | Non-text surfaces are independently gated and currently fail-closed before provider delivery; no surface is advertised without an adapter and parity fixtures. High compatibility/bounds risk. |
 | Request IDs and routing headers | chat handler | gateway tests | `wayfinder-gateway::response_meta` | 12 lower-case hex ID; score is two decimals; model/mode and optional served-by/failover/budget/offline/cache/decision-only plus rate headers. Critical exact-header parity. |
 | Route scope | `extract_prompt` | gateway tests, ADR-0021 | `decision_policy::scope` | `turn`, `last_user`, `user`, `all`; text parts joined; role-filter fallback to last message. High multi-turn parity. |
 | Pin, threshold, sticky, tuning, slash overrides | gateway policy helpers | gateway tests, ADRs 0011/0022/0023/0036 | `decision_policy` | Pin > slash pin > threshold/scoring/sticky, with exact accepted aliases and modes. Unknown model IDs score normally. Critical precedence parity. |

ios/WayfinderIOS/WayfinderIOS/AppModel.swift

@@ -436,7 +436,8 @@ final class AppModel {
           contextTokens: nil,
           imageInput: false,
           tools: false,
-          streaming: true
+          streaming: true,
+          surface: .text
         )
       )
       let plan: RoutePlan
@@ -1240,7 +1241,12 @@ struct RoutingDestination: Identifiable, Hashable {
         text: true,
         streaming: true,
         imageInput: false,
-        tools: false
+        tools: false,
+        embeddings: false,
+        imageGeneration: false,
+        audioInput: false,
+        audioOutput: false,
+        batch: false
       ),
       automaticEligible: automaticEligible
     )
@@ -1261,7 +1267,12 @@ struct RoutingDestination: Identifiable, Hashable {
         text: true,
         streaming: true,
         imageInput: false,
-        tools: false
+        tools: false,
+        embeddings: false,
+        imageGeneration: false,
+        audioInput: false,
+        audioOutput: false,
+        batch: false
       ),
       automaticEligible: true
     )

roadmaps/WF-ROADMAP-0010-verified-efficiency-evidence-engine.md

@@ -75,6 +75,14 @@ usage and streaming events, and fail closed on unsupported fields. Input,
 output, event, response, and accounting bounds are explicit; duplicate terminal
 events or a second route decision are not permitted. See WF-ADR-0066.
 
+Issue #154 begins the modality compatibility sequence. The shared runtime and
+Apple bridge now carry explicit embeddings, image-generation, audio, and batch
+capabilities, and the gateway rejects known non-text payloads before delivery
+while every flag remains disabled by default. Each surface will ship through a
+separate provider adapter and parity/bounds/accounting gate; batch persistence
+and reconciliation are intentionally not present in this foundation slice.
+See WF-ADR-0067.
+
 WF-ADR-0052 adds bounded workspace policy above virtual keys: shared
 process-local RPM/TPM, inherited model allowlists, stable public model aliases,
 and explicit workspace response attribution. It also fixes the enterprise

rust/crates/wayfinder-apple-ffi/src/lib.rs

@@ -118,6 +118,11 @@ pub struct DestinationCapabilities {
     pub streaming: bool,
     pub image_input: bool,
     pub tools: bool,
+    pub embeddings: bool,
+    pub image_generation: bool,
+    pub audio_input: bool,
+    pub audio_output: bool,
+    pub batch: bool,
 }
 
 /// Requirements that must be satisfied before complexity scoring.
@@ -127,6 +132,17 @@ pub struct RoutingRequirements {
     pub image_input: bool,
     pub tools: bool,
     pub streaming: bool,
+    pub surface: InferenceSurface,
+}
+
+/// Provider execution surface required by a mobile request.
+#[derive(Clone, Copy, Debug, uniffi::Enum)]
+pub enum InferenceSurface {
+    Text,
+    Embeddings,
+    ImageGeneration,
+    Audio,
+    Batch,
 }
 
 /// Bounded request supplied by an Apple host.
@@ -166,6 +182,10 @@ pub enum ExclusionReason {
     ImageInputUnsupported,
     ToolsUnsupported,
     StreamingUnsupported,
+    EmbeddingsUnsupported,
+    ImageGenerationUnsupported,
+    AudioUnsupported,
+    BatchUnsupported,
     AutomaticNotAllowed,
 }
 
@@ -461,6 +481,19 @@ impl From<RoutingRequirements> for CoreRoutingRequirements {
             image_input: value.image_input,
             tools: value.tools,
             streaming: value.streaming,
+            surface: value.surface.into(),
+        }
+    }
+}
+
+impl From<InferenceSurface> for wayfinder_routing_core::InferenceSurface {
+    fn from(value: InferenceSurface) -> Self {
+        match value {
+            InferenceSurface::Text => Self::Text,
+            InferenceSurface::Embeddings => Self::Embeddings,
+            InferenceSurface::ImageGeneration => Self::ImageGeneration,
+            InferenceSurface::Audio => Self::Audio,
+            InferenceSurface::Batch => Self::Batch,
         }
     }
 }
@@ -484,6 +517,13 @@ impl From<DestinationCapabilities> for CoreDestinationCapabilities {
             streaming: value.streaming,
             image_input: value.image_input,
             tools: value.tools,
+            modalities: wayfinder_routing_core::ModalityCapabilities {
+                embeddings: value.embeddings,
+                image_generation: value.image_generation,
+                audio_input: value.audio_input,
+                audio_output: value.audio_output,
+                batch: value.batch,
+            },
         }
     }
 }
@@ -556,6 +596,10 @@ impl From<CoreExclusionReason> for ExclusionReason {
             CoreExclusionReason::ImageInputUnsupported => Self::ImageInputUnsupported,
             CoreExclusionReason::ToolsUnsupported => Self::ToolsUnsupported,
             CoreExclusionReason::StreamingUnsupported => Self::StreamingUnsupported,
+            CoreExclusionReason::EmbeddingsUnsupported => Self::EmbeddingsUnsupported,
+            CoreExclusionReason::ImageGenerationUnsupported => Self::ImageGenerationUnsupported,
+            CoreExclusionReason::AudioUnsupported => Self::AudioUnsupported,
+            CoreExclusionReason::BatchUnsupported => Self::BatchUnsupported,
             CoreExclusionReason::AutomaticNotAllowed => Self::AutomaticNotAllowed,
         }
     }
@@ -622,6 +666,11 @@ mod tests {
                 streaming: true,
                 image_input: false,
                 tools: false,
+                embeddings: false,
+                image_generation: false,
+                audio_input: false,
+                audio_output: false,
+                batch: false,
             },
             automatic_eligible: true,
         }
@@ -638,6 +687,7 @@ mod tests {
                 image_input: false,
                 tools: false,
                 streaming: true,
+                surface: InferenceSurface::Text,
             },
         }
     }