Bound connected-service operation exposure before LLM tool catalog materialization

[louis4li]

Background

Channel runtime can fail while preparing the initial LLM turn when a broad connected-service OpenAPI surface is expanded into too many LLM-visible tools:

Aevatar.AI.Abstractions.ToolProviders.AgentTurnToolCatalogException:
Final tool catalog exceeds its typed safety budget
(schema_bytes=123316/49152, connected_reads=341/2147483647, connected_writes=151/2147483647)

This is not primarily a schema budget size problem. The current path effectively treats selected connected-service API surface as final LLM tool catalog material:

ChannelRuntimeConfigProof
-> resolve tool set refs
-> discover tools
-> select visible route tools
-> select connected operations by service selector / endpoint selector
-> union all selected names
-> build AgentTurnToolCatalog
-> proof budget check

On current feature/integrate, the important behavior is:

• NyxIdConnectedServiceToolSource materializes matching service endpoints into IAgentTool instances.
• ChannelRuntimeToolCatalogMaterializer then selects connected operation names by service selector / endpoint_names.
• NyxidDefault + no selectors selects every visible connected operation.
• A selector with empty endpoint_names selects every matching operation for that service.
• AgentTurnToolCatalog finally sums canonical schema bytes and rejects the catalog if it exceeds the ordinary budget.

For broad services such as Google Workspace, one service may include Drive, Calendar, Gmail, and other endpoint groups. Treating the service catalog as a per-turn LLM tool catalog can exceed context and safety budgets even when the service authorization itself is legitimate.

Problem

The current design conflates two different concepts:

connected service capability catalog
==
LLM-visible per-turn tool catalog

That creates several issues:

• Service-level selection can implicitly expose hundreds of endpoint tools.
• Endpoint schemas are materialized before operation narrowing.
• Budget enforcement happens as a final reject instead of guiding catalog reduction.
• The model sees too many similar tools, increasing choice noise.
• Write/destructive operation exposure is wider than necessary even when approval exists later.
• Execution authority is still too coupled to provider tool-calling shape rather than Aevatar command/event/actor execution.

Recommended Direction

Use Option 1 + Option 5 as the near-term migration layer, then progressively move toward Option 3 as the long-term execution model.

In other words:

Option 1 + Option 5 = first production-safe step
Four phases = complete migration path from current exact tools to typed command/event execution
Option 3 = long-term architecture target

Short term: keep exact LLM tools, but only after server-side operation selection has narrowed the operation set.

Long term: connected-service execution should be owned by typed command/event/actor lifecycle. Tool calling should become an interaction adapter, not the authoritative execution model.

Phase 1: Select Before Materializing Exact Tools

Goal: preserve the current exact-tool LLM UX while only materializing a bounded selected operation set.

OpenAPI / NyxID catalog
-> lightweight operation index cards
-> runtime operation selection
-> materialize selected exact tools only
-> AgentTurnToolCatalog budget proof

Operation index cards can include:

service_slug
operation_id / endpoint_id
method + path
tag/group
summary
short description
readOnly / write / destructive
requiresApproval
input field names
schema digest
required auth scopes

Completion requirements:

• Connected service read/write operations still execute.
• Final catalog contains only bounded selected operations.
• endpoint_names limits materialization, not just final selected names.
• NyxidDefault + no selectors no longer means expose every visible connected operation.
• Empty endpoint_names has explicit bounded semantics: disallow, use runtime selection, or return disambiguation.
• Over-budget connected-service selections return structured diagnostics instead of surfacing as reply-generation exceptions.

Phase 2: Selected Operation Set As Server Authority

Goal: the model sees a projection of the server-selected operation set; the selected set is the authority.

operation selector
-> selected operation set + digest/proof
-> exact tools carry selected operation proof
-> tool call validates operation belongs to selected set
-> NyxID proxy execution

Completion requirements:

• Each turn has a selected operation set digest/proof.
• Tool calls must target operations in the selected set.
• Operation id, method, path, request schema, and risk policy come from server-side proof.
• The model cannot construct arbitrary URLs, methods, headers, or unvalidated request bodies.
• Validation failures are explicit business errors that can be repaired or clarified.
• Approval, risk, and audit use selected operation proof rather than model-provided names.

Phase 3: Exact Tool As Command Adapter

Goal: exact tools remain available to the LLM, but tools no longer execute connected-service side effects directly. They submit typed commands.

model calls exact selected tool
-> tool adapter submits OperationInvocationCommand
-> actor owns execution lifecycle
-> actor emits result / receipt events
-> response observes result

Completion requirements:

• Add typed protobuf command/event/result contracts.
• Actor owns idempotency, approval, retry, timeout, execution result, and failure state.
• Read operations return bounded results.
• Write operations return durable receipts.
• Audit, approval, and failure results are part of command/event lifecycle.
• The old exact tool path becomes an adapter and no longer owns execution authority.

Phase 4: Structured Operation Request Path

Goal: high-risk, workflow, and skill scenarios do not need provider tool-calling. The model emits a structured operation request that runtime validates and dispatches.

model emits structured operation request
-> validate selected operation proof
-> dispatch OperationInvocationCommand
-> actor executes
-> result through event/readmodel observed

Completion requirements:

• Structured output validation and repair are supported.
• Missing arguments can trigger clarification.
• Selected operation proof validation still applies.
• Structured request path and exact tool adapter share the same command/event execution trunk.
• Tool calling remains available as an interaction adapter, not as the core execution model.

Non-goals

• Do not raise MaximumSchemaBytes as the primary fix.
• Do not pass raw OpenAPI specs to the LLM on every turn.
• Do not let the model construct URLs, HTTP methods, headers, or unvalidated request bodies.
• Do not treat service authorization as equivalent to per-turn execution authorization.
• Do not add an unconstrained generic invoke tool as an escape hatch.
• Do not make provider-specific tool calling the only connected-service execution abstraction.

Implementation Notes

Near-term work should focus on Phase 1:

• Add an operation index / candidate selection step before connected-service tool materialization.
• Apply endpoint filters before materialization.
• Replace NyxidDefault + no selectors => all visible connected operations with bounded semantics.
• Replace empty endpoint_names => all service operations with bounded selection, explicit opt-in, or disambiguation.
• Add diagnostics for selected endpoint count, read/write count, schema bytes, and largest schemas.
• Convert catalog-over-budget in channel materialization into structured diagnostics.

Acceptance Criteria

• Broad connected services do not default to materializing all endpoints into the final LLM tool catalog.
• Final AgentTurnToolCatalog schema bytes stay within the ordinary budget for broad service registrations.
• Explicit endpoint allowlists reduce candidates before materialization.
• Empty selector semantics are covered by tests.
• Selection misses return structured diagnostics or clarification instead of runtime catalog exceptions.
• Existing connected-service read/write happy paths still work.
• Approval, risk, audit, and validation contracts are not weakened.
• Tests cover:
  • explicit endpoint_names materializes only selected endpoints;
  • empty endpoint_names does not expose all service operations by default;
  • nyxid_default without selectors does not expose all connected operations by default;
  • over-budget connected service selection returns diagnostics rather than a reply-generation exception;
  • selected operation proof blocks unselected operation invocation.