Skip to content

AgentSandbox: optional modelEgress: gateway (the direct-runtime + governed-egress quadrant) #38

Description

@stxkxs

What

Add an optional modelEgress: direct | gateway field to AgentSandboxSpec
(default direct = current behavior). When gateway, a direct/sandbox agent
routes its model calls through the per-Platform agentgateway instead of
straight to Bedrock — so an attributable runtime (its own tenant IRSA, runs
its own loop) still gets governed, cost-observable egress (per-route model
pin, Guardrails, rate-limit, prompt-cache/cost metrics).

Why

The platform has two agent-runtime shapes today, and they sit on the two
matching corners of a 2-axis space:

  • AgentFleet = kagent runtime + agentgateway egress (managed/governed).
  • AgentSandbox = direct runtime (fab SdkRuntime) + direct-to-Bedrock egress
    (ephemeral/attributable).

The two axes — where the loop runs (kagent | direct) and where models egress
(gateway | direct) — are independent, but egress is currently coupled to the
runtime by CRD type. The valuable missing corner is direct + gateway: today
the attribution-sensitive path is also the spend/guardrail-blind path, which is
backwards. The opposite off-diagonal (kagent + direct) has no real use case and
is explicitly out of scope.

Keep the two CRDs separate — they model genuinely different lifecycle shapes
(always-on autoscaled fleet vs. ephemeral session pod). This is not a merge
into one CRD with a runtime: switch, and not a new runtime: axis field.
It's one optional egress field on the existing sandbox.

Current state (why it doesn't work today)

  • AgentSandboxSpec has no egress field — operators/api/agents/v1alpha1/agentsandbox_types.go:21.
  • The sandbox NetworkPolicy permits only DNS + broad HTTPS (0.0.0.0/0 except
    the metadata endpoint, TCP 443) — sandboxEgressRules(),
    operators/internal/controller/hardened_pod.go:73. It does not allow the
    in-cluster agentgateway:8080 hop, so even pointing fab at the gateway baseUrl
    would be blocked at the NetworkPolicy. The missing field is necessary but not
    sufficient — the NP is the real blocker.
  • The gateway URL + egress rule live only on the fleet path: gwHost
    construction operators/internal/controller/agentfleet_reconcile.go:163,
    ModelConfig baseUrl :189, fleet NP egress to agentgateway:8080
    (ensureFleetNetworkPolicy) :132-137.

Shape of the change (operator side)

  1. Add to AgentSandboxSpec (after Env): // +kubebuilder:validation:Enum=direct;gateway
    // +kubebuilder:default=direct ModelEgress string. Regenerate CRDs.
  2. Lift the gateway host/route URL construction out of agentfleet_reconcile.go:163/189
    into a shared helper (e.g. gatewayBaseURL(platform, route)) both reconcilers call.
  3. When ModelEgress == "gateway": in the sandbox reconciler — (a) append an
    egress rule to agentgateway:8080 to the pod's NetworkPolicy (a sandbox
    variant of sandboxEgressRules() that adds the agentgateway-namespace TCP 8080
    peer), and (b) inject the gateway baseUrl into the pod env so fab's runtime
    egresses through it. Default direct leaves the NP + env exactly as today.
  4. Optional tightening: when gateway, the broad 0.0.0.0/0:443 rule can be
    dropped (the agent no longer needs direct internet for models). Leave as a
    follow-up decision; don't touch default-path behavior.

Open question / cross-repo (do NOT couple)

agentgateway exposes an OpenAI-compatible endpoint; fab's SdkRuntime egresses via
FAB_INFERENCE (anthropic / anthropic-aws / bedrock). The direct+gateway path
needs fab to honor an OpenAI-compatible base URL for model calls — a fab
capability, tracked separately. The operator change here only opens the NP +
injects the env; it must not import or assume anything fab-version- or
consumer-specific.

Consumer (independent)

The first consumer is the crossbearing accountability demo (attributable agent
identity + governed/observable egress in one path). This stays a standalone
platform capability
— no crossbearing dependency lands in eks-agent-platform;
crossbearing consumes the capability, nothing more. Gate the build on that demo
need.

Out of scope

  • Merging AgentFleet/AgentSandbox, or adding a runtime: kagent | direct field.
  • The kagent + direct off-diagonal.
  • Making direct the tenant chart default (the chart still emits AgentFleet).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions