hasna · andrei-hasna · Jul 24, 2026 · Jun 22, 2026
diff --git a/.env.example b/.env.example
@@ -7,6 +7,20 @@ ANTHROPIC_API_KEY=
 GOOGLE_GENERATIVE_AI_API_KEY=
 OPENROUTER_API_KEY=
 
+# Gateway providers
+AI_GATEWAY_API_KEY=
+LITELLM_PROXY_BASE_URL=
+LITELLM_API_KEY=
+PORTKEY_API_KEY=
+PORTKEY_CONFIG_ID=
+PORTKEY_PROVIDER=
+PORTKEY_VIRTUAL_KEY=
+CLOUDFLARE_API_TOKEN=
+CLOUDFLARE_AI_GATEWAY_BASE_URL=
+HELICONE_API_KEY=
+KONG_AI_GATEWAY_BASE_URL=
+KONG_AI_GATEWAY_API_KEY=
+
 # Chinese and China-focused providers
 DEEPSEEK_API_KEY=
 DASHSCOPE_API_KEY=

diff --git a/README.md b/README.md
@@ -1,6 +1,6 @@
 # Hasna Gateway
 
-Hasna Gateway is the open-source AI gateway core for Hasna apps and self-hosted teams. It exposes one stable OpenAI-compatible API while routing requests across providers, including OpenAI, Google Gemini, OpenRouter, DeepSeek, Qwen/DashScope, Kimi/Moonshot, Z.AI/GLM, and SiliconFlow.
+Hasna Gateway is the open-source AI gateway core for Hasna apps and self-hosted teams. It exposes one stable OpenAI-compatible API while routing requests across providers, including OpenAI, Google Gemini, OpenRouter, Vercel AI Gateway, LiteLLM Proxy, Portkey, Cloudflare AI Gateway, Helicone AI Gateway, Kong AI Gateway, DeepSeek, Qwen/DashScope, Kimi/Moonshot, Z.AI/GLM, and SiliconFlow.
 
 The open-source package is useful on its own. Anyone can run it locally or on their own server, bring their own provider keys, define routing policy, and point applications at one endpoint. The hosted Hasna gateway can build on the same core while keeping accounts, billing, pooled provider contracts, discounts, tenant policy, and hosted observability private.
 
@@ -9,7 +9,7 @@ The open-source package is useful on its own. Anyone can run it locally or on th
 - OpenAI-compatible HTTP API first, starting with `/v1/chat/completions`.
 - One gateway key for clients, many provider keys behind the gateway.
 - Bring-your-own-key mode for self-hosted users.
-- Routing by model alias, provider allowlist/blocklist, region policy, price ceilings, fallback, and capability.
+- Routing by model alias, provider allowlist/blocklist, region policy, price ceilings, fallback, capability, and smart cost/quality/latency hints.
 - Explicit China/provider policy so requests are never silently routed to a region or provider class the caller did not allow.
 - Usage normalization, estimated cost hooks, route decision metadata, and optional local JSONL usage ledger.
 - Hard or soft budgets by gateway key, tenant, and model alias across USD plus input/output/total tokens.
@@ -88,6 +88,28 @@ Required config examples:
 
 Provider keys are loaded from environment variables only. Do not put provider secrets in config files.
 
+Providers can use `baseUrl`, `baseUrlEnv`, `apiKeyEnv`, custom `auth`, and static or env-derived `headers`. This keeps OpenAI-compatible gateways on the generic adapter instead of adding hardcoded adapter forks. The built-in presets include:
+
+- Direct/provider presets: `openai`, `openrouter`, `deepseek`, `qwen`, `kimi`, `zai`, `siliconflow`.
+- Gateway presets: `vercel-ai-gateway`, `litellm-proxy`, `portkey`, `cloudflare-ai-gateway`, `helicone-ai-gateway`, `kong-ai-gateway`.
+
+Smart routing is available with route mode `smart` or request `gateway.routing: "smart"`. It filters by policy first, then scores eligible candidates using configured prices, context, capabilities, quality/latency/success/throughput hints, and deterministic fallback ordering when metrics are missing.
+
+```json
+{
+  "model": "coding",
+  "messages": [{ "role": "user", "content": "Refactor this function." }],
+  "gateway": {
+    "routing": "smart",
+    "priority": "quality",
+    "cost_quality_tradeoff": 3,
+    "required_capabilities": ["tools", "json"],
+    "min_context_tokens": 128000,
+    "sticky_session_id": "thread-123"
+  }
+}
+```
+
 Budgets live in the same JSON config and spend is calculated from the usage ledger. JSONL append through `storage.usageLedgerPath` is the local-first default. Daily, monthly, and lifetime budgets require either `storage.usageLedgerPath` or an explicit `storage.cloud` backend; per-request budgets can run without cumulative storage. Use `mode: "hard"` to block exhausted budgets with an OpenAI-compatible `402` error, or `mode: "soft"` to keep serving while exposing warnings in gateway metadata and ledger records.
 
 ### Runtime Modes
@@ -105,6 +127,8 @@ Set `runtime.mode` to `production-cloud` when running the gateway behind a cloud
 
 Production cloud mode does not create DNS, ACM, API Gateway, secrets, provider keys, or cloud infrastructure. Those deployment steps require an operator-owned deployment workflow outside this package.
 
+The companion `open-router` repo is currently documented as the future extraction point for prompt-aware routing and eval harnesses. The deterministic routing implementation lives in this package today because it is tightly coupled to gateway policy, provider config, budgets, attempts, and ledger metadata.
+
 ## Documentation
 
 - [Product requirements](docs/product-requirements.md)

diff --git a/docs/README.md b/docs/README.md
@@ -31,9 +31,21 @@ The first implementation should prioritize a small working gateway over broad in
 - Config validation.
 - Model aliases.
 - Fallback routing.
+- Smart cost/quality/latency routing.
 - Explicit provider policy.
+- Config-driven provider auth and headers.
 - Streaming.
 - Usage normalization.
 - Tests.
 
-Provider breadth should come after the request lifecycle is reliable.
+Provider breadth should stay on the generic OpenAI-compatible adapter when the upstream gateway uses standard chat completions plus headers or documented request-body provider options.
+
+## Gateway Examples
+
+- [OpenRouter Auto Router](../examples/openrouter-auto/README.md)
+- [Vercel AI Gateway](../examples/vercel-ai-gateway/README.md)
+- [Portkey AI Gateway](../examples/portkey/README.md)
+- [Cloudflare AI Gateway](../examples/cloudflare-ai-gateway/README.md)
+- [LiteLLM Proxy](../examples/litellm-proxy/README.md)
+- [Helicone AI Gateway](../examples/helicone-ai-gateway/README.md)
+- [Kong AI Gateway](../examples/kong-ai-gateway/README.md)
diff --git a/docs/api-contract.md b/docs/api-contract.md
@@ -89,15 +89,26 @@ Example:
   ],
   "stream": true,
   "gateway": {
-    "routing": "fallback",
+    "routing": "smart",
+    "priority": "quality",
+    "cost_quality_tradeoff": 3,
+    "required_capabilities": ["tools"],
+    "min_context_tokens": 128000,
     "allowed_providers": ["deepseek", "qwen", "openai"],
     "blocked_regions": ["cn"],
     "max_output_usd_per_million_tokens": 10
   }
 }
 ```
 
-The optional `gateway` field is a gateway-specific extension. It should be ignored before forwarding to providers.
+The optional `gateway` field is a gateway-specific extension. It is ignored before forwarding to direct providers. For gateway providers with documented request-body routing controls, Hasna Gateway maps only supported fields:
+
+- OpenRouter: `provider.order`, `only`, `ignore`, `sort`, `max_price`, `allow_fallbacks`, `zdr`, `data_collection`, and Auto Router plugin options such as `allowed_models` and `cost_quality_tradeoff`.
+- Vercel AI Gateway: `providerOptions.gateway.order`, `only`, `caching`, and `providerTimeouts`.
+
+Unsupported gateway-only fields and secrets are stripped.
+
+Smart routing fields include `task`, `priority`, `cost_quality_tradeoff`, `sticky_session_id`, `min_quality`, `min_context_tokens`, `expected_input_tokens`, `required_capabilities`, `provider_order`, `provider_only`, and `provider_ignore`. Policy is applied before scoring.
 
 ## Response Shape
 
@@ -129,7 +140,19 @@ Non-streaming responses should match OpenAI chat completion shape:
     "provider_model": "deepseek-chat",
     "route_mode": "fallback",
     "attempts": 1,
-    "estimated_cost_usd": 0.00012
+    "estimated_cost_usd": 0.00012,
+    "route_decision": {
+      "requested_model": "coding",
+      "selected": "deepseek/deepseek-chat",
+      "scores": [
+        {
+          "provider": "deepseek",
+          "model": "deepseek/deepseek-chat",
+          "score": 0.82,
+          "reason": "highest cost, quality, latency, and success weighted score among eligible models"
+        }
+      ]
+    }
   }
 }
 ```

diff --git a/docs/architecture.md b/docs/architecture.md
@@ -59,9 +59,12 @@ The router receives a normalized request and eligible model candidates. It choos
 - `lowest-latency`: choose lowest recent p95 or configured latency.
 - `highest-throughput`: choose provider with best recent success and throughput.
 - `balanced`: weighted score from cost, latency, success rate, and quality hints.
+- `smart`: request-tunable scoring across cost, quality, latency, success, context, and capability hints.
 
 Routing should always produce a route decision object that can be logged and tested.
 
+Policy filtering happens before scoring. The router must not let a high score route outside region, data, BYOK, credential, capability, or cost policy.
+
 ### Provider Adapter Layer
 
 Adapters convert the internal request to provider-specific requests and normalize responses back into the gateway response shape. A provider adapter owns:
@@ -130,6 +133,15 @@ Recommended internal modules:
 - `src/errors`: provider error taxonomy.
 - `src/sdk`: embeddable TypeScript API.
 
+## open-router Companion
+
+The companion `open-router` repository is intended for reusable prompt-aware routing, provider scoring, and evaluation harnesses. It is currently a companion placeholder, not a package dependency.
+
+The deterministic smart routing layer belongs in `open-gateway` for now because it depends directly on gateway config, data policy, credentials, route metadata, budgets, usage ledger records, and provider attempt accounting. Once `open-router` has reusable package code, the boundary should be:
+
+- `open-gateway`: policy, credentials, budgets, provider attempts, ledger, and OpenAI-compatible HTTP surface.
+- `open-router`: optional prompt-aware scoring/evals that receive already-policy-filtered candidates and return explainable ranking metadata.
+
 ## Existing Hasna Code To Reuse
 
 - Provider conversion ideas from `open-aicopilot`.

diff --git a/docs/provider-adapters.md b/docs/provider-adapters.md
@@ -46,6 +46,67 @@ The built-in `google` preset is intentionally conservative about data policy. Op
 
 OpenRouter can be supported as a provider adapter and as a routing backend. It should not be the only gateway strategy. Hasna Gateway should still be able to call providers directly.
 
+The current implementation keeps OpenRouter on the OpenAI-compatible adapter and maps only documented request body controls to `provider`, `plugins`, and `session_id`. This avoids an adapter fork while still supporting provider selection, ZDR, data collection, max price, and Auto Router options.
+
+## Generic Gateway Provider Config
+
+OpenAI-compatible providers and gateways can be configured without code changes:
+
+```json
+{
+  "id": "example-gateway",
+  "displayName": "Example Gateway",
+  "kind": "openai-compatible",
+  "baseUrlEnv": "EXAMPLE_GATEWAY_BASE_URL",
+  "auth": {
+    "type": "header",
+    "apiKeyEnv": "EXAMPLE_GATEWAY_KEY",
+    "headerName": "x-api-key",
+    "prefix": ""
+  },
+  "headers": {
+    "x-config-id": { "env": "EXAMPLE_GATEWAY_CONFIG_ID" },
+    "x-static": "static-value"
+  },
+  "dataPolicy": {
+    "allowTraining": false,
+    "allowLogging": false,
+    "byokOnly": true
+  }
+}
+```
+
+Supported provider config fields:
+
+- `baseUrl`: static OpenAI-compatible base URL.
+- `baseUrlEnv`: environment variable containing the base URL.
+- `apiKeyEnv`: shorthand for bearer auth.
+- `auth.type`: `bearer`, `header`, or `none`.
+- `auth.apiKeyEnv`, `auth.headerName`, `auth.prefix`: custom credential header settings.
+- `headers`: static values or `{ "env": "...", "prefix": "...", "required": true }`.
+
+Built-in gateway presets:
+
+- `vercel-ai-gateway`
+- `litellm-proxy`
+- `portkey`
+- `cloudflare-ai-gateway`
+- `helicone-ai-gateway`
+- `kong-ai-gateway`
+
+These remain normal route candidates. If the upstream gateway performs its own fallback or load balancing, Hasna Gateway records that upstream as one provider attempt unless the route config lists additional Hasna candidates.
+
+## Provider Option Mapping
+
+Direct providers receive only OpenAI-compatible request fields. Gateway-specific fields are stripped.
+
+Mapped gateway bodies:
+
+- OpenRouter: `provider.order`, `only`, `ignore`, `sort`, `max_price`, `allow_fallbacks`, `zdr`, `data_collection`, `quantizations`, `preferred_min_throughput`, and Auto Router plugin `allowed_models` / `cost_quality_tradeoff`.
+- Vercel AI Gateway: `providerOptions.gateway.order`, `only`, `caching`, and `providerTimeouts`.
+
+Portkey, Cloudflare, LiteLLM, Helicone, and Kong are supported through provider config headers/auth and OpenAI-compatible model IDs. Their own routing/load-balancing configs stay in those systems.
+
 ## Chinese Provider Priority
 
 These providers should be first-class because they are important for cost, coding, and international model access:

diff --git a/docs/provider-references.md b/docs/provider-references.md
@@ -1,6 +1,6 @@
 # 2026 Provider References
 
-These notes capture provider docs checked during project setup and rechecked during implementation on 2026-06-16. Model names change quickly, so release smoke tests should still prefer provider `/models` APIs when credentials are available.
+These notes capture provider docs checked during project setup, rechecked during implementation on 2026-06-16, and rechecked for multi-gateway routing on 2026-06-22. Model names change quickly, so release smoke tests should still prefer provider `/models` APIs when credentials are available.
 
 ## DeepSeek
 
@@ -41,3 +41,59 @@ These notes capture provider docs checked during project setup and rechecked dur
 - Auth: `Authorization: Bearer <api-key>`
 - Example model: `Pro/zai-org/GLM-4.7`
 - Usage includes normal OpenAI token fields plus reasoning and cache details on some models.
+
+## OpenRouter
+
+- Docs: https://openrouter.ai/docs/guides/routing/provider-selection
+- Auto Router docs: https://openrouter.ai/docs/guides/routing/routers/auto-router
+- OpenAI-compatible base URL: `https://openrouter.ai/api/v1`
+- Provider routing uses a `provider` object with fields such as `order`, `only`, `ignore`, `sort`, `max_price`, `allow_fallbacks`, `zdr`, and `data_collection`.
+- Auto Router uses model `openrouter/auto`; per-request Auto Router settings use an `auto-router` plugin with `allowed_models` and `cost_quality_tradeoff`.
+- Hasna Gateway maps only these documented fields and strips other gateway-only fields for direct providers.
+
+## Vercel AI Gateway
+
+- Docs: https://vercel.com/docs/ai-gateway/models-and-providers/provider-options
+- OpenAI-compatible base URL: `https://ai-gateway.vercel.sh/v1`
+- Provider options use `providerOptions.gateway` with `order`, `only`, `caching`, and `providerTimeouts`.
+- BYOK credentials are managed in Vercel AI Gateway settings; requests should not include provider secrets through Hasna Gateway.
+
+## LiteLLM Proxy
+
+- Docs: https://docs.litellm.ai/docs/routing
+- Proxy load balancing docs: https://docs.litellm.ai/docs/proxy/load_balancing
+- OpenAI-compatible proxy base URL is deployment-specific, commonly `http://127.0.0.1:4000/v1`.
+- LiteLLM owns its internal routing strategies such as weighted pick, latency-based, cost-based, and order fallback. Hasna Gateway treats the LiteLLM proxy as one upstream candidate unless route config adds additional candidates.
+
+## Portkey AI Gateway
+
+- Config docs: https://portkey.ai/docs/product/ai-gateway/configs
+- Load balancing docs: https://portkey.ai/docs/product/ai-gateway/load-balancing
+- OpenAI-compatible gateway URL: `https://api.portkey.ai/v1`
+- Gateway config selection can be passed with `x-portkey-config`; generic header auth supports `x-portkey-api-key` and optional provider/virtual-key headers.
+
+## Cloudflare AI Gateway
+
+- REST API docs: https://developers.cloudflare.com/ai-gateway/usage/rest-api/
+- OpenAI-compatible REST base URL: `https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/v1`
+- Deprecated compat base URL: `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat`
+- Auth uses a Cloudflare API token in `Authorization`.
+
+## Helicone AI Gateway
+
+- Docs: https://docs.helicone.ai/gateway/overview
+- Integration docs: https://docs.helicone.ai/gateway/integrations/overview
+- OpenAI-compatible base URL: `https://ai-gateway.helicone.ai/v1`
+- The preset uses `Helicone-Auth: Bearer <HELICONE_API_KEY>`.
+
+## Kong AI Gateway
+
+- Docs: https://developer.konghq.com/ai-gateway/
+- Load balancing docs: https://developer.konghq.com/ai-gateway/load-balancing/
+- Base URL and auth depend on the deployed Kong route and plugins.
+- Kong can perform its own load balancing, retries, fallback, and semantic routing. Hasna Gateway records Kong as one upstream attempt unless configured with additional local fallback candidates.
+
+## RouteLLM
+
+- Repo: https://github.com/lm-sys/routellm
+- RouteLLM is useful for prompt-aware routing and evaluations, but it is not embedded in this implementation. The current smart routing is deterministic and config-driven inside Hasna Gateway. The `open-router` companion repo is the future extraction point for prompt-aware/eval routing when reusable package code exists.
diff --git a/docs/routing-and-policy.md b/docs/routing-and-policy.md
@@ -11,6 +11,7 @@ The default self-hosted behavior should be conservative:
 - Do not call a provider without a configured key.
 - Do not call a hosted Hasna endpoint unless the user configured it.
 - Do not route to China-region or China-owned providers unless the route or config allows them.
+- Do not score or prefer a candidate until region, data, BYOK, credential, capability, and cost policy filters have passed.
 
 Production cloud behavior must be explicit instead of inferred from deployment context. A config with `runtime.mode: "production-cloud"` should bind to a non-loopback interface, keep gateway auth required, require runtime secrets and route readiness for `/health`, and constrain provider discovery to HTTPS provider URLs plus any exact origins listed in `runtime.serviceDiscovery.allowedProviderBaseUrls`.
 
@@ -115,6 +116,35 @@ Fallbacks should not hide:
 - User input errors.
 - Unsafe region or data policy mismatches.
 
+## Smart Routing
+
+Smart routing is an ordered policy and scoring layer:
+
+1. Resolve the requested model or alias into configured candidates.
+2. Apply policy filters first: provider allow/block lists, region, China opt-in, data retention, training/logging, BYOK, credentials, model capability, context window, and price ceilings.
+3. Score only the remaining eligible candidates.
+4. Return the route decision, skipped reasons, scores, and selected model in gateway metadata and ledger records.
+
+Supported route modes:
+
+- `fallback`: first eligible candidate in configured order.
+- `cheapest`: lowest configured input plus output token price. If no eligible candidate has prices, the route fails closed.
+- `lowest-latency`: latency-weighted score using configured `averageLatencyMs` when present.
+- `highest-throughput`: throughput and success weighted score using configured `throughputTokensPerSecond` and `successRate`.
+- `balanced`: weighted score across cost, quality, latency, and success.
+- `smart`: same score inputs as `balanced`, adjusted by request hints such as `priority` and `cost_quality_tradeoff`.
+
+Request hints under `gateway` can reduce the eligible set or tune scoring:
+
+- `priority`: `cost`, `quality`, `latency`, or `balanced`.
+- `cost_quality_tradeoff`: `0` favors quality, `10` favors cost.
+- `sticky_session_id` or `session_id`: deterministic tie-breaking for repeated conversations.
+- `required_capabilities`: capabilities such as `tools`, `json`, `vision`, or `reasoning`.
+- `min_quality` and `min_context_tokens`.
+- `provider_order`, `provider_only`, and `provider_ignore`.
+
+When configured metrics are missing, smart routing uses deterministic fallback values and original candidate order. It does not read the usage ledger inside synchronous route resolution; future runtime metric injection can pass precomputed latency/success data into routing without changing the fail-closed policy order.
+
 ## Cost Controls
 
 Cost controls should support: