Self-hosted AI gateway — route, observe, and control your LLM API traffic.
One binary. Zero dependencies. Full dashboard. No vendor lock-in.
Why • Quick Start • Dashboard • Features • Configuration • Contributing
One endpoint for all providers. Protocol translation between OpenAI and Anthropic. Cost-based smart routing. Per-user budgets. Full request inspector.
Save Money — Smart routing picks the cheapest provider for each model. Set monthly budgets per user. See exactly where every dollar goes.
Stay Simple — One endpoint for all providers. Send OpenAI-format requests to Claude, or Anthropic-format to GPT. Protocol translation is transparent.
Keep Control — Scoped API keys, provider-level access control, full request logging, and separate admin/user dashboards give the right view to the right person.
Zero Ops — Single Node.js process, embedded SQLite, no Redis, no Postgres, no Docker required. Just npx tokenparty and you're running.
┌─────────────┐ ┌──────────────┐ ┌──────────────────┐
│ Your Apps │──────▶│ TokenParty │──────▶│ OpenAI / Claude │
│ (any SDK) │◀──────│ Gateway │◀──────│ / DeepSeek / ... │
└─────────────┘ └──────────────┘ └──────────────────┘
│
┌─────┴─────┐
│ Dashboard │
└───────────┘
npx @tokenparty/tokenpartynpm install -g @tokenparty/tokenparty
tokenparty# Create docker-compose.yaml, then:
docker compose up -ddocker-compose.yaml
services:
tokenparty:
image: nfqlt/node22
entrypoint: sh -c "npm install -g @tokenparty/tokenparty && tokenparty"
ports:
- "3456:3456"
volumes:
- npm_global:/usr/local
- ./tokenparty-data:/root/.tokenparty
restart: unless-stopped
volumes:
npm_global:Open http://localhost:3456 — configure providers and tokens from the dashboard.
Point any OpenAI-compatible SDK at your gateway:
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:3456/v1",
api_key="tp-***"
)
# Routes to the cheapest provider automatically
response = client.chat.completions.create(
model="claude-sonnet-4-20250514",
messages=[{"role": "user", "content": "Hello!"}]
)Or use the Anthropic SDK:
import anthropic
client = anthropic.Anthropic(
base_url="http://localhost:3456/anthropic",
api_key="tp-***"
)
# GPT-4o through the Anthropic SDK — protocol converted transparently
message = client.messages.create(
model="gpt-4o",
max_tokens=1024,
messages=[{"role": "user", "content": "Hello!"}]
)
Admin Portal — Overview with cost/usage charts, request inspector with full prompt/response detail, provider management with groups, user & budget management, settings.
User Portal — Personal cost dashboard, budget progress, cache hit rate, model-level breakdown, request history.
Agent Aware — Automatically detects Claude Code, OpenClaw, and other AI agents. Shows per-agent usage breakdown in the dashboard.
| Category | Feature |
|---|---|
| Routing | Cost-based smart routing across providers |
| Routing | Provider fallback & automatic retry |
| Routing | Multi-key load balancing (round-robin) |
| Routing | Model-level priority with ordered fallback chain |
| Protocol | OpenAI ↔ Anthropic bidirectional conversion |
| Protocol | Full SSE streaming with protocol translation |
| Protocol | OpenAI Chat, Responses, and Models API support |
| Cost | Per-model pricing configuration (input/output/cache) |
| Cost | Monthly budget enforcement per user |
| Cost | Cost analytics by user, provider, model, and tag |
| Cost | Cache hit rate tracking |
| Cost | USD/CNY dual currency support |
| Access | Scoped API keys with provider-level access control |
| Access | Provider groups for bulk access rules |
| Access | Usage quotas per token |
| Access | Admin authentication |
| Observability | Real-time usage dashboard with charts |
| Observability | Full request/response JSONL audit logs |
| Observability | Route trace — see exactly how each request was routed |
| Observability | Custom tags via x-tkparty-tags header |
| Observability | Request filtering by user, provider, model, status, tags |
| Observability | AI agent detection (Claude Code, OpenClaw) |
| Operations | Hot-reload config (no restart needed) |
| Operations | Environment variable interpolation in config |
| Operations | Log storage management with auto-cleanup |
| Operations | Version update check |
| Operations | Keep-alive connection pooling |
| Deployment | npm / npx — single command, zero config |
| Deployment | Docker / docker-compose ready |
| Deployment | Zero external dependencies (SQLite, no Redis/Postgres) |
| UX | Separate admin & user portals |
| UX | Multi-account quick switching |
providers:
- id: anthropic-main
type: anthropic
name: "Anthropic"
apiKey: ${ANTHROPIC_API_KEY}
baseUrl: https://api.anthropic.com
group: production
models:
- id: claude-sonnet-4-20250514
inputPrice: 3 # $ per 1M tokens
outputPrice: 15
cacheReadPrice: 0.3
- claude-opus-4-20250514
- id: openai-main
type: openai
name: "OpenAI"
apiKey: ${OPENAI_API_KEY}
baseUrl: https://api.openai.com/v1
group: production
models:
- gpt-4o
- gpt-4o-mini
tokens:
- key: tp-team-alice
name: "Alice"
allowedProviders: ["group:production"]
monthlyBudget: 100 # USD — enforced by the proxy
enabled: true
- key: tp-team-bob
name: "Bob"
allowedProviders: [openai-main]
monthlyBudget: 50
enabled: trueUnconfigured model prices are treated as free and routed with highest priority. Environment variables in ${VAR} format are resolved at startup.
TokenParty/
├── packages/
│ ├── proxy/ # Hono reverse proxy (TypeScript)
│ │ └── src/
│ │ ├── adapters/ # OpenAI ↔ Anthropic translators
│ │ ├── proxy/ # Auth, routing, forwarding
│ │ ├── metrics/ # Usage collection → SQLite
│ │ ├── routes/ # Admin & user API handlers
│ │ ├── tags/ # Agent detection & tag extraction
│ │ └── store/ # Database & log writer
│ └── dashboard/ # React + Vite + Tailwind + Recharts
│ └── src/
│ ├── layouts/ # Admin & User layouts
│ └── pages/ # Overview, Requests, Providers, Users, Settings
| Component | Tech | Role |
|---|---|---|
| Gateway | Hono + Node.js | Reverse proxy with streaming SSE |
| Storage | better-sqlite3 (WAL) | Usage aggregation & request index |
| Logs | JSONL files | Full request/response audit trail |
| Dashboard | React 19 + Vite 6 | Admin & user monitoring UI |
| Config | YAML + chokidar | Hot-reloadable setup |
- Add providers — Configure your OpenAI, Anthropic, DeepSeek, etc. API keys in the dashboard or YAML
- Create tokens — Generate scoped API keys for your team with budgets and provider access rules
- Point your apps — Any OpenAI or Anthropic SDK, just change the
base_url - Observe — Watch costs, usage, and full request traces in real-time
- npm package & Docker deployment
- Multi-provider support with load balancing
- OpenAI ↔ Anthropic protocol translation
- Full SSE streaming with protocol conversion
- Cost-based smart routing
- Per-user monthly budget enforcement
- Separate admin & user portals
- Multi-account quick switching
- Real-time cost & usage dashboard
- Request inspector with full audit trail
- Custom tags & filtering
- Provider groups & access control
- Provider fallback / retry
- Log storage management
- AI agent detection & per-agent detail panel (Claude Code, OpenClaw)
- Model-level priority with ordered fallback chain across providers
- Route trace — visual routing diagram for each request
- Upstream model auto-discovery
- Connection pooling (keep-alive HTTP agents) — prevents TIME_WAIT port exhaustion
- Provider config validation (rejects invalid baseUrl etc. at save time)
- Dashboard copy-as-cURL on request route trace
- Shareable request URLs (filters, pagination, and detail id in URL)
- Automatic prompt cache optimization (Anthropic cache_control injection)
- Cost savings report ("TokenParty saved you $XX")
- Tag-based cost analysis (cost breakdown by project/feature)
- Usage alerts (webhook/email at budget thresholds)
- Model downgrade strategy (budget-aware auto-routing)
- Rate limiting enforcement
- Session tracking
- Export billing reports (CSV/PDF)
See the full roadmap and open issues for what's next.
| TokenParty | LiteLLM | One API | Portkey | |
|---|---|---|---|---|
| Self-hosted | ✅ | ✅ | ✅ | ❌ (SaaS) |
| Language | TypeScript / Node.js | Python | Go | — |
| Zero dependencies | ✅ (SQLite, no Redis/PG) | ❌ (Postgres/Redis) | ✅ | — |
| Protocol translation | OpenAI ↔ Anthropic | OpenAI only | OpenAI only | OpenAI only |
| Built-in dashboard | ✅ (React) | ❌ (separate) | ✅ (basic) | ✅ |
| Per-user budgets | ✅ | ❌ | ✅ (basic) | ✅ |
| Agent detection | ✅ | ❌ | ❌ | ❌ |
Contributions are welcome! See CONTRIBUTING.md for setup and guidelines.
MIT © 2026 Zhou Zhengchang
Built with way too many tokens.