gate

Hellas Gate is a user-aligned llm inference server/proxy/router

in-progress

some of most of this exists, read as a wishlist rather than supported though

Private local inference via built-in catgrad engine
End-to-end encrypted peer-to-peer mesh overlay to aggregate capacity across networked nodes
Use LetsEncrypt to generate a private cert for a public https://<your-node-id>.private.hellas.ai endpoint, Hellas will relay it over p2p
Upstream support for any openai/anthropic-compatible providers, openrouter, vllm, ollama etc
Supports Cursor, Codex and Claude-Code, open-webui, charm, etc via local http server
API key management, cost-tracking, rate-limiting, quotas, permissions, etc
Smart router can optimize routing for cost, latency, etc via 'virtual models'
Capture/Log/Export all requests, responses, metadata through the gateway

Once catgrad ZK backend is implemented, we can support verifying responses- check request was serviced correctly without quantization, context injection, tampered weights, etc
Once protoproto consensus protocol is implemented, we can support settlement and thus create decentralized, trustless and permissionless markets for llm inference