Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 64 additions & 0 deletions gateways/higress/data/ingresses/bench.gen.yaml.tmpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# One Ingress, six prefix paths, one destination: each path is a wire dialect's own canonical
# route (the same paths engine ingress.rs probes), so each diagonal cell is verbatim passthrough -
# the request leaves in the dialect it arrived in, and the mock's shape recording can tell. The
# gemini and bedrock paths embed their cell's model, which the manifest declares as constants and
# mirrors into cell_paths so the probe and this route agree by construction, not by copy-paste.
# Shape mirrors the image's own all-in-one/config/ingresses/*.yaml (labels, timestamps and the
# resource-definer marker included - the file-based apiserver serves these as k8s objects).
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: bench-diagonal
namespace: higress-system
creationTimestamp: "2000-01-01T00:00:00Z"
resourceVersion: "1"
labels:
higress.io/resource-definer: higress
annotations:
higress.io/destination: mock.static
spec:
rules:
- http:
paths:
- backend:
resource:
apiGroup: networking.higress.io
kind: McpBridge
name: default
path: /v1/chat/completions
pathType: Prefix
- backend:
resource:
apiGroup: networking.higress.io
kind: McpBridge
name: default
path: /v1/responses
pathType: Prefix
- backend:
resource:
apiGroup: networking.higress.io
kind: McpBridge
name: default
path: /v1/messages
pathType: Prefix
- backend:
resource:
apiGroup: networking.higress.io
kind: McpBridge
name: default
path: /v2/chat
pathType: Prefix
- backend:
resource:
apiGroup: networking.higress.io
kind: McpBridge
name: default
path: "/v1beta/models/{HIGRESS_GEMINI_MODEL}:generateContent"
pathType: Prefix
- backend:
resource:
apiGroup: networking.higress.io
kind: McpBridge
name: default
path: "/model/{HIGRESS_BEDROCK_MODEL}/converse"
pathType: Prefix
19 changes: 19 additions & 0 deletions gateways/higress/data/mcpbridges/default.gen.yaml.tmpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# The upstream registry this run drives: the harness's mock, on the host loopback. This file
# replaces the image's own default McpBridge (whose only registry is the console at 8001); the
# all-in-one bootstrap copies its defaults into /data with cp -n, so a file that already exists at
# this path is left alone and this one wins. 127.0.0.1 reaches the mock because the harness
# launches every container with host networking - the same fact apisix's override.endpoint and
# kong's upstream_url rely on. Shape mirrors the image's own all-in-one/config/mcpbridges/default.yaml.
apiVersion: networking.higress.io/v1
kind: McpBridge
metadata:
name: default
namespace: higress-system
creationTimestamp: "2000-01-01T00:00:00Z"
resourceVersion: "1"
spec:
registries:
- domain: "127.0.0.1:{MOCK_PORT}"
name: mock
port: {MOCK_PORT}
type: static
68 changes: 68 additions & 0 deletions gateways/higress/definition.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
{
"auth": "dummy",
"cell_paths": {
"bedrock>bedrock": "/model/{HIGRESS_BEDROCK_MODEL}/converse",
"gemini>gemini": "/v1beta/models/{HIGRESS_GEMINI_MODEL}:generateContent"
},
"class": "API gateway",
"config": [],
"config_files": [
{
"output": "data/mcpbridges/default.gen.yaml",
"template": "data/mcpbridges/default.gen.yaml.tmpl"
},
{
"output": "data/ingresses/bench.gen.yaml",
"template": "data/ingresses/bench.gen.yaml.tmpl"
}
],
"constants": {
"HIGRESS_BEDROCK_MODEL": "anthropic.claude-3-sonnet-20240229-v1:0",
"HIGRESS_GEMINI_MODEL": "gpt-4o-mini-gemini"
},
"display": "Higress",
"egress": [
"openai",
"openai-responses",
"anthropic",
"gemini",
"cohere",
"bedrock"
],
"lang": "Other",
"launch": {
"args": [],
"image": "higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/all-in-one:2.2.4",
"kind": "docker",
"mounts": [
{
"container_path": "/data/mcpbridges/default.yaml",
"host_path": "data/mcpbridges/default.gen.yaml",
"read_only": true
},
{
"container_path": "/data/ingresses/bench.yaml",
"host_path": "data/ingresses/bench.gen.yaml",
"read_only": true
}
]
},
"matrix": [
"100000",
"010000",
"001000",
"000100",
"000010",
"000001"
],
"matrix_note": "Higress 2.2.4 runs as the official all-in-one image with file-based config: one McpBridge static registry aimed at the mock and one six-path Ingress to it, so every served cell is same-dialect verbatim passthrough on the diagonal. The cross-dialect cells need the ai-proxy wasm plugin's per-ingress provider bindings (WasmPlugin matchRules), which this one-static-config manifest has not been verified against, so they are grey by that configuration limit rather than measured false. Upstream selection by request header - the mechanism other entrants use to light up cross columns on one path - is not configured because the Ingress resource Higress's file config speaks has no header matcher.",
"model": "gpt-4o-mini",
"name": "higress",
"path": "/v1/chat/completions",
"port": 8080,
"repo": "https://github.com/alibaba/higress",
"runtime": {
"container": "higress-bench",
"kind": "docker"
}
}
7 changes: 7 additions & 0 deletions gateways/higress/env
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# The all-in-one image starts Envoy with --concurrency=${GATEWAY_CONCURRENCY:-16}, so the image
# default is 16 workers regardless of how many cores the container is pinned to - on the 4 pinned
# cores that is 12 extra spinning workers fighting the mock and the load generator. This is the
# gateway's OWN knob (not one of the runtime-standard names the harness sets centrally for every
# entrant), so it is declared here, from {NCORE}, exactly once.
GATEWAY_CONCURRENCY={NCORE}
USE_PLUGIN_SERVER=off
67 changes: 67 additions & 0 deletions gateways/higress/why
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Why each declared setting exists. One claim per setting; the config lint fails on any setting
# that is declared and not claimed here.

image higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/all-in-one:2.2.4
The project's official standalone artifact (one container: file-based apiserver + controller +
pilot + Envoy gateway + console), pinned to the 2.2.4 release. Verified multi-arch against the
registry manifest on 2026-08-25: linux/amd64 + linux/arm64, so it runs natively on both arches
this benchmark fields. The registry is Aliyun Hangzhou; a pull from an AWS box is slow but
unauthenticated and works.

port 8080
The image's own GATEWAY_HTTP_PORT default. Nothing in this harness depends on the number; we do
not restate the env var, we just bind the gateway where it already listens.

env GATEWAY_CONCURRENCY={NCORE}
bind. start-gateway.sh launches Envoy with --concurrency=${GATEWAY_CONCURRENCY:-16}; the image
default of 16 workers on the 4 pinned cores would oversubscribe the box the mock and load
generator share. This is the gateway's own knob (the harness sets only runtime-standard names
like GOMAXPROCS centrally), so it is declared per-gateway, from {NCORE}, exactly as the manifest
spec's one exception allows.

env USE_PLUGIN_SERVER=off
measure. The all-in-one image defaults this to on, which keeps the built-in plugin server - a
resident nginx listening on 8002 inside the container - serving Wasm plugin images to Envoy over
local HTTP; base.sh normalizes off/false/no to off, and with it off start-plugin-server.sh exits
at boot. This manifest configures no Wasm plugin anywhere - no WasmPlugin resource, no plugin
annotation on the Ingress, and the McpBridge is used purely as a static registry - so that nginx
can never answer a request, and the project's own deploy-by-docker table says the alternative
source it enables (the public image server) is only consulted when a Wasm plugin must be
fetched, which nothing here ever does. This is the manifest's one turn-off of a default-on
component, made on the same minimality principle the rest of the config lives by: the process
exists to serve a capability this deployment does not configure, and leaving it on would count
an idle nginx's RSS in figures that already honestly carry the console JVM's.

mount data/mcpbridges/default.gen.yaml -> /data/mcpbridges/default.yaml (read-only)
upstream. The all-in-one bootstrap (start-apiserver.sh) copies its defaults into /data with
cp -n, so an existing file at this path survives and ours - pointing the sole static registry at
the mock - is the config the controller actually loads. Read-only because the gateway has no
reason to rewrite its own input, and a writable mount would let a run mutate what the next run
reads. The rest of /data stays container-writable, which the bootstrap needs.

mount data/ingresses/bench.gen.yaml -> /data/ingresses/bench.yaml (read-only)
ingress. One Ingress with the six canonical dialect paths, all to mock.static: each is a prefix
route in the dialect's own shape, so every served cell is same-dialect passthrough. The gemini
and bedrock paths embed their cell's model constants, and cell_paths mirrors the same constants
into the probe path, so route and probe agree by construction.

matrix: the diagonal only
This manifest configures no ai-proxy plugin, so it makes no translation claim: an openai request
leaves as openai, /v1/messages in leaves as /v1/messages. The cross-dialect cells would need a
WasmPlugin with per-ingress provider bindings; that is a real capability of the project but one
this one-static-config manifest has not been verified against, and an unverified green is the
one thing this benchmark refuses to publish. They are grey (undeclared), not red.

auth dummy
The routes forward whatever credential the client presents; the mock accepts any. The value
satisfies the manifest's own field, it is not a secret.

RSS includes the console JVM
MODE=full is required for a self-contained container (MODE=gateway hangs waiting for a pilot that
is not running), and full mode includes the console - a JVM. The memory figures therefore report
the whole all-in-one process tree, which is the honest cost of running Higress the way this
artifact ships; quoting an Envoy-only number would describe a deployment that does not exist.

lang Other
The data plane is Envoy (C++) and the control plane is Go; the board colours by one
implementation language, and mixed does not get to pick the flattering half.
Loading