demos: per-app token budgets (mixed sliding_window/token_bucket) on real k8s - #7
Open
jordigilh wants to merge 13 commits into
Open
demos: per-app token budgets (mixed sliding_window/token_bucket) on real k8s#7jordigilh wants to merge 13 commits into
jordigilh wants to merge 13 commits into
Conversation
|
Non-conforming commit subjects (expected
Amend with |
|
PR too large: 2165 lines added (limit: 750, excludes Cargo files, tests, docs, examples, and benchmarks). Please split into smaller PRs. Add |
Author
k8s-real-pods-token-rate-limit.mp4 |
jordigilh
force-pushed
the
jordigilh/token-rate-limit-per-app-demo
branch
from
August 20, 2026 15:45
8c44864 to
45fd2ac
Compare
praxis-bot
reviewed
Aug 20, 2026
praxis-bot
left a comment
praxis-bot
left a comment
Collaborator
There was a problem hiding this comment.
PR Review
Thorough demo with excellent documentation and evidence-backed recording. The README, config, k8s manifests, and dashboard are well-structured, and the end-to-end verification approach (live requireGate assertions during recording, not after the fact) is strong. The RBAC scoping is minimal and appropriate. A few findings below.
Findings
[Large] recording/slides/proof-agenda.html loads Red Hat brand fonts from Google Fonts (Red Hat Display, Red Hat Text, Red Hat Mono) and uses Red Hat's brand palette (--rh-red-50: #ee0000) with CSS comments explicitly referencing redhat.com/en/about/brand/standards. The PR description and RECORDING.md state the dashboard is "deliberately kept free of any company branding since this repo is meant for community/contributor adoption," but the intro slide deck that opens the recording video is fully Red Hat-branded. If the goal is an unbranded community repo, the slides should use the same generic navy/cyan palette as the dashboard, or the PR description should clarify that the intro slides are intentionally branded while only the dashboard is generic.
[Medium] k8s/04-apps.yaml _stream_gateway_logs reads the Kubernetes pods/log HTTP response one byte at a time (resp.read(1) in a while True loop, line-buffered manually). This is correct but needlessly expensive on CPU for a streaming endpoint. Using for line in resp: or resp.readline() would achieve the same line-by-line behavior with buffered I/O and significantly less overhead.
[Medium] k8s/04-apps.yaml do_POST catches urllib.error.HTTPError but not urllib.error.URLError. If a gateway pod is unreachable (connection refused, DNS failure, timeout), the handler raises an unhandled exception, which crashes the request and returns a raw 500 traceback to the dashboard. Catching URLError and returning a structured JSON error (e.g. {"status": 503, ...}) would keep the dashboard functional during partial failures and make the demo more resilient to pod scheduling races at startup.
praxis-bot
reviewed
Aug 22, 2026
praxis-bot
left a comment
praxis-bot
left a comment
Collaborator
There was a problem hiding this comment.
Re-review (2 new commits)
The RECORDING.md branding-rationale removal is clean -- no orphaned references. One new finding in the added "Run on Kubernetes (kind)" section.
Findings
|
Unsigned commits: faaee4e. Please sign your commits. |
Member
|
@jordigilh mind a rebase and sign the commit. Ty! |
Adds a single-process demo of the token_rate_limit filter's bucket_key_header option (ai#129): independent per-app token budgets on one gateway, reservation-based admission, and reconciliation against actual usage. Points to the source branch on jordigilh/ai rather than vendoring the filter code here, and calls out the token-bucket-vs-sliding-window divergence from ai#658's current design doc explicitly, since that question is still unresolved on the proposal's review thread. Signed-off-by: Jordi Gil <jgil@redhat.com>
Rework the per-app token budgets demo around the "final scenario" for ai#129: two independent Praxis AI gateway instances (docker compose, Valkey, a minimal stub backend) sharing one sliding-window budget per app, rather than a single in-process gateway. Verified live: an app's budget exhausted on one gateway instance is denied on the other, while an unrelated app's budget on the same instance is unaffected. Updates config.yaml to the window/capacity schema (replacing the now-removed rate/burst token-bucket fields) and adds a backend: valkey block, matching the source branch's sliding-window rewrite. Rewrites the README around the two-instance topology and adds an "Alternative implementations considered" section (Envoy-style external rate-limit service, the token-bucket predecessor of this same demo, the sibling Grid-routing demo's broader scope, and in-process-only state) so the design trade-offs are visible without re-deriving them from source. Signed-off-by: Jordi Gil <jgil@redhat.com>
Shortens the demo window to 10s so a full deny -> recover cycle is watchable, adds a 6th scenario step (app-a re-admitted once its earlier reservation ages out of the window), and records a new narrated walkthrough proving it end-to-end against the live stack. Also adds a static dashboard + nginx reverse proxy (recording-only convenience, not part of the Praxis AI filter chain) and gitignores for local Cursor/Serena tooling. Signed-off-by: Jordi Gil <jgil@redhat.com>
praxis#551 now has confirmation that mixed per-app algorithm choice (sliding window for one app, token bucket for another, same deployment) is a real requirement -- update the open design questions to reflect that the direction is settled even though this demo's source branch still only implements sliding window. Also add a concrete grid#83 alignment checklist (met/not met/not exercised/config gap) now that grid#83 is published as the authoritative Valkey-backed quota spec, point the recording section at the actual committed file name, and make explicit that the source branch isn't upstream yet so "build from this branch" isn't optional. Signed-off-by: Jordi Gil <jgil@redhat.com>
Extends the per-app-budgets demo to exercise the source branch's newly landed per-rule algorithm choice (ai#789/praxis#551) instead of the single fixed sliding-window algorithm this demo previously used unconditionally: - config.yaml: two rules matched by a new x-tier header -- gold-tier (sliding_window, app-a/app-c, unchanged behavior/timing from before) and silver-tier (token_bucket, app-b, new), both Valkey-backed in the same namespace. - dashboard/index.html: sends x-tier alongside x-app-id (required now that config.yaml matches on it), labels each app's tier/algorithm, and extends the scripted scenario to also prove app-b's budget is shared across gateway replicas for token_bucket, then recovers via continuous refill rather than a window slide. - README.md: rewritten walkthrough, architecture diagram, and curl commands reflecting both rules; "Current scope" and "Open design questions" updated now that per-rule algorithm choice is implemented on the source branch (still not merged upstream); flags that the branch's commits aren't pushed to the fork yet and that the compose stack itself wasn't re-verified in this environment (a local container-runtime issue), only the built gateway binaries directly against a live Valkey. The recorded video (recording/output/sliding-window-based-token-rate- limit.mp4) still only covers the previous single-algorithm scenario -- flagged explicitly in the README pending a re-record. Signed-off-by: Jordi Gil <jgil@redhat.com>
Record the mixed-algorithm (sliding_window + token_bucket) scenario end to end against a live two-gateway + Valkey stack, then redo the dashboard after review feedback that the first cut front-loaded all 7 requests into the first ~15s of a ~77s narration and then sat static for the rest. dashboard/index.html now paces each request against narration.srt's cue timestamps so on-screen action tracks the narration for the full clip, and renders a live per-app gauge plus a 30s rolling sparkline driven by a client-side model that replays the same reserve/reconcile arithmetic as the Lua scripts -- so gold-tier's sliding-window "flat until the window slides" recovery and silver-tier's token-bucket "continuous ramp" recovery are visually distinct, not just narrated. Every transition the model predicts is still cross-checked against each request's real HTTP status/headers via requireGate; the model is a visualization layer on top of that evidence, not a replacement for it. Also catches and documents a timing bug found while dry-running the new offsets with curl before spending a browser recording on them: the original gap between silver-tier's exhaustion and its cross-instance denial check (5.3s) exceeded the bucket's exact full-refill time (5.0s), so the check returned 200 instead of the expected 429; fixed by tightening the gap to 2.0s. See recording/RECORDING.md for the full dry-run transcript and the recording environment notes (RHEL 9 lab host, Podman + SELinux workaround). Signed-off-by: Jordi Gil <jgil@redhat.com>
Move the per-app token budgets demo off docker-compose onto a kind cluster (valkey, backend, 2 gateways, 3 apps, dashboard as real Deployments/Services) so the dashboard can show literal Kubernetes pod names instead of static app-a/b/c labels, closing the "this looks like a mock" gap raised on the prior recording. - k8s/: namespace + RBAC (app pods get get/list on pods and get on pods/log), and Deployments/Services for every component. - Dashboard now resolves real pod identities via the K8s API (through each app pod, using its ServiceAccount token) and renders live per-request telemetry (timestamps, pod-to-pod routing, HTTP status/latency, request IDs, rate-limit headers) instead of synthetic values. - Add a live gateway-log panel per gateway, streamed via SSE from the in-cluster pods/log subresource (kubectl logs -f equivalent), with client-side reformatting so DENY/ADMIT lines lead with the fields that matter (app, rule, estimate) and client_ip is resolved back to app name. - Drop the "Gateway A"/"Gateway B" topology cards and their manual "Send as app-X" buttons: they only existed for ad hoc manual testing (the recorded scenario always calls sendRequest() directly, never the buttons) and duplicated identity info the new gwlogs panel headers already show. Frees a full row of vertical space for the logs. - Fix two real bugs found while verifying app-attribution in the gwlogs panel: (1) log streams could start before the client-side IP->app map finished resolving, so early lines showed raw pod IPs; sequence identity resolution before opening the SSE connections. (2) the log endpoint replayed the last 15 lines of pod history on connect, which could reference long-recycled IPs from earlier app-pod generations; changed to tailLines=0 so panels start clean and fill live as the scenario runs. - Also carries the earlier narration/intro-slide polish (three-part intro: title cards, architecture diagram, animated topology preview) and the matching record.mjs timing/output-path updates. Signed-off-by: Jordi Gil <jgil@redhat.com>
Completes the retuned 8-request mixed-algorithm scenario (estimate_tokens 15/40 gold-tier, 7/10 silver-tier, both exactly matched to the now tier-aware stub backend's usage.total_tokens) that RECORDING.md already described but wasn't fully wired into config.yaml/docker-compose.yml/k8s manifests yet, and re-records the narrated walkthrough end-to-end against a real kind cluster to match it. - Re-record `output/k8s-real-pods-token-rate-limit.mp4` (~143s, matches the current Ava-voice narration.wav and dashboard v2's narration-paced timing) against a fresh kind deployment of the real two-gateway + Valkey + three-app stack -- every pod name, gateway log line, and HTTP status shown is live, not simulated. - Ship the dashboard's generic (unbranded, navy/cyan) skin rather than a Red Hat-branded alternative, since this repo is meant for community/contributor adoption; both skins were recorded and validated identically, and the branded one is being held back pending a separate team decision (see RECORDING.md). - Add a "References" section to the README, splitting "Related work" into the specific Praxis issues this demo's config/behavior derives from (ai#121, ai#658, ai#789/praxis#551, ai#129, grid#83) versus external prior art for logic not yet specified anywhere in Praxis (token bucket, sliding-window rate limiting, Envoy's external rate-limit service pattern, Redis's atomic Lua rate-limiter pattern, and the nerdalert spike branch this filter's ledger was adapted from). Signed-off-by: Jordi Gil <jgil@redhat.com>
k8s/deploy.sh has existed since the Kubernetes migration, and its own comment already pointed at a "Run on Kubernetes (kind)" README section that was never actually written -- the only kind/k8s mentions were the "Recorded walkthrough" note that the video used it, and an "Alternative implementations considered" bullet about the sibling Grid demo's stack, neither of which told a reader how to reproduce it themselves. Add that section: build the gateway image, create the kind cluster and load the image (imagePullPolicy: Never expects it on the node already, not a registry), run deploy.sh, and port-forward the same ports "Validate the request flow" and the dashboard already assume, so both work unmodified against either backend. Signed-off-by: Jordi Gil <jgil@redhat.com>
praxis-bot flagged that the intro slide deck (recording/slides/ proof-agenda.html, spliced into the shipped video) is fully Red Hat-branded (fonts, palette, brand-standards citation), contradicting the impression left by RECORDING.md's claim that this demo was kept generic for community/contributor adoption. That claim was only ever scoped to dashboard/index.html's skin, not the intro deck, but rather than caveat it further, just drop the branding rationale entirely -- nothing about the skin choice needs stating here. Signed-off-by: Jordi Gil <jgil@redhat.com>
`docker save`/`docker load` has no direct counterpart in the kind instructions above it -- the podman path uses `podman save` + `kind load image-archive`, not `podman load`. More importantly, Docker users don't need the intermediate tar file at all: `kind load docker-image` loads straight from Docker's image store in one step. Signed-off-by: Jordi Gil <jgil@redhat.com>
praxis-bot flagged two issues in the app pod's dashboard-facing HTTP server (k8s/04-apps.yaml): - /gw-logs read the K8s log stream one byte at a time to find each newline. Iterate the response object directly instead, which uses Python's buffered readline() under the hood. - do_POST caught HTTPError (non-2xx from the gateway) but not URLError (gateway unreachable, DNS failure, connection refused, timeout), so a gateway restart or scheduling race surfaced as a raw traceback / 500 to the dashboard instead of a clean 503. Signed-off-by: Jordi Gil <jgil@redhat.com>
jordigilh
force-pushed
the
jordigilh/token-rate-limit-per-app-demo
branch
from
September 1, 2026 17:52
faaee4e to
b0848d3
Compare
praxis-bot flagged that slides/proof-agenda.html loads Red Hat's brand fonts and palette while the rest of the demo (dashboard/index.html) is generic. Rather than just dropping the contradictory claim (90596aa), actually align the deck: replace the three Red Hat Google Fonts with the system font stack dashboard/index.html already uses, and rename/ recolor the --rh-* palette to a neutral blue accent instead of Red Hat red. No layout changes; verified all three sections still render correctly (checked via a headless screenshot of each <section>). The already-recorded video predates this change, so its intro still shows the old Red Hat-styled cards -- noted in RECORDING.md pending a re-record. Signed-off-by: Jordi Gil <jgil@redhat.com>
Author
|
Done! 😊 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
demos/token-rate-limit-per-app-budgets/demo: two independent Praxis AI gateway replicas sharing one Valkey-backedtoken_rate_limitbudget per application, with per-rule algorithm choice (gold-tier=sliding_window,silver-tier=token_bucket) matched by anx-tierheader, per ai#789/praxis#551 and ai#129'sbucket_key_headerproposal.kindKubernetes cluster (k8s/,deploy.sh) — real pod identities, live gateway stdout via SSE, live namespace/pod-status strip, all pulled from the actual Kubernetes API, not simulated. Seerecording/RECORDING.mdfor how it was produced.ai#121,ai#658,ai#789/praxis#551,ai#129,grid#83)Recording
recording/output/k8s-real-pods-token-rate-limit.mp4(1920x1080, h264/aac, ~143s) — GitHub doesn't inline-preview binary diffs over ~10MB in the "Files changed" tab, so click through to watch/download it:Important
This is early exploratory work; the
token_rate_limitfilter code this demo exercises lives only on a personal fork branch, not onpraxis-aimain. See the README's warning banner, "Current scope", and "Open design questions" for what's settled vs. still open upstream.Test plan
markdownlint/lycheeCI checks pass on the updated README/RECORDING.md (all new external links manually verified to return HTTP 200)recording/output/k8s-real-pods-token-rate-limit.mp4and confirms it matches the README's narrated walkthrough