This repository is a semester's worth of teaching material disguised as a working system. It is a five-runtime platform (Python, JavaScript, Go, Rust, Kotlin/JVM) with contract-first gRPC, shaped load, seamless historical data, and three-layer monitoring -- all under one observability roof and one uniform service contract.
The thesis: done means operable, not 200 OK. Every module makes one
operational difference visible on a dashboard, then asks you to break it and
watch which layer notices. The process -- the RFCs that argued each decision,
the ADRs that froze it, the runbooks that operate it, and the exercises that
stress it -- is the material, not a wrapper around it.
The semester is ten modules over roughly fourteen weeks: the seven build phases of RFC-0001, the sibling RFC-0002 integration exhibit, and two upcoming modules (Kubernetes on Kind, then an authentication capstone) that are in design.
The model is hybrid: run the present, read the past.
- Run current
main. Every module's guided path is the live system onmain. Start withmake doctor(toolchain, Docker, RAM), then bring up the profile the module needs --make upfor the core stack,make up-fullfor every optional profile, or a specific--profileselection. Then do the module's exercise against the running stack. - Read how it grew. Every phase argued its case before it shipped: read the phase's RFC-0001 decision records and the ADRs that landed with it, then read the pull requests that delivered it. RFC-0001 Section 9 lists what each phase built and in what order.
- Operate it. The runbooks in
docs/runbooks/are the ops half of every module: an alert fires, you open the runbook, you triage.
main is the single source of truth, and the only thing you run. Bug fixes,
doc corrections, and new exercises always land there. The repo keeps no
historical snapshots to run instead: an old tree would carry every defect
fixed since, so history is for reading, not for bringing up.
| Module | Phase / RFC | What lands | Read (RFC + ADR) | Run | Do | Operate (runbooks) |
|---|---|---|---|---|---|---|
| 0. Baseline and operability | Phase 0 | Repo layout, CI, toolchain | D6, D8, D14; ADR 1/4/9/12 | make up |
ex 00 | -- |
| 1. Three-layer monitoring | Phase 1 | blackbox + canary v1 | D6, D9, D10; ADR 7/8 | make up-full |
ex 01, 08 | probe-down, canary-journey-failing |
| 2. Contract-first gRPC | Phase 2 | proto/ + gRPC server + buf gate | D3, D11; ADR 2/10 | make up |
ex 02 | -- |
| 3. Analytics and the event stream | Phase 3 | Go analytics + canary v2 | D1, D3, D7; ADR 5/2 | make up-full |
ex 03 | analytics-stream-down, canary-pipeline-lag-high |
| 4. Load and incident-to-inbox | Phase 4 | loadgen + Alertmanager + e2e gate | D4, D5; ADR 3/6/7/8 | make up-full + make incident |
ex 04 | all four (triage) |
| 5. Historical data and seams | Phase 5 | seeder + history dashboards | D5, D7; ADR 5/3 | make up-workshop + seed |
ex 05 | -- |
| 6. The JVM showcase | Phase 6 | Kotlin reports + canary v3 | D2, D6, D10, D12; ADR 4 | make up-full |
ex 06 | -- |
| 7. A UI over the API | RFC-0002 | reports-ui Caddy SPA (:8084) |
RFC-0002; ADR 13 | make up-full |
-- | -- |
| 8. Into Kind (UPCOMING) | RFC-0003 | K8s/Kind, Gateway API, Helm | RFC-0003 (in design) | -- | -- | -- |
| 9. Capstone: authN/authZ (UPCOMING) | RFC-0004 | OIDC across the polyglot mesh | RFC-0004 (in design) | make up-full |
ex 07 | -- |
ex NN is docs/exercises/NN-*.md; D<n> is the
decision record of that number in RFC-0001 Section 6-12;
ADR n is docs/adr/000n-*.md.
The lesson: a service that returns 200 is not done -- it is done when it is
operable. Read RFC-0000 and RFC-0001
decisions D6 (uniform contract), D8 (monorepo layout), D14 (toolchain); ADRs
0001,
0004,
0009,
0012. Bring up the core stack
with make up and do exercise 00. Discussion: what
would you need to see before you would page someone at 3am?
The lesson: whitebox, blackbox, and synthetic layers each notice different
failures. Read RFC-0001 D6, D9, D10; ADRs
0007,
0008. make up-full adds the
synthetic profile (blackbox_exporter + Rust canary v1). Do
exercise 01; operate with
probe-down and
canary-journey-failing. Discussion: which
layer fires first, and why is that the wrong one to alert on alone?
The dashboards are a deliverable here, not just a view onto one: exercise 08 edits one in Grafana, shows the repo never hearing about it, and closes the loop through a diff somebody can review.
The lesson: the contract is the artifact; code is downstream of it. Read
RFC-0001 D3, D11; ADRs 0002,
0010. The core stack (make up) already
serves the gRPC port. Do exercise 02 -- call
the contract, then break it and watch the buf CI gate reject it. Discussion:
what does "contract-first" buy you that a shared client library does not?
The lesson: a service that owns its own store, and the durability gap that
choice exposes. Read RFC-0001 D1, D3, D7; ADRs
0005,
0002. make up-full (or add
--profile analytics) brings up Go analytics and its own Postgres; canary v2
now measures pipeline lag. Do
exercise 03; operate with
analytics-stream-down and
canary-pipeline-lag-high. Discussion:
break the stream -- which monitoring layer fires first?
The lesson: dashboards without traffic are screenshots; an incident should end
in someone's inbox. Read RFC-0001 D4, D5; ADRs
0003,
0006,
0007,
0008. make up-full brings up
shaped k6 load, Alertmanager, and Mailpit; make incident overlays a fault.
Do exercise 04 and use all four runbooks
as a triage drill. Discussion: trace one alert from metric to inbox -- where
could it silently drop?
The lesson: seeded history must stitch into live traffic with no visible seam.
Read RFC-0001 D5, D7; ADRs
0005,
0003. Use workshop mode:
make up-workshop (hardcodes DEMO_TIME_SCALE=24, so one profile-day is one
wall-clock hour), then DEMO_TIME_SCALE=24 SEED_DAYS=3 make seed-history at
the matching scale. Do exercise 05
-- find the three seeded anomalies. Discussion: where is the seam, and how
would you prove there is not one?
The lesson: same operational requirements, a very different runtime. Read
RFC-0001 D2, D6, D10, D12; ADR
0004. make up-full adds the reports
profile -- Kotlin/Spring Boot on the JVM (:8083), canary v3's report step,
and a report k6 scenario. Do
exercise 06 -- watch the GC sawtooth
under report load. Discussion: what does the JVM make visible that the Go and
Rust services do not?
The lesson: a static SPA is still an operable service -- health, metrics, and a
reverse proxy included. Read RFC-0002; ADR
0013. make up-full adds the
reports-ui profile: a Caddy-served SPA on :8084 that proxies the reports
API and re-exposes its own /healthz and /metrics. No dedicated exercise yet
-- explore the UI and read the reverse-proxy and metrics story. Discussion:
what makes a "just static files" service page-worthy?
The deployment and operability platform. Kubernetes on Kind, Gateway API via Envoy, Helm charts, the D6 health endpoints wired to probes, the measured per-service footprint turned into resource requests and limits, and the Prometheus Operator with ServiceMonitors. This is where the whole polyglot stack becomes production-like. Currently in design.
The platform today has zero auth -- everything is open with demo credentials (ADR-0011 deferred it; RFC-0001 Section 10 names a security capstone). This capstone adds authentication and authorization across the polyglot mesh, riding on the Kind/Kubernetes platform from Module 8: OIDC login on the SPAs (Authorization Code + PKCE), then JWT/JWKS validation and role/scope authz in every service -- each language its own way, a cross-language exhibit like the D6 contract -- then edge OIDC at the Envoy Gateway (SecurityPolicy) and service-to-service auth (client-credentials / mTLS). It is provider-agnostic: everything integrates against the standard OIDC discovery/JWKS contract, so any compliant provider drops in. The repo ships three interchangeable provider deployments -- Keycloak, Zitadel, Dex -- and students choose one; swapping providers is itself the lesson, the same "two implementations of one contract" pedagogy as the nginx/Caddy contrast. Currently in design. Discussion: where does each language draw the line between authentication and authorization, and what breaks when you swap the provider?
One piece of this module is runnable today, ahead of RFC-0004: exercise 07 undoes the compose stack's network segmentation and loopback port bindings on purpose, shows an analytics-tier database reading the core application's rows across the flat network, and puts it back. Do it before the capstone: authentication is the control you add after you know what is reachable without it, and the exercise makes that reachability concrete rather than theoretical.
- Pacing. Roughly one to two weeks per module fills a ~14-week semester: seven RFC-0001 phases, the RFC-0002 sibling, and the two upcoming modules. Front-load Modules 0-1; Modules 3-4 carry the most operational depth.
- Compressed sessions. For in-class demos use workshop mode
(
make up-workshop,DEMO_TIME_SCALE=24): a 90-day history plays out in hours. Seed at the same scale or loadgen's guard refuses to start against a mismatched marker. - Hardware. Point students at
make doctorfirst -- it checks Docker and RAM before anything starts. The core stack (make up) runs ~10 containers; the full profile set (make up-full) runs ~18, and the Kind module in Module 8 will need noticeably more headroom. - Assessment. Each exercise ends in a Discussion questions section --
those double as assignments, quiz prompts, or lab write-ups.
make ciis the same gate the platform holds itself to, so "make CI green" is a fair grading bar.
Every build phase is a run of pull requests and a matching RFC-0001 decision
trail. To read the construction history end to end: walk git log --oneline
on main (each merge names its PR), and read each phase's section of
RFC-0001 beside the ADRs that froze
its decisions. The
engineering principles doc explains why the
RFC-then-ADR lifecycle exists at all.