Tracking: VPN gateway operator

[jacaudi]

Umbrella issue for turning this operator into a Kubernetes-native VPN gateway: a WireGuard server that terminates client tunnels and forwards their traffic onward through gluetun to a commercial VPN provider, with clients declared explicitly or injected into existing workloads by label.

Full design rationale, architecture and leak analysis: docs/ROADMAP.md.

Phase 0 — Foundations

Blocks everything else. #1 and #2 in particular: until there is one Deployment builder and declarative reconciliation, every new spec field needs a bespoke drift check, and anything implemented in internal/resources has no runtime effect at all.

• [P0] Consolidate resource rendering onto internal/resources builders #1 Consolidate the two divergent Deployment builders
• [P0] Replace trigger-based Deployment drift checks with declarative reconciliation #2 Replace trigger-based Deployment drift checks with declarative reconciliation
• [P0] Rework iptables generation: non-flushing, operator-owned chains, correct egress interface #3 Rework iptables generation: non-flushing, operator-owned chains, correct egress interface
• [P0] Add PersistentKeepalive to generated peer configs #4 Add PersistentKeepalive to generated peer configs — independently useful today

Phase 1 — Pod extension points

• [P1] Add spec.initContainers and spec.extraContainers #5 Add spec.initContainers and spec.extraContainers
• [P1] Add spec.volumes and volume mounts for user-supplied containers #6 Add spec.volumes and volume mounts for user-supplied containers
• [P1] Validate user-supplied containers against reserved names #7 Validate user-supplied containers against reserved names

Phase 2 — gluetun

• [P2] Design the spec.gluetun API #8 Design the spec.gluetun API
• [P2] Render the gluetun sidecar #9 Render the gluetun sidecar
• [P2] Provider credentials via secretRef #10 Provider credentials via secretRef
• [P2] Auto-compute gluetun firewall input ports and outbound subnets #11 Auto-compute gluetun firewall input ports and outbound subnets
• [P2] Route peer egress through the gluetun tunnel #12 Route peer egress through the gluetun tunnel
• [P2] Reconcile DNS between gluetun's resolver and spec.dns #13 Reconcile DNS between gluetun's resolver and spec.dns
• [P2] Gate Wireguard readiness on gluetun tunnel health #14 Gate Wireguard readiness on gluetun tunnel health
• [P2] Surface gluetun's forwarded port and exit IP in Wireguard status #15 Surface gluetun's forwarded port and exit IP in status
• [P2] E2E: egress exits via the VPN and fails closed when the tunnel drops #16 E2E: egress exits via the VPN and fails closed when the tunnel drops

Phase 3 — Client paths and ingress

• [P3] Emit an internal peer config flavor pointing at the ClusterIP #17 Emit an internal peer config flavor pointing at the ClusterIP
• [P3] LoadBalancer UDP path for mobile and stock clients #18 LoadBalancer UDP path for mobile and stock clients
• [P3] Render an HTTPRoute for the wstunnel path #19 Render an HTTPRoute for the wstunnel path
• [P3] Optional UDPRoute rendering for the direct path #20 Optional UDPRoute rendering for the direct path
• [P3] Derive peer endpoints from spec.gateway.hostname and watch Gateway status #21 Derive peer endpoints from spec.gateway.hostname and watch Gateway status
• [P3] Per-path MTU for direct vs tunnel configs #22 Per-path MTU for direct vs tunnel configs

Phase 4 — Webhook infrastructure

• [P4] Webhook scaffolding #23 Webhook scaffolding
• [P4] Webhook certificate management and failure-policy scoping #24 Webhook certificate management and failure-policy scoping

Phase 5 — Implicit clients

• [P5] Define the label-based opt-in contract for implicit clients #25 Define the label-based opt-in contract
• [P5] Pod mutating webhook: inject the WireGuard client sidecar #26 Pod mutating webhook: inject the WireGuard client sidecar
• [P5] Pod controller: mint and reap a WireguardPeer per pod #27 Pod controller: mint and reap a WireguardPeer per pod
• [P5] Split the aggregated peer-config Secret into per-peer Secrets #28 Split the aggregated peer-config Secret into per-peer Secrets
• [P5] Cross-namespace wireguardRef with bidirectional consent #29 Cross-namespace wireguardRef with bidirectional consent
• [P5] Make peer IPAM instance-scoped rather than namespace-scoped #30 Make peer IPAM instance-scoped rather than namespace-scoped
• [P5] Replicate a peer's config Secret into the workload's namespace #31 Replicate a peer's config Secret into the workload's namespace
• [P5] Client sidecar: kernel and userspace paths as a native sidecar #32 Client sidecar: kernel and userspace paths as a native sidecar
• [P5] In-pod egress lockdown for injected clients #33 In-pod egress lockdown for injected clients
• [P5] Operator-generated CNI network policy for injected clients #34 Operator-generated CNI network policy for injected clients

Phase 6 — Project

• [P6] Docs and examples for all three client paths #35 Docs and examples for all three client paths
• [P6] Fork housekeeping: Go module path #36 Fork housekeeping: Go module path

---

Decisions

Recorded so they don't get relitigated. Rationale in docs/ROADMAP.md.

Single server, not HA. A generic WireGuard client already reconnects on its own after the server pod reschedules, with no client-side logic, given a stable endpoint address and a persisted server key — both of which already hold. WireGuard has no connection to lose. Recovery is ~20–40s for a pod crash and tunable for node failure via tolerationSeconds. HA would have cost multi-peer config generation, a failover sidecar, metric semantics, per-server credential management, and a permanent split between how internal and external clients fail over. The CRD stays forward-compatible so HA remains additive.

Three client paths. ClusterIP for in-cluster pods, LoadBalancer UDP for mobile and stock clients, Gateway plus wstunnel for obfuscated desktop clients. Mobile cannot use the tunnel path — PreUp and friends are wg-quick features and are unsupported on iOS and Android.

LoadBalancer by default for raw UDP, UDPRoute opt-in. Envoy Gateway supports UDPRoute, but it proxies non-transparently and puts every VPN packet through a userspace proxy. Gateway API is used unconditionally for the wstunnel path, where L7 termination is the actual value.

Implicit clients are minted per Pod, not per Deployment. A peer identity shared across replicas misdelivers silently: WireGuard tracks one endpoint per public key, so the server sends pod A's return traffic to whichever pod handshook last.

Opt-in is a label, not an annotation. Webhook objectSelector matches labels only.

Killswitch in two layers. In-pod rules work on any CNI; the CNI policy survives sidecar crash or interface teardown, which is the failure that actually leaks.

[jacaudi]

Engineering practices

These apply to every issue on this roadmap. Individual plans state how they apply to that specific change rather than restating them.

TDD

Write the test first, watch it fail for the right reason, then make it pass. A test that has never failed has proven nothing.

For changes that add behaviour this is literal. For pure refactors it is not — there is no new behaviour to drive out, so writing new tests first would be cargo-culting. On a refactor, TDD means: establish a green baseline before touching anything, keep it green after every step, and if an existing test fails, the refactor changed behaviour. Fix the code, never the assertion.

DRY

The single hardest-won lesson in this codebase: internal/resources and the inline reconciler functions are the same code twice, and they silently drifted apart. That is what #1 exists to undo. Do not create a second copy of anything — a constant, a rendering path, a default value. One source of truth, referenced.

Note the limit: two things that look alike but change for different reasons are not duplication. Do not collapse them to satisfy the acronym.

12-Factor

Most factors are satisfied by Kubernetes itself. The ones that genuinely bite here:

• Config — configuration comes from the CRD or environment, never hardcoded in more than one place. A default belongs in the CRD's +kubebuilder:default, with the Go constant existing only as a nil-fallback.
• Processes — builders stay pure functions of their inputs. No caching on the reconciler, no hidden state between calls.
• Logs — event streams to stdout via the existing logr. No log files, no log rotation.
• Disposability — fast startup, graceful shutdown. Matters most for the injected client sidecar, where slow teardown means a leaking tunnel.

YAGNI

Build what the issue asks for. Do not add a config knob because someone might want it, do not generalise for a second implementation that does not exist, and do not pre-build for HA — that was explicitly deferred, and the CRD only needs to stay forward-compatible, not pre-plumbed.

If you find yourself writing an abstraction with exactly one caller, stop.

KISS

Prefer the boring solution. If a reviewer needs the plan open to understand the diff, it is too clever. Explicit repetition of three short lines beats a helper that takes four parameters to avoid it.

Cold review before the PR

Before opening a PR, get a review from a reviewer that has none of the context that produced the change. A fresh session or agent, given only:

• the diff
• the issue text, including its acceptance criteria
• the repository

and explicitly not given the implementation plan, the conversation that produced it, or any reasoning about why choices were made.

Ask it to answer:

1. Does the diff do what the issue's acceptance criteria say, no more and no less?
2. Is there scope creep — anything changed that the issue did not ask for?
3. Is anything here wrong, unclear, or surprising to someone seeing it for the first time?
4. Are the tests actually testing the behaviour, or asserting on implementation details?

The point is that an author who has been reasoning about a change for an hour cannot see what is unexplained in it. If the cold reviewer misreads the diff, that is a finding about the diff, not about the reviewer. Address the findings, then open the PR.

---

Generated by Claude Code

[jacaudi]

The engineering practices above now live in CONTRIBUTING.md (added in #39), so contributors reading the repository can find them rather than having to dig through this tracker.

Treat the file as canonical from here — by its own DRY rule, this comment is the copy that will drift.

---

Generated by Claude Code