Skip to content

feat(srv6): SRv6-native NAT-less (DSR) ClusterIP services#10

Open
ryskn wants to merge 1 commit into
fix/srv6-nodeip-dt4from
feat/srv6-native-dsr
Open

feat(srv6): SRv6-native NAT-less (DSR) ClusterIP services#10
ryskn wants to merge 1 commit into
fix/srv6-nodeip-dt4from
feat/srv6-native-dsr

Conversation

@ryskn

@ryskn ryskn commented Jun 7, 2026

Copy link
Copy Markdown
Owner

Summary

Experimental SRv6-native data path for pod-backed ClusterIP services that replaces cnat DNAT/un-DNAT with Direct Server Return (DSR).

With SRv6 as the pod transport, the cnat-based service path does not compose: on a cross-node pod-backed ClusterIP the reverse cnat session lives in fib 0 and the cnat input arc never runs on the SRv6-decapped inner packet, so the reply is never un-DNAT'd (SYN-ACK keeps the backend pod IP → client RST). Instead of patching cnat×SRv6, this steers the VIP over SRv6 and lets the backend accept the VIP directly (DSR): the inner packet keeps dst=VIP end to end, the backend replies with src=VIP, so no reverse translation is needed and the failure class disappears. Source IP is preserved (no SNAT).

How it works

  • Opt-in: feature gate srv6NativeServicesEnabled + per-service annotation cni.projectcalico.org/vppSRv6Native: "true". Eligible = ClusterIP type, all ports port==targetPort, no ExternalIP/LB. Everything else keeps the cnat path.
  • services: classify eligible services, skip cnat for their ClusterIPs, publish DSRService events; reconcile-based diff/withdraw.
  • connectivity/SRv6: per-service weighted-ECMP SR policy over the backend nodes' End.DT6 SIDs + steering (main table); End.DT decaps into PodVRF (via sw_if_index). BSID derived from the policy pool, collision-checked against per-node BSIDs and cached for stability.
  • cni: bind the VIP on local backend pods (lo) + allow it through uRPF + install the PodVRF ECMP delivery route; per-pod cleanup tracking; periodic + event-driven reconcile with retain-on-failure retry.

Verification

End-to-end on an IPv6 single-stack cluster: cross-node client → annotated ClusterIP reaches the backend with no cnat translation, no RST, and the client source IP preserved; vppctl trace shows SRv6 decap → delivery with no cnat-output session lookup.

Security review (spec-anchored)

Addressed here:

  • Per-service BSID could collide with IPAM-allocated per-node SR-policy BSIDs (would clobber pod-connectivity policies on install/withdraw) → BSID is now collision-checked against in-use BSIDs and cached/stable.
  • ExternalIP/LoadBalancer IPs on a DSR service would still hit the cnat un-DNAT issue → such services are excluded from DSR and kept fully on cnat.

Follow-up (separate): pod-backed endpoint detection so a mis-annotated host-network service falls back to cnat instead of silently breaking.

Scope / future work

First cut: pod-backed ClusterIP, port==targetPort only. Future: port remapping (stateless L4), host-network/NodePort/External, per-pod End.DX6 service SIDs (RFC 9256 single-tier weighted LB), and making DSR the default when SRv6 is enabled.

@ryskn

ryskn commented Jun 7, 2026

Copy link
Copy Markdown
Owner Author

Follow-up for F-DSR-2 (pod-backed endpoint detection / cnat fallback for host-network services) tracked in ryskn/vpp-dataplane-research#32.

Replace cnat DNAT/un-DNAT with Direct Server Return for pod-backed
ClusterIP services in the SRv6 data path. Traffic keeps dst=VIP end to
end; backend pods accept the VIP on lo and reply from it, so no reverse
translation is needed and the cross-node SRv6 un-DNAT failure disappears.
Source IP is preserved (no SNAT).

- config: SRv6NativeServicesEnabled feature gate + vppSRv6Native annotation
- services: classify eligible ClusterIP services (gate + annotation +
  ClusterIP + port==targetPort), skip cnat, publish DSRService events,
  reconcile-based diff/withdraw
- connectivity/SRv6: per-service ECMP SR policy over backend nodes'
  End.DT6 SIDs + steering; decap into PodVRF via sw_if_index; reconcile
  with retain-on-failure retry and periodic re-reconcile
- cni: bind VIP on local backend pods (lo) + uRPF allow + PodVRF ECMP
  delivery route; per-pod cleanup tracking; reconcile on pod add/del

Experimental; requires SRv6Enabled. Other services (NodePort/External,
port-remapped, host-network backed) keep the cnat path.

Signed-off-by: Ryosuke Nakayama <ryosuke.nakayama@ryskn.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant