fix(srv6): steer pod-to-node service traffic through End.DT6 SID#9
Closed
ryskn wants to merge 1 commit into
Closed
fix(srv6): steer pod-to-node service traffic through End.DT6 SID#9ryskn wants to merge 1 commit into
ryskn wants to merge 1 commit into
Conversation
c0d0c2f to
4c7b5ec
Compare
There was a problem hiding this comment.
Pull request overview
IPv6 single-stack + SRv6 環境で SRv6 が有効になった際に、host-network backed な ClusterIP Service への Pod 発トラフィックが SRv6 を迂回して非対称経路になり drop される問題を、PodVRF 上でリモートノード IP(/128)宛を End.DT6 SID に steer することで解消する変更です。
Changes:
- リモートノード自身の IPv6 Node IP(/128) 宛トラフィックを、当該ノードの End.DT6 policy(BSID) に steering するヘルパーを追加
AddConnectivityの SRv6 構築完了タイミングで上記 steering を追加し、host-network backed ClusterIP の往復対称性を確保
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| } | ||
| prefix, err := ip_types.ParsePrefix(nodeIP.String() + "/128") | ||
| if err != nil { | ||
| p.log.Errorf("SRv6Provider steerNodeIPViaSID parse prefix %s: %s", nodeip, err) |
| Bsid: policy.Bsid, | ||
| } | ||
| if err := p.vpp.AddSRv6Steering(srSteer); err != nil { | ||
| p.log.Errorf("SRv6Provider steerNodeIPViaSID AddSRv6Steering %s", err) |
| } | ||
| policy, err := p.getPolicyNode(nodeip, types.SrBehaviorDT6) | ||
| if err != nil || policy == nil { | ||
| p.log.Debugf("SRv6Provider steerNodeIPViaSID: no DT6 policy for %s yet, retried later", nodeip) |
48e48da to
bcfa493
Compare
Under SRv6, a pod accessing a host-network-backed ClusterIP (e.g. the kubernetes Service fd30::1 -> the apiserver node IP fd00:1::10) is dropped on the backend node. cnat DNATs the Service VIP to the node IP, which is directly connected in the uplink subnet, so the packet bypasses SRv6 and is forwarded over the uplink as plain IPv6. cnat also excludes node-IP destinations from SNAT, so the pod source is preserved. On the receiving node that pod prefix is only reachable through the SRv6 overlay, so the unencapsulated pod-sourced packet fails the source/reverse-path lookup and is dropped (ip6 source lookup miss). Pod-backed ClusterIPs are unaffected: the backend is a pod address whose FIB resolution is an SRv6 tunnel, so the path stays symmetric. Steer pod-originated traffic destined to a remote node's own IPv6 address (/128) through that node's already-advertised End.DT6 SID (steerNodeIPViaSID in AddConnectivity). The steering entry is installed in PodVRFIndex so node-to-node control traffic in the main table is untouched, and the pod source identity is preserved (no SNAT). It reuses the remote node's existing End.DT6 local SID (setEndDT, FibTable 0); no new dataplane primitive is needed. The DT6 policy is already installed by the CreateSRv6Tunnel pass over the node's pod prefixes, so we only add the steering and do not re-add the policy (AddModSRv6Policy is delete-then-add and would churn the live policy shared with the pod-prefix steering).
bcfa493 to
ab2e7ca
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
In an IPv6 single-stack + SRv6 setup, once SRv6 is actually active (the
localsid/policy pools are markedallowedUses: [Tunnel]so Pods are allocated from the default pool), host-network-backed ClusterIP Services break.A concrete example is a Pod on one node accessing the
kubernetesService:The packet is dropped on the backend node. Pod-backed ClusterIP Services are unaffected.
Root cause
For a host-network-backed Service, cnat DNATs the Service VIP to a node IP. That node IP is directly connected in the uplink subnet, so the packet bypasses SRv6 and is forwarded over the uplink as plain IPv6.
At the same time, cnat excludes node-IP destinations from SNAT, so the Pod source address is preserved.
On the receiving node, that Pod prefix is only reachable through the SRv6 overlay. As a result, a Pod-sourced packet arriving unencapsulated from the uplink fails the source/reverse-path lookup and is dropped (
ip6 source lookup miss).Pod-backed Services do not hit this path because the backend is a Pod address; its FIB resolution is an SRv6 tunnel, so traffic stays symmetric.
Fix
Steer Pod-originated traffic destined to a remote node's own IPv6 address (
/128) through that node's already-advertised End.DT6 SID. This addssteerNodeIPViaSIDinAddConnectivity.PodVRFIndex, so node-to-node control traffic in the main table is unaffected.setEndDTwith FibTable 0); no new dataplane primitive is required.The resulting path is:
The DT6 policy is already installed in VPP by the
CreateSRv6Tunnelpass over the node's Pod prefixes, so this change only adds the steering entry and does not re-add the policy. This avoids churn of a live policy shared with the Pod-prefix steering, sinceAddModSRv6Policyis delete-then-add.Notes
SrSteerIPv4handling.DelConnectivityis currently a no-op, matching the existing limitation for Pod-prefix steering. Once fix(srv6): implement DelConnectivity with NLRI-key teardown and candidate-path failover projectcalico/vpp-dataplane#1025 lands, this steering should be removed from the same teardown path.