Skip to content

fix(srv6): steer pod-to-node service traffic through End.DT6 SID#1028

Merged
sknat merged 1 commit into
projectcalico:masterfrom
ryskn:fix/srv6-host-backed-clusterip
Jun 1, 2026
Merged

fix(srv6): steer pod-to-node service traffic through End.DT6 SID#1028
sknat merged 1 commit into
projectcalico:masterfrom
ryskn:fix/srv6-host-backed-clusterip

Conversation

@ryskn

@ryskn ryskn commented May 27, 2026

Copy link
Copy Markdown
Contributor

Summary

In an IPv6 single-stack + SRv6 setup, once SRv6 is actually active (the localsid/policy pools are marked allowedUses: [Tunnel] so Pods are allocated from the default pool), host-network-backed ClusterIP Services break.

A concrete example is a Pod on one node accessing the kubernetes Service:

fd30::1 -> apiserver node IP fd00:1::10

The packet is dropped on the backend node. Pod-backed ClusterIP Services are unaffected.

Root cause

For a host-network-backed Service, cnat DNATs the Service VIP to a node IP. That node IP is directly connected in the uplink subnet, so the packet bypasses SRv6 and is forwarded over the uplink as plain IPv6.

At the same time, cnat excludes node-IP destinations from SNAT, so the Pod source address is preserved.

On the receiving node, that Pod prefix is only reachable through the SRv6 overlay. As a result, a Pod-sourced packet arriving unencapsulated from the uplink fails the source/reverse-path lookup and is dropped (ip6 source lookup miss).

Pod-backed Services do not hit this path because the backend is a Pod address; its FIB resolution is an SRv6 tunnel, so traffic stays symmetric.

Fix

Steer Pod-originated traffic destined to a remote node's own IPv6 address (/128) through that node's already-advertised End.DT6 SID. This adds steerNodeIPViaSID in AddConnectivity.

  • The steering entry is installed in PodVRFIndex, so node-to-node control traffic in the main table is unaffected.
  • It reuses the remote node's existing End.DT6 local SID (created by setEndDT with FibTable 0); no new dataplane primitive is required.
  • The Pod source identity is preserved; no SNAT is needed.

The resulting path is:

pod source preserved
  -> SRv6 encap toward the remote node's End.DT6 SID
  -> remote node decapsulates
  -> lookup in table 0
  -> delivered to the host-network backend

The DT6 policy is already installed in VPP by the CreateSRv6Tunnel pass over the node's Pod prefixes, so this change only adds the steering entry and does not re-add the policy. This avoids churn of a live policy shared with the Pod-prefix steering, since AddModSRv6Policy is delete-then-add.

Notes

  • IPv4 node IPs / DT4 are out of scope for this PR. Dual-stack support would need the corresponding End.DT4 and SrSteerIPv4 handling.
  • Explicit teardown of this node-IP steering is not implemented here because DelConnectivity is currently a no-op, matching the existing limitation for Pod-prefix steering. Once fix(srv6): implement DelConnectivity with NLRI-key teardown and candidate-path failover #1025 lands, this steering should be removed from the same teardown path.

Under SRv6, a pod accessing a host-network-backed ClusterIP (e.g. the
kubernetes Service fd30::1 -> the apiserver node IP fd00:1::10) is dropped on
the backend node. cnat DNATs the Service VIP to the node IP, which is directly
connected in the uplink subnet, so the packet bypasses SRv6 and is forwarded
over the uplink as plain IPv6. cnat also excludes node-IP destinations from
SNAT, so the pod source is preserved. On the receiving node that pod prefix is
only reachable through the SRv6 overlay, so the unencapsulated pod-sourced
packet fails the source/reverse-path lookup and is dropped (ip6 source lookup
miss).

Pod-backed ClusterIPs are unaffected: the backend is a pod address whose FIB
resolution is an SRv6 tunnel, so the path stays symmetric.

Steer pod-originated traffic destined to a remote node's own IPv6 address
(/128) through that node's already-advertised End.DT6 SID (steerNodeIPViaSID
in AddConnectivity). The steering entry is installed in PodVRFIndex so
node-to-node control traffic in the main table is untouched, and the pod
source identity is preserved (no SNAT). It reuses the remote node's existing
End.DT6 local SID (setEndDT, FibTable 0); no new dataplane primitive is needed.
The DT6 policy is already installed by the CreateSRv6Tunnel pass over the
node's pod prefixes, so we only add the steering and do not re-add the policy
(AddModSRv6Policy is delete-then-add and would churn the live policy shared
with the pod-prefix steering).
@sknat

sknat commented May 28, 2026

Copy link
Copy Markdown
Collaborator

/codebuild_run(ab2e7ca)

@sknat sknat merged commit f64163d into projectcalico:master Jun 1, 2026
5 checks passed
ryskn added a commit to ryskn/vpp-dataplane that referenced this pull request Jun 16, 2026
Per-node integration of the EgressPolicy CRD inside calico-vpp-agent. The
package watches EgressPolicy resources, decides which steering entries
should be installed on this node (selector → local pods × destinationCIDRs),
and delegates the actual VPP install/remove to a VPPInterface seam — which
is wired in production to connectivity.SRv6Provider's AddSRv6Steering /
DelSRv6Steering (the same pattern as PR projectcalico#1028's steerNodeIPViaSID).

Layout:
- doc.go              package overview + integration notes
- types.go            VPPInterface + SteeringRequest + policyState
- manager.go          per-node coordinator, idempotent diff/reconcile
- watcher.go          EgressPolicy CRD watcher (controller-runtime client.NewWithWatch)
- manager_test.go     lifecycle + key stability + deferred-when-not-ready
- README.md           wire-up steps for the agent's main bootstrap

A policy is only acted on once status.conditions[type=Ready]=True is set by
the bgp-controller and status.srPolicy.bsid is populated. Until then the
manager defers (no VPP writes).

This PR is the skeleton — the actual wiring into the agent's main lifecycle
and the srv6Provider adapter are left as follow-ups (clearly documented in
README.md).

Closes #13
ryskn added a commit to ryskn/vpp-dataplane that referenced this pull request Jun 17, 2026
Per-node integration of the EgressPolicy CRD inside calico-vpp-agent. The
package watches EgressPolicy resources, decides which steering entries
should be installed on this node (selector → local pods × destinationCIDRs),
and delegates the actual VPP install/remove to a VPPInterface seam — which
is wired in production to connectivity.SRv6Provider's AddSRv6Steering /
DelSRv6Steering (the same pattern as PR projectcalico#1028's steerNodeIPViaSID).

Layout:
- doc.go              package overview + integration notes
- types.go            VPPInterface + SteeringRequest + policyState
- manager.go          per-node coordinator, idempotent diff/reconcile
- watcher.go          EgressPolicy CRD watcher (controller-runtime client.NewWithWatch)
- manager_test.go     lifecycle + key stability + deferred-when-not-ready
- README.md           wire-up steps for the agent's main bootstrap

A policy is only acted on once status.conditions[type=Ready]=True is set by
the bgp-controller and status.srPolicy.bsid is populated. Until then the
manager defers (no VPP writes).

This PR is the skeleton — the actual wiring into the agent's main lifecycle
and the srv6Provider adapter are left as follow-ups (clearly documented in
README.md).

Closes #13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants