-
Notifications
You must be signed in to change notification settings - Fork 72
[vpp] VXLAN L3 dataplane: inner-aware encap hash, underlay ECMP, source-independent decap (patches 0015-0017) #262
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
yue-fred-gao
merged 9 commits into
sonic-net:master
from
aaronber0614:vpp-vxlan-enablement
Aug 18, 2026
+501
−1
Merged
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
d86b662
[vpp] VXLAN dataplane enablement: inner-aware encap hash, underlay EC…
aaronber0614 49b076e
vxlan: harden source-independent (decap-any) VNET decap
aaronber0614 fdfd03d
Merge master into vpp-vxlan-enablement; renumber VXLAN patches to 001…
aaronber0614 c3eeb5e
Merge branch 'master' into vpp-vxlan-enablement
yejianquan 738d1df
Address review: correct stale patch reference in 0016 description
aaronber0614 2a72c38
[vpp] Revert VPP_VERSION to master 2606-0.3 (no per-PR bump)
aaronber0614 ab51dda
[vpp] Drop patch 0016 (global multipath tolerance) from VXLAN series
aaronber0614 7da2a84
[vpp] 0015: make VXLAN encap inner flow hash VLAN-aware
aaronber0614 9c78e1d
Merge branch 'master' into vpp-vxlan-enablement
aaronber0614 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
107 changes: 107 additions & 0 deletions
107
vppbld/patches/0015-vxlan-encap-inner-aware-flow-hash.patch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,107 @@ | ||
| From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001 | ||
| From: Aaron Bernardino <aaronber@microsoft.com> | ||
| Date: Fri, 10 Jul 2026 00:00:00 +0000 | ||
| Subject: [PATCH] vxlan: inner-aware flow hash in the encap node | ||
|
|
||
| The VXLAN encap node derives the outer UDP source port (and the buffer | ||
| flow hash used for underlay ECMP) from vnet_l2_compute_flow_hash(). That | ||
| helper reads the inner ethertype at current_data + l2.l2_len. For an | ||
| L3-routed VXLAN tunnel the inner packet arrives already routed (the | ||
| ip4/ip6-rewrite adjacency prepends the inner Ethernet header) and | ||
| l2.l2_len is 0/unset, so the helper misparses the frame and returns a | ||
| value that does not depend on the inner L3/L4 headers. Every inner flow | ||
| to a given tunnel endpoint then maps to the same outer UDP source port | ||
| and collapses onto a single underlay ECMP path / bond member. | ||
|
|
||
| This is the exact case that patch 0011 (inner-aware flow hash) left out of | ||
| scope on the assumption that "VXLAN ... their outer UDP source port is the | ||
| standard-mandated entropy carrier (RFC 7348 section 4.2), so the existing | ||
| outer-5-tuple hash already distributes inner flows." That assumption only | ||
| holds if the encap node actually populates the outer UDP source port from | ||
| the inner flow -- which it does not for L3-routed tunnels. This patch | ||
| fixes the entropy carrier at its source rather than adding the peek-inner | ||
| machinery of 0011 to the underlay. | ||
|
|
||
| Read the inner Ethernet ethertype directly at the buffer current pointer | ||
| (the inner Ethernet header is present for both L2 and L3 tunnels at the | ||
| encap call sites) and hash the inner IPv4/IPv6 5-tuple. Tagged inner | ||
| frames are handled by skipping up to two 802.1Q/802.1ad VLAN tags before | ||
| reading the inner ethertype; this cannot be delegated to | ||
| vnet_l2_compute_flow_hash() because that helper depends on l2.l2_len, | ||
| which is not valid on the routed path (it shares storage with | ||
| ip.flow_hash in the buffer opaque union). Anything that is not plain | ||
| inner IPv4/IPv6 still falls back to the generic vnet_l2_compute_flow_hash(). | ||
|
|
||
| Signed-off-by: Aaron Bernardino <aaronber@microsoft.com> | ||
| --- | ||
| diff --git a/src/plugins/vxlan/encap.c b/src/plugins/vxlan/encap.c | ||
| index 9d304b7..475239d 100644 | ||
| --- a/src/plugins/vxlan/encap.c | ||
| +++ b/src/plugins/vxlan/encap.c | ||
| @@ -57,6 +57,44 @@ format_vxlan_encap_trace (u8 * s, va_list * args) | ||
| } | ||
| #endif | ||
|
|
||
| +static_always_inline u32 | ||
| +vxlan_encap_inner_flow_hash (vlib_buffer_t * b) | ||
| +{ | ||
| + /* For L3 VXLAN tunnels the inner packet is routed (not bridged) into this | ||
| + * encap node, so vnet_buffer(b)->l2.l2_len is not set. The generic | ||
| + * vnet_l2_compute_flow_hash() then misparses the frame and returns a value | ||
| + * that does not depend on the inner L3/L4 headers, collapsing every inner | ||
| + * flow for a given tunnel endpoint onto a single outer UDP source port and | ||
| + * a single underlay ECMP path. The inner Ethernet header is present at the | ||
| + * current data pointer, so read its ethertype directly and hash the inner | ||
| + * IP 5-tuple. Skip up to two VLAN tags so tagged inner frames are hashed on | ||
| + * their real inner headers too. Fall back to the generic L2 hash for | ||
| + * anything that is not plain inner IPv4/IPv6. */ | ||
| + ethernet_header_t *eh = vlib_buffer_get_current (b); | ||
| + u16 ethertype = clib_net_to_host_u16 (eh->type); | ||
| + u8 *l3h = (u8 *) (eh + 1); | ||
| + | ||
| + if (ethertype == ETHERNET_TYPE_VLAN || ethertype == ETHERNET_TYPE_DOT1AD) | ||
| + { | ||
| + ethernet_vlan_header_t *vlan = (ethernet_vlan_header_t *) l3h; | ||
| + ethertype = clib_net_to_host_u16 (vlan->type); | ||
| + l3h = (u8 *) (vlan + 1); | ||
| + if (ethertype == ETHERNET_TYPE_VLAN) | ||
| + { | ||
| + vlan = (ethernet_vlan_header_t *) l3h; | ||
| + ethertype = clib_net_to_host_u16 (vlan->type); | ||
| + l3h = (u8 *) (vlan + 1); | ||
| + } | ||
| + } | ||
| + | ||
| + if (ethertype == ETHERNET_TYPE_IP4) | ||
| + return ip4_compute_flow_hash ((ip4_header_t *) l3h, IP_FLOW_HASH_DEFAULT); | ||
| + else if (ethertype == ETHERNET_TYPE_IP6) | ||
| + return ip6_compute_flow_hash ((ip6_header_t *) l3h, IP_FLOW_HASH_DEFAULT); | ||
| + | ||
| + return vnet_l2_compute_flow_hash (b); | ||
| +} | ||
| + | ||
| always_inline uword | ||
| vxlan_encap_inline (vlib_main_t *vm, vlib_node_runtime_t *node, | ||
| vlib_frame_t *from_frame, u8 is_ip4) | ||
| @@ -125,8 +163,8 @@ vxlan_encap_inline (vlib_main_t *vm, vlib_node_runtime_t *node, | ||
| vlib_buffer_t *b1 = b[1]; | ||
| b += 2; | ||
|
|
||
| - u32 flow_hash0 = vnet_l2_compute_flow_hash (b0); | ||
| - u32 flow_hash1 = vnet_l2_compute_flow_hash (b1); | ||
| + u32 flow_hash0 = vxlan_encap_inner_flow_hash (b0); | ||
| + u32 flow_hash1 = vxlan_encap_inner_flow_hash (b1); | ||
|
|
||
| /* Get next node index and adj index from tunnel next_dpo */ | ||
| if (sw_if_index0 != vnet_buffer (b0)->sw_if_index[VLIB_TX]) | ||
| @@ -351,7 +389,7 @@ vxlan_encap_inline (vlib_main_t *vm, vlib_node_runtime_t *node, | ||
| vlib_buffer_t *b0 = b[0]; | ||
| b += 1; | ||
|
|
||
| - u32 flow_hash0 = vnet_l2_compute_flow_hash (b0); | ||
| + u32 flow_hash0 = vxlan_encap_inner_flow_hash (b0); | ||
|
|
||
| /* Get next node index and adj index from tunnel next_dpo */ | ||
| if (sw_if_index0 != vnet_buffer (b0)->sw_if_index[VLIB_TX]) | ||
| -- | ||
| 2.34.1 | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we support tagged routed packets? today it will fall back to vnet_l2_compute_flow_hash, which doesn't have proper l2.l2_len set. So the problem still exists for such flows. Also, should we check l2.l2_len first? If it is properly set, we can use vnet_l2_compute_flow_hash, which makes uses properly parsed l2 headers. If it is not set, we fall back to this path to parse the l2 header.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@yue-fred-gao
You are right that tagged routed frames still fall through today, I will fix that.
On why I would not check l2.l2_len first: on a routed frame it is not reliable. The packet reaches the encap node through ip4/ip6-rewrite, and l2.l2_len sits on the same memory as ip.flow_hash in the buffer opaque union. The IP lookup always writes that field before we get here, so l2_len ends up as 0 or a hash byte, not a real header length, and there is no valid bit to tell the difference. Trusting it would sometimes send routed frames back into the same misparse we are fixing, which is why 0015 reads the inner header directly.
So instead of gating on l2_len, I will make the direct parse skip VLAN tags and hash the real inner IP:
if (ethertype == ETHERNET_TYPE_VLAN || ethertype == ETHERNET_TYPE_DOT1AD)
{
ethernet_vlan_header_t *vlan = (ethernet_vlan_header_t *) l3h;
ethertype = clib_net_to_host_u16 (vlan->type);
l3h = (u8 ) (vlan + 1); / handles single and QinQ tags */
}
The bridged path stays identical, tagged routed frames now hash correctly, and anything unusual still falls back to vnet_l2_compute_flow_hash. I will fold this into 0015 and revalidate on the testbed before pushing.
Co-authored with CoPilot