[DONT MERGE] Test VPP ACL IN_PORTS support by lolyu · Pull Request #29248 · sonic-net/sonic-buildimage

lolyu · 2026-08-29T09:38:06Z

Why I did it

Do not merge. This PR exists only to produce a test image; it will not be merged.

SAI ACL IN_PORTS is not honoured by the VPP dataplane. saivpp accepts the qualifier and reports success, then discards the scope and programs the entry as if it were absent. An entry meant to drop on one port drops on every port the table is bound to.

On a dual-ToR testbed this inverts MuxOrch's standby drop rule into a box-wide IPv4 outage: with one port in standby, acl-index 3 (ipv4 deny any -> any) was applied inbound on all 32 front-panel ports.

The fix spans two repositories, which is why this PR points at both:

PR	Change
sonic-net/sonic-platform-vpp#278	Add `in_sw_if_index` to the VPP ACL rule so a rule can match the ingress interface
sonic-net/sonic-sairedis#2064	Fan a scoped SAI entry out into one VPP rule per named port
sonic-net/sonic-platform-vpp#280	Stop counting ACL policy denies as interface drops, so a standby mux port does not report a ~99% `RX_DRP` rate

#280 is independent of IN_PORTS and is in review on its own, but the image needs it: without it dualtor_io/test_normal_op.py::test_upstream_standby_rx_drop_check fails on the very drop rules this feature installs.

HLD: sonic-net/SONiC#2506

Work item tracking

Microsoft ADO (number only):

How I did it

Repointed the submodules at the PR branches:

platform/vpp       -> lolyu/sonic-platform-vpp @ test_acl_in_ports_combined (02abe7d)
src/sonic-sairedis -> lolyu/sonic-sairedis     @ vpp_acl_in_port_match      (e0de9e7)
src/sonic-swss     -> lolyu/sonic-swss         @ test_vpp_acl_in_ports_swss (8855e07)

test_acl_in_ports_combined exists only for CI. The two sonic-platform-vpp changes were originally stacked on one branch, but they are independent and are now in review separately, so no single review branch carries both. It merges #278 and #280 with the patches numbered 0018 (IN_PORTS) and 0019 (policy drop), and VPP_VERSION at 2606-0.7.

That version needs care. Both PR branches set 2606-0.6, and because the edit is textually identical on each side git auto-resolves rules/vpp.mk without reporting a conflict — only patches/series conflicts. Left alone, the merge would publish a deb under a version minted for one patch series while actually containing two, which is precisely the stale-cache failure rules/vpp.mk warns about. The bump to 0.7 is therefore deliberate and manual, and it matches the image already validated on the testbed.

The two must move together. Adding in_sw_if_index to the acl_rule type changes the layout of the acl_add_replace message and therefore its CRC, which VPP uses to detect API mismatch. A syncd built against one side cannot program ACLs on the other, so neither branch is testable alone.

src/sonic-swss carries an unrelated intfsorch VLAN RIF MAC fix (sonic-net/sonic-swss#4816) needed for this testbed to come up; it is not part of the IN_PORTS feature.

VPP is rebuilt from source. rules/vpp.mk carries VPP_VERSION = 2606-0.7 for the combined patch series. That deb is not published (404 on packages.buildkite.com), so the curl -L -f download step fails and vppbld/Makefile falls back to build_locally, which applies the full patch series and builds VPP. The image build therefore also verifies the patch applies cleanly and compiles under the real build flags (-Werror -Wall -O3). Expect the VPP build to add significant time to this job.

How to verify it

Deploy the resulting image to a dualtor-vpp testbed and confirm the drop rule is scoped to the standby ports.

The binding is deliberately unchanged — the ACL stays bound to every front-panel port, and the scope now comes from the rules:

vppctl show acl-plugin acl
  acl-index 3:
    rule 0: ipv4 deny any -> any  in-port bobm2     <-- one rule per standby port
    rule 1: ipv4 deny any -> any  in-port bobm7
    ...

vppctl show acl-plugin interface
  sw_if_index 2 (Ethernet4,  STANDBY):  input acl(s): 2, 3, 1
  sw_if_index 3 (Ethernet8,  ACTIVE ):  input acl(s): 2, 3, 1   <-- same list, different verdict

Check	Expectation
Rule count	Equals the standby port count, and each rule's `in-port` is a standby port
Binding	ACL still bound to all front-panel ports
Downstream forwarding	Traffic to a server behind an active mux port is forwarded; behind a standby port it is dropped
Mux toggle	standby → active → standby converges, with the rule set tracking the standby membership each time
Mux health	`show mux status` reports `healthy`
No-`IN_PORTS` regression	A table with no scoped entries produces the same ACL as before the change

Pre-fix, the same commands showed acl-index 3 on sw_if_index 1 .. 32 with a single unscoped deny any -> any rule.

Which release branch to backport (provide reason below if selected)

Tracking issue/work item for backport/cherry-pick request (GitHub issue or Microsoft ADO):
Failure type:

Tested branch

master

Test result

Image build pending.

Rule-level state was verified on vlab-vpp-03 (18 standby ports → 18 ingress-scoped deny rules) and vlab-vpp-04 (6 → 6), with ACL 3 still bound to all 32 interfaces on both.

A 2606-0.7 image carrying this same patch pair was deployed to both DUTs and dualtor_io/test_normal_op.py::test_upstream_standby_rx_drop_check passed — RX_DRP diff=0 against a threshold of 10, on a port with 5,762 RX_OK, where it had previously failed at +5706. Denies remain visible at the ACL node (vppctl show errors → 11398 acl-plugin-in-ip4-l2 ACL deny packets), so the counter is corrected rather than the drops hidden.

Downstream forwarding under mux toggles is what this image is being built to exercise more broadly.

Description for the changelog

N/A — test-only PR, not for merge.

Link to config_db schema for YANG module changes

N/A — no schema change. IN_PORTS is an existing CONFIG_DB ACL_RULE field.

azure-pipelines · 2026-08-29T09:38:16Z

Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run.

mssonicbld · 2026-08-29T09:38:25Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-08-29T09:38:42Z

Azure Pipelines: Successfully started running 1 pipeline(s).

mssonicbld · 2026-08-31T05:37:37Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-08-31T05:37:43Z

Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command.

lolyu · 2026-08-31T05:38:26Z

First test image failed every dualtor server-traffic case — root-caused, not an ACL issue

The image built from the first revision (master-29248.1206265) was deployed to vlab-vpp-03/04 and dualtor_io failed with zero packets received, e.g. test_normal_op_active_server_to_standby_server:

sent_packets: 5710,  received_packets: 0
Sniffer failed to filter any traffic from DUT

The ACL change is working correctly

Both fan-outs are exactly right. Upper ToR, with only Ethernet64 standby:

acl-index 3 count 1 tag {sonic_acl_oid:0x700000002}
  0: ipv4 deny src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0 dport 0 in-port bobm16
  applied inbound on sw_if_index: 1, 26, 27, ... 25      <- still all 32 ports

Peer ToR, with the other 23 mux ports standby, has 23 rules — bobm16 correctly absent.

Live 20-packet replay, 192.168.0.22 (active bobm21) → 192.168.0.17 (standby bobm16):

DUT	role for ingress port	`acl-plugin-in-ip4-l2`
`vlab-vpp-03`	active	20 permit / 20 checked
`vlab-vpp-04`	standby	20 deny / 20 checked

That is precisely the intended behaviour: the active ToR forwards, the standby ToR drops the Y-cable copy on the port it is standby for.

The real cause

VPP trace on the upper ToR — the ACL permits, then the frame dies two nodes later:

acl-plugin-in-ip4-l2  action: 1, match: acl 1 rule 0
                      lsb_of_sw_if_index 22          <- new match field populated
l2-fwd                dst 00:aa:bb:cc:dd:ee  result [0xffffffffffffffff, -1]
error-drop            rx:bobm21
drop                  l2-fwd: L2 forward misses

The bridge domain's BVI carries the switch MAC, not the shared VLAN gateway MAC:

Vlan1000 (kernel) link/ether 00:aa:bb:cc:dd:ee   <- what servers address
bvi1000  (VPP)    Ethernet address 22:ff:ab:76:97:c0
show l2fib bd_id 1000 | grep -c 00:aa:bb:cc:dd:ee  ->  0
BD 1000 UU-Flood: drop

So every frame a server sends to its default gateway misses the L2 FIB and is dropped.

Isolation test — same packet, same ports, same ACL state, only the destination MAC changed:

dst MAC	delivered
`00:aa:bb:cc:dd:ee` (VLAN gateway)	0 / 20
`22:ff:ab:76:97:c0` (BVI)	10 / 10

The delivered copies arrive with src_mac 22:ad:a2:34:08:76, the peer ToR's switch MAC, so the whole path — upper ToR, IPinIP, T1, peer decap, egress on Ethernet64 — is healthy. Note that source MAC is also the wrong one: it should be the shared VLAN MAC, which is why the test's received_pkt_src_mac filter would reject it even if it arrived.

Both symptoms are sonic-net/sonic-swss#4816: IntfsOrch::doTask() creates the VLAN RIF with the VLAN MAC and then immediately overwrites it with gMacAddress.

Fix

I originally left src/sonic-swss at master reasoning that no swss change is involved in ACL IN_PORTS. That was wrong — the dualtor VPP testbed cannot route any server traffic without that fix, so the ACL change was untestable. Reference PR #29062 carried it for this reason.

src/sonic-swss now points at lolyu/test_vpp_acl_in_ports_swss, which is the current buildimage master pointer 93fc008c with the two fix commits cherry-picked on top. The PR branch itself is 18 commits behind that pointer, so pointing at it directly would have regressed swss.

mssonicbld · 2026-08-31T07:41:00Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-08-31T07:41:14Z

Azure Pipelines: Successfully started running 1 pipeline(s).

mssonicbld · 2026-09-01T12:38:52Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-09-01T12:39:00Z

Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command.

…RTS branches Build a test image carrying the SAI ACL IN_PORTS support so it can be deployed to a dual-ToR VPP testbed. The two changes are a matched pair and cannot be tested apart. Adding in_sw_if_index to the VPP acl_rule type changes the layout of the acl_add_replace message and therefore its CRC, so a syncd built against one side cannot program ACLs on the other. platform/vpp -> lolyu/acl_match_in_port sonic-net/sonic-platform-vpp#278 src/sonic-sairedis -> lolyu/vpp_acl_in_port_match sonic-net/sonic-sairedis#2064 vpp.mk carries VPP_VERSION 2606-0.6 for the new patch series. That deb is not published, so the download step fails and vppbld/Makefile falls back to build_locally, which applies the patch series and builds VPP from source. The image build therefore also verifies that the patch applies and compiles under the real build flags. Signed-off-by: Longxiang Lyu <lolv@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Longxiang Lyu <lolv@microsoft.com>

The first test image built from this PR failed every dualtor server traffic case: servers sent to their gateway and nothing came back. The ACL work is not at fault. A VPP trace on vlab-vpp-03 shows the frame is permitted by the ACL and then dropped two nodes later: acl-plugin-in-ip4-l2 action: 1 (permit), acl 1 rule 0 lsb_of_sw_if_index 22 l2-fwd dst 00:aa:bb:cc:dd:ee result [0xffff..., -1] error-drop l2-fwd: L2 forward misses bvi1000 carries the switch MAC 22:ff:ab:76:97:c0 while Vlan1000 in the kernel carries the shared gateway MAC 00:aa:bb:cc:dd:ee, and the bridge domain has unknown-unicast flooding set to drop. Every frame a server sends to its default gateway therefore misses the L2 FIB and is dropped. That is sonic-net/sonic-swss#4816: IntfsOrch::doTask() creates the RIF with the VLAN MAC and then immediately overwrites it with gMacAddress. The reference PR 29062 carried that branch; omitting it here left the testbed unable to route any server traffic. Point src/sonic-swss at test_vpp_acl_in_ports_swss, which is the current buildimage master pointer 93fc008c with the two fix commits cherry-picked on top. The PR branch itself is 18 commits behind that pointer, so using it directly would have regressed swss. Signed-off-by: Longxiang Lyu <lolv@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Longxiang Lyu <lolv@microsoft.com>

Bump platform/vpp to 3f6f3f8, which adds VPP patch 0019 so an ACL deny no longer increments the ingress interface drop counter. The previous test image passed dualtor traffic correctly but failed dualtor_io/test_normal_op.py::test_upstream_standby_rx_drop_check: Ethernet16 on the standby ToR reported RX_DRP +5706 against a threshold of 10, because VPP charged every intentional ACL deny to /if/drops, which surfaces as RX_DRP. VPP_VERSION moves to 2606-0.7 with this bump, so the image build will again build VPP from source rather than fetching a cached deb. Signed-off-by: Longxiang Lyu <lolv@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Longxiang Lyu <lolv@microsoft.com>

mssonicbld · 2026-09-01T12:40:39Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-09-01T12:40:58Z

Azure Pipelines: Successfully started running 1 pipeline(s).

sonic-platform-vpp: the branch this pointed at carried both the IN_PORTS plugin patch and the ACL policy-drop counter fix. Those have since been split into two independent PRs (sonic-net#278 and sonic-net#280), so no single review branch holds both any more. Point at test_acl_in_ports_combined, which merges them for CI only, with the patches renumbered 0018/0019 and VPP_VERSION at 2606-0.7 - the same combination already validated on the vms-kvm-dual-vpp-t0-1 testbed. Note that merging the two branches conflicts only in patches/series; rules/vpp.mk auto-resolves silently because both sides set 2606-0.6 identically, so the version was bumped to 0.7 by hand. Leaving it at 0.6 would have published a deb under a version minted for one patch series while containing two. sonic-sairedis: pick up the egress-stage guard, so an IN_PORTS entry in an egress table emits no rule instead of being matched against the TX interface. Signed-off-by: Longxiang Lyu <lolv@microsoft.com>

mssonicbld · 2026-09-02T05:58:46Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-09-02T05:59:04Z

Azure Pipelines: Successfully started running 1 pipeline(s).

The IN_PORTS patch had been renumbered to 0019 when the policy-drop fix was split into its own PR and took 0018. 0018 is the next free slot after master's 0017, and it belongs to the feature PR, so the two have been swapped back: sonic-platform-vpp#278 is 0018 and sonic-net#280 is 0019. Numbers only; the patch contents are unchanged and the two touch disjoint files, so the combined series still applies in either order. VPP_VERSION stays at 2606-0.7 for the CI image. Signed-off-by: Longxiang Lyu <lolv@microsoft.com>

mssonicbld · 2026-09-03T04:19:58Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-09-03T04:20:04Z

Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command.

sonic-sairedis: an entry whose IN_PORTS cannot be resolved now emits no rule for that entry instead of failing the whole ACL table. Relevant to this image because IngressTableDrop is shared between MuxOrch and the PFC watchdog, so the old behaviour could freeze the mux drop table on a startup port-ordering race. Signed-off-by: Longxiang Lyu <lolv@microsoft.com>

mssonicbld · 2026-09-03T07:59:54Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-09-03T08:00:01Z

Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command.

sonic-platform-vpp#278 has merged, so the hand-merged test_acl_in_ports_combined branch is no longer needed: patch 0018 is on master and sonic-net#280 has been rebased onto it, which leaves that branch carrying both patches at VPP_VERSION 2606-0.7 by itself. Point the submodule at acl_policy_drop_not_if_drop instead. The tree is identical to the combined branch apart from one stray blank line in series, so this changes nothing about what is built, but the image now tracks a branch that is actually under review rather than a throwaway merge that has to be re-created by hand every time either side moves. Signed-off-by: Longxiang Lyu <lolv@microsoft.com>

mssonicbld · 2026-09-03T12:07:44Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-09-03T12:07:51Z

Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command.

Resolves the src/sonic-sairedis conflict. Both sides moved the pointer from f6344f0b7: master to 1205916dd, this branch to its feature commit. The feature branch has been rebased onto 1205916dd, so its head is a descendant of master's pointer and is the correct resolution - the image keeps the l2-vlan-filter FDB change rather than silently dropping it. Signed-off-by: Longxiang Lyu <lolv@microsoft.com>

mssonicbld · 2026-09-03T12:10:56Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-09-03T12:11:11Z

Azure Pipelines: Successfully started running 1 pipeline(s).

sonic-sairedis: warn when an ACL entry's attribute dump hits MAX_ACL_ATTRS, since resolving IN_PORTS now depends on that qualifier surviving a cap that drops attributes by name order. Signed-off-by: Longxiang Lyu <lolv@microsoft.com>

mssonicbld · 2026-09-05T07:23:35Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-09-05T07:23:42Z

Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command.

lolyu force-pushed the test_vpp_acl_in_ports branch from 6b71afd to b003690 Compare August 31, 2026 07:40

lolyu and others added 3 commits September 1, 2026 12:39

lolyu force-pushed the test_vpp_acl_in_ports branch from 8a9c421 to 8835222 Compare September 1, 2026 12:40

lolyu closed this Sep 9, 2026

Uh oh!

Conversation

lolyu commented Aug 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why I did it

Work item tracking

How I did it

How to verify it

Which release branch to backport (provide reason below if selected)

Tested branch

Test result

Description for the changelog

Link to config_db schema for YANG module changes

Uh oh!

azure-pipelines Bot commented Aug 29, 2026

Uh oh!

mssonicbld commented Aug 29, 2026

Uh oh!

azure-pipelines Bot commented Aug 29, 2026

Uh oh!

mssonicbld commented Aug 31, 2026

Uh oh!

azure-pipelines Bot commented Aug 31, 2026

Uh oh!

lolyu commented Aug 31, 2026

First test image failed every dualtor server-traffic case — root-caused, not an ACL issue

The ACL change is working correctly

The real cause

Fix

Uh oh!

mssonicbld commented Aug 31, 2026

Uh oh!

azure-pipelines Bot commented Aug 31, 2026

Uh oh!

mssonicbld commented Sep 1, 2026

Uh oh!

azure-pipelines Bot commented Sep 1, 2026

Uh oh!

mssonicbld commented Sep 1, 2026

Uh oh!

azure-pipelines Bot commented Sep 1, 2026

Uh oh!

mssonicbld commented Sep 2, 2026

Uh oh!

azure-pipelines Bot commented Sep 2, 2026

Uh oh!

mssonicbld commented Sep 3, 2026

Uh oh!

azure-pipelines Bot commented Sep 3, 2026

Uh oh!

mssonicbld commented Sep 3, 2026

Uh oh!

azure-pipelines Bot commented Sep 3, 2026

Uh oh!

mssonicbld commented Sep 3, 2026

Uh oh!

azure-pipelines Bot commented Sep 3, 2026

Uh oh!

mssonicbld commented Sep 3, 2026

Uh oh!

azure-pipelines Bot commented Sep 3, 2026

Uh oh!

mssonicbld commented Sep 5, 2026

Uh oh!

azure-pipelines Bot commented Sep 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

lolyu commented Aug 29, 2026 •

edited

Loading