Skip to content

[caclmgrd] Own the redfish docker0 syslog INPUT exception - #429

Merged
yxieca merged 2 commits into
sonic-net:masterfrom
nexthop-ai:caclmgrd-own-redfish-docker0-syslog
Sep 3, 2026
Merged

yxieca merged 2 commits into
sonic-net:masterfrom
nexthop-ai:caclmgrd-own-redfish-docker0-syslog

Conversation

@shreyansh-nexthop

@shreyansh-nexthop shreyansh-nexthop commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Why I did it

redfish runs bridge-networked on platforms that enable it, so its rsyslog forwards to the docker0 gateway over RELP (tcp/2514) rather than to 127.0.0.1.

caclmgrd flushes and rebuilds the INPUT chain on every control-plane ACL change and appends a catch-all DROP. That forwarded syslog arrives on docker0, not lo, so it is not covered by the loopback ACCEPT and gets swept into the catch-all DROP. Logs emitted inside the redfish container never reach the host /var/log/syslog, including a process going FATAL at startup, which is exactly when they are most needed.

Installing the exception from the container start script does not work, for two independent reasons:

  1. caclmgrd owns the INPUT chain and flushes it on every reconcile, so a raw iptables -I is wiped on the next control-plane ACL change.
  2. redfish.service runs as the sonicadmin user, so iptables in ExecStartPre fails with "you must be root" and the rule is never installed at all.

This mirrors the dhcp_server docker0 syslog exception added in #412, which resolved the same problem for that container.

How I did it

Generalize the existing single dhcp_server exception into a loop over the bridged-container features, so caclmgrd owns the rule and re-emits it on every rebuild.

  • Emit iptables -A INPUT -i docker0 -p tcp --dport 2514 -j ACCEPT -m comment --comment redfish_syslog in get_acl_rules_and_translate_to_iptables_commands(), immediately after the loopback ACCEPTs and therefore before the catch-all DROP.
  • Host namespace and IPv4 only, since docker0 exists only in the host namespace.
  • Gated on the existing RedfishAllowed flag, which is already seeded from FEATURE|redfish state at init and updated on FEATURE-table events, so no new flag or event plumbing is needed.
  • Tagged with --comment redfish_syslog so the rule is identifiable in iptables -S and can be matched by a container-side iptables -C presence check.

Unit tests cover flag seeding from feature state, the rule being emitted when enabled and absent when disabled or when the FEATURE entry is missing, ordering strictly before the catch-all DROP, host-namespace-only emission, and the FEATURE-table event transitions across a runtime feature toggle.

How to verify it

On a platform with redfish enabled:

  1. Confirm the rule is present:
sudo iptables -S INPUT | grep redfish_syslog
  1. Configure a CTRLPLANE ACL (a table plus at least one rule, so caclmgrd programs its catch-all DROP), then confirm the exception is ordered above the DROP:
sudo iptables -S INPUT | grep -n -E 'redfish_syslog|-A INPUT -j DROP'
  1. Confirm it survives a reconcile:
sudo systemctl restart caclmgrd
sudo iptables -S INPUT | grep redfish_syslog
  1. With the companion buildimage change in place, force a new RELP connection so that an existing conntrack ESTABLISHED entry cannot mask the result, then probe end to end:
docker exec redfish supervisorctl restart rsyslogd
docker exec redfish logger -t probe HELLO
sudo grep probe /var/log/syslog

Verified on a platform with a CTRLPLANE ACL configured so caclmgrd programs its catch-all DROP:

  • redfish_syslog ACCEPT present and owned by caclmgrd.
  • Ordered above the catch-all DROP: exception at INPUT line 4, catch-all DROP at line 18.
  • Survived systemctl restart caclmgrd, still at line 4 with the DROP at line 18.
  • End to end: the container's rsyslogd was restarted first to force a new RELP connection. The connection established through the DROP and docker exec redfish logger was delivered to the host /var/log/syslog.
  • A REDFISH CTRLPLANE ACL table was exercised in the same run and produced its expected -A INPUT -s 10.0.0.0/8 -p tcp --dport 443 -j ACCEPT.

Note that this change alone does not make redfish syslog work. It only opens the firewall path. A companion sonic-buildimage change is required for delivery, pointing SYSLOG_TARGET_IP at the docker0 gateway and adding the host rsyslog docker0 listener.

Which release branch to port

  • master

Description for the changelog

[caclmgrd] Own the redfish docker0 syslog (RELP tcp/2514) INPUT exception so it survives the control-plane ACL flush-and-rebuild.

Link to config_db schema for YANG module changes

N/A, no schema change.

Signed-off-by: shreyansh-nexthop <shreyansh@nexthop.ai>
@mssonicbld

Copy link
Copy Markdown

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

…log tests

caclmgrd_redfish_acl_test.py already drives handle_feature_state_events through
the same four cases: enable transition, disable transition, unrelated key
ignored, and same-state no-op. The copy added alongside the syslog tests called
the pre-rename name and duplicated coverage that already exists.

Signed-off-by: Shreyansh Jain <shreyansh@nexthop.ai>
@mssonicbld

Copy link
Copy Markdown

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

nh-grecs Bot pushed a commit to nexthop-ai/sonic-host-services that referenced this pull request Sep 1, 2026
rebuild-source: sonic-net#429 @ nexthop-ai/sonic-host-services 48324e6 [case: upstream:open]
@shreyansh-nexthop
shreyansh-nexthop marked this pull request as ready for review September 1, 2026 06:27
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@yxieca yxieca left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve.

Makes caclmgrd own the redfish docker0 syslog (RELP tcp/2514) INPUT exception, consolidating the previously dhcp_server-only rule into a per-feature loop. Since redfish is bridge-networked, its rsyslog arrives on docker0 rather than lo and would otherwise be swept into the control-plane catch-all DROP.

Verified:

  • Fail-closed: RedfishAllowed defaults False and is seeded from FEATURE state, so the port only opens when redfish is explicitly enabled.
  • The exception is re-emitted on every rebuild immediately before the catch-all DROP, so there's no rebuild window where a DROP exists without it (covered by test_rule_reinserted_before_catch_all_drop).
  • Runtime enable/disable reprograms the rule via the FEATURE-table event handler; dhcp_server semantics are unchanged.
  • The test's rule constant is pinned byte-for-byte to the container-side -C check, keeping the caclmgrd/container contract in sync.

Non-blocking: the exception accepts any bridge container's RELP to docker0:2514, not strictly redfish's rsyslog — inherent to the shared docker0 bridge and identical to the existing dhcp_server exception, scoped by feature-enable.

Reviewed with AI assistance on behalf of Ying.

nh-grecs Bot pushed a commit to nexthop-ai/sonic-host-services that referenced this pull request Sep 3, 2026
rebuild-source: sonic-net#429 @ nexthop-ai/sonic-host-services 48324e6 [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-host-services that referenced this pull request Sep 3, 2026
rebuild-source: sonic-net#429 @ nexthop-ai/sonic-host-services 48324e6 [case: upstream:open]
@yxieca
yxieca merged commit 0cc4625 into sonic-net:master Sep 3, 2026
6 checks passed
@mssonicbld

Copy link
Copy Markdown

The label Approved for msft-202608 Branch can only be added by a member of the @sonic-net/release-manager-202608 team. Removing the label. Please contact one of the release managers: @yejianquan @yxieca @judyjoseph to approve the cherry pick.

@mssonicbld

Copy link
Copy Markdown

The change is not in msft-202608 yet. @shreyansh-nexthop, please manually create the cherry pick PR for branch msft-202608.
You can ping the release branch owner(github account: yxieca) to approve your cherry pick PR.
If this change is already in msft-202608, please comment "already in msft-202608". Thanks!

---Powered by SONiC BuildBot

@mssonicbld

Copy link
Copy Markdown

This PR has backport request label(s) for branch(es): msft-202608, but is missing required test information. Please make sure you tick the tested branch(es) in the Tested branch section and provide test evidence (e.g., 202608: <test result>) in the Test result section as well in your PR description.

---Powered by SONiC BuildBot

@weiguo-nvidia

Copy link
Copy Markdown

Hi @shreyansh-nexthop ,

Could you help resolve the cherry-pick conflict to 202608 branch

@shreyansh-nexthop

Copy link
Copy Markdown
Contributor Author

Hi @shreyansh-nexthop ,

Could you help resolve the cherry-pick conflict to 202608 branch

Have raised manual PR for 202608: Azure/sonic-host-services.msft#23 and Azure/sonic-buildimage-msft#3137. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants