feat(install): apply the Workerman kernel tuning on install and --update - #602
Conversation
Implements the applicable parts of Workerman's kernel-optimization appendix (https://www.workerman.net/doc/workerman/appendices/kernel-optimization.html) in scripts/install.sh, on both the fresh-install path and `--update`, so an install that predates this picks the tuning up on the next update. The finding that motivated this: on the live box both phlix daemons were running with a soft RLIMIT_NOFILE of 1024. The appendix prescribes /etc/security/limits.conf for this, but systemd does not read limits.conf at all — the daemon's ceiling comes from LimitNOFILE= in the unit, which was unset, so the service inherited systemd's soft default of 1024. Workerman holds one fd per live connection, and an HTTP worker serving HLS also holds the open segment files, so 1024 is reachable under load and surfaces as "Too many open files" with clients dropped mid-stream. Applied: - LimitNOFILE=1048576 in the unit template, plus an idempotent `--update` retrofit that only fires when the directive is absent (an operator who tuned it by hand keeps their value). Anchored after LimitMEMLOCK= when present, else inserted before ExecStart=. - /etc/sysctl.d/99-phlix-net.conf: somaxconn 4096 -> 65535 (this value silently clamps every listen() backlog), tcp_max_syn_backlog 4096 -> 262144, netdev_max_backlog 1000 -> 30000, ip_local_port_range 32768-60999 -> 16384-65535. - /etc/security/limits.d/99-phlix.conf: nofile soft 65536 / hard 1048576 for root and the service user. PAM-only, so this covers login shells and hand-run CLI scripts (scripts/*.php) but explicitly NOT the daemon. Deliberately NOT applied, with the reason recorded inline in the generated sysctl file so a future reader does not "restore" them: - fs.file-max=6815744 — the live default is 9223372036854775807; writing the page's value is a ~10^12x downgrade. - tcp_max_tw_buckets=20000 — the live default is 262144, and this is a cap: once exceeded the kernel kills TIME_WAIT sockets early and logs an overflow. 13x downgrade. - tcp_tw_recycle=0 — removed from Linux in 4.12; the key does not exist and listing it makes `sysctl --system` exit non-zero. - nf_conntrack_max=2621440 — only meaningful behind a stateful firewall (ufw is inactive here, conntrack_count is 107 against a max of 262144), pins ~1 GB of unswappable kernel memory, and needs a matching nf_conntrack_buckets bump or it just lengthens the hash chains. The sysctl/limits drop-ins are owned outright and rewritten wholesale rather than editing the shared /etc/sysctl.conf, which makes re-runs idempotent; --uninstall removes them. Containers get a read-only or namespaced /proc/sys, so the function detects that and skips with a warning instead of failing the install. The PAM drop-in's account is passed to the function as an ARGUMENT rather than by reassigning $SERVICE_USER, because do_update's chown steps on either side of the call site also read that global. Verified: the retrofit sed was extracted verbatim from this script and run against the real live unit files in three shapes (has LimitMEMLOCK, lacks LimitMEMLOCK, already patched). All assert one LimitNOFILE line, correct [Service] placement, a purely additive diff (zero deleted lines), idempotency on re-run, and a clean `systemd-analyze verify`. All four sysctl keys were confirmed to exist on the target kernel, so `sysctl -p` cannot fail on them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Coverage variation | ✅ -0.05% coverage variation (-1.00%) |
| Diff coverage | ✅ ∅ diff coverage |
Coverage variation details
Coverable lines Covered lines Coverage Common ancestor commit (d620c87) 63701 42022 65.97% Head commit (43d55e4) 63701 (+0) 41991 (-31) 65.92% (-0.05%) Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch:
<coverage of head commit> - <coverage of common ancestor commit>
Diff coverage details
Coverable lines Covered lines Diff coverage Pull request (#602) 0 0 ∅ (not applicable) Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified:
<covered lines added or modified>/<coverable lines added or modified> * 100%
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #602 +/- ##
============================================
- Coverage 66.80% 66.76% -0.05%
Complexity 21657 21657
============================================
Files 668 668
Lines 67864 67864
============================================
- Hits 45339 45307 -32
- Misses 22525 22557 +32 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Implements the applicable parts of Workerman's kernel-optimization appendix in
scripts/install.sh, on both the fresh-install path and--update, so an install that predates this picks the tuning up on the next update.The finding
On the live box both phlix daemons were running with a soft
RLIMIT_NOFILEof 1024:The appendix prescribes
/etc/security/limits.conffor this — but systemd does not readlimits.confat all. The daemon's ceiling comes fromLimitNOFILE=in the unit, which was unset, so the service inherited systemd's soft default of 1024. Workerman holds one fd per live connection, and an HTTP worker serving HLS also holds the open segment files, so 1024 is reachable under load and surfaces as "Too many open files" with clients dropped mid-stream.Applied
LimitNOFILE=(systemd unit)net.core.somaxconnnet.ipv4.tcp_max_syn_backlognet.core.netdev_max_backlognet.ipv4.ip_local_port_range32768 60999(28,232 ports)16384 65535(49,152 ports)nofile(root + service user)somaxconnwas the other silent one — it clamps everylisten()backlog, visible asSend-Q 4096on all phlix listen sockets inss -ltn.The port range is a widening (+74%); the floor stops at 16384 rather than the page's 10240 so the registered-port band — which contains every port Phlix listens on (2206, 8096, 8097, 8800, 8802-8805) — stays outside the ephemeral range, making an
ip_local_reserved_portsentry unnecessary.Deliberately NOT applied
Each with the reason recorded inline in the generated sysctl file, so a future reader doesn't "restore" them:
fs.file-max=6815744— live default is9223372036854775807. Writing the page's value is a ~10¹²× downgrade.tcp_max_tw_buckets=20000— live default is 262144, and this is a cap: once exceeded the kernel destroys TIME_WAIT sockets early and logs an overflow. 13× downgrade.tcp_tw_recycle=0— removed from Linux in 4.12 (2017). The key doesn't exist; listing it makessysctl --systemexit non-zero.nf_conntrack_max=2621440— only meaningful behind a stateful firewall (ufw is inactive;nf_conntrack_countis 107 against a max of 262144). Pins ~1 GB of unswappable kernel memory and needs a matchingnf_conntrack_bucketsbump or it just lengthens the hash chains.Design notes
/etc/sysctl.conf, which makes re-runs idempotent.--uninstallremoves them.--updateLimitNOFILEretrofit fires only when the directive is absent, so an operator who tuned it by hand keeps their value./proc/sys; the function detects that and skips with a warning rather than failing the install.$SERVICE_USER—do_update's chown steps on either side of the call site read that global, and moving it would silently re-target them.Verification
The retrofit
sedwas extracted verbatim from this script (so the test can't drift from shipped code) and run against the real live unit files in three distinct shapes — hasLimitMEMLOCK, lacksLimitMEMLOCK, already patched:Also verified: all four sysctl keys exist on the target kernel (so
sysctl -pcannot fail on them); the generatedlimits.confis well-formed (4 fields/line); the user argument is honoured with no global leak;bash -nclean; and the--uninstallloop is safe underset -euo pipefailin all three file-presence cases.Paired with detain/phlix-hub#(same change).
🤖 Generated with Claude Code