From 91068efff77e1a799b9408fe308cc32eef396966 Mon Sep 17 00:00:00 2001 From: Aybo <265313202+Aybook@users.noreply.github.com> Date: Wed, 15 Jul 2026 18:36:14 +0200 Subject: [PATCH 01/18] feat(server): poll() event loop on POSIX, lifting the 1024-socket ceiling (#310) (#436) The hub's single event loop watches every connected socket in one socket.select call; luasocket's select.c builds a fixed 1024-bit fd_set, so at ~1024 sockets the call raised an uncaught error and crashed the whole hub, dropping every user - the Linux sibling of the Windows #416 crash at 64. luasocket/src/select.c gains a global_poll backend (variable-length struct pollfd[] via lua_newuserdata, no FD_SETSIZE cap) registered under the same `select` name behind a #if defined(_WIN32) guard: POSIX gets poll, Windows keeps fd_set/select (FD_SETSIZE=1024, #416), the hub side is untouched. Contract parity with select is exact - check_dirty force-include of buffered-decrypted TLS sockets, empty-not-nil result tables on timeout, a socket reportable in both read+write lists in one tick, errored fds surfaced into every list they were watched in. hub/hub.c raises RLIMIT_NOFILE soft->hard at boot so the default ulimit -n = 1024 does not become the new silent cap; getfdlimit() exposes the resulting ceiling and the boot log reports the backend + ceiling. The #416 smoke guard became backend-aware (the Linux leg asserts the poll line, failing pre-fix; Windows keeps FD_SETSIZE >= 1024). Co-authored-by: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 2 + core/hub.lua | 35 +++-- docs/BUILDING.md | 32 +++++ hub/hub.c | 72 ++++++++++ luasocket/src/select.c | 308 ++++++++++++++++++++++++++++++++++------- tests/smoke/run.py | 83 ++++++----- 6 files changed, 438 insertions(+), 94 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3665fabf..83a6cebf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -144,6 +144,8 @@ land on `release/3.1.x` per ### Features +- [#310](https://github.com/luadch-ng/luadch/issues/310) - **the POSIX event loop now uses `poll()` instead of `select()`, removing the ~1024 concurrent-socket ceiling.** The hub's single event loop (`core/server.lua` `tick()`) watches every connected socket in one `socket.select` call; the bundled luasocket `select.c` builds a fixed 1024-bit `fd_set`, so once the hub held ~1024 sockets (users + v4/v6 listeners + in-handshake conns + HBRI + HTTP) the call raised an uncaught error and the whole hub crashed, dropping every user - the Linux sibling of the Windows [#416](https://github.com/luadch-ng/luadch/issues/416) crash at 64. `luasocket/src/select.c` gains a `global_poll` backend (variable-length `struct pollfd[]` via `lua_newuserdata`, no FD_SETSIZE cap) registered under the same `select` name behind a `#if defined(_WIN32)` guard, so the hub side is untouched on POSIX and Windows keeps its `fd_set`/`select` path (FD_SETSIZE=1024, #416). Contract parity with the select backend is exact: the `check_dirty` force-include of TLS sockets holding buffered-decrypted data (LuaSec), empty-not-nil result tables on timeout, a socket reportable in both read and write lists in one tick, and errored fds surfaced into every list they were watched in. `hub/hub.c` raises `RLIMIT_NOFILE` soft->hard at boot so the default `ulimit -n = 1024` does not become the new silent cap; the practical ceiling is now the process file-descriptor limit (raise further via `ulimit -n` / systemd `LimitNOFILE` / Docker `--ulimit nofile`). The boot line reports the backend and the real fd ceiling; the #416 smoke guard became backend-aware (the Linux leg asserts the poll line, which fails pre-fix; Windows keeps the `FD_SETSIZE >= 1024` assertion). No behavioural change on Windows. 3.2.x only. + - **#78 allowlist, Phase D: HTTP API for the whitelist (`etc_whitelist` v0.02).** Four endpoints mirroring the blocklist HTTP API: `GET /v1/whitelist` (read, filter/sort/paginate via `http_filter`), `GET /v1/whitelist/counts` (read), `POST /v1/whitelist` (admin, body `{cidr, source?, reason?, expires_at?}`, source enum `{manual, pinger}`, host-bits-set CIDRs rejected), `DELETE /v1/whitelist/{id}` (admin). HTTP admin tokens bypass the ADC hierarchy guard (the token is the trust surface; entries attributed `by_nick = ` / `by_level = 100`). `etc_whitelist_test` gains 32 HTTP checks (registration + scope + schema + all four handlers, happy + error paths); a seed-aware `test_http_phase_d_whitelist` smoke does a real token-authed POST -> GET -> DELETE roundtrip (baseline count captured so it stays robust to the pinger seed). Docs: `SCRIPTS.md` (new `etc_whitelist` section) + `BLOCKLIST.md` (allowlist layer). Completes the 4-PR #78 allowlist arc. 3.2.x only. - **#78 allowlist, Phase C: the IP-blocking plugins now consult the whitelist first.** `etc_geoip` (country/ASN kick), `etc_proxydetect` (proxy kick + the paid provider query) and `usr_hubs` (invalid-hubcount kick + hub-limit ban) each gained a one-line `if whitelist.is_whitelisted(ip) then return end` guard at the earliest point of their per-connection decision, so a whitelisted IP (hublist pinger / trusted infra) is exempt from these three - and for `etc_proxydetect` the guard sits BEFORE the cache / quota / HTTP so a trusted IP never costs a provider query. This silences the recurring `PROXYDETECT` / `GEOIP` / `USER HUBS` log lines for whitelisted IPs. Scope: this covers the IP-reputation blockers (plus the unified blocklist store via Phase A) and the hub-limit ban; the share / slots / nick-policy plugins (`usr_share` / `usr_slots` / `usr_nick_*`) are NOT yet whitelist-aware (a whitelisted IP still faces those), a possible follow-up. `etc_geoip_test` and `etc_proxydetect_test` gain whitelist-exemption regressions that provably fail without the guard (§1a.7); `usr_hubs` has no unit harness so its guard is the identical one-line mirror (diff-is-self-evidence). 3.2.x only. diff --git a/core/hub.lua b/core/hub.lua index ef2c4112..5dad5ffa 100755 --- a/core/hub.lua +++ b/core/hub.lua @@ -1877,15 +1877,32 @@ end loop = function() signal_set( "external_exit_request", false ) signal_set( "hub", "run" ) - -- One-time boot line: the compile-time select() capacity (luasocket - -- FD_SETSIZE). server.tick() selects over every connected socket at once, - -- so once the hub holds this many sockets socket.select raises "too many - -- sockets" and this loop dies. On Windows this is the Winsock default 64 - -- unless the luasocket build raises it (luasocket/CMakeLists.txt, #416); - -- on Linux glibc it is 1024. Logged (event.log) so operators can diagnose - -- that class of crash; watching more sockets needs the select->poll port - -- (#310). - out_put( "hub.lua: select() capacity (FD_SETSIZE): ", ( use "socket" )._SETSIZE, " sockets" ) + -- One-time boot line: which event-loop backend luasocket was built with + -- and its socket ceiling. server.tick() watches every connected socket at + -- once. On the POSIX poll backend (#310) there is no fixed ceiling - the + -- limit is the process file-descriptor limit (hub/hub.c raises + -- RLIMIT_NOFILE at boot; raise further via ulimit -n / systemd LimitNOFILE + -- / Docker --ulimit). On the Windows select backend the ceiling is the + -- compile-time FD_SETSIZE (Winsock default 64 unless the luasocket build + -- raises it - luasocket/CMakeLists.txt, #416); once the hub holds that many + -- sockets socket.select raises "too many sockets" and this loop dies. + -- Logged (event.log) so operators can diagnose that class of crash. + local _sockmod = use "socket" + if _sockmod._EVENTBACKEND == "poll" then + -- getfdlimit is a C launcher primitive, present in every coherent + -- POSIX build (same build that produced the poll socket module), so + -- call it directly like any other `use "X"` primitive. + local _fdlimit = ( use "getfdlimit" )( ) + if _fdlimit == -1 then + out_put( "hub.lua: event loop backend: poll (socket ceiling: unlimited fds; raise via ulimit -n)" ) + elseif _fdlimit > 0 then + out_put( "hub.lua: event loop backend: poll (socket ceiling ~= ", _fdlimit, " open files; raise via ulimit -n / systemd LimitNOFILE / Docker --ulimit)" ) + else + out_put( "hub.lua: event loop backend: poll (socket ceiling = OS file-descriptor limit; raise via ulimit -n)" ) + end + else + out_put( "hub.lua: select() capacity (FD_SETSIZE): ", _sockmod._SETSIZE, " sockets" ) + end while signal_get "hub" == "run" do server.tick() if not signal_get( "external_exit_request" ) then diff --git a/docs/BUILDING.md b/docs/BUILDING.md index a2993ae5..270230a1 100644 --- a/docs/BUILDING.md +++ b/docs/BUILDING.md @@ -246,6 +246,38 @@ is already running in this directory` (also written to --- +## Concurrent connections (file-descriptor limit) + +On Linux/BSD the hub's event loop uses `poll()`, so there is no fixed +software cap on how many clients can be connected at once +([#310](https://github.com/luadch-ng/luadch/issues/310)). The practical +ceiling is the process **open-file-descriptor limit** (each connection is +one socket, plus a handful for listeners, HBRI and the HTTP API). The +launcher raises the soft limit to the hard limit at boot, and the boot +line in `log/event.log` reports the resulting ceiling: + +``` +hub.lua: event loop backend: poll (socket ceiling ~= 1024 open files; raise via ulimit -n / systemd LimitNOFILE / Docker --ulimit) +``` + +To go higher than the hard limit (needed only for very large hubs), raise +it at the OS level - the hub then picks up the higher hard limit on the +next start: + +- **bare metal / shell:** `ulimit -Hn 65535` (as root) before launching, + or set `nofile` in `/etc/security/limits.conf`. +- **systemd:** `LimitNOFILE=65535` in the service unit's `[Service]` + section. +- **Docker:** `--ulimit nofile=65535:65535` (or the `ulimits:` key in a + compose file). + +Windows uses the `select()` backend, capped at `FD_SETSIZE = 1024` +sockets (raised from the Winsock default of 64 in +[#416](https://github.com/luadch-ng/luadch/issues/416)); the boot line +there reports `select() capacity (FD_SETSIZE): 1024 sockets`. + +--- + ## File permissions for secrets `cfg/user.tbl` (registered users with their cleartext passwords - see diff --git a/hub/hub.c b/hub/hub.c index a63f6ec8..7210384a 100755 --- a/hub/hub.c +++ b/hub/hub.c @@ -12,6 +12,7 @@ #include #include #include /* flock */ +#include /* getrlimit / setrlimit RLIMIT_NOFILE */ #include #include #include @@ -229,6 +230,71 @@ static int makedir(lua_State *L) return 1; } +/* Resulting soft RLIMIT_NOFILE after raise_fd_limit(): the practical + * concurrent-socket ceiling on the POSIX poll backend. 0 = unknown (Windows, + * or getrlimit failed), -1 = unlimited (RLIM_INFINITY), else the fd count. + * Exposed to Lua via getfdlimit() for the hub boot log. */ +static long g_fd_limit = 0; + +/* Raise the open-file-descriptor soft limit to the hard limit at boot. + * + * The POSIX poll() event-loop backend (luasocket/src/select.c, #310) can watch + * as many sockets as the process may open, so the concurrent-connection + * ceiling is RLIMIT_NOFILE, not FD_SETSIZE. Distros default the SOFT limit to + * 1024 while the HARD limit is far higher; raising soft->hard needs no + * privilege and lifts the ceiling with zero operator action, right where the + * poll port removed the old 1024 select cap - otherwise `ulimit -n = 1024` + * would silently become the new cap. Going beyond the hard limit still needs + * the OS (ulimit -Hn as root / systemd LimitNOFILE / Docker --ulimit nofile). + * No-op on Windows: the select backend is FD_SETSIZE-bound (1024, #416), not + * fd-limit-bound. Best-effort: a probe/set failure warns and leaves the limit + * at the OS default rather than aborting the hub. */ +static void raise_fd_limit(void) +{ +#ifdef __unix__ + struct rlimit rl; + if (getrlimit(RLIMIT_NOFILE, &rl) != 0) + { + fprintf(stderr, "warning: getrlimit(RLIMIT_NOFILE) failed (%s); " + "file-descriptor limit left at the OS default\n", + strerror(errno)); + fflush(stderr); + return; + } + if (rl.rlim_cur < rl.rlim_max) + { + rl.rlim_cur = rl.rlim_max; + if (setrlimit(RLIMIT_NOFILE, &rl) != 0) + { + fprintf(stderr, "warning: setrlimit(RLIMIT_NOFILE) failed (%s); " + "concurrent-socket ceiling stays at the soft limit\n", + strerror(errno)); + fflush(stderr); + } + } + /* Re-read ground truth: the soft limit may or may not have moved. */ + if (getrlimit(RLIMIT_NOFILE, &rl) == 0) + { + g_fd_limit = (rl.rlim_cur == RLIM_INFINITY) ? -1 : (long)rl.rlim_cur; + } +#endif +} + +/* + * getfdlimit() -> integer + * + * The concurrent-socket ceiling established by raise_fd_limit(): the soft + * RLIMIT_NOFILE on POSIX (-1 = unlimited), or 0 when unknown (Windows, or the + * probe failed). Read once by the hub boot log so operators see the real + * limit the poll backend gives them. Set before the first run_lua and static, + * so it survives +reload (run_lua re-entry) unchanged. + */ +static int getfdlimit(lua_State *L) +{ + lua_pushinteger(L, (lua_Integer)g_fd_limit); + return 1; +} + static void run_lua(void); static int restart(lua_State *L) @@ -272,6 +338,7 @@ static void run_lua(void) lua_register(L, "doexit", doexit); lua_register(L, "requestexit", requestexit); lua_register(L, "makedir", makedir); + lua_register(L, "getfdlimit", getfdlimit); int err = luaL_loadfile(L, "core/init.lua") || lua_pcall(L, 0, 0, 0); if (err) { @@ -505,6 +572,11 @@ int main(int argc, char **argv) "directory; refusing to start."); return EXIT_FAILURE; } + // Lift the fd soft limit to the hard limit so the POSIX poll() event loop + // (#310) is not silently re-capped at the default ulimit -n. Inherited + // across the daemonize() fork, so raising it here (once, in the parent) is + // enough. No-op on Windows. + raise_fd_limit(); daemonize(); write_lock_pid(); run_lua(); diff --git a/luasocket/src/select.c b/luasocket/src/select.c index bb47c459..66a8a0ac 100644 --- a/luasocket/src/select.c +++ b/luasocket/src/select.c @@ -1,6 +1,14 @@ /*=========================================================================*\ * Select implementation * LuaSocket toolkit +* +* luadch-ng: on POSIX the event-loop primitive (socket.select) is backed by +* poll() instead of select(), removing the fixed FD_SETSIZE=1024 ceiling that +* crashes the hub's single event loop (core/server.lua tick()) once it watches +* ~1024 sockets at once. The Lua-facing contract is byte-for-byte identical on +* both backends, so the hub side is untouched. Windows keeps the fd_set/select() +* path (FD_SETSIZE=1024, see luasocket/CMakeLists.txt + luadch-ng/luadch#416). +* See luadch-ng/luadch#310. \*=========================================================================*/ #include "luasocket.h" @@ -10,22 +18,43 @@ #include +#if !defined(_WIN32) +#include +#include +#endif + +/* Backend selection: both bodies register under the same Lua name "select"; + * only one is compiled per platform. */ +#if defined(_WIN32) +#define LS_SELECT_IMPL global_select +#else +#define LS_SELECT_IMPL global_poll +#endif + /*=========================================================================*\ * Internal function prototypes. \*=========================================================================*/ static t_socket getfd(lua_State *L); static int dirty(lua_State *L); +static void make_assoc(lua_State *L, int tab); +#if defined(_WIN32) static void collect_fd(lua_State *L, int tab, int itab, fd_set *set, t_socket *max_fd); static int check_dirty(lua_State *L, int tab, int dtab, fd_set *set); static void return_fd(lua_State *L, fd_set *set, t_socket max_fd, int itab, int tab, int start); -static void make_assoc(lua_State *L, int tab); static int global_select(lua_State *L); +#else +static void collect_read(lua_State *L, int tab, int itab, int rtab, + struct pollfd *pfds, int *nfds, int *ndirty); +static void collect_write(lua_State *L, int tab, int itab, + struct pollfd *pfds, int *nfds); +static int global_poll(lua_State *L); +#endif /* functions in library namespace */ static luaL_Reg func[] = { - {"select", global_select}, + {"select", LS_SELECT_IMPL}, {NULL, NULL} }; @@ -38,13 +67,82 @@ int select_open(lua_State *L) { lua_rawset(L, -3); lua_pushstring(L, "_SOCKETINVALID"); lua_pushinteger(L, SOCKET_INVALID); + lua_rawset(L, -3); + /* luadch-ng: which event-loop backend this build uses - "poll" on POSIX, + * "select" on Windows. The hub boot log reads this; _SETSIZE is only a + * hard socket cap on the select backend (poll has none). See #310. */ + lua_pushstring(L, "_EVENTBACKEND"); +#if defined(_WIN32) + lua_pushstring(L, "select"); +#else + lua_pushstring(L, "poll"); +#endif lua_rawset(L, -3); luaL_setfuncs(L, func, 0); return 0; } /*=========================================================================*\ -* Global Lua functions +* Shared helpers (backend-independent) +\*=========================================================================*/ +static t_socket getfd(lua_State *L) { + t_socket fd = SOCKET_INVALID; + lua_pushstring(L, "getfd"); + lua_gettable(L, -2); + if (!lua_isnil(L, -1)) { + lua_pushvalue(L, -2); + lua_call(L, 1, 1); + if (lua_isnumber(L, -1)) { + double numfd = lua_tonumber(L, -1); + fd = (numfd >= 0.0)? (t_socket) numfd: SOCKET_INVALID; + } + } + lua_pop(L, 1); + return fd; +} + +static int dirty(lua_State *L) { + int is = 0; + lua_pushstring(L, "dirty"); + lua_gettable(L, -2); + if (!lua_isnil(L, -1)) { + lua_pushvalue(L, -2); + lua_call(L, 1, 1); + is = lua_toboolean(L, -1); + } + lua_pop(L, 1); + return is; +} + +/* Turn an array result table into one that is ALSO indexable by socket + * object (result[sock] = i), and return that augmented table on the stack + * top - the value the Lua caller receives. Backend-independent. */ +static void make_assoc(lua_State *L, int tab) { + int i = 1, atab; + lua_newtable(L); atab = lua_gettop(L); + for ( ;; ) { + lua_pushnumber(L, i); + lua_gettable(L, tab); + if (!lua_isnil(L, -1)) { + lua_pushnumber(L, i); + lua_pushvalue(L, -2); + lua_settable(L, atab); + lua_pushnumber(L, i); + lua_settable(L, atab); + } else { + lua_pop(L, 1); + break; + } + i = i+1; + } +} + +#if defined(_WIN32) +/*=========================================================================*\ +* Windows backend: fd_set + select() +* +* fd_set here is a Winsock SOCKET array sized by FD_SETSIZE (raised to 1024 in +* luasocket/CMakeLists.txt, #416). Unchanged from upstream LuaSocket. \*=========================================================================*/ /*-------------------------------------------------------------------------*\ * Waits for a set of sockets until a condition is met or timeout. @@ -82,38 +180,6 @@ static int global_select(lua_State *L) { } } -/*=========================================================================*\ -* Internal functions -\*=========================================================================*/ -static t_socket getfd(lua_State *L) { - t_socket fd = SOCKET_INVALID; - lua_pushstring(L, "getfd"); - lua_gettable(L, -2); - if (!lua_isnil(L, -1)) { - lua_pushvalue(L, -2); - lua_call(L, 1, 1); - if (lua_isnumber(L, -1)) { - double numfd = lua_tonumber(L, -1); - fd = (numfd >= 0.0)? (t_socket) numfd: SOCKET_INVALID; - } - } - lua_pop(L, 1); - return fd; -} - -static int dirty(lua_State *L) { - int is = 0; - lua_pushstring(L, "dirty"); - lua_gettable(L, -2); - if (!lua_isnil(L, -1)) { - lua_pushvalue(L, -2); - lua_call(L, 1, 1); - is = lua_toboolean(L, -1); - } - lua_pop(L, 1); - return is; -} - static void collect_fd(lua_State *L, int tab, int itab, fd_set *set, t_socket *max_fd) { int i = 1, n = 0; @@ -133,13 +199,8 @@ static void collect_fd(lua_State *L, int tab, int itab, fd = getfd(L); if (fd != SOCKET_INVALID) { /* make sure we don't overflow the fd_set */ -#ifdef _WIN32 if (n >= FD_SETSIZE) luaL_argerror(L, tab, "too many sockets"); -#else - if (fd >= FD_SETSIZE) - luaL_argerror(L, tab, "descriptor too large for set size"); -#endif FD_SET(fd, set); n++; /* keep track of the largest descriptor so far */ @@ -193,22 +254,169 @@ static void return_fd(lua_State *L, fd_set *set, t_socket max_fd, } } -static void make_assoc(lua_State *L, int tab) { - int i = 1, atab; - lua_newtable(L); atab = lua_gettop(L); +#else +/*=========================================================================*\ +* POSIX backend: poll() +* +* poll() takes a variable-length pollfd array instead of the fixed 1024-bit +* fd_set, so there is no FD_SETSIZE ceiling on how many sockets the single +* hub event loop can watch (bounded only by the process fd limit; hub/hub.c +* raises RLIMIT_NOFILE at boot). See luadch-ng/luadch#310. +* +* Contract parity with global_select (the hub relies on all of these): +* - Returns two array-iterable tables (read-ready, write-ready), each also +* indexable by socket object via make_assoc; EMPTY (never nil) on timeout, +* so tick()'s ipairs() never sees nil. +* - Preserves the check_dirty() force-include: a socket whose :dirty() is +* true (LuaSec TLS record already decrypted/buffered at the SSL layer, with +* no fresh kernel readable event) is pre-recorded as readable and NOT +* polled - without this every TLS client with buffered app-data stalls. +* - The same fd may be reported in BOTH lists in one tick (TLS want-write / +* partial-send re-arm): one POLLIN pollfd and one POLLOUT pollfd carry it. +* - An errored fd (POLLERR/POLLHUP/POLLNVAL) surfaces into every list it was +* watched in, so the hub reads/writes it, gets the error, and cleans up. +\*=========================================================================*/ + +/* Collect the read watch-set. One POLLIN pollfd per live socket in `tab`, + * appended before any write entry. A socket whose :dirty() returns true is + * pre-recorded into `rtab` (result read table) and NOT polled - this fuses + * collect + check_dirty for the read side into one pass. Stack-neutral. */ +static void collect_read(lua_State *L, int tab, int itab, int rtab, + struct pollfd *pfds, int *nfds, int *ndirty) { + int i = 1; + if (lua_isnil(L, tab)) return; + luaL_checktype(L, tab, LUA_TTABLE); for ( ;; ) { + t_socket fd; lua_pushnumber(L, i); lua_gettable(L, tab); - if (!lua_isnil(L, -1)) { - lua_pushnumber(L, i); + if (lua_isnil(L, -1)) { + lua_pop(L, 1); + break; + } + fd = getfd(L); + if (fd != SOCKET_INVALID) { + /* map descriptor back to the object for the result tables */ + lua_pushnumber(L, (lua_Number) fd); lua_pushvalue(L, -2); - lua_settable(L, atab); - lua_pushnumber(L, i); - lua_settable(L, atab); - } else { + lua_settable(L, itab); + if (dirty(L)) { + /* already-ready: pre-include as readable, do not poll it + * (mirrors select.c check_dirty + FD_CLR) */ + lua_pushnumber(L, ++(*ndirty)); + lua_pushvalue(L, -2); + lua_settable(L, rtab); + } else { + pfds[*nfds].fd = fd; + pfds[*nfds].events = POLLIN; + pfds[*nfds].revents = 0; + *nfds = *nfds + 1; + } + } + lua_pop(L, 1); + i = i + 1; + } +} + +/* Collect the write watch-set. One POLLOUT pollfd per live socket in `tab`, + * appended after all read entries. Stack-neutral. */ +static void collect_write(lua_State *L, int tab, int itab, + struct pollfd *pfds, int *nfds) { + int i = 1; + if (lua_isnil(L, tab)) return; + luaL_checktype(L, tab, LUA_TTABLE); + for ( ;; ) { + t_socket fd; + lua_pushnumber(L, i); + lua_gettable(L, tab); + if (lua_isnil(L, -1)) { lua_pop(L, 1); break; } - i = i+1; + fd = getfd(L); + if (fd != SOCKET_INVALID) { + lua_pushnumber(L, (lua_Number) fd); + lua_pushvalue(L, -2); + lua_settable(L, itab); + pfds[*nfds].fd = fd; + pfds[*nfds].events = POLLOUT; + pfds[*nfds].revents = 0; + *nfds = *nfds + 1; + } + lua_pop(L, 1); + i = i + 1; } } + +/*-------------------------------------------------------------------------*\ +* Waits for a set of sockets until a condition is met or timeout. +\*-------------------------------------------------------------------------*/ +static int global_poll(lua_State *L) { + int itab, rtab, wtab, ret, ndirty = 0, nfds = 0; + int rcount, wcount, k; + size_t cap = 0; + struct pollfd *pfds; + t_timeout tm; + double t = luaL_optnumber(L, 3, -1); + lua_settop(L, 3); + /* Upper bound on pollfd entries = array length of both watch tables (a + * socket watched for both read and write gets two entries; getfd filters + * out non-socket rows, so this only over-allocates, never under). */ + if (!lua_isnil(L, 1)) cap += lua_rawlen(L, 1); + if (!lua_isnil(L, 2)) cap += lua_rawlen(L, 2); + /* GC-managed scratch buffer: survives the luaL_error longjmp below and is + * reclaimed by the collector (unlike malloc, which would leak on error). + * lua_newuserdata never returns NULL (it raises on OOM). */ + pfds = (struct pollfd *) lua_newuserdata(L, + (cap ? cap : 1) * sizeof(struct pollfd)); + lua_newtable(L); itab = lua_gettop(L); + lua_newtable(L); rtab = lua_gettop(L); + lua_newtable(L); wtab = lua_gettop(L); + collect_read(L, 1, itab, rtab, pfds, &nfds, &ndirty); + collect_write(L, 2, itab, pfds, &nfds); + /* dirty sockets are already known-ready: poll immediately, don't block */ + t = ndirty > 0 ? 0.0 : t; + timeout_init(&tm, t, -1); + timeout_markstart(&tm); + do { + double s = timeout_getretry(&tm); + int ms = (s >= 0.0) ? (int)(s * 1.0e3) : -1; + ret = poll(pfds, (nfds_t) nfds, ms); + } while (ret < 0 && errno == EINTR); + if (ret > 0 || ndirty > 0) { + /* read results start after the pre-included dirty sockets */ + rcount = ndirty; + wcount = 0; + for (k = 0; k < nfds; k++) { + short re = pfds[k].revents; + if (re == 0) continue; + if (pfds[k].events & POLLIN) { + /* read watch entry: readable, or errored while read-watched */ + if (re & (POLLIN | POLLERR | POLLHUP | POLLNVAL)) { + lua_pushnumber(L, ++rcount); + lua_pushnumber(L, (lua_Number) pfds[k].fd); + lua_gettable(L, itab); + lua_settable(L, rtab); + } + } else { + /* write watch entry: writable, or errored while write-watched */ + if (re & (POLLOUT | POLLERR | POLLHUP | POLLNVAL)) { + lua_pushnumber(L, ++wcount); + lua_pushnumber(L, (lua_Number) pfds[k].fd); + lua_gettable(L, itab); + lua_settable(L, wtab); + } + } + } + make_assoc(L, rtab); + make_assoc(L, wtab); + return 2; + } else if (ret == 0) { + lua_pushstring(L, "timeout"); + return 3; + } else { + luaL_error(L, "poll failed"); + return 3; + } +} +#endif diff --git a/tests/smoke/run.py b/tests/smoke/run.py index 54c3f36c..6959523a 100644 --- a/tests/smoke/run.py +++ b/tests/smoke/run.py @@ -12326,24 +12326,29 @@ def test_no_script_errors(log_path: Path, error_log_path: Path = None): ) -def test_select_capacity_logged(staging_dir: Path): - """Regression guard for #416 (Windows FD_SETSIZE=64 hub crash). - - The event loop selects over every connected socket at once; on Windows - luasocket's select.c raises "too many sockets" at n >= FD_SETSIZE, and - the bundled build inherited the Winsock default of 64 - so the hub - crashed once it held ~64 sockets. The luasocket Windows build now defines - FD_SETSIZE=1024 (parity with Linux glibc). - - hub.loop() logs the compile-time capacity (socket._SETSIZE, the exact - macro the guard checks) once at boot via out.put; assert it is >= 1024. - This proves the compile define reached the shipped socket module. Fails - pre-fix on the Windows leg (logs 64); the Linux leg already reports 1024. - It goes through the real hub because a standalone `require('socket')` - under msys2 lua hits the bundled-liblua ABI clash (see smoke.yml). Read - from event.log (out.put's target, persisted open-append-close per line; - the harness sets log_events=true) - not smoke-hub.log, whose stdout is - block-buffered and lost when the hub is terminated at teardown.""" +def test_event_loop_backend_logged(staging_dir: Path): + """Regression guard for the event-loop backend and its socket ceiling. + + hub.loop() logs one boot line naming the luasocket event-loop backend and + the ceiling on how many sockets the single event loop can watch. The line + (and the assertion) differ by platform: + + - POSIX poll backend (#310): "event loop backend: poll (socket ceiling + ...)". poll() takes a variable-length pollfd array, so there is NO + fixed FD_SETSIZE cap; the ceiling is the process fd limit (hub/hub.c + raises RLIMIT_NOFILE at boot). This test REQUIRES the poll line on the + Linux leg, which fails pre-fix: the old select.c backend logged the + "select() capacity (FD_SETSIZE)" line instead and had no _EVENTBACKEND. + - Windows select backend (#416): "select() capacity (FD_SETSIZE): N + sockets". select.c raises "too many sockets" at n >= FD_SETSIZE; the + Winsock default 64 crashed the hub at ~64 sockets, so the Windows build + must define FD_SETSIZE=1024. Assert N >= 1024 (unchanged by #310). + + Goes through the real hub because a standalone require('socket') under msys2 + lua hits the bundled-liblua ABI clash (see smoke.yml). Reads event.log + (out.put's target, persisted open-append-close per line; the harness sets + log_events=true) - not smoke-hub.log, whose stdout is block-buffered and + lost when the hub is terminated at teardown.""" log_path = staging_dir / "log" / "event.log" if not log_path.exists(): raise TestFailure( @@ -12351,19 +12356,27 @@ def test_select_capacity_logged(staging_dir: Path): f"enable log_events in the cfg.tbl override for this assertion" ) text = log_path.read_text(encoding="utf-8", errors="replace") - m = re.search(r"select\(\) capacity \(FD_SETSIZE\): (\d+) sockets", text) - if not m: - raise TestFailure( - "event.log has no 'select() capacity (FD_SETSIZE): N sockets' " - "line - hub.loop() should log it once at startup" - ) - cap = int(m.group(1)) - if cap < 1024: - raise TestFailure( - f"select() FD_SETSIZE capacity is {cap}, expected >= 1024 - the " - f"Windows luasocket build must define FD_SETSIZE=1024; the default " - f"64 crashes the event loop at ~64 sockets (#416)" - ) + if sys.platform.startswith("win"): + m = re.search(r"select\(\) capacity \(FD_SETSIZE\): (\d+) sockets", text) + if not m: + raise TestFailure( + "event.log has no 'select() capacity (FD_SETSIZE): N sockets' " + "line - the Windows select backend should log it once at startup" + ) + cap = int(m.group(1)) + if cap < 1024: + raise TestFailure( + f"select() FD_SETSIZE capacity is {cap}, expected >= 1024 - the " + f"Windows luasocket build must define FD_SETSIZE=1024; the default " + f"64 crashes the event loop at ~64 sockets (#416)" + ) + else: + if not re.search(r"event loop backend: poll", text): + raise TestFailure( + "event.log has no 'event loop backend: poll' line - the POSIX " + "luasocket build must use the poll() event-loop backend (#310); " + "pre-fix it logged 'select() capacity (FD_SETSIZE)' instead" + ) # ----------------------------------------------------------------------------- @@ -12548,12 +12561,12 @@ def run_tests(staging_dir: Path): log("PASS no script errors in log") try: - test_select_capacity_logged(staging_dir) + test_event_loop_backend_logged(staging_dir) except Exception as e: - log(f"FAIL select() FD_SETSIZE capacity >= 1024 (#416): {e}") - failed.append("select() FD_SETSIZE capacity >= 1024 (#416)") + log(f"FAIL event loop backend logged (#310/#416): {e}") + failed.append("event loop backend logged (#310/#416)") else: - log("PASS select() FD_SETSIZE capacity >= 1024 (#416)") + log("PASS event loop backend logged (#310/#416)") return failed From 94a130f80c64da07f4d96f18bab697dcd0eb695d Mon Sep 17 00:00:00 2001 From: Kcchouette Date: Thu, 16 Jul 2026 15:10:33 +0000 Subject: [PATCH 02/18] Remove redundant save call in setlevel function (#437) Co-authored-by: Aybook <265313202+Aybook@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) --- core/hub_user_object.lua | 2 -- 1 file changed, 2 deletions(-) diff --git a/core/hub_user_object.lua b/core/hub_user_object.lua index 414936f8..deef7ac6 100644 --- a/core/hub_user_object.lua +++ b/core/hub_user_object.lua @@ -624,8 +624,6 @@ local function createuser( _client, _sid ) level = tonumber( level ) if utf.match( level, _regex.reguser.level ) then profile.level = level - cfg_saveusers( _regusers ) - --return true return cfg_saveusers( _regusers ) end return false, "invalid level" From 816eaf9a6ea019ae9086ade075ca1bebcc58ec7b Mon Sep 17 00:00:00 2001 From: Kcchouette Date: Thu, 16 Jul 2026 15:10:37 +0000 Subject: [PATCH 03/18] Refactor bool function in adc.lua (#438) Removed duplicate 'bool' function and adjusted indentation. Co-authored-by: Aybook <265313202+Aybook@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) --- core/adc.lua | 3 --- 1 file changed, 3 deletions(-) diff --git a/core/adc.lua b/core/adc.lua index a67083cf..3f1be36f 100755 --- a/core/adc.lua +++ b/core/adc.lua @@ -170,9 +170,6 @@ _regex = { sid = function( str ) return string_match( str, _sid ) end, - bool = function( str ) - return string_match( str, _sid ) - end, bool = function( str ) return string_match( str, _bool ) end, From eddacc2b1375fce0dedf818ea5a191d9fca77f20 Mon Sep 17 00:00:00 2001 From: Kcchouette Date: Thu, 16 Jul 2026 15:10:41 +0000 Subject: [PATCH 04/18] Fix type check for number in util.lua (#439) Co-authored-by: Aybook <265313202+Aybook@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) --- core/util.lua | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/core/util.lua b/core/util.lua index d421c3c1..e45d9514 100755 --- a/core/util.lua +++ b/core/util.lua @@ -635,7 +635,7 @@ formatseconds = function( t, hubstart ) err = "util.lua: error: number expected, got nil" return nil, err end - if not type( t ) == "number" then + if type( t ) ~= "number" then err = "util.lua: error: number expected, got " .. type( t ) return nil, err end @@ -668,7 +668,7 @@ formatbytes = function( bytes ) err = "util.lua: error: number expected, got nil" return nil, err end - if not type( bytes ) == "number" then + if type( bytes ) ~= "number" then err = "util.lua: error: number expected, got " .. type( bytes ) return nil, err end From 1fbf735184e703fad4b016632082d3f314567a7c Mon Sep 17 00:00:00 2001 From: Aybo <265313202+Aybook@users.noreply.github.com> Date: Thu, 16 Jul 2026 17:37:42 +0200 Subject: [PATCH 05/18] docs: add CONTRIBUTING.md documenting the dev-branch policy (#440) Three external PRs (#437, #438, #439) all arrived against master - and correctly so from the outside: nothing documented the GitFlow-A branch policy, master is the repository's default branch so GitHub offers it, and README described 3.2.x as "active development (master)" without mentioning dev anywhere. CONTRIBUTING.md leads with the branch table (dev = target this, master = release substrate, release/3.1.x = security backports only), records that a mistargeted PR just gets retargeted by a maintainer with the contributor's authorship intact, routes vulnerability reports to docs/SECURITY.md, and links out for build / tests / the core restricted-env use "X" contract / the plugin sandbox / the companion scripts repo rather than duplicating docs/DEVELOPMENT.md. README gains a dev-branch pointer directly under the release-lines table (the sentence that pointed contributors at the wrong branch) plus CONTRIBUTING.md and the previously-unlisted docs/DEVELOPMENT.md in its documentation list; DEVELOPMENT.md's orientation table gains a "your first PR" row. Also records the #437 / #438 / #439 cleanups in the changelog. Co-authored-by: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 4 +++ CONTRIBUTING.md | 70 +++++++++++++++++++++++++++++++++++++++++++++ README.md | 9 ++++++ docs/DEVELOPMENT.md | 1 + 4 files changed, 84 insertions(+) create mode 100644 CONTRIBUTING.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 83a6cebf..cf2f2778 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -49,6 +49,8 @@ land on `release/3.1.x` per ### Documentation +- **new [`CONTRIBUTING.md`](CONTRIBUTING.md) - a contributor entry point, closing a real doc gap.** Three external PRs ([#437](https://github.com/luadch-ng/luadch/pull/437) / [#438](https://github.com/luadch-ng/luadch/pull/438) / [#439](https://github.com/luadch-ng/luadch/pull/439)) all arrived against `master`, and correctly so from the outside: nothing documented the GitFlow-A branch policy, `master` is the repository's default branch (so GitHub offers it), and `README.md` literally described 3.2.x as "active development (`master`)" without mentioning `dev` at all. The new page leads with the branch table (`dev` = target this, `master` = release substrate, `release/3.1.x` = security backports only), records that a mistargeted PR just gets retargeted by a maintainer with the contributor's authorship intact (no close-and-reopen), routes vulnerability reports to [`docs/SECURITY.md`](docs/SECURITY.md), and links out for the rest - build, tests, the core restricted-env `use "X"` contract, the plugin sandbox, the companion scripts repo - rather than duplicating them; the engineering how-to stays in [`docs/DEVELOPMENT.md`](docs/DEVELOPMENT.md). `README.md` gains a `dev`-branch pointer directly under the release-lines table (the sentence that was actively pointing contributors at the wrong branch) plus `CONTRIBUTING.md` and the previously-unlisted `docs/DEVELOPMENT.md` in its documentation list; `docs/DEVELOPMENT.md`'s orientation table gains a "your first PR" row. Docs-only, no code change. 3.2.x only. + - Contributor-doc currency + pattern capture (post-#398 webhook arc). **`CLAUDE.md`:** latest-release marker v3.1.12 -> **v3.1.13**; the §3 module map gains `core/hmac.lua` (HMAC-SHA256, #398) and `core/geoip_update.lua` (was omitted); the §5 current-era note re-framed from "bug triage" to feature-work + triage; §8 gains a "drop-in plugin patch as a lighter alternative to a 3.1.x release" note (the `usr_uptime` v0.10.1 case). **`docs/DEVELOPMENT.md`:** six durable patterns added - `scope="none"` custom-auth HTTP routes, verifying a signed body with `hmac.sha256` + dynamically-named per-instance secret keys, bots-at-module-load / routes-in-`onStart` (reload safety), bulky operator config in its own `cfg/.tbl`, the `getpeername()`-nil-after-accept guard (#401), and poll-don't-fixed-settle for a smoke assertion whose hub state installs asynchronously and fails open (#408). **`docs/HTTP_API.md`:** the §4.4 `none` scope row and the §5.1 plugin-registration `scope` arg now document `scope="none"` for plugin-owned auth (`etc_webhook` is the first user); the `meta` key list gains `plugin`. Plus: two unit tests that existed but were never wired into CI (`etc_blocklist_test`, `cmd_usercleaner_orphan_test` - both pass) are now registered on both smoke legs, so `tests/README`'s "the whole set runs in CI" holds. Docs / test-wiring only. 3.2.x only. - [#364](https://github.com/luadch-ng/luadch/issues/364) (Sopor) - **clarified the `kill_wrong_ips` wording so it no longer reads as if it relates to CGNAT traffic/download blocking.** `kill_wrong_ips` only governs the login-time *mismatch* kick (client-advertised INF IP vs authenticated TCP source); it does not affect how the hub blocks users - e.g. the Traffic Manager, which on 3.x decides on level / share / account-nick, not IP (the actual fix for the #364 CGNAT block-bypass reported on 2.24). The old wording listed bare "CGNAT users" as a `kill_wrong_ips = false` beneficiary and claimed peers "reach them correctly in CGNAT-egress scenarios", which both overstated the effect (genuine carrier-NAT users get no inbound active-mode P2P regardless) and invited conflation with #364. Tightened in `docs/SECURITY.md`, the `kill_wrong_ips` `cfg_defaults` comment, `examples/cfg/cfg.tbl`, and the Breaking entry above: the beneficiary is now described by the actual mismatch mechanism (stale VPN IP, hand-set wrong WAN IP, dual-stack family mismatch), with an explicit "does not change how the hub blocks users" note. Docs-only, no code change. 3.2.x only. @@ -61,6 +63,8 @@ land on `release/3.1.x` per ### Refactors +- **three core cleanups from an external contributor** (@Kcchouette - [#437](https://github.com/luadch-ng/luadch/pull/437), [#438](https://github.com/luadch-ng/luadch/pull/438), [#439](https://github.com/luadch-ng/luadch/pull/439)). (1) `core/hub_user_object.lua` `user.setlevel` called `cfg_saveusers( _regusers )` **twice** - once bare, then again as the return expression - so every level change wrote `user.tbl` to disk two times; the bare call and a stale `--return true` comment are gone and the return contract is unchanged (the sibling `setrank` already had the clean shape). (2) `core/adc.lua` `_regex` declared the key `bool` twice: the first matched `_sid` (the 4-char base32 SID pattern, wrong for a boolean), the second `_bool`. A Lua table constructor applies both in order, so the last already won - the correct `_bool` variant was live all along and the first was dead code. Removing it is a no-op, but the duplicate was a trap for anyone reordering the table. (3) `core/util.lua` `formatseconds` / `formatbytes` guarded with `not type( t ) == "number"`, which parses as `( not type( t ) ) == "number"` -> `false == "number"` -> always false, so the guard never fired; now `type( t ) ~= "number"`. Also a no-op today: the `local t = tonumber( t )` rebind directly above already reduces every non-numeric argument to `nil`, which the preceding `not t` branch rejects, so the check is unreachable either way - fixed because the expression becomes a live bug the moment that rebind is refactored away (the same file already used the correct form at a third site). Net effect: (2) and (3) are behaviour-neutral hardening; (1) halves the disk writes on a level change. 3.2.x only. + - [#301](https://github.com/luadch-ng/luadch/issues/301) follow-up - move the bundled core language tables from `examples/lang/` to `lang/` at source-tree root. Matches the install-tree layout (CMake was already installing them to `lang/`), matches the precedent of `scripts/lang/` living under its consumer rather than under `examples/`, and removes a "why is this under examples/?" stumbling block for new contributors / Weblate onboarding. Pure `git mv` so blame history is preserved; CMakeLists.txt updated to point at the new source path (1 line); `tests/unit/lang_test.lua` + smoke.yml comment updated. Install output is byte-identical (verified). 3.2.x only, not backported. ### Bugfixes diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..d0ed193d --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,70 @@ +# Contributing to luadch + +Thanks for wanting to contribute. This page covers the things that are easy to +get wrong from the outside. The engineering how-to (core modules, plugins, +testing, security checklists, Definition of Done) lives in +[`docs/DEVELOPMENT.md`](docs/DEVELOPMENT.md). + +## Open pull requests against `dev`, not `master` + +GitHub offers `master` by default because it is the repository's default +branch - but `master` is the release substrate and only receives changes that +have already been validated. New work goes to `dev` first. + +| Branch | What it is | Target it? | +|---|---|---| +| **`dev`** | Staging. Everything lands here first and is validated on a test hub (`ghcr.io/luadch-ng/luadch:dev` is rebuilt on every push). | **Yes - this one** | +| `master` | Release substrate for the 3.2.x line. Promoted from `dev` once validated; release tags are cut here. | No (maintainers) | +| `release/3.1.x` | Maintenance line, critical security backports only. | No (maintainers) | + +If you already opened a PR against `master`, nothing is lost - a maintainer +can retarget it to `dev` in one click, and it stays your PR with your +authorship intact. No need to close and reopen it. + +**Found a security vulnerability?** Do not open a public issue or PR - see +[`docs/SECURITY.md`](docs/SECURITY.md) for the reporting process. + +## Before opening the PR + +- **Build it.** [`docs/BUILDING.md`](docs/BUILDING.md) has the Linux, Windows + and ARM recipes. +- **Run the tests.** [`tests/README.md`](tests/README.md) - a Lua unit suite + plus a protocol-level smoke harness. Both run in CI on Linux *and* Windows + for every PR, so running them locally first saves a round trip. +- **One logical change per PR**, referencing the issue it addresses. Unrelated + fixes belong in separate PRs - it keeps review honest and makes reverts + surgical. +- **Match the surrounding code.** Lua style follows the file you are editing. + Comments explain *why*, not *what*. Avoid drive-by refactors: if you spot + something unrelated, open an issue instead. +- **Bug fixes should come with a test** that fails before the fix and passes + after. The exception is a diff that is self-evidently its own proof (a typo, + dead code, a redundant call). See + [`docs/DEVELOPMENT.md` §4](docs/DEVELOPMENT.md). + +## Where things live + +- **`core/*.lua`** - the hub itself. It runs in a restricted environment where + every global must be imported (`local X = use "X"`); a bare global passes + unit tests but fails at hub boot. Read + [`docs/DEVELOPMENT.md` §2](docs/DEVELOPMENT.md) before touching core. +- **`scripts/*.lua`** - bundled plugins, running in a sandbox. + [`docs/PLUGIN_API.md`](docs/PLUGIN_API.md) is the API reference; + [`docs/DEVELOPMENT.md` §3](docs/DEVELOPMENT.md) adds the conventions on top + of it. +- **Additional plugins** that do not ship with the hub live in the companion + repo [`luadch-ng/scripts`](https://github.com/luadch-ng/scripts). +- **`tests/`** - unit tests (`tests/unit/`) and the smoke harness + (`tests/smoke/run.py`). + +## Reporting bugs + +Open an issue with the hub version (`+hubinfo` output or the boot line in +`log/event.log`), the platform, and what you did to trigger it. Log excerpts +from `log/error.log` help. If the report is about an older release, say which +one - a good share of reports turn out to be already fixed on the current +line. + +## License + +Contributions are made under the project's [GPL-3.0](LICENSE) license. diff --git a/README.md b/README.md index 477272d7..6165bf94 100644 --- a/README.md +++ b/README.md @@ -24,6 +24,10 @@ The 3.1.x line concludes the modernisation programme (Phases 1-7 + ADC-coverage closure). The active 3.2.x line picks up the Phase 8 feature work. +**Contributing?** New work is staged on the `dev` branch before it is +promoted to `master`, so pull requests go against **`dev`** - see +[CONTRIBUTING.md](CONTRIBUTING.md). + ## Original Features - TLS 1.3 with AES-128 / AES-256 cipher suites @@ -65,6 +69,11 @@ and operator guidance. TLS-only deployments, troubleshooting - **[docs/PLUGIN_API.md](docs/PLUGIN_API.md)** - plugin scripting API reference (listeners, modules, objects, conventions, pitfalls) +- **[CONTRIBUTING.md](CONTRIBUTING.md)** - which branch to target, what a + PR needs, how to report a bug +- **[docs/DEVELOPMENT.md](docs/DEVELOPMENT.md)** - engineering how-to: + authoring core modules, plugin conventions, testing, security + checklists, Definition of Done ## Quick start diff --git a/docs/DEVELOPMENT.md b/docs/DEVELOPMENT.md index c53e25fe..59b9a3fe 100644 --- a/docs/DEVELOPMENT.md +++ b/docs/DEVELOPMENT.md @@ -15,6 +15,7 @@ it. | You want to... | Read | |---|---| +| Open your first PR (which branch to target, PR scope) | [`../CONTRIBUTING.md`](../CONTRIBUTING.md) | | Know the rules / architecture / roadmap | `CLAUDE.md` | | Build from source (Linux / Windows / ARM) | [`BUILDING.md`](BUILDING.md) | | Deploy / operate a built hub | [`INSTALLING.md`](INSTALLING.md), [`CONFIGURATION.md`](CONFIGURATION.md), [`DOCKER.md`](DOCKER.md) | From 4c033e459f5986a95f0a946cd3df8fce16681c0e Mon Sep 17 00:00:00 2001 From: Aybo <265313202+Aybook@users.noreply.github.com> Date: Thu, 16 Jul 2026 18:32:32 +0200 Subject: [PATCH 06/18] fix(sysinfo): CIM -> WMI fallback so old Windows (PS 2.0) reports OS/CPU/RAM (#432) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit +hubinfo showed for OS/CPU/RAM on Server 2008 R2 / Windows 7 because core/sysinfo.lua queried via Get-CimInstance (PowerShell 3.0+), which PowerShell 2.0 (the default there) does not have. Each Windows probe now runs `try { Get-CimInstance ... } catch { Get-WmiObject ... }` in one PowerShell call - Get-WmiObject exists in every Windows PowerShell (2.0-5.1; powershell.exe, not PS-7/Core pwsh), so PS 2.0 hosts take the catch branch and report real values. Validated live on Windows 11 (+ Get-WmiObject directly). A win_cim(class, property) helper keeps the 4 probes DRY; its args are hardcoded Win32_* literals (no injection). - core/sysinfo.lua: new win_cim helper + 4 call sites. - tests/unit/sysinfo_test.lua (new, 21 checks): win/unix parsing + a regression that the WMI fallback stays in every Windows command (provably fails if removed, §1a.7). Both smoke.yml legs. Sopor-reported: a hubowner on 3.1.14 additionally CRASHED here (cmd_hubinfo.lua nil-concat) because the pre-refactor 3.1.x plugin had no `or msg_unknown` guard; that gets a v0.30 drop-in per §8. 3.2.x. Co-authored-by: Claude Opus 4.8 (1M context) --- .github/workflows/smoke.yml | 7 ++ CHANGELOG.md | 2 + core/sysinfo.lua | 32 ++++++-- tests/unit/sysinfo_test.lua | 160 ++++++++++++++++++++++++++++++++++++ 4 files changed, 194 insertions(+), 7 deletions(-) create mode 100644 tests/unit/sysinfo_test.lua diff --git a/.github/workflows/smoke.yml b/.github/workflows/smoke.yml index 7b846364..5d32e0b4 100644 --- a/.github/workflows/smoke.yml +++ b/.github/workflows/smoke.yml @@ -46,6 +46,9 @@ jobs: - name: Run bloom unit test run: lua5.4 tests/unit/bloom_test.lua + - name: Run sysinfo unit test + run: lua5.4 tests/unit/sysinfo_test.lua + # Phase 1b of #82 HTTP API: router unit tests cover the pure-Lua # bits (constant_time_eq, schema validator, envelope helpers, # token resolution, request-id shape, register guards). @@ -366,6 +369,10 @@ jobs: shell: msys2 {0} run: lua5.4 tests/unit/bloom_test.lua + - name: Run sysinfo unit test + shell: msys2 {0} + run: lua5.4 tests/unit/sysinfo_test.lua + - name: Run http_router unit test shell: msys2 {0} run: lua5.4 tests/unit/http_router_test.lua diff --git a/CHANGELOG.md b/CHANGELOG.md index cf2f2778..6ad65083 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -69,6 +69,8 @@ land on `release/3.1.x` per ### Bugfixes +- **`core/sysinfo.lua`: `+hubinfo` now works on old Windows (Server 2008 R2 / Windows 7) instead of showing ``** (Sopor-reported). The OS / CPU / RAM probes used `Get-CimInstance`, which is PowerShell 3.0+; PowerShell 2.0 (the default on Server 2008 R2 / Win7) raises `CommandNotFoundException`, so the queries returned empty / non-numeric output. On 3.2.x `cmd_hubinfo` already wrapped that in `or msg_unknown` (so it degraded to ``, no crash), but the values were simply unavailable on old Windows. Each Windows probe now runs `try { Get-CimInstance ... } catch { Get-WmiObject ... }` in one PowerShell call - `Get-WmiObject` exists in every Windows PowerShell (2.0-5.1; `powershell.exe`, not the PS-7/Core `pwsh`), so PS 2.0 hosts take the catch branch and report real values (validated live on Windows 11 + against `Get-WmiObject` directly). `tests/unit/sysinfo_test.lua` (new, 21 checks incl. a regression that the WMI fallback stays in every Windows command - provably fails if removed, §1a.7); registered on both smoke.yml legs. The pre-refactor 3.1.x `cmd_hubinfo.lua` (v0.29) additionally CRASHED here (`attempt to concatenate a nil value 'cache_check_ram_total'`, because it had no `or msg_unknown` guard); a v0.30 drop-in for the 3.1.x line carries both the nil-guard and the WMI fallback (no `v3.1.15` release, per the §8 drop-in-plugin path). 3.2.x. + - **clear "port already in use" message + Windows port-hijack fix** (Sopor follow-up). When a listener could not bind its port the hub was unhelpful: on Linux it logged only the cryptic luasocket line (`...bind: address already in use`) then silently retried every 30s; on Windows the pre-bind `SO_REUSEADDR` (added for the Linux #128 TIME_WAIT fast-restart) has `SO_REUSEPORT` semantics, so a second process re-bound the SAME port with no error at all - two hubs then silently shared it, and any local process could hijack a luadch port. Two changes: (1) `core/server.lua` sets `SO_REUSEADDR` only on Linux (detected via `util.path_sep()`); on Windows it is skipped both pre- and post-bind, so a same-port second bind now fails visibly with `WSAEADDRINUSE` and the port can no longer be hijacked. Linux keeps the #128 fast-restart behaviour unchanged. (2) `core/hub.lua` `add_server_handler` now logs a clear operator line (`server: IPv4 TLS port 5001 is already in use - another process (or a second hub) holds it. Stop it or change the port in cfg/cfg.tbl; retrying every 30s.`) plus a generic message for any other listener-start failure that was previously swallowed. Verified that dropping `SO_REUSEADDR` on Windows does NOT regress the fast-restart: the full Windows smoke restart battery (plaintext / tier / `+reload` / dual-stack-same-port #107) stays green. Smoke: a new stage starts a second hub (separate dir, same ports as the live first hub) and asserts the guidance appears, provably failing pre-fix (§1a.7 - the assertion is on the new `change the port in cfg/cfg.tbl` phrase that the pre-fix cryptic line lacks; pre-fix Windows silently re-binds and logs nothing). Complements the single-instance lock (same Sopor thread): the lock stops a second copy in the SAME dir; this makes a second hub in a DIFFERENT dir with clashing ports fail loudly instead of silently. 3.2.x only. - **single-instance lock: a second hub started against the same install tree now refuses to start** (Sopor). Nothing stopped two `Luadch.exe` (or `./luadch`) processes running from one install directory at once - on Windows especially, where `core/server.lua` sets `SO_REUSEADDR` before bind (a deliberate fast-restart fix, #128) and Windows gives `SO_REUSEADDR` `SO_REUSEPORT` semantics, so the second process re-binds the same port and both hubs then share (and race on) `cfg/user.tbl`, `master.key`, the plugin `scripts/data/*.tbl` stores and the logs; interleaved `saveusers()`/`savetable()` writes can corrupt them (data loss). Fixed in the C launcher (`hub/hub.c`) with a per-install lock file (`luadch.lock` in the install dir): unix `flock(LOCK_EX|LOCK_NB)` + `O_CLOEXEC`, Windows exclusive `CreateFile` (deny-share, non-inheritable handle, delete-on-close). Held open for the whole process lifetime and released by the OS on crash (no stale-lock cleanup step), retained by the daemon child across the `-d` fork and across `+reload` (the fd survives the `restart()`/atexit re-entry). Scope is per install directory - a test hub and a prod hub in separate directories on one box still run in parallel; only a second copy of the SAME install is refused (non-zero exit + "another instance is already running" to stderr and `log/exception.txt`). Fail-open: only the definitive already-held signal (unix `EWOULDBLOCK`, Windows `ERROR_SHARING_VIOLATION`) blocks boot; an un-creatable lock file (read-only fs and the like) warns and continues. Deployment note: a supervisor that starts the replacement before the old process has finished its shutdown drain sees the refusal - stop must complete before start (systemd / `docker restart` are sequential and unaffected). Smoke: a new stage starts a second hub against the live staging tree and asserts it exits non-zero with the refusal while the first stays up, provably failing pre-fix (§1a.7 - on the unpatched Windows binary the second instance boots and runs concurrently). Two-pass review (§1a.6) caught the missing `O_CLOEXEC` (a spawned child could otherwise inherit the fd and brick the next restart) and the `ERROR_ACCESS_DENIED`-is-not-a-live-instance classification. 3.2.x only, not backported (hardening, not a security/crash fix; per §8 the 3.1.x bar is high). Operator guide: [`docs/BUILDING.md`](docs/BUILDING.md). diff --git a/core/sysinfo.lua b/core/sysinfo.lua index b1f02ee5..2f081ba9 100644 --- a/core/sysinfo.lua +++ b/core/sysinfo.lua @@ -71,6 +71,24 @@ local capture = function( cmd ) return trim( s ) end +-- Windows: Get-CimInstance is PowerShell 3.0+ and is ABSENT on the +-- PowerShell 2.0 that ships with Server 2008 R2 / Windows 7 (it raises +-- a catchable CommandNotFoundException there). Get-WmiObject exists in +-- EVERY Windows PowerShell (2.0 through 5.1); `powershell.exe` is +-- Windows PowerShell, not the PS-7/Core `pwsh` where the WMI cmdlets +-- were dropped. So try the modern CIM cmdlet and fall back to WMI in +-- one popen: PS 3.0+ takes the try branch, PS 2.0 the catch. Both expose +-- the identical `().` shape. Fixes "" OS/CPU/RAM +-- (and the pre-refactor cmd_hubinfo nil-concat crash) on old Windows. +local win_cim = function( wmi_class, property ) + return capture( + 'powershell -NoProfile -Command "try { (Get-CimInstance ' + .. wmi_class .. ').' .. property + .. ' } catch { (Get-WmiObject ' + .. wmi_class .. ').' .. property .. ' }"' + ) +end + -- Pull a colon-delimited field from /proc/cpuinfo-style output. -- Matches cmd_hubinfo's old `split( s, ":", "\n" )` shape: -- take the first line, strip everything up to and including the @@ -100,9 +118,9 @@ local os_kind = function( ) return _os_kind end local os_name = function( ) if _os_kind == "win" then - -- wmic was removed in Windows 11 24H2+; use PowerShell's - -- CIM cmdlets instead (matches cmd_hubinfo v0.29). - local s = capture( 'powershell -NoProfile -Command "(Get-CimInstance Win32_OperatingSystem).Caption"' ) + -- CIM with a WMI fallback (see win_cim) so PS 2.0 hosts + -- (Server 2008 R2 / Win7) report a real caption, not the default. + local s = win_cim( "Win32_OperatingSystem", "Caption" ) return s or "Microsoft Windows" elseif _os_kind == "unix" then local s = capture( "uname -s -r -v -m" ) @@ -114,7 +132,7 @@ end local cpu_info = function( ) if _os_kind == "win" then - local s = capture( 'powershell -NoProfile -Command "(Get-CimInstance Win32_Processor).Name"' ) + local s = win_cim( "Win32_Processor", "Name" ) return s elseif _os_kind == "unix" then -- Try /proc/cpuinfo with three known label variants: @@ -138,7 +156,7 @@ end local ram_total = function( ) if _os_kind == "win" then - local s = capture( 'powershell -NoProfile -Command "(Get-CimInstance Win32_ComputerSystem).TotalPhysicalMemory"' ) + local s = win_cim( "Win32_ComputerSystem", "TotalPhysicalMemory" ) if not s then return nil end return util.formatbytes( tonumber( s ) or s ) elseif _os_kind == "unix" then @@ -158,8 +176,8 @@ end -- `check_ram_total`. Preserved deliberately. local ram_free = function( ) if _os_kind == "win" then - -- FreePhysicalMemory is in KiB on both PowerShell paths. - local s = capture( 'powershell -NoProfile -Command "(Get-CimInstance Win32_OperatingSystem).FreePhysicalMemory"' ) + -- FreePhysicalMemory is in KiB on both the CIM and WMI paths. + local s = win_cim( "Win32_OperatingSystem", "FreePhysicalMemory" ) if not s then return nil end return util.formatbytes( ( tonumber( s ) or 0 ) * 1024 ) elseif _os_kind == "unix" then diff --git a/tests/unit/sysinfo_test.lua b/tests/unit/sysinfo_test.lua new file mode 100644 index 00000000..219bcca2 --- /dev/null +++ b/tests/unit/sysinfo_test.lua @@ -0,0 +1,160 @@ +--[[ + + tests/unit/sysinfo_test.lua + + Unit tests for core/sysinfo.lua (host OS / CPU / RAM probes). + + Coverage: + - Windows path: os_name / cpu_info / ram_total / ram_free parse a + stubbed popen result; each Windows popen command carries the + Get-CimInstance -> Get-WmiObject fallback (regression guard: PS + 2.0 hosts - Server 2008 R2 / Win7 - have no Get-CimInstance, so + the fallback MUST stay in the command). + - Windows query failure (empty popen) degrades: os_name -> default, + ram_total -> nil (the caller wraps nil with msg_unknown). + - Unix path: uname / /proc/meminfo / /proc/cpuinfo parsing. + + io.popen is stubbed to record the command + return canned output, so + the test is platform-independent (forces os_kind via util.path_sep). + + Run: lua5.4 tests/unit/sysinfo_test.lua + +]]-- + +local _cmds -- recorded popen commands (this load) +local _canned -- substring -> output ("" = empty = capture nil) +local _path_sep = "\\" -- "\\" -> win, "/" -> unix (read once at load) + +local function _stub_use( name ) + if name == "io" then + return { popen = function( cmd ) + _cmds[ #_cmds + 1 ] = cmd + local out + for sub, val in pairs( _canned ) do + if cmd:find( sub, 1, true ) then out = val; break end + end + if not out or out == "" then + -- emulate a handle whose read yields "" (capture -> nil) + return { read = function() return out or "" end, close = function() end } + end + local done = false + return { read = function() if done then return nil end done = true; return out end, + close = function() end } + end } + end + if name == "tostring" then return tostring end + if name == "tonumber" then return tonumber end + if name == "string" then return string end + if name == "util" then + return { + path_sep = function() return _path_sep end, + formatbytes = function( b ) + local n = tonumber( b ) + if not n then return nil end + return string.format( "%.2f GB", n / 1073741824 ) + end, + } + end + error( "sysinfo_test stub: missing dep " .. tostring( name ) ) +end + +local function load_sysinfo( ) + _cmds = { } + _G.use = _stub_use + return assert( loadfile( "core/sysinfo.lua" ) )( ) +end + +---------------------------------------------------------------------- +-- harness +---------------------------------------------------------------------- + +local _passes, _fails = 0, 0 +local function eq( what, got, want ) + if got == want then _passes = _passes + 1 + else _fails = _fails + 1 + io.stderr:write( string.format( "FAIL: %s\n got: %s\n want: %s\n", + what, tostring( got ), tostring( want ) ) ) end +end +local function truthy( what, v ) + if v then _passes = _passes + 1 + else _fails = _fails + 1; io.stderr:write( "FAIL: " .. what .. "\n" ) end +end + +local function any_cmd_has( needle ) + for _, c in ipairs( _cmds ) do if c:find( needle, 1, true ) then return true end end + return false +end + +---------------------------------------------------------------------- +-- Windows path + the CIM/WMI fallback regression guard +---------------------------------------------------------------------- + +_path_sep = "\\" +_canned = { + Caption = "Windows 10 Pro", + Win32_Processor = "Intel(R) Core(TM) i7", + TotalPhysicalMemory = "17179869184", -- 16 GiB in bytes + FreePhysicalMemory = "8388608", -- 8 GiB in KiB +} +do + local si = load_sysinfo( ) + eq( "win os_kind", si.os_kind(), "win" ) + eq( "win os_name parsed", si.os_name(), "Windows 10 Pro" ) + eq( "win cpu_info parsed", si.cpu_info(), "Intel(R) Core(TM) i7" ) + eq( "win ram_total formatted", si.ram_total(), "16.00 GB" ) + eq( "win ram_free formatted", si.ram_free(), "8.00 GB" ) + + -- The load-bearing regression: every Windows probe must try + -- Get-CimInstance AND carry the Get-WmiObject fallback. + truthy( "os command has Get-CimInstance", any_cmd_has( "Get-CimInstance Win32_OperatingSystem" ) ) + truthy( "os command has Get-WmiObject fallback", any_cmd_has( "Get-WmiObject Win32_OperatingSystem" ) ) + truthy( "cpu command has WMI fallback", any_cmd_has( "Get-WmiObject Win32_Processor" ) ) + truthy( "ram_total command has WMI fallback", any_cmd_has( "Get-WmiObject Win32_ComputerSystem" ) ) + -- ram_free re-shells on call, so its command lands in _cmds too + truthy( "ram_free command has WMI fallback", + any_cmd_has( "(Get-WmiObject Win32_OperatingSystem).FreePhysicalMemory" ) ) + truthy( "commands wrapped in try/catch", any_cmd_has( "try {" ) and any_cmd_has( "catch {" ) ) +end + +---------------------------------------------------------------------- +-- Windows query failure degrades cleanly (no crash, nil for RAM) +---------------------------------------------------------------------- + +_path_sep = "\\" +_canned = { } -- every popen yields "" -> capture returns nil +do + local si = load_sysinfo( ) + eq( "win os_name default on empty", si.os_name(), "Microsoft Windows" ) + eq( "win ram_total nil on empty", si.ram_total(), nil ) + eq( "win ram_free nil on empty", si.ram_free(), nil ) + eq( "win cpu_info nil on empty", si.cpu_info(), nil ) +end + +---------------------------------------------------------------------- +-- Unix path +---------------------------------------------------------------------- + +_path_sep = "/" +_canned = { + ["uname"] = "Linux host 5.4.0 x86_64", + ["MemTotal"] = "16384000", -- KiB + ["MemFree"] = "8192000", -- KiB + ["model name"] = "model name\t: Intel(R) Core(TM) i5", +} +do + local si = load_sysinfo( ) + eq( "unix os_kind", si.os_kind(), "unix" ) + eq( "unix os_name", si.os_name(), "Linux host 5.4.0 x86_64" ) + eq( "unix cpu_info (first_colon_field)", si.cpu_info(), "Intel(R) Core(TM) i5" ) + eq( "unix ram_total", si.ram_total(), string.format( "%.2f GB", 16384000 * 1024 / 1073741824 ) ) + eq( "unix ram_free", si.ram_free(), string.format( "%.2f GB", 8192000 * 1024 / 1073741824 ) ) + truthy( "unix uses no powershell", not any_cmd_has( "powershell" ) ) +end + +---------------------------------------------------------------------- + +if _fails > 0 then + io.stderr:write( string.format( "\nFAIL: %d/%d checks failed\n", _fails, _passes + _fails ) ) + os.exit( 1 ) +end +print( string.format( "OK: %d checks passed", _passes ) ) From 852270ce507e30173d3987bfb3c1a64e72f149e9 Mon Sep 17 00:00:00 2001 From: Aybo <265313202+Aybook@users.noreply.github.com> Date: Thu, 16 Jul 2026 20:18:01 +0200 Subject: [PATCH 07/18] fix(ci): install zlib dev package on all three release legs (#441) CMakeLists.txt does find_package(ZLIB REQUIRED) unconditionally for the Phase 8 S4b zlib_stream (ZLIF) module, but release.yml installed it on none of its three legs: - linux: build-essential cmake libssl-dev - aarch64: build-essential perl wget ca-certificates patchelf file git (debian:bullseye-slim, --no-install-recommends) - msys2: gcc cmake make openssl zip Every other build path already had it - smoke.yml on both legs, docker/Dockerfile, and the prerequisites documented in docs/BUILDING.md. Only the release pipeline drifted. The trap is that libz.so.1 (runtime) is present on virtually every Linux box because much of userspace links it, so zlib looks installed - but zlib.h and the libz.so link ship only in zlib1g-dev, which build-essential does not depend on. Verified from the package database: `apt-cache depends build-essential` lists no zlib, and a box with gcc, cmake and OpenSSL headers still had no /usr/include/zlib.h. Never caught because release.yml only fires on v* tags and release/** pushes, no v3.2.x tag exists yet, and release/3.1.x predates zlib_stream - so the release pipeline has never run against a tree containing find_package(ZLIB REQUIRED). The first v3.2.0 tag would have failed the aarch64 leg at configure. Also corrects the CMakeLists.txt comment that legitimised the gap ("aarch64 Bullseye container has zlib as a base-system package - no change needed") and lists every install site to keep in sync. Found by a dead-code audit sweep of the build glue. Co-authored-by: Claude Opus 4.8 (1M context) --- .github/workflows/release.yml | 11 +++++++++-- CHANGELOG.md | 2 ++ CMakeLists.txt | 21 ++++++++++++++++++--- 3 files changed, 29 insertions(+), 5 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 7d5d3bbf..a3f719b3 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -35,9 +35,15 @@ jobs: - uses: actions/checkout@v4 - name: Install build dependencies + # zlib1g-dev is REQUIRED, not optional: CMakeLists.txt does + # find_package(ZLIB REQUIRED) for the zlib_stream module, and the + # dev package is what ships zlib.h + the libz.so link. The runtime + # zlib1g is present on every box (half of userspace links it), but + # build-essential does NOT depend on zlib1g-dev - so configure fails + # without this line. Keep in sync with smoke.yml + docker/Dockerfile. run: | sudo apt-get update - sudo apt-get install -y build-essential cmake libssl-dev + sudo apt-get install -y build-essential cmake libssl-dev zlib1g-dev - name: Configure run: cmake -B build -DCMAKE_BUILD_TYPE=Release @@ -129,7 +135,7 @@ jobs: done apt-get install -y --no-install-recommends \ build-essential perl wget ca-certificates \ - patchelf file git + patchelf file git zlib1g-dev - name: Install CMake ${{ env.CMAKE_VERSION }} (Kitware aarch64 binary) # Bullseye's stock cmake (3.18) is too old for cmake_minimum_required(3.20). @@ -291,6 +297,7 @@ jobs: mingw-w64-ucrt-x86_64-cmake mingw-w64-ucrt-x86_64-make mingw-w64-ucrt-x86_64-openssl + mingw-w64-ucrt-x86_64-zlib zip - name: Configure diff --git a/CHANGELOG.md b/CHANGELOG.md index 6ad65083..9b751acd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -69,6 +69,8 @@ land on `release/3.1.x` per ### Bugfixes +- **`release.yml` never installed zlib's development package - the first `v3.2.0` tag would have failed the aarch64 build at configure.** `CMakeLists.txt` does `find_package(ZLIB REQUIRED)` (unconditional, for the Phase 8 S4b `zlib_stream` ZLIF module), but none of the three release legs installed it: the Linux leg had `build-essential cmake libssl-dev`, the aarch64 `debian:bullseye-slim` container had `build-essential perl wget ca-certificates patchelf file git` under `--no-install-recommends`, and the msys2 leg had no zlib package. Every *other* build path already had it (`smoke.yml` on both legs, `docker/Dockerfile`, and `docs/BUILDING.md`'s documented prerequisites) - only the release pipeline drifted. The trap: the runtime library `libz.so.1` is present on effectively every Linux box (much of userspace links it), so zlib "looks" installed, but `zlib.h` and the `libz.so` link ship **only** in `zlib1g-dev`, and `build-essential` does not depend on it - verified from the package database (`apt-cache depends build-essential` has no zlib entry; a box with gcc/cmake/OpenSSL headers still had no `/usr/include/zlib.h`). This was never caught because `release.yml` only fires on `v*` tags and on `release/**` pushes, no `v3.2.x` tag exists yet, and the `release/3.1.x` line predates `zlib_stream` entirely - so the release pipeline has never once been run against a tree containing `find_package(ZLIB REQUIRED)`. Fixed by adding `zlib1g-dev` to both apt legs and `mingw-w64-ucrt-x86_64-zlib` to the msys2 leg. The `CMakeLists.txt` comment that legitimised the gap ("aarch64 Bullseye container has zlib as a base-system package - no change needed") is corrected and now lists every install site to keep in sync. Found by a dead-code audit sweep of the build glue. 3.2.x only (the 3.1.x line has no `zlib_stream` and is unaffected). + - **`core/sysinfo.lua`: `+hubinfo` now works on old Windows (Server 2008 R2 / Windows 7) instead of showing ``** (Sopor-reported). The OS / CPU / RAM probes used `Get-CimInstance`, which is PowerShell 3.0+; PowerShell 2.0 (the default on Server 2008 R2 / Win7) raises `CommandNotFoundException`, so the queries returned empty / non-numeric output. On 3.2.x `cmd_hubinfo` already wrapped that in `or msg_unknown` (so it degraded to ``, no crash), but the values were simply unavailable on old Windows. Each Windows probe now runs `try { Get-CimInstance ... } catch { Get-WmiObject ... }` in one PowerShell call - `Get-WmiObject` exists in every Windows PowerShell (2.0-5.1; `powershell.exe`, not the PS-7/Core `pwsh`), so PS 2.0 hosts take the catch branch and report real values (validated live on Windows 11 + against `Get-WmiObject` directly). `tests/unit/sysinfo_test.lua` (new, 21 checks incl. a regression that the WMI fallback stays in every Windows command - provably fails if removed, §1a.7); registered on both smoke.yml legs. The pre-refactor 3.1.x `cmd_hubinfo.lua` (v0.29) additionally CRASHED here (`attempt to concatenate a nil value 'cache_check_ram_total'`, because it had no `or msg_unknown` guard); a v0.30 drop-in for the 3.1.x line carries both the nil-guard and the WMI fallback (no `v3.1.15` release, per the §8 drop-in-plugin path). 3.2.x. - **clear "port already in use" message + Windows port-hijack fix** (Sopor follow-up). When a listener could not bind its port the hub was unhelpful: on Linux it logged only the cryptic luasocket line (`...bind: address already in use`) then silently retried every 30s; on Windows the pre-bind `SO_REUSEADDR` (added for the Linux #128 TIME_WAIT fast-restart) has `SO_REUSEPORT` semantics, so a second process re-bound the SAME port with no error at all - two hubs then silently shared it, and any local process could hijack a luadch port. Two changes: (1) `core/server.lua` sets `SO_REUSEADDR` only on Linux (detected via `util.path_sep()`); on Windows it is skipped both pre- and post-bind, so a same-port second bind now fails visibly with `WSAEADDRINUSE` and the port can no longer be hijacked. Linux keeps the #128 fast-restart behaviour unchanged. (2) `core/hub.lua` `add_server_handler` now logs a clear operator line (`server: IPv4 TLS port 5001 is already in use - another process (or a second hub) holds it. Stop it or change the port in cfg/cfg.tbl; retrying every 30s.`) plus a generic message for any other listener-start failure that was previously swallowed. Verified that dropping `SO_REUSEADDR` on Windows does NOT regress the fast-restart: the full Windows smoke restart battery (plaintext / tier / `+reload` / dual-stack-same-port #107) stays green. Smoke: a new stage starts a second hub (separate dir, same ports as the live first hub) and asserts the guidance appears, provably failing pre-fix (§1a.7 - the assertion is on the new `change the port in cfg/cfg.tbl` phrase that the pre-fix cryptic line lacks; pre-fix Windows silently re-binds and logs nothing). Complements the single-instance lock (same Sopor thread): the lock stops a second copy in the SAME dir; this makes a second hub in a DIFFERENT dir with clashing ports fail loudly instead of silently. 3.2.x only. diff --git a/CMakeLists.txt b/CMakeLists.txt index 5c772d13..94560ec2 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -62,9 +62,24 @@ message(STATUS "luadch: OpenSSL ${OPENSSL_VERSION} at ${OPENSSL_INCLUDE_DIR}") # ----------------------------------------------------------------------------- # zlib: Phase 8 S4b (ADC-EXT ZLIF stream compression). Used by the # zlib_stream C module that wraps deflate / inflate streams with the -# Z_SYNC_FLUSH semantic ADC mandates. System zlib on Linux + MinGW -# (Ubuntu CI installs zlib1g-dev). aarch64 Bullseye container has zlib -# as a base-system package - no change needed. +# Z_SYNC_FLUSH semantic ADC mandates. This is the one dependency that is +# NOT bundled - it must come from the build environment. +# +# EVERY build path must install the zlib DEVELOPMENT package explicitly; +# there is no environment where it can be assumed present. The runtime +# library (libz.so.1) is on virtually every Linux box because half of +# userspace links it, but the headers and the libz.so link ship only in +# zlib1g-dev, and build-essential does NOT depend on it - so this +# find_package fails on an otherwise complete toolchain. An earlier +# version of this comment claimed the aarch64 Bullseye container had zlib +# "as a base-system package - no change needed"; that was wrong and left +# all three release.yml legs without it (never noticed, because the +# release workflow only fires on v* tags and the 3.1.x line predates +# zlib_stream). Install sites, keep in sync: +# .github/workflows/release.yml - zlib1g-dev (x2), mingw-w64-ucrt-x86_64-zlib +# .github/workflows/smoke.yml - zlib1g-dev, mingw-w64-ucrt-x86_64-zlib +# docker/Dockerfile - zlib-dev (alpine) +# Building from source? See docs/BUILDING.md. # ----------------------------------------------------------------------------- find_package(ZLIB REQUIRED) message(STATUS "luadch: ZLIB ${ZLIB_VERSION_STRING} at ${ZLIB_INCLUDE_DIRS}") From bc4d5589c097ded850c701d81fa712e83a0a9e0a Mon Sep 17 00:00:00 2001 From: Aybo <265313202+Aybook@users.noreply.github.com> Date: Thu, 16 Jul 2026 20:52:43 +0200 Subject: [PATCH 08/18] fix(scripts): resolve plugin lang lookups; guard the class repo-wide (#442) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit etc_cmdlog read lang.failmsg1 / lang.failmsg2, but its lang files define those messages as msg_denied / msg_nofile. Both lookups returned nil, the `or ""` fallback fired every time, and a German hub printed English on +cmdlog show regardless of cfg.language - silently, no error ever raised. Pre-existing since the SourceForge -> GitHub migration. etc_cmdlog v1.3 -> v1.4. This is the second instance of the class (#301 PR-2 fixed usr_share's lang.msg_minmax vs msg_sharelimits), and it surfaced in a plugin the one-off usr_share_lang_test.lua could not see. Per CLAUDE.md §1a.1 the guard is now repo-wide: tests/unit/plugin_lang_test.lua enumerates the shipped plugins from examples/cfg/cfg.tbl's scripts whitelist and asserts every lang.X reference resolves in BOTH .lang.en and .lang.de (68 plugins, 1034 distinct references). It supersedes usr_share_lang_test.lua, which is deleted - usr_share is one of the 68 swept, so coverage is a strict superset and the msg_sharelimits regression stays guarded. Registered on both CI legs in its place. The sweep found a third instance, also fixed: cmd_delreg declared `local msg_reason = lang.msg_reason or "No reason."`, a key no cmd_delreg lang file defines - copy-pasted from cmd_ban, where the key exists and the local is used. Harmless (the local was never referenced) but dead either way, so it is removed. cmd_delreg v0.32 -> v0.33. Two traps recorded in the test header for anyone extending it: - Asserting type(value) == "string" is wrong. ucmd_menu* menu structures, month_name and cmd_ascii.pics are legitimately tables; an early draft asserting string produced 149 false positives of 151 hits. Existence is the invariant. - Enumerating via io.popen("ls") is wrong. A native Windows Lua routes popen through cmd.exe, finds no `ls`, scans zero plugins and passes VACUOUSLY. Hence the cfg-whitelist enumeration (pure loadfile) plus explicit minimum-count guards. Provably fails pre-fix on all three findings (§1a.7): 2271/2277 checks, exit 1; 2275/2275 and exit 0 with the fix. Co-authored-by: Claude Opus 4.8 (1M context) --- .github/workflows/smoke.yml | 23 ++-- CHANGELOG.md | 2 + scripts/cmd_delreg.lua | 10 +- scripts/etc_cmdlog.lua | 13 ++- tests/unit/plugin_lang_test.lua | 165 +++++++++++++++++++++++++++++ tests/unit/usr_share_lang_test.lua | 105 ------------------ 6 files changed, 199 insertions(+), 119 deletions(-) create mode 100644 tests/unit/plugin_lang_test.lua delete mode 100644 tests/unit/usr_share_lang_test.lua diff --git a/.github/workflows/smoke.yml b/.github/workflows/smoke.yml index 5d32e0b4..327ff037 100644 --- a/.github/workflows/smoke.yml +++ b/.github/workflows/smoke.yml @@ -115,13 +115,18 @@ jobs: - name: Run lang unit test run: lua5.4 tests/unit/lang_test.lua - # #301 PR-2: usr_share lang-key consistency. Scans the plugin - # source for every lang.X reference and asserts X is defined in - # both .lang.de and .lang.en. Catches typos like the pre-fix - # lang.msg_minmax / msg_sharelimits drift that silently dropped - # the German translation. - - name: Run usr_share lang unit test - run: lua5.4 tests/unit/usr_share_lang_test.lua + # Repo-wide plugin lang-key consistency. For every bundled plugin + # that ships lang files, scans the source for each lang.X reference + # and asserts X is defined in both .lang.de and .lang.en. Catches + # the drift class that silently kills a translation: the lookup + # returns nil, the `or ""` fallback fires, and no error is + # ever raised. Supersedes the former usr_share-only test (#301 PR-2, + # lang.msg_minmax vs msg_sharelimits) - usr_share is one of the ~68 + # plugins swept here, so coverage is a strict superset. Generalised + # after the same bug reappeared in etc_cmdlog (lang.failmsg1/2 vs + # msg_denied/msg_nofile), which the one-plugin test could not see. + - name: Run plugin lang unit test + run: lua5.4 tests/unit/plugin_lang_test.lua # #327 etc_aliases: HTTP API surface + reload-safety. Exercises # all four +addalias reject branches (bad_alias / conflict_command @@ -413,9 +418,9 @@ jobs: shell: msys2 {0} run: lua5.4 tests/unit/lang_test.lua - - name: Run usr_share lang unit test + - name: Run plugin lang unit test shell: msys2 {0} - run: lua5.4 tests/unit/usr_share_lang_test.lua + run: lua5.4 tests/unit/plugin_lang_test.lua - name: Run etc_aliases unit test shell: msys2 {0} diff --git a/CHANGELOG.md b/CHANGELOG.md index 9b751acd..f0bb5f8e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -69,6 +69,8 @@ land on `release/3.1.x` per ### Bugfixes +- **`etc_cmdlog`: the German translation was dead - the plugin read lang keys that no lang file defines.** `scripts/etc_cmdlog.lua` looked up `lang.failmsg1` / `lang.failmsg2`, but `scripts/lang/etc_cmdlog.lang.{en,de}` define those messages as `msg_denied` / `msg_nofile`. Both lookups therefore returned nil, the `or ""` fallback fired every time, and a German hub printed English on `+cmdlog show` regardless of `cfg.language` - silently, with no error ever raised. Pre-existing since the SourceForge -> GitHub migration (both sides landed already-mismatched); plugin bumped to v1.4. **This is the second instance of the same class** ([#301](https://github.com/luadch-ng/luadch/issues/301) PR-2 fixed `usr_share`'s `lang.msg_minmax` vs `msg_sharelimits`), and it reappeared in a plugin the one-off `usr_share_lang_test.lua` could not see - so per §1a.1 the guard is now repo-wide: new `tests/unit/plugin_lang_test.lua` enumerates the shipped plugins from `examples/cfg/cfg.tbl`'s `scripts` whitelist and asserts every `lang.X` reference in each plugin resolves in **both** `.lang.en` and `.lang.de` (68 plugins, 1034 distinct references). It **supersedes** `usr_share_lang_test.lua` (deleted - `usr_share` is one of the 68 swept, so coverage is a strict superset, and the `msg_sharelimits` regression stays guarded); registered on both CI legs in its place. The sweep surfaced a third instance, also fixed here: `cmd_delreg` declared `local msg_reason = lang.msg_reason or "No reason."` - a key no `cmd_delreg` lang file defines - copy-pasted from `cmd_ban`, where the key exists and the local is used; harmless because the local was never referenced, but the lookup was dead either way, so the dead local is gone (v0.33). Two traps worth recording for anyone extending the test: asserting `type(value) == "string"` is wrong (`ucmd_menu*` menu structures, `month_name`, `cmd_ascii.pics` are legitimately **tables** - an early draft produced 149 false positives out of 151 hits), and enumerating via `io.popen("ls")` is wrong (a native Windows Lua routes popen through cmd.exe, finds no `ls`, scans zero plugins and passes **vacuously** - hence the cfg-whitelist enumeration plus explicit minimum-count guards). Provably fails pre-fix on all three findings (§1a.7). 3.2.x only, not backported. + - **`release.yml` never installed zlib's development package - the first `v3.2.0` tag would have failed the aarch64 build at configure.** `CMakeLists.txt` does `find_package(ZLIB REQUIRED)` (unconditional, for the Phase 8 S4b `zlib_stream` ZLIF module), but none of the three release legs installed it: the Linux leg had `build-essential cmake libssl-dev`, the aarch64 `debian:bullseye-slim` container had `build-essential perl wget ca-certificates patchelf file git` under `--no-install-recommends`, and the msys2 leg had no zlib package. Every *other* build path already had it (`smoke.yml` on both legs, `docker/Dockerfile`, and `docs/BUILDING.md`'s documented prerequisites) - only the release pipeline drifted. The trap: the runtime library `libz.so.1` is present on effectively every Linux box (much of userspace links it), so zlib "looks" installed, but `zlib.h` and the `libz.so` link ship **only** in `zlib1g-dev`, and `build-essential` does not depend on it - verified from the package database (`apt-cache depends build-essential` has no zlib entry; a box with gcc/cmake/OpenSSL headers still had no `/usr/include/zlib.h`). This was never caught because `release.yml` only fires on `v*` tags and on `release/**` pushes, no `v3.2.x` tag exists yet, and the `release/3.1.x` line predates `zlib_stream` entirely - so the release pipeline has never once been run against a tree containing `find_package(ZLIB REQUIRED)`. Fixed by adding `zlib1g-dev` to both apt legs and `mingw-w64-ucrt-x86_64-zlib` to the msys2 leg. The `CMakeLists.txt` comment that legitimised the gap ("aarch64 Bullseye container has zlib as a base-system package - no change needed") is corrected and now lists every install site to keep in sync. Found by a dead-code audit sweep of the build glue. 3.2.x only (the 3.1.x line has no `zlib_stream` and is unaffected). - **`core/sysinfo.lua`: `+hubinfo` now works on old Windows (Server 2008 R2 / Windows 7) instead of showing ``** (Sopor-reported). The OS / CPU / RAM probes used `Get-CimInstance`, which is PowerShell 3.0+; PowerShell 2.0 (the default on Server 2008 R2 / Win7) raises `CommandNotFoundException`, so the queries returned empty / non-numeric output. On 3.2.x `cmd_hubinfo` already wrapped that in `or msg_unknown` (so it degraded to ``, no crash), but the values were simply unavailable on old Windows. Each Windows probe now runs `try { Get-CimInstance ... } catch { Get-WmiObject ... }` in one PowerShell call - `Get-WmiObject` exists in every Windows PowerShell (2.0-5.1; `powershell.exe`, not the PS-7/Core `pwsh`), so PS 2.0 hosts take the catch branch and report real values (validated live on Windows 11 + against `Get-WmiObject` directly). `tests/unit/sysinfo_test.lua` (new, 21 checks incl. a regression that the WMI fallback stays in every Windows command - provably fails if removed, §1a.7); registered on both smoke.yml legs. The pre-refactor 3.1.x `cmd_hubinfo.lua` (v0.29) additionally CRASHED here (`attempt to concatenate a nil value 'cache_check_ram_total'`, because it had no `or msg_unknown` guard); a v0.30 drop-in for the 3.1.x line carries both the nil-guard and the WMI fallback (no `v3.1.15` release, per the §8 drop-in-plugin path). 3.2.x. diff --git a/scripts/cmd_delreg.lua b/scripts/cmd_delreg.lua index 23ded694..08a6e282 100755 --- a/scripts/cmd_delreg.lua +++ b/scripts/cmd_delreg.lua @@ -6,6 +6,13 @@ - usage: [+!#]delreg nick | [+!#]delreg nick + v0.33: + - removed the unused "msg_reason" local: it read + "lang.msg_reason", a key no cmd_delreg lang file defines + (copy-pasted from cmd_ban, where the key exists and the + local IS used). Nothing degraded because the local was + never referenced, but the lookup was dead either way + v0.32: - #243 family-wide consistency sweep: ADC `+delreg nick` path now uses the `activate and prefix_table` guard + @@ -138,7 +145,7 @@ -------------- local scriptname = "cmd_delreg" -local scriptversion = "0.32" +local scriptversion = "0.33" local cmd = "delreg" @@ -161,7 +168,6 @@ local block = hub.import( "etc_trafficmanager" ) local lang, err = cfg.loadlanguage( scriptlang, scriptname ); lang = lang or {}; err = err and hub.debug( err ) local msg_denied = lang.msg_denied or "[ DELREG ]--> You are not allowed to use this command or to delreg targets with this level." -local msg_reason = lang.msg_reason or "No reason." local msg_usage = lang.msg_usage or "Usage: [+!#]delreg nick / or del with blacklist entry: [+!#]delreg nick " local msg_error = lang.msg_error or "[ DELREG ]--> An error occurred: " local msg_del = lang.msg_del or "[ DELREG ]--> You were delregged." diff --git a/scripts/etc_cmdlog.lua b/scripts/etc_cmdlog.lua index a2fa2917..0960848e 100644 --- a/scripts/etc_cmdlog.lua +++ b/scripts/etc_cmdlog.lua @@ -6,6 +6,13 @@ Usage: [+!#]cmdlog show + v1.4: + - fix: the language lookups read "lang.failmsg1" / "lang.failmsg2", + but the lang files define the keys as "msg_denied" / "msg_nofile" + - both lookups returned nil, the "or" fallback to the hardcoded + English literal fired every time, and the German translations + were unreachable regardless of cfg.language + v1.3: - HTTP API: GET /v1/log/cmd?lines=N (admin scope) #82 Phase 3 PR-4 @@ -59,7 +66,7 @@ -------------- local scriptname = "etc_cmdlog" -local scriptversion = "1.3" +local scriptversion = "1.4" -- HTTP API tail-style cap per docs/HTTP_API.md §6.4. Same value -- as cmd_errors.lua for consistency across log endpoints. @@ -102,8 +109,8 @@ local help_title = lang.help_title or "etc_cmdlog.lua" local help_usage = lang.help_usage or "[+!#]cmdlog show" local help_desc = lang.help_desc or "Shows the command log" -local msg_denied = lang.failmsg1 or "You are not allowed to use this command." -local msg_nofile = lang.failmsg2 or "No 'cmd.log' found." +local msg_denied = lang.msg_denied or "You are not allowed to use this command." +local msg_nofile = lang.msg_nofile or "No 'cmd.log' found." local msg_usage = lang.msg_usage or "Usage: [+!#]cmdlog show" local msg1 = lang.msg1 or " | Command: [+!#]" diff --git a/tests/unit/plugin_lang_test.lua b/tests/unit/plugin_lang_test.lua new file mode 100644 index 00000000..d89d07e7 --- /dev/null +++ b/tests/unit/plugin_lang_test.lua @@ -0,0 +1,165 @@ +--[[ + + tests/unit/plugin_lang_test.lua + + Repo-wide lang-key consistency check for the bundled plugins. + + Every plugin binds its language table as `local lang, err = + cfg.loadlanguage( scriptlang, scriptname )` and then reads keys with + the `local msg_x = lang.some_key or ""` idiom. If the + source reads a key the lang file does not define, the lookup returns + nil, the `or` fallback fires, and the plugin silently serves the + hardcoded English literal forever - the translation is dead and no + error is ever raised. cfg.language makes no difference. That failure is + invisible at runtime, which is exactly why it needs a test. + + This has now bitten twice: + - #301 PR-2: scripts/usr_share.lua read `lang.msg_minmax` while the + lang files defined `msg_sharelimits`. + - scripts/etc_cmdlog.lua read `lang.failmsg1` / `lang.failmsg2` while + the lang files defined `msg_denied` / `msg_nofile`. + The first was fixed with a one-plugin test (usr_share_lang_test.lua), + and the bug promptly reappeared in a plugin that test did not look at. + So this supersedes it with a sweep over EVERY bundled plugin - per + CLAUDE.md §1a.1, fix the pattern everywhere, not just where it was + noticed. usr_share is one of the plugins scanned here, so coverage is a + strict superset of the test this replaces. + + Method: enumerate the shipped plugins from `examples/cfg/cfg.tbl`'s + `scripts` whitelist, and for each one that ships lang files, load both + the .en and .de table, scan the plugin source for every `lang.X` + reference, and assert X exists in BOTH tables. + + Three traps this deliberately avoids: + - Do NOT assert the value is a string. Plenty of legitimate keys are + TABLES (`ucmd_menu*` right-click menu structures, `month_name`, + cmd_ascii's `pics`). An earlier draft asserted `type(v)=="string"` + and produced 149 false positives out of 151 hits. Existence is the + invariant; the type is the plugin's business. + - Comments are stripped first, so a `lang.X` mentioned in a header + changelog cannot trip a false positive. + - No shell/`io.popen` globbing. A native Windows Lua routes popen + through cmd.exe, where `ls` does not exist - the enumeration would + silently yield nothing and the whole test would pass vacuously. + Reading the cfg whitelist is pure `loadfile` and works everywhere, + and it ties this test to the plugin set we actually ship. + + Provably fails pre-fix: on the unpatched tree it reports + etc_cmdlog lang.failmsg1 / lang.failmsg2 and cmd_delreg lang.msg_reason + as missing (CLAUDE.md §1a.7). + + Run: lua tests/unit/plugin_lang_test.lua (any Lua 5.4, from repo root) + Exit code 0 = pass, 1 = failure (CI-friendly). + +]]-- + +local CFG_TBL = "examples/cfg/cfg.tbl" +local LANG_DIR = "scripts/lang/" +local PLUGIN_DIR = "scripts/" + +-- Vacuity guard. If the cfg whitelist ever fails to load or its shape +-- changes, this test would scan nothing and every assertion below would +-- pass trivially - a green test that checks nothing is worse than no test. +-- The tree ships ~78 plugins, ~68 of them with lang files. Fail loudly if +-- we ever see implausibly few. Lower only alongside a real drop. +local MIN_PLUGINS = 60 +local MIN_WITH_LANG = 50 + +local function read_text( path ) + local f = io.open( path, "rb" ) + if not f then return nil end + local s = f:read( "*a" ) + f:close( ) + return s +end + +local function load_table( path ) + local chunk = loadfile( path ) + if not chunk then return nil, "cannot load" end + local ok, t = pcall( chunk ) + if not ok or type( t ) ~= "table" then return nil, "did not return a table" end + return t +end + +-- Strip block comments `--[[ ... ]]` and line comments so a `lang.X` in a +-- header changelog or an explanatory note cannot register as a lookup. +local function strip_comments( s ) + s = s:gsub( "%-%-%[%[.-%]%]", "" ) + s = s:gsub( "%-%-[^\n]*", "" ) + return s +end + +local failures, checks = 0, 0 +local function check( label, ok ) + checks = checks + 1 + if not ok then + failures = failures + 1 + io.write( "FAIL " .. label .. "\n" ) + end +end + +-- cfg.scripts entries come in two shapes: a bare "name.lua" string, and a +-- `{ "name.lua", enabled = true }` table (the per-plugin toggle form). +local function entry_name( v ) + local file = ( type( v ) == "table" ) and v[ 1 ] or v + if type( file ) ~= "string" then return nil end + return file:match( "^(.+)%.lua$" ) +end + +local cfg, cfg_err = load_table( CFG_TBL ) +check( CFG_TBL .. " loads (" .. tostring( cfg_err ) .. ")", cfg ~= nil ) + +local names = { } +if cfg and type( cfg.scripts ) == "table" then + for _, v in ipairs( cfg.scripts ) do + local n = entry_name( v ) + if n then names[ #names + 1 ] = n end + end +end +table.sort( names ) + +check( string.format( "cfg.scripts lists at least %d plugins (found %d)", + MIN_PLUGINS, #names ), + #names >= MIN_PLUGINS ) + +local scanned, total_refs = 0, 0 + +for _, name in ipairs( names ) do + local en_path = LANG_DIR .. name .. ".lang.en" + -- Not every plugin ships lang files; those are simply out of scope + -- here (nothing to be inconsistent with). + if read_text( en_path ) then + local source = read_text( PLUGIN_DIR .. name .. ".lua" ) + local en, en_err = load_table( en_path ) + local de, de_err = load_table( LANG_DIR .. name .. ".lang.de" ) + + check( name .. ": plugin source exists", source ~= nil ) + check( name .. ": .lang.en loads (" .. tostring( en_err ) .. ")", en ~= nil ) + check( name .. ": .lang.de exists and loads (" .. tostring( de_err ) .. ")", de ~= nil ) + + if source and en and de then + scanned = scanned + 1 + local seen = { } + for key in strip_comments( source ):gmatch( "lang%.([%w_]+)" ) do + if not seen[ key ] then + seen[ key ] = true + total_refs = total_refs + 1 + check( name .. ": lang." .. key .. " defined in .lang.en", en[ key ] ~= nil ) + check( name .. ": lang." .. key .. " defined in .lang.de", de[ key ] ~= nil ) + end + end + end + end +end + +check( string.format( "scanned at least %d plugins with lang files (scanned %d)", + MIN_WITH_LANG, scanned ), + scanned >= MIN_WITH_LANG ) + +io.write( string.format( "\n%d/%d checks passed (%d plugins scanned, %d distinct lang.X references)\n", + checks - failures, checks, scanned, total_refs ) ) +if failures > 0 then + io.write( "FAIL " .. failures .. " check(s) failed\n" ) + os.exit( 1 ) +end +io.write( "OK plugin_lang_test\n" ) diff --git a/tests/unit/usr_share_lang_test.lua b/tests/unit/usr_share_lang_test.lua deleted file mode 100644 index 0f7b5b29..00000000 --- a/tests/unit/usr_share_lang_test.lua +++ /dev/null @@ -1,105 +0,0 @@ ---[[ - - tests/unit/usr_share_lang_test.lua - - Lang-key consistency check for scripts/usr_share.lua (#301 PR-2). - - Pre-fix: scripts/usr_share.lua read `lang.msg_minmax`, but - scripts/lang/usr_share.lang.{de,en} defined the key as - `msg_sharelimits`. The German translation was therefore DEAD - the - lookup returned nil, the `or` fallback to the hardcoded English - literal fired every time, and a German hub showed the English - message regardless of cfg.language. - - Test: scan the plugin source for every `lang.X` reference and assert - X exists in BOTH the .lang.de and .lang.en tables. Provably fails on - the pre-fix code (`lang.msg_minmax` -> nil -> assertion error); - passes once the lookup is renamed to `lang.msg_sharelimits`. - - Generic-enough that the same pattern can be lifted into a sweeping - check across all plugins if the broader #301 cleanup wants it. - - Run: lua tests/unit/usr_share_lang_test.lua (any Lua 5.4) - Exit code 0 = pass, 1 = failure (CI-friendly). - -]]-- - -local PLUGIN_SOURCE = "scripts/usr_share.lua" -local LANG_DE = "scripts/lang/usr_share.lang.de" -local LANG_EN = "scripts/lang/usr_share.lang.en" - -local function read_text( path ) - local f, err = io.open( path, "rb" ) - if not f then - io.stderr:write( "FATAL: cannot open " .. path .. ": " .. tostring( err ) .. "\n" ) - os.exit( 1 ) - end - local s = f:read( "*a" ) - f:close( ) - return s -end - -local function load_lang( path ) - local chunk, err = loadfile( path ) - if not chunk then - io.stderr:write( "FATAL: cannot load " .. path .. ": " .. tostring( err ) .. "\n" ) - os.exit( 1 ) - end - local ok, t = pcall( chunk ) - if not ok or type( t ) ~= "table" then - io.stderr:write( "FATAL: " .. path .. " did not return a table: " .. tostring( t ) .. "\n" ) - os.exit( 1 ) - end - return t -end - -local source = read_text( PLUGIN_SOURCE ) -local de = load_lang( LANG_DE ) -local en = load_lang( LANG_EN ) - --- Strip block comments `--[[ ... ]]` and line comments so a stray --- `lang.X` in a comment cannot trip a false positive. -local function strip_comments( s ) - s = s:gsub( "%-%-%[%[.-%]%]", "" ) - s = s:gsub( "%-%-[^\n]*", "" ) - return s -end -source = strip_comments( source ) - --- Collect every `lang.X` reference. Lua identifier = [A-Za-z_][A-Za-z0-9_]*. -local refs = { } -for key in source:gmatch( "lang%.([%w_]+)" ) do - refs[ key ] = true -end - -local failures, checks = 0, 0 -local function check( label, ok ) - checks = checks + 1 - if not ok then - failures = failures + 1 - io.write( "FAIL " .. label .. "\n" ) - else - io.write( "ok " .. label .. "\n" ) - end -end - --- The reference set must be non-empty - otherwise the test is vacuous --- (a refactor that drops every lang.X lookup would make the assertions --- below trivially pass). -local n = 0; for _ in pairs( refs ) do n = n + 1 end -check( "found at least one lang.X reference in " .. PLUGIN_SOURCE, n > 0 ) - -for key in pairs( refs ) do - check( "de[" .. key .. "] defined", - type( de[ key ] ) == "string" and de[ key ] ~= "" ) - check( "en[" .. key .. "] defined", - type( en[ key ] ) == "string" and en[ key ] ~= "" ) -end - -io.write( string.format( "\n%d/%d checks passed (%d lang.X reference(s) scanned)\n", - checks - failures, checks, n ) ) -if failures > 0 then - io.write( "FAIL " .. failures .. " check(s) failed\n" ) - os.exit( 1 ) -end -io.write( "OK usr_share_lang_test\n" ) From c42b1e4d607bd070e3453a2c2b81a858c37b8f77 Mon Sep 17 00:00:00 2001 From: Aybo <265313202+Aybook@users.noreply.github.com> Date: Thu, 16 Jul 2026 21:57:37 +0200 Subject: [PATCH 09/18] fix(cmd_ban): reject a bantime below 1 instead of silently losing the ban (#443) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `+ban nick X -5 reason` stored bantime = -300. At X's next login the expiry check computed an always-negative remaining, treated the entry as expired and PRUNED it - so X was kicked once and then walked straight back in, while the operator believed X was banned. The only time validation was is_integer( time ), and -5 == math.floor( -5 ), so negatives sailed through. Meanwhile help_desc and both lang files advertised "negative values means ban forever" - a feature deliberately removed in v0.15 ("removed the ban forever crap"), leaving the promise stranded ever since. Fixed by rejecting < 1 (new msg_badtime, en + de) at the existing validation site, which sits before target resolution. The stale promise is gone from help_desc and both lang files. Zero is rejected for the same reason (a 0-minute ban expires instantly). This also closes a divergence per §1a.1: the HTTP path has enforced min = 1 since #82 (request_schema + a duration_minutes < 1 guard). Only the older ADC path was broken. Also removes the commented-out ban-forever block: it referenced msg_forever, which no lang file defines, so it could never have been revived by uncommenting anyway. Note the originally-suspected fix - dropping `[-]?` from the parser - is a no-op: `%S+` already matches `-5` (verified in Lua 5.4). Validation, not parsing, was the gap. A real permanent ban is deliberately NOT reintroduced here; it is tracked as a separate feature (ADC STA 231 + TL-1, an explicit keyword rather than a magic negative). cmd_ban v0.42 -> v0.43. §1a.7: a new smoke stage sends `+ban nick -5|0 spam` and asserts the rejection, with a positive control that a valid time still reaches the target lookup. Fails pre-fix - the hub answers "User not found.", proving the negative was accepted and the command ran on to target resolution. Found by the dead-code audit. Co-authored-by: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 2 + scripts/cmd_ban.lua | 59 ++++++++++++--------- scripts/lang/cmd_ban.lang.de | 3 +- scripts/lang/cmd_ban.lang.en | 3 +- tests/smoke/run.py | 99 ++++++++++++++++++++++++++++++++++++ 5 files changed, 139 insertions(+), 27 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f0bb5f8e..2b024516 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -69,6 +69,8 @@ land on `release/3.1.x` per ### Bugfixes +- **`cmd_ban`: a negative `