test(freestanding): probe the parse-depth guard, and correct the soak's stale premise

[Nitjsefnie]

What does this PR do?

Replaces the fossil comment in tools/embed_stack_soak.sh, documents what the depth guards are actually sized for, and adds a second soak run that exercises PARSE_MAX_DEPTH — so "the guard fires cleanly" is tested rather than assumed.

Related Issues and Pull Requests

Fixes #758

Takes items 1 and 3 of your Suggested list, and the second branch of item 2.

Changes

• tools/embed_stack_soak.sh: header rewritten — 64 KiB is now described as a deliberately tight per-level regression budget rather than EigenOS's stack. New second run: embed_stack_soak depth-guard under ulimit -s 256.
• tools/embed_stack_soak_main.c: a depth-guard mode reached only by argv, building its nested source in .bss so the probe isn't charged for its own input.
• docs/FREESTANDING.md: new "Stack budget and the depth guards" subsection under "Build profile" — that section already said "Two CI gates keep it honest:" with a bullet per gate, and the soak was the one freestanding gate missing from the list. Now three. (grep -rn "PARSE_MAX_DEPTH\|COMPILE_MAX_DEPTH" docs/ returned nothing beforehand — there was no prior home.)
• CHANGELOG.md: one ### Changed bullet.

On item 2, I took the "no" branch: the guards are documented as sized for the 1 MiB boot stack, with the constrained profile relying on the canary. Making them stack-derived is a much larger behavioural change to shipped guards, whereas the soak is explicitly valuable as a per-level cost detector — it caught #361 — and a tighter-than-reality budget is right for that job. Overrule me if you'd rather have the guards reachable everywhere.

Testing

Your premise holds: the guard is unreachable at 64 KiB, and the comment is a fossil. Three of your numbers I could not reproduce, and since they're yours I'd rather show the measurements than quietly restate them. Every figure below is a 10–20 trial pass-rate, because single-shot ulimit bisects are not repeatable here — ASLR moves the initial stack pointer, and my first sweep came back non-monotonic (40→segv, 38→ok, 37→segv, 36→ok), which is what forced the method change.

The floor is a band, not a threshold. Hosted release CLI, print of 1:

rlimit	48K	46K	44K	42K	40K	38K	36K	34K
pass	20/20	20/20	20/20	14/20	14/20	3/20	6/20	0/20

Reliable floor 44 KiB, not 32–40 — your range is the flaky band's lower half. On the soak harness (the binary the gate actually runs, which is different and lower) the baseline is ~32 KiB, i.e. half the 64 KiB budget rather than 62%. The doc and script use the soak's own numbers, since that's the binary the budget applies to.

PARSE_MAX_DEPTH fires at 128 source levels, not 256. parse_expression and parse_unary each bump g_parse_depth, so the counter runs ~2 per parenthesis level:

levels=127 stack=8192K rc=0
levels=128 stack=8192K rc=1
  Parse error line 1: expression nesting too deep

So the death-vs-guard gap is ~2.3× (56 vs 128), not the four-to-six× in the issue. Nesting death under 64 KiB is clean to 54 levels, banded 56–60, dead from 62 — your "between 40 and 60" is right at the top but 40–54 was 10/10 clean.

ulimit -s 128 is not clean. It's 18/20 on the soak binary and 5/10 on the CLI. The reliable floor is 144 KiB, which is why the new probe runs at 256 KiB — ~1.8× margin. At 64 KiB the same input is SIGSEGV 20/20, which is the direct evidence the guard was untestable under the existing gate.

The new run asserts three things together: exit 0 (ruling out the 139), nesting too deep on stderr (so an unrelated rejection can't pass it), and depth-guard OK on stdout, which is emitted only after the harness confirms the eval returned NULL and a following 6 * 7 still evaluates to 42 — i.e. the guard unwound cleanly rather than wedging the runtime. GUARD_LEVELS is 300, past 256 whether that constant is read as source levels or parser recursion, so the probe stays valid if the bump rate changes.

Two mutants confirm it isn't vacuous: running the probe at 64 KiB instead of 256 fails with rc=139, and dropping GUARD_LEVELS to 100 fails with 100 nested levels were ACCEPTED — the parse-depth guard did not fire.

All four steps of the freestanding job pass, plus CC=gcc ./build.sh and the suite at 3367/3367, exit 0. doc_drift_check.sh exit 0.

Follow-ups / Known Limitations

• I could not verify the 1 MiB EigenOS figure and did not adopt it as ours. EigenOS is not a submodule here (git submodule status empty, no .gitmodules), and gh api repos/InauguralSystems/EigenOS 404s for me — it isn't in the org's 22 public repos, so it's private or renamed, and docs/FREESTANDING.md:5 links to a URL I can't reach. Every mention is attributed to PARSE_MAX_DEPTH is unreachable under a 64 KiB stack, and the soak gate's premise is stale (EigenOS is 1 MiB now) #758 and explicitly marked as uncheckable from this repo — which is itself part of the argument for not describing the soak budget in terms of it. I did soften the pre-existing EigenOS #UD heisenbug root cause: 12.7 KiB compiler stack frames + the EIGS_POISON hunter #361 sentence to "the 64 KiB boot stack EigenOS used at the time", since that past-tense claim is one the issue supports.
• .github/workflows/ci.yml:214 is now stale: name: Embedded-stack soak (REPL soak in a 64 KiB stack rlimit) no longer describes a step that also runs a 256 KiB probe. It's a display string and I didn't want to touch CI config unilaterally in a PR about a stale comment — but say the word and I'll fold in the one-line rename.
• 256 KiB is calibrated on this machine. 144 KiB was the reliable floor here; CI is a different container and CPU. If run 2 ever flakes, raise the number rather than weakening the assertion.
• COMPILE_MAX_DEPTH (128) is left alone. Your -fstack-usage figure implies ~194 KiB for it on its own, so the same reachability question applies, but testing it needs a different probe shape and belongs in its own change.

Checklist

• make test passes locally — 3367/3367, plus all four freestanding steps
• New builtins have signature comments and docs in docs/BUILTINS.md — n/a
• New library functions follow conventions in docs/STDLIB.md — n/a
• New examples have a comment header explaining what they demonstrate — n/a
• CHANGELOG.md updated (if user-facing change)

Generated by Claude Opus 5 (brief, implementation, review)

[InauguralPhysicist]

Approving. Verified on current main (83ff452, post-#811/#814 — merges cleanly):

• Both soak runs pass on the merged tree: the original 64 KiB REPL soak (104 steps) and the new 256 KiB depth-guard probe (nesting too deep on stderr, clean unwind, 6 * 7 still evaluates after).
• Non-vacuity mutant reproduced: GUARD_LEVELS 300→100 fails with 100 nested levels were ACCEPTED — the parse-depth guard did not fire, exactly as documented. The three-part assertion (rc, stderr marker, post-guard eval) is the right shape — none of the legs can pass for the wrong reason.
• The measurement work is the part I want to call out: banded pass-rates instead of single-shot ulimit bisects is the correct method under ASLR, and correcting the issue's numbers with the method shown (44 KiB reliable floor, ×2 parse-depth bump rate, 144 KiB floor for the guard probe) is exactly what the issue deserved. Same for refusing to restate the 1 MiB EigenOS figure you couldn't check from this repo — it's private, so your caveat-and-attribute handling was right.
• Taking the "no" branch on item 2 (guards stay boot-stack-sized, soak stays a deliberately tight per-level budget) is the decision I'd have made — the soak's value is as a cost detector, and #361 is the proof.

The stale ci.yml:214 display name: don't respin for it — I'll fix the one-line name string on main directly after this merges.

Merging.