Skip to content

fix(web): exempt planned --max-requests recycles from the crash-loop guard#523

Open
mirchaemanuel wants to merge 1 commit into
illegalstudio:mainfrom
mirchaemanuel:fix/516-max-requests-recycle-fast-death
Open

fix(web): exempt planned --max-requests recycles from the crash-loop guard#523
mirchaemanuel wants to merge 1 commit into
illegalstudio:mainfrom
mirchaemanuel:fix/516-max-requests-recycle-fast-death

Conversation

@mirchaemanuel

Copy link
Copy Markdown
Contributor

Fixes the crash-loop-guard half of #516 (Caveat 2): the --web master counts scheduled --max-requests worker recycles as startup deaths, so sustained traffic with a small quota trips MAX_FAST_DEATHS and shuts the whole server down.

What

  • The worker child now exits with a dedicated RECYCLE_EXIT_CODE (86) after serve() returns — which only happens on a planned --max-requests recycle. Documented as distinct from 0 (clean exit), 1 (worker errors), and 2 (usage errors).
  • The master's reaper classifies a reaped status via a new is_planned_recycle() (WIFEXITED && WEXITSTATUS == RECYCLE_EXIT_CODE) and exempts planned recycles from the fast-death accounting. A recycle neither increments nor resets the streak: healthy recycle churn cannot trip the guard, but it also cannot mask a real crash loop interleaved with it. Respawn behavior is unchanged — every dead worker is replaced.
  • Signal deaths and every other exit code keep feeding the guard exactly as before.

Why the "neutral" semantics

Resetting the streak on recycle would let a crash-looping handler hide behind recycle churn; counting it (status quo) kills a healthy server. Skipping the accounting entirely for planned recycles preserves both properties.

Verification

  • E2E before/after (compiled examples/web-hello, --workers 1 --max-requests 5, 80 sequential requests with client retries at recycle boundaries):
  • Guard still trips on real crash loops: fixed binary with an unbindable --listen 8.8.8.8:… → master gives up and exits within seconds, as before.
  • Unit tests (3 new, on the status-word classification): recycle exit is planned; exit codes 0/1/2 are not; signal deaths (SIGSEGV/SIGKILL/SIGTERM) are not — WIFEXITED gates the code check so a raw signal status can never alias the recycle code. cargo test -p elephc-web: 8 passed.
  • E2E regression test web_max_requests_survives_sustained_recycle_churn in tests/web_tests.rs: 30 requests through a quota of 2 (~15 recycles > MAX_FAST_DEATHS), asserts the master is still alive. cargo test --test web_tests web_max_requests: 2 passed.
  • cargo build --release clean under the zero-warnings gate.

Notes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant