fix(transform): suspend on yields in nested generator loop headers

[proggeramlug]

Summary

A generator never suspended on a yield in a loop header (while / do…while condition, for condition / update) when the loop was nested inside if / else, try / catch / finally, a switch case, a label or another loop. #5933 fixed only loops that are direct statements of the generator body. The residual yield did not suspend, so [...g()] was empty, a sent-value loop never terminated, and .return() / .throw() hit a generator that had already finished. Async generators were affected the same way.

Minifiers emit this shape. lru-cache 11.5.2 resolves import + node to dist/esm/node/index.min.js, where *#A / *#z are if(this.#n)for(let t=…;this.#V(t)&&(…(yield t),t!==this.#a);)t=…, so every iteration API came back empty.

Root cause

• The linearizer only splits a compound statement into states when body_contains_yield finds a suspend point in it. This decides the If (linearize.rs:1355), Try (:1130), Labeled (:1594), Switch (:1665) and loop-body arms.
• body_contains_yield (crates/perry-transform/src/generator/break_continue.rs:276) looked at loop bodies only (While :326, DoWhile :332, For :338) and never at loop headers.
• So an enclosing statement whose only yield sat in a nested loop header was emitted inline. The await/yield in loop condition or update position is evaluated once instead of per-iteration (async polling loops never re-await) #5933 header arms (linearize.rs:686 / :729 / :769) never ran on that loop.

While covering every header position, three more gaps turned up. All three fail even for top-level loops on the baseline:

1. For-init top-level yield (for (let t = yield x; …), for (yield x; …)): hoist_yields.rs:104 hoisted only the yields nested inside the init. It left a top-level one in the init, and no arm splits it there. After the await→yield rewrite, plain async functions have the same shape: for (let x = await p; …) produced an empty loop.
2. Yielding for-update with a continue inside try/finally: the header arm skips this shape on purpose (the update must run after the finally). The body arm then copied the update into update_state without splitting it (linearize.rs:956). If the body had no other yield, the loop fell through to the catch-all.
3. Async generator header-yield operand: await_async_generator_yield_operands (lower/yield_await.rs) runs before linearization and never looks inside loop headers. As a result, yield promise in a loop condition delivered the promise itself instead of its value.

Fix (crates/perry-transform/src/generator/)

• break_continue.rs: body_contains_yield also checks While / DoWhile conditions and For conditions and updates, using hoist_yields::expr_contains_yield, which does not descend into nested closures.
• hoist_yields.rs: a for-init's let-initializer or expression-statement is hoisted fully, including a top-level yield, ahead of the loop. The init runs once, so hoisting it is exact.
• linearize.rs:
  • The header arms now go through a new normalize_loop_header_stmts. It hoists yields, and in an async generator it also awaits each yield operand. The await is added only to the statements built from the header; loop bodies were already processed.
  • The For body arm now also fires when the update the header arm left in place contains a yield. That update is linearized starting at update_state, which is still the continue target.
  • With no update yield, state numbering and emission order are unchanged.

Tests

• test-files/test_gap_10419_generator_loop_header_yield.ts matches Node 26.5.1 byte-for-byte. It covers:

  • the issue repro
  • 5 header positions (while-cond, do-while-cond, for-cond, for-update, for-init) × 12 containers (top-level control, if, else, try, catch, finally, switch, labeled block, labeled loop, if + labeled loop, nested loop, if > try > switch)
  • a zero-iteration / untaken-branch control
  • a minified lru-cache-style private generator class
  • sent values that terminate while / do…while / for-update / for-init loops
  • return() / throw() while suspended in a header yield (inner and outer catch)
  • break in a switch
  • continue outer with a yielding update
  • continue inside try/finally with a yielding update
  • nested header yields and per-iteration closures
  • async generators with for await (cond in if, while in try, do-while in switch, update in label, promise operand, sent values, return() mid-loop)
  • a plain async function's for (let x = await p; …)
  • controls that already worked

  Every loop calls a guard that throws after 1000 steps, and every consumer caps its pulls, so a regression prints a wrong line instead of hanging.

  • Baseline 7661bc0: 83 lines differ, and the harness reports Parity Fail: 1.
  • This branch: identical output and Parity Pass: 1, with and without PERRY_NO_AUTO_OPTIMIZE=1.

• crates/perry-transform/src/generator/loop_header_yield_tests.rs (5 unit tests):

  • detection at depth, with a negative control for a yield inside a closure
  • no residual Expr::Yield after the full transform for 7 nested shapes × sync/async, with a check that each fixture starts with a yield
  • the for-init hoist
  • the async operand await happens before the yield

  Against the baseline transform sources, 4 of 5 fail; the closure negative control passes either way, as it should.

Validation (perrybuilder, Linux x64)

check	result
cargo test --release -p perry-transform --tests	142 passed, 0 failed (includes the 5 new tests)
cargo test --release -p perry --test issue_5933_loop_condition_await --test issue_5868_switch_state_machine --test issue_5975_labeled_continue_in_yielding_switch --test issue_6709_async_generator_pending_await --test for_await_continue --test for_await_block_scope_leak --test issue_6067_generator_local_string_index_oob (separate target dir)	35 passed, 0 failed across the 7 suites
./scripts/run_lint_gates.sh	82 of 83 gates pass, including the full compile tier (-D warnings check, clippy, API-docs regen and drift). The one red gate, benchmarks/ci_public_baseline_check.py ("public artifact benchmark inputs changed"), fails the same way on a pristine 7661bc0 tree, so it predates this change.
gap suite (PERRY_SKIP_BUILD=1 ./scripts/run_gap_tests.sh)	GAP_EXIT=0, "Gap snapshot OK — 820 tests match". 814 pass, 6 fail (2159, 2514, json_lazy_defineproperty_index, perfhooks_3088…, prop_plan_cache_invalidation, v8_2_3680plus); the baseline 7661bc0 run fails the same 6. No new failures. The new test passes.
python3 scripts/check_test_registration.py	OK

Performance (perf stat -e instructions:u, 3 runs each, median; Node 26.5.1 wall median for context)

workload	baseline	this PR	Δ	Node wall
generator-heavy loop, yields in bodies only (range / while / if generators, 6M steps)	29,547,584,109	29,548,120,360	+0.002%	173 ms
header yields at top level (for / while cond, 4M steps)	20,410,835,724	20,410,926,584	+0.000%	196 ms
benchmarks/tls-budget/asyncpipe.ts (async functions use the same linearizer)	8,306,599,496	8,306,140,732	−0.006%	282 ms
issue repro scaled (header yields nested in if / try, 4M steps)	1,378,696,829 (yields nothing)	21,443,822,676	n/a	198 ms

Emitted LLVM IR (--trace llvm) for the first three workloads is byte-identical between the two compilers, except for the embedded entry-file path. The fix changes only which statements get linearized. Generators without header yields compile to the same code. A nested header-yield loop now costs about the same as the top-level form: 21.44B vs 20.41B instructions, with the extra ~5% coming from the enclosing try/if states.

In absolute terms, Perry generators are far slower than Node: about 4,900 instructions per generator step, or 3–4.5 s CPU on the first workload against Node's 173 ms wall. That gap was there before this change, including for top-level and body-only yields, and this PR neither causes nor fixes it.

Package check

lru-cache 11.5.2, default (minified) entry, compiled through perry.compilePackages with the audit script (keys / values / entries / rkeys / forEach / for…of / dump / clear() disposal / maxSize / subclass): 14 diff lines vs Node on the baseline, 0 on this branch.

Not verified / not addressed

• Throughput of lru-cache iteration compiled from source is still far from Node. 300 passes of keys() / values() / forEach over 10k entries take ~26 s CPU (349B instructions) against ~0.4 s in Node. The baseline already spent 193B instructions while yielding nothing. The profile is dominated by private-member access (js_object_get_own_field_or_undef + memcmp, ic_miss::private_instance_element_is_present, and a format! of the private storage name on every access), not by the generator transform. A plain method doing 3M #field / #method accesses costs 57.5B instructions, against 83 ms in Node. I have not filed that separately.
• macOS was not tested.

Fixes #10419

Summary by CodeRabbit

• Bug Fixes

  • Fixed generator suspension for yield expressions in while, do…while, and for loop headers, including initializers, conditions, and updates.
  • Improved async generator handling so yield operands are awaited correctly.
  • Preserved correct behavior across nested control flow, exception handling, break/continue, and iterator cleanup.

• Tests

  • Added comprehensive regression coverage for synchronous and asynchronous generators, including nested loops and control-flow scenarios.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 755d0810-7c0d-4d1f-adef-2f863f8976be

📥 Commits

Reviewing files that changed from the base of the PR and between 5030e6e and 9f9d799.

📒 Files selected for processing (7)

• changelog.d/10537-generator-loop-header-yield.md
• crates/perry-transform/src/generator/break_continue.rs
• crates/perry-transform/src/generator/hoist_yields.rs
• crates/perry-transform/src/generator/linearize.rs
• crates/perry-transform/src/generator/loop_header_yield_tests.rs
• crates/perry-transform/src/generator/mod.rs
• test-files/test_gap_10419_generator_loop_header_yield.ts

Included review availability: Your plan provides up to 8 included reviews per hour; 3 remain after this review.

---

📝 Walkthrough

Walkthrough

Generator transformation now detects yields in loop headers at nested depths, hoists for initializer yields, and creates suspension states for yielding conditions and updates. Async generator operands are awaited. Unit and integration tests cover control flow, sent values, exceptions, closures, and async behavior.

Changes

Generator loop-header yields

Layer / File(s)	Summary
Header detection and initializer hoisting crates/perry-transform/src/generator/break_continue.rs, crates/perry-transform/src/generator/hoist_yields.rs	body_contains_yield now checks loop conditions and updates. Top-level for initializer yields are fully hoisted.
Loop-header state linearization crates/perry-transform/src/generator/linearize.rs	Loop headers use shared normalization. Yielding for updates receive suspension states and return to the loop condition. Async generator operands are awaited.
Regression and integration coverage crates/perry-transform/src/generator/loop_header_yield_tests.rs, crates/perry-transform/src/generator/mod.rs, test-files/test_gap_10419_generator_loop_header_yield.ts, changelog.d/10537-generator-loop-header-yield.md	Tests cover nested containers, initializer hoisting, async awaiting, sent values, control flow, exceptions, closures, lru-cache iteration, and async generators. The changelog records the fix.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Bug fix · Severity of issue fixed: Medium

Sequence Diagram(s)

sequenceDiagram
  participant GeneratorBody
  participant body_contains_yield
  participant linearize_body
  participant GeneratorState
  GeneratorBody->>body_contains_yield: inspect loop condition or update
  body_contains_yield-->>linearize_body: identify a yielding loop
  linearize_body->>GeneratorState: normalize header and emit suspension state
  GeneratorState-->>GeneratorBody: resume at header continuation

Loading

Merge Risk: ⚪ Minimal · up to 9f9d7

The loop-header suspension changes have broad regression coverage and no identified merge-blocking issue.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 52.17% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 23 functions across 6 files. (1 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely identifies the main change: suspending on yields in nested generator loop headers.
Description check	✅ Passed	The description is detailed and covers the change summary, root cause, implementation, related issue, tests, validation results, performance, package verification, and known limitations. It does not r…
Linked Issues check	✅ Passed	Issue #10419 requires nested loop-header yields to suspend and resume correctly. The change updates body_contains_yield to inspect while/do…while conditions and for conditions/updates, so encl…
Out of Scope Changes check	✅ Passed	The production changes implement issue #10419's loop-header suspension behavior. The unit tests, regression test, changelog entry, async-generator handling, and plain-async handling support or documen…

Full details: Docstring Coverage

Explanation

Docstring coverage is 52.17% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 23 functions across 6 files. (1 skipped: 1 unsupported.)

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[proggeramlug]

Landed via merge train #10559 (v0.5.1592). All source commits preserve authorship; merged main matches the validated train exactly.