Skip to content

Block-scoped handle, callable handler values, declared operation modes and interned effect dispatch - #239

Draft
MelbourneDeveloper wants to merge 9 commits into
mainfrom
codex/handler-values
Draft

MelbourneDeveloper wants to merge 9 commits into
mainfrom
codex/handler-values

Conversation

@MelbourneDeveloper

@MelbourneDeveloper MelbourneDeveloper commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

TL;DR

Replaces the handle E … in body handler surface with a block-scoped handle E { … } statement and a callable handler E { … } value, adds declared operation modes (control) and handler return clauses, and moves runtime dispatch from string lookup to interned operation ids with O(1) evidence slots.

Details

Surface (both flavors, no compatibility aliases). handle E { arms } is a statement that governs the rest of its block — the statements and final expression that follow it — so a handled region needs no in/do and no extra nesting; ML writes the same thing with indented arms. handler E { arms } is the handler itself: a value with no region attached, which can be bound, captured, passed and called with a zero-argument computation, so one handler serves many regions. Both forms accept a return p => body clause that transforms the region's completion value. The in and do applications are deleted outright, as is file-scope handle, and a handle with nothing after it is an error. tree-sitter-osprey/grammar.js gains handler_value_expression, handler_return and operation_mode; crates/osprey-syntax/src/default/handle_rest.rs is new, and the ML parser and both lowerers follow.

Declared operation modes. An operation is now declared control or left undecorated. A control arm receives the performer's continuation, returns the handler's answer and may resume; an undecorated operation is a value operation whose arm supplies the operation's result and owns no continuation, so resume in one is refused by name. Mode is part of the operation's interface rather than inferred from an arm body, so substituting one handler for another cannot change it. abort/once/many and replayable now attach to a control operation, and a multiplicity on a value operation is rejected (crates/osprey-ast/src/multiplicity.rs, crates/osprey-types/src/expr.rs).

Runtime dispatch. Codegen interns every (effect instantiation, operation) pair a program can perform into a dense i32 and refuses a program needing more than OSP_MAX_OPERATION_IDS (4096); the thread-local handler stack keeps one evidence slot per id, so a perform is one array read instead of a string-keyed search. Arm activation is now scoped — __osprey_handler_push_scoped, __osprey_handler_suspend_scope, __osprey_handler_restore_scope — so suspending an arm removes its activation and the scopes it nests inside, and restores them in order (compiler/runtime/effects_runtime.c/.h).

Effect identity and static selection. An effect's identity is its base name plus resolved arguments, so Stash<int> and Stash name one effect everywhere: the operation table, requirement rows and handler lookup all key on it. handle static E { … } is discharged during compilation; lower_static is split into bindings.rs, hygiene.rs and requirements.rs, with lexical resolution run before static terms move between scopes.

Specs and corpus. 0017-AlgebraicEffects and 0035-StagedEffects are substantially rewritten (plus 0002, 0003, 0013, 0020, 0022, 0023, 0024, 0025, 0034, 0036, 0037), and the handle-form snippets in docs, website, blog posts, playground and VS Code snippets are migrated. examples/failscompilation gains handle_with_nothing_to_handle (and its ML twin) and generic_effect_instantiation_cannot_cross, and drops the cases whose forms no longer exist.

CI repairs in the final commit. A dead OperationMode::as_str (hawk); a reindented multi-line string literal in examples/wasm/studio.osp that made the program print four leading spaces per :: marker and failed the wasm golden (website E2E); three installed-VSIX fixtures still written in the deleted handle … in form; and osprey-ast coverage below its 97% floor. lower_static also carried a second copy of the uncovered-operation and arm-arity diagnostics that stage::validate and the type checker already emit — unreachable under [STAGE-LOWER-ORDER-PHASE] and one defect with two wordings — so those duplicates are gone.

Checking got faster, not slower. base_env rebuilt every built-in from scratch on each call, and builtin_signature calls it once per call node inside the effect-row fixpoint — which redundant_annotations re-runs per candidate annotation. This branch's runtime_group added ten bound-name set constructions to each rebuild, taking one 1357-line program from 7.0s to 13.5s and blowing the language server's 60s hover budget on a two-core runner. The environment is a pure function of the compiler binary, so it is now built once behind a OnceLock; read-only callers borrow it. The same file checks in 3.2s, and the CHUNKY: hover + symbols work on the actual list corpus file test runs in 14.1s against 55.5s before.

How Do The Automated Tests Prove It Works?

The surface is pinned, including what it rejects. crates/osprey-syntax/tests/handler_syntax.rs: a_handler_over_the_rest_of_a_block_is_accepted_in_both_flavors, in_and_do_handler_forms_are_rejected_in_both_flavors, both_flavors_build_the_same_handler_value, a_handler_that_names_no_body_and_handles_nothing_is_rejected, a_file_scope_handler_is_rejected_by_name_in_both_flavors, ml_resume_takes_one_argument_not_the_rest_of_the_expression. handler_contract.rs adds removed_handler_applications_are_syntax_errors, value_arms_have_no_continuation_even_inside_a_control_arm and block_scoped_and_callable_handlers_have_one_contract — the last asserting the two forms answer identically, which is the claim that makes handler values a refactor and not a second semantics.

Modes, returns and values are executed, not just parsed. Every test in handler_values.rs, handler_returns.rs and operation_modes.rs compiles and runs its program under all three memory backends and compares stdout byte-for-byte, with the ARC runs asserting a zero live-object exit. They cover captured and isolated handler state (returned_handler_closures_preserve_shared_and_isolated_state), completion transformation on both value and control answers (return_clauses_transform_completion_and_resume_but_not_control_answers, repeated_deep_resume_and_value_dispatch_transform_completion_once), ownership transfer of managed answers (managed_body_answers_transfer_ownership_to_return_clauses) and forwarding to an outer activation (value_and_control_arms_forward_to_the_outer_activation, return_requests_forward_outside_their_own_activation).

Static discharge keeps its meaning. static_selection.rs (10 programs) holds discharge through helpers, callbacks, alias chains, closures and kernels, and rejects what must not be selected statically (mixed_and_control_effects_cannot_be_selected_statically, unused_static_arms_reject_transitive_dynamic_requests). staged_hygiene.rs (12 programs) proves the rewrite never captures or renames the wrong binding — names_resolve_through_namespace_and_module_scopes, record_union_and_list_binders_survive_discharge, result_payload_binders_survive_discharge, captured_mutable_state_is_shared_with_specialized_helpers.

The scoped handler stack is tested in C. compiler/runtime/effects_scope_tests.h adds t_scope_forwarding, t_scope_nested_forwarding, t_scope_snapshot_preserves_activation, t_scope_coro_resume_and_abandon and two death tests — death_scope_missing_handler, death_scope_unbalanced_restore — so an unbalanced restore aborts instead of silently dispatching to the wrong arm.

Whole-language regression. The differential corpus runs 213/213 with 213/213 byte-exact goldens under default, gc and arc, and 147/147 under wasm32 via Node's WASI, with every ML twin held to its Default twin's golden. examples/failscompilation is checked against recorded diagnostics, so each removed form and each new refusal has a pinned message. Coverage gates pass with osprey-ast at 97.4% (floor 97%), and the C runtime libraries all meet their thresholds.

Everything CI runs was run locally on this branch: fmt, clippy -D warnings, hawk (0 findings), deslop (4.4% ≤ 5%), cargo test under llvm-cov with every crate threshold, the four corpus runs above, the C runtime suites, the full VS Code extension suite (324 + 32 passing), the website Playwright suite (125/125), the wasm examples under Node and the browser shim, bank/profiler/mobile-tools/docs-HTML acceptance, bank e2e (17/17), the Docker web compiler, the iOS device and simulator suites, and the Android gate on an emulator (132/132 device goldens) with lintDebug.

@MelbourneDeveloper MelbourneDeveloper changed the title Replace inferred effect behavior with declared control and scoped handlers Replace effect modes and implement scoped handler answer transformations Sep 17, 2026
@MelbourneDeveloper MelbourneDeveloper changed the title Replace effect modes and implement scoped handler answer transformations Replace algebraic effect handlers and delete legacy forms Sep 17, 2026
MelbourneDeveloper and others added 4 commits September 19, 2026 07:33
…(1) evidence slots

Codegen interns every (effect instantiation, operation) pair to a dense id;
the runtime keeps one evidence slot per id, so a perform is an array index
instead of a locked linear strcmp scan. The handler stack is thread-local, so
the mutex protected nothing and is gone; wasm32 no longer needs a pthread shim.
…tity, resume and block-lowering defects

Delete every remaining in/do handler consumer: the corpus, examples,
website playground and blog snippets are rewritten in the block-statement
and handler-value forms, with byte-exact goldens under default, gc, arc
and wasm32.

Compiler defects the migration exposed, each pinned by a test:
- effect identity is the base name plus resolved arguments everywhere:
  Requirement::new, OperationTable lookups and static answers key on it,
  so handle Choice<int> reads Choice's declared modes
- structural staging rules (stage::validate) run before inference so a
  kernel or arity defect is named, not reported as an unhandled operation
- a kernel under an enclosing static handler filters statically answered
  operations from its dynamic residual row
- a many arm's own operation is not replayed external work
- resume outside an arm or in a lambda names the missing continuation
  truthfully; value arms name the control declaration that would own one
- written braces stay a block (a doc above { … } documents the block);
  only a handle region and ML layout unwrap a lone value

Specs, CLAUDE.md, plan 0016 and design notes now match the shipped
syntax and diagnostics.
CI on codex/handler-values was red on `Dead-code gate (hawk)` and
`Tests: Website E2E (Playwright)`, and two more gates failed only when
run locally. Each is fixed at its cause, with no gate weakened.

- hawk: `OperationMode::as_str` was added on this branch and is reachable
  from nothing. Deleted.

- Website E2E: the handler migration mechanically reindented the
  multi-line string literal in examples/wasm/studio.osp, so the program
  printed four leading spaces on every `::` marker line and the wasm
  smoke test failed against the shared golden. The literal is restored;
  studio.osp again matches studio.expectedoutput byte-for-byte in both
  flavors, natively and under WASI.

- VS Code extension: three installed-VSIX fixtures still wrote the
  deleted `handle E ... in body` form, so the LSP reported syntax errors
  where the tests expected unused-symbol diagnostics. The fixture
  programs are migrated to the rest-of-block form and `Pick.choose` is
  declared `control`, which `resume` now requires. No assertion changed.

- Coverage: osprey-ast fell to 95.7% against its 97% floor, because the
  branch added hygiene, bindings and requirements with nothing reaching
  them. Three staged-hygiene programs now cover namespace and module
  scopes, record/union/list binders and Result payload binders, and two
  rejection tests cover the partial and undeclared static region. The
  floor is unchanged.

`lower_static` also carried a second copy of two diagnostics: the
uncovered-operation message `stage::validate` already emits and the arm
arity message the type checker already emits. Both are unreachable under
[STAGE-LOWER-ORDER-PHASE] and gave one defect two wordings, so the
duplicates are deleted and an unrewritable request is left standing for
the residual row. Both failscompilation goldens are unchanged.

Verified locally: fmt, clippy (all targets, -D warnings), hawk, deslop,
cargo test with coverage and every crate threshold, the corpus at
213/213 under default, gc and arc and 147/147 under wasm32, the C
runtime suites and thresholds, the full VS Code extension suite, the
website Playwright suite, the wasm examples under Node and the browser
shim, bank, profiler, mobile tools, docs-HTML acceptance, bank e2e, the
Docker web compiler, the iOS device and simulator suites, and the
Android gate on an emulator (132/132) with lintDebug.
@MelbourneDeveloper MelbourneDeveloper changed the title Replace algebraic effect handlers and delete legacy forms Block-scoped handle, callable handler values, declared operation modes and interned effect dispatch Sep 19, 2026
CI's `Tests: VS Code extension (coverage)` failed on a 60s timeout in
`CHUNKY: hover + symbols work on the actual list corpus file`, and on
`deactivate stops the running language client without throwing` behind
it. Neither is a flake: checking that one 1357-line program took 13.5s
on this branch against 7.0s on main, and the language server pays that
per request, so a two-core runner cannot answer a hover inside the test
budget.

`base_env` rebuilt every built-in from scratch on each call, and
`builtin_signature` calls it once per call node inside the effect-row
fixpoint, which `redundant_annotations` re-runs per candidate
annotation. This branch made each rebuild dearer still: `runtime_group`
takes the whole bound-name set before and after each of five groups and
diffs them, so ten set constructions joined every rebuild.

The environment is a pure function of the compiler binary — no source
input, no fresh type variables, quantified binders spelled `Var(0)`..
`Var(2)` — so it is built once behind a `OnceLock`. `base_env` still
hands out the copy its callers extend with source bindings;
`builtin_signature` and `builtin_callback_type` only read a binding, so
they borrow the built environment instead of rebuilding and cloning it.

The same file now checks in 3.2s — faster than main — and the hover test
runs in 14.1s against the 55.5s it took before, which is what left it no
margin on a runner.

Verified: fmt, clippy, hawk, cargo test with coverage and every crate
threshold, the corpus at 213/213 under default, gc and arc and 147/147
under wasm32, and the full VS Code extension suite (324 + 32 passing).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant