Skip to content

fix(hu): stop charging host I/O waits to the guest's epoch budget - #319

Merged
YuanYuYuan merged 3 commits into
mainfrom
fix/hu-epoch-budget
Aug 21, 2026
Merged

YuanYuYuan merged 3 commits into
mainfrom
fix/hu-epoch-budget

Conversation

@YuanYuYuan

@YuanYuYuan YuanYuYuan commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Slice 3 of 5, splitting #311. #320 sits on it.

What this does

hu runs each plugin as a WASM component. This change stops the host from charging its own I/O waits to the plugin's compute budget.

What fails without this

wasmtime preempts a runaway guest with an epoch deadline. A background ticker increments the epoch every 100 ms. Each dispatch calls set_epoch_deadline(30). A guest therefore gets about 3 seconds of wall clock.

The ticker measures wall clock. It does not measure guest execution. A host call that blocks on the network spends the budget while the guest does not run. wasmtime then traps the guest as soon as the host call returns. The guest never runs the code that the host call gave it.

Four host calls block for seconds on a cold graph:

call site what it waits for
encode_yaml_to_cdr topic schema discovery — 2 s budget
service_call service schema discovery — 2 s budget
both replies.recv() the caller's own timeout

Two discoveries in one dispatch already exceed the budget.

The self-directed trap in the last step is the defect:

sequenceDiagram
    autonumber
    participant G as Guest (plugin)
    participant H as Host (hu)
    participant T as Epoch ticker
    participant N as Network

    Note over G,T: dispatch starts, set_epoch_deadline(30), about 3 s of wall clock
    G->>H: service_call
    activate H
    H->>N: query
    loop every 100 ms, for 5 s
        T->>T: increment_epoch()
    end
    Note over T: 50 ticks charged, so the budget of 30 is gone
    N-->>H: reply
    H-->>G: Ok(response)
    deactivate H
    G--xG: wasmtime traps the guest on return
    Note over G: the guest never runs the code the host just gave it
Loading

The guest is not at fault in that diagram. It never ran while its budget drained.

With the guard, the ticker stops for the length of the wait:

sequenceDiagram
    autonumber
    participant G as Guest (plugin)
    participant H as Host (hu)
    participant T as Epoch ticker
    participant N as Network

    Note over G,T: dispatch starts, set_epoch_deadline(30), about 3 s of wall clock
    G->>H: service_call
    activate H
    H->>H: HostBlockGuard::enter(), counter 0 to 1
    H->>N: query
    loop every 100 ms, for 5 s
        T->>T: epoch_should_tick() is false, so no increment
    end
    N-->>H: reply
    H->>H: guard drops, counter 1 to 0
    Note over T: ticking resumes and the budget still reads 30
    H-->>G: Ok(response)
    deactivate H
    G->>G: runs the response, or the error branch
Loading

HostBlockGuard suspends the ticker for the length of the wait. The guard counts its references, so nesting is safe. The test host_block_guard_suspends_the_epoch_ticker pins that behaviour. A later edit can delete one fetch_add and remove the guarantee. No other test finds that.

The guest must not switch the watchdog off

Two of the four waits use a fixed 2 s constant. The service reply waits use timeout-ms. The guest chooses that value. It crosses the WIT boundary as a u32.

An unclamped u32 allows about 49 days. A plugin can therefore choose how long the host stops preempting plugins. The watchdog exists to preempt a guest that does not yield. The guest must not control it.

clamp_guest_timeout caps the value at 60 s. The host writes a warning when it reduces the value. The plugin author then sees why the timeout changed.

Why the process-wide scope is acceptable

The host runs one engine and one ticker. The suspension therefore applies to every guest. It does not apply only to the guest that called the host.

Serialised dispatch makes this acceptable today:

path why only one guest runs at a time
CLI one plugin, sequential event loop
TUI iterates the plugins in order
hu web one mutex over the plugin vector, per request

No second guest exists to preempt. The serialisation bounds the cost. The length of the wait does not bound it. A reader can easily confuse these two reasons, and only the first one holds.

A later change can remove the serialisation. Per-plugin locks or a plugin pool would do so. The gap then becomes real. No code links that change to this consequence.

The per-store alternative. wasmtime supplies Store::epoch_deadline_callback. Host functions already hold &mut PluginState, which is the store data. A flag on PluginState and a callback that returns UpdateDeadline::Continue extend the deadline of the blocked guest alone. Every other guest stays preemptible. That design is better, and it also costs more work. This counter behaves the same way while dispatch stays serialised.

One artefact of the split

Three call-site comments say "See subscribe". subscribe takes the guard in #311, not here. The text comes from the branch that this slice was cut from. A change here would force #311 to change it back. The five slices would then no longer sum to that branch. HostBlockGuard documents the mechanism in full.

Breaking changes

None.

Every guest dispatch runs under a ~3 s wall-clock epoch deadline, and the
ticker measures wall clock. Time a host call spends waiting on the
network is therefore charged to the guest even though the guest is not
running, so a host call that blocks longer than the remaining budget
traps the guest the moment it returns.

Four host calls already block for seconds: schema discovery on the
publish path, service schema discovery, and both service reply waits.
`HostBlockGuard` suspends the ticker for the duration of the wait. It
counts its references and carries a unit test, because nothing else
would catch its removal.

The suspension is process-wide, so a different runaway guest is not
preempted while a blocking call is in flight. That window is bounded by
the host call's own timeout, and trapping a well-behaved guest for
waiting on the network is strictly worse.
The host holds a HostBlockGuard across the service reply wait, and that
wait is bounded by `timeout-ms`, which crosses the WIT boundary as a u32
the guest picks. Unclamped, a plugin could ask for ~49 days and so decide
how long the host stops preempting plugins -- including itself. The epoch
watchdog exists to preempt a guest that will not yield; it must not be a
guest-operated switch.

Cap it at 60 s, far above any real service call, and warn when the cap
actually bites so a plugin author is not left wondering.

Also corrects what the trade-off comment claims. The process-wide scope is
acceptable because dispatch is serialised -- one plugin at a time in the
CLI, the TUI and behind hu web's mutex -- not because the waits are short.
The comment now says that, names the per-store alternative
(Store::epoch_deadline_callback plus a flag on PluginState), and says what
would make the gap real.
@YuanYuYuan
YuanYuYuan force-pushed the fix/hu-epoch-budget branch from 1a6629d to aa68097 Compare August 21, 2026 12:17
…nglish

Applies the ASD-STE100 sentence rules to the comments this branch adds:
one idea per sentence, active voice with a named actor, present tense for
what the code does.

Measured on the changed comment lines: longest sentence 16 to 14 words,
passive constructions 5 to 1, sentences joining two independent clauses
1 to 0. The remaining passive, "The waits themselves are bounded", hides
no actor -- the next two sentences name every bound.

The three "See `subscribe`" call-site comments are deliberately untouched.
They carry the wording of the branch this slice was cut from, and changing
them here would break that correspondence.
@YuanYuYuan
YuanYuYuan merged commit 9b10746 into main Aug 21, 2026
30 checks passed
@YuanYuYuan
YuanYuYuan deleted the fix/hu-epoch-budget branch August 21, 2026 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant