Conversation
Queuing an entire publication burst before writing could close subscribers at the 64 KiB cap while their sockets still had room. ChunkedResponse::deliver attempts to drain output after each accepted push and returns the required readiness interests. Its drain_into method also serves the readiness handler, preserving FIFO order and send-deadline accounting. The beacon API applies registration changes and closes connections on cap, write or registration errors. When a write drains an existing backlog, restore READABLE alone. This also handles publication before the response head's writable event. Subscribers whose buffers were already empty keep their registration. The cap remains 64 KiB and is checked before each push. Keep PENDING_MAX private to httpcore. Test delivery through impl Write with a scripted writer, so burst survival does not depend on kernel buffer capacity or reader scheduling. The tests cover a 40-frame burst exceeding the cap, ordered output, and returned interests after draining the head or a blocked write. They also exercise partial writes, interruptions, stall-clock updates, cap errors, zero-length writes and write errors. The beacon API checks delivery and ordering over a Unix socket using a burst that fits entirely in the application buffer. Two Linux-only integration tests check epoll registration after draining an idle subscriber's frame or its unsent response head. These retain their Linux readiness assumptions without requiring capacity for a large socket burst. Formatting and all-feature Clippy passed. All 108 HTTP-core, 141 beacon API and 14 application-boundary tests passed. The workspace suite's only failure was finalized_state_loads rejecting an expired checkpoint. Assisted-by: Claude:claude-fable-5-1 Assisted-by: Codex:GPT-6
One block's 128 data_column_sidecar events are expected to total about 290 KiB when each carries 21 Fulu commitments. Eager writes can reduce pending output, but the 64 KiB cap cannot hold that burst if the socket accepts no bytes. Raise the cap to 512 KiB to accommodate the burst without relying on kernel send-buffer capacity. Earlier pending events or repeated publications can still exhaust the allowance. At 64 subscriptions, the pending-output allowance grows from 4 MiB to 32 MiB. Buffers retain their allocations until the subscriptions close, including after a full drain. Add a scripted-writer test that accepts all 128 representative frames while every write returns WouldBlock. The response head also counts against the cap. Derive the eager-write test's burst length from the cap so queuing without writing would still exceed it after this increase. Expand the real-socket delivery test to 128 frames and check their order. Add a small publication test that checks for no pending bytes before pumping, independently of whether the larger burst fits the buffer. Document the allowance and allocation lifetime in ADR-0004. Assisted-by: Claude:claude-fable-5-1 Assisted-by: Codex:GPT-6
vladimir-ea
reviewed
Sep 14, 2026
| let backlog = !self.pending_write().is_empty(); | ||
| if !self.push(chunk, now) { | ||
| return Err(Closed::AtCap { pending: self.pending_write().len() }); | ||
| } |
Collaborator
There was a problem hiding this comment.
wouldn't it make sense to drain pending (and free space) before trying to push to it?
Bronek
added this pull request to stack #257
September 14, 2026 16:36
vladimir-ea
reviewed
Sep 14, 2026
| Err(Closed::AtCap { pending }) => { | ||
| tracing::warn!( | ||
| "beacon api subscriber would exceed send cap with {pending} bytes already pending, closing" | ||
| ); |
Collaborator
There was a problem hiding this comment.
is this how other beacon clients behave? the alternative is to just drop the current message and carry on - if other clients happen to all behave like that we should consider having the same behaviour
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
/eth/v1/eventssubscriptions queued frames untilhandle_event. A publication burst could exceed the 64 KiB send cap infan_outand disconnect subscribers while their sockets still had room.This PR attempts socket writes as frames are published inside
fan_outand raises the cap to 512 KiB. With no earlier event backlog, that accommodates one block’s 128 column events carrying 21 Fulukzg_commitmentseach (about 290 KiB) even when the socket accepts no bytes. It prepares the transport for the follow-up PR adding those commitments.The changes are split into two commits: eager writes, then the cap increase. At 64 subscriptions, the pending-output allowance grows from 4 MiB to 32 MiB. ADR-0004 records the sizing decision.