feat(proto): add batch send/recv APIs by ifdario · Pull Request #735 · n0-computer/noq

ifdario · 2026-07-01T09:24:46Z

Description

Adds functions to send and receive batches of datagrams. Particularly the
locking overhead in noq stands out and is reduced. But also adds batch
APIs into noq-proto and the driver needs to be woken fewer times.

API Changes

New APIs:

noq_proto
- Datagrams::send_many
- Datagrams::recv_many
noq
- Connection::read_many_datagrams
- Connection::send_many_datagrams

Notes & open questions

n/a

Change checklist

Self-review.
Documentation updates following the style guide, if relevant.
Tests if relevant.
All breaking changes documented. (none)

Add Connection::send_datagrams / read_datagrams and the underlying DatagramState::recv_many / Datagrams::send_many to amortize the per-datagram connection-mutex cost, which is the dominant overhead at high pps for a mesh VPN forwarding bursts of TUN packets. - DatagramState::recv_many drains all buffered incoming datagrams in one VecDeque drain instead of N pop_fronts. - Datagrams::send_many queues a whole batch under one logical operation: peer-support check once, drop-oldest backpressure pass once, oversized datagrams skipped individually (not aborting the batch). - Connection::send_datagrams takes the mutex once, wakes the driver once. - Connection::read_datagrams is a ReadDatagrams future that parks on the datagram_received notify then drains everything in one lock hold. - New SendMany { queued, rejected } result struct. - 2 new unit tests: batch send/recv roundtrip, oversized-skip behavior. - bench: datagram benchmark harness (BenchOpt/DatagramOpt, DatagramCounters). Baseline (loopback, release, 2-core): 440 MiB/s @ 1200B, 4.17M pps @ 64B.

flub

Probably not an unreasonable request, left some initial notes from a quick skim. I haven't looked at tests or benchmarks at all yet.

flub · 2026-07-01T14:03:09Z

Please run cargo make locally before pushing so that the obvious CI failures are avoided. And please follow up on fixing CI failures of course :)

Resolves flub's PR n0-computer#735 review: - Fix send_many over-allocation: evict oldest per-element before each push (not push-all-then-evict), so the outgoing deque never transiently holds the whole batch. A datagram larger than the budget but under max_size is still enqueued once the queue is emptied, matching send(drop=true). Locked in by datagram_batch_send_drop_oldest_per_element. - send_many now stops at the first oversized datagram and returns Ok(count) accepted from the front (mirrors write_chunks / sendmmsg "first K succeeded"). No longer returns Err(TooLarge); callers recover via &datagrams[count..]. Empty batch returns Ok(0) (datagram_batch_send_empty_is_ok). - Rename to the chunks-style API: send_many_datagrams / read_many_datagrams, returning ReadManyDatagrams. Drop the SendMany struct. - Take slices for consistency with read_many_chunks / write_chunks: send_many_datagrams(&[Bytes]) -> Result<usize, _>; read_many_datagrams(&mut [Bytes]) (fill-slots, returns count drained). recv_many return-count bug fixed (now reports slots filled, not bool). - Extract over_send_budget / would_exceed_send_budget helpers, deduped across send and send_many. - CI: fmt (project unstable config), clippy, rustdoc, all tests green.

- send_many takes a `drop` flag, mirroring send. Reject the whole batch with TooLarge up front if any datagram is oversized, so a size error is never a partial send. With drop = false, stop when the buffer is full and return the count queued. - Share the drop-oldest loop between send and send_many. - Rename the send-buffer helpers to is_send_buffer_exceeded and has_send_buffer_capacity, and drain in recv_many instead of popping. - Focus the docs and drop the lock/implementation details.

flub · 2026-07-14T17:57:39Z

Please use the (admittedly tiny, well hidden) re-request review button whenever the PR is ready for another round of review. It is next each reviewer's name at the top right.

flub · 2026-07-16T11:54:21Z

+
+/// Direction of the datagram flood.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum Direction {


Have you found the direction to be important? In theory the direction could also be done for the stream bench, but for some reason it is not. Which is what makes me wonder if this reveals anything interesting.

Am I crazy or my comments where deleted?

The future no longer forces the caller's out slice borrow to match the connection borrow.

An unpaced flood queues datagrams far faster than the driver transmits, so drop-oldest evicts almost the whole run locally and the benchmark measures queueing speed instead of throughput. Wait for send buffer space before each send so every queued datagram reaches the wire and loss reflects the network.

…rker Derive derive_more::Display on Direction, SendMode, and Congestion so the JSON report prints them directly instead of hand-mapping strings. Replace the uni done-stream and the drain grace timeout with an in-band 1-byte DONE datagram: the sender floods, then resends the marker every 250ms until the receiver closes; the receiver counts until it sees the marker. The duplex coordination stream keeps only the S/F handshake, where F now means the peer's DONE was received.

flub

basically LGTM I think, but would be nice to clean up the last nit.

Thanks for you patience!

# Conflicts: # bench/src/lib.rs # noq-proto/src/connection/datagrams.rs

The flood sized its payload once from max_datagram_size() and kept it for the whole run. MTU discovery raises the estimate within a few ms of connecting and black hole detection drops it back to the minimum MTU, so a flood that sampled the raised value aborted with TooLarge as soon as the estimate came down. Duplex runs hit this most often, since the loss they generate is what triggers black hole detection. Sends now re-clamp to the current limit and carry on instead of failing.

…f/datagram-batching

Take inbound packets from a peer connection with read_many_datagrams, so a burst is drained under one lock hold instead of a wake per packet, and hand the lazy-dial backlog to send_many_datagrams in per-route runs. The batch APIs come from noq via the rayfish forks (n0-computer/noq#735, n0-computer/iroh#4383), so ray patches noq alongside iroh and netwatch. The drop-newest policy is kept by tracking the bytes already staged in a run against the connection's send buffer space.

flub · 2026-09-02T16:50:57Z

Could you apply the last changes I suggest in a new commit please? I don't seem to be able to commit those myself as I was hoping. And keep it to just those so I don't have to worry about any other changes to check again. :)

flub · 2026-09-02T16:53:11Z

well, you can merge main again too, to fix the cargo deny and perf test failures as well.

…f/datagram-batching

ifdario mentioned this pull request Jul 1, 2026

feat(iroh): expose batch datagram send/recv APIs n0-computer/iroh#4383

Open

4 tasks

n0bot Bot added this to iroh Jul 1, 2026

github-project-automation Bot moved this to 🚑 Needs Triage in iroh Jul 1, 2026

flub requested changes Jul 1, 2026

View reviewed changes

Comment thread noq-proto/src/connection/datagrams.rs Outdated

Comment thread noq-proto/src/connection/datagrams.rs Outdated

Comment thread noq/src/connection.rs Outdated

Comment thread noq/src/connection.rs Outdated

github-project-automation Bot moved this from 🚑 Needs Triage to 🏗 In progress in iroh Jul 1, 2026

ifdario commented Jul 2, 2026

View reviewed changes

Comment thread noq-proto/src/connection/datagrams.rs Outdated

ifdario requested a review from flub July 3, 2026 13:01

flub requested changes Jul 8, 2026

View reviewed changes

ifdario requested a review from flub July 11, 2026 09:19

flub requested changes Jul 13, 2026

View reviewed changes

Comment thread noq-proto/src/connection/datagrams.rs Outdated

Comment thread noq-proto/src/connection/datagrams.rs

Comment thread noq/src/connection.rs Outdated

flub requested changes Jul 13, 2026

View reviewed changes

Comment thread bench/src/bin/datagram.rs Outdated

Comment thread bench/src/bin/datagram.rs Outdated

perf(datagrams): address review, fix bench close races

671d2fd

docs: tighten batch API docs, style cleanups

8ceeebd

ifdario requested a review from flub July 15, 2026 08:27

flub requested changes Jul 16, 2026

View reviewed changes

ifdario added 2 commits July 16, 2026 16:13

refactor(noq): decouple the out buffer lifetime in ReadManyDatagrams

076c324

The future no longer forces the caller's out slice borrow to match the connection borrow.

ifdario requested a review from flub July 16, 2026 15:28

flub requested changes Jul 28, 2026

View reviewed changes

Comment thread bench/src/lib.rs Outdated

ifdario added 4 commits August 11, 2026 12:01

Merge remote-tracking branch 'upstream/main' into perf/datagram-batching

1edde75

# Conflicts: # bench/src/lib.rs # noq-proto/src/connection/datagrams.rs

refactor(bench): shorten Direction Display with rename_all

6b2d4f3

refactor(bench): shorten SendMode and Congestion Display with rename_all

22db9b8

ifdario requested a review from flub August 15, 2026 17:08

ifdario and others added 5 commits September 1, 2026 14:27

Merge branch 'main' into perf/datagram-batching

f3a2fb6

Merge commit 'refs/tmp/nq-main' into perf/datagram-batching

a517e35

style: rewrap batch datagram doc comments to comment_width

fc6c0c3

Merge remote-tracking branch 'origin/perf/datagram-batching' into per…

b2261e3

…f/datagram-batching

refactor(datagrams): focus batch API change

2650940

flub reviewed Sep 2, 2026

View reviewed changes

Comment thread noq-proto/src/connection/datagrams.rs Outdated

Comment thread noq-proto/src/connection/datagrams.rs Outdated

Comment thread noq-proto/src/connection/datagrams.rs Outdated

ifdario and others added 3 commits September 2, 2026 23:03

Merge branch 'main' into perf/datagram-batching

3171b23

docs(datagrams): apply review wording and link style

cfc697e

Merge remote-tracking branch 'origin/perf/datagram-batching' into per…

ebba1b8

…f/datagram-batching

flub changed the title ~~perf(datagrams): add batch send/recv APIs (N2+N4)~~ perf(datagrams): add batch send/recv APIs Sep 3, 2026

flub enabled auto-merge September 3, 2026 09:22

flub disabled auto-merge September 3, 2026 09:23

flub changed the title ~~perf(datagrams): add batch send/recv APIs~~ feat(proto): add batch send/recv APIs Sep 3, 2026

flub approved these changes Sep 3, 2026

View reviewed changes

flub added this pull request to the merge queue Sep 3, 2026

github-merge-queue Bot removed this pull request from the merge queue due to no response for status checks Sep 3, 2026

flub added this pull request to the merge queue Sep 3, 2026

Merged via the queue into n0-computer:main with commit bde8f94 Sep 3, 2026
63 of 64 checks passed

github-project-automation Bot moved this from 🏗 In progress to ✅ Done in iroh Sep 3, 2026

ifdario deleted the perf/datagram-batching branch September 4, 2026 22:32

Conversation

ifdario commented Jul 1, 2026 • edited by flub Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

API Changes

Notes & open questions

Change checklist

Uh oh!

flub left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

flub commented Jul 1, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

flub commented Jul 14, 2026

Uh oh!

Uh oh!

flub Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

ifdario Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

flub left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

flub commented Sep 2, 2026

Uh oh!

flub commented Sep 2, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ifdario commented Jul 1, 2026 •

edited by flub

Loading