perf(proto): investigate opt-in first-flight controls for large handshakes

[domenkozar]

ML-DSA-65 raw-public-key authentication produces much larger TLS flights than Ed25519 in n0-computer/iroh#4534. At 100 ms RTT without loss, mutual authentication takes about 288 ms versus 152 ms in untraced runs.

Full trace-level log: both endpoints, Ed25519 followed by ML-DSA-65. This log uses unmodified noq/noq-proto 1.3.0; tracing adds a little overhead. Source and reproduction commands.

In the PQ handshake:

• The server queues 5,556 Handshake CRYPTO bytes, versus 392 for Ed25519 (line 293). Its first packets are paced about 27 ms apart, then it blocks on the amplification limit (line 348). The returning client ACK allows it to finish the flight.
• The client subsequently queues 5,356 Handshake CRYPTO bytes, versus 181, spread across another ~28 ms (line 433).

An external prototype tests two opt-in changes: one additional 1,200-byte padded client Initial to supply server send allowance, and finite extra Handshake pacing credits. Amplification, congestion and explicit send-rate checks remain in place. Together these reduce the clean-link PQ median to about 152 ms, but short queues can make bursts slower than the control.

Would configurable Initial-flight size and bounded Handshake pacing credits fit noq, or would you approach this differently? Defaults would stay unchanged. Prototype/test diff and comparison results are available for review; real-network follow-up timings are exploratory because other host work overlapped.

Related: #66, #507. Investigation and preparation used AI assistance.

[flub]

This issue is written rather confusingly. IIUC when using an ML-DSA-65 signature algorithm for the raw_public_key authentication the QUIC handshake takes much longer to complete. And this is attributed to pacing and anti-amplification limits?

And IIUC the two solutions you are proposing are:

• Send more unsolicited data by the client so the server in the Initial has a higher anti-amplification budget.
• Make sure the pacer has larger window so the handshake is not subject to pacing.

Could you please share the trace-level logs of such a handshake? The included table and descriptions are very vague.

Please do try and write issues shorter and to the point. LLM output is not suitable. I'd appreciate if you could edit down the issue to be shorter and clearer. Think about the reader.

[domenkozar]

Yes, that’s correct. The larger ML-DSA-65 handshake is delayed by pacing and the server’s anti-amplification limit. The prototype sends an extra padded client Initial and gives Handshake packets a bounded pacing allowance, while preserving amplification and congestion checks.

I’ve shortened the issue body and added the full trace-level log. The PQ handshake starts at line 279, and the amplification block is at line 348.

[flub]

Thanks for the logs. Adding a client and server span to these makes them even nicer to read, I can recommend that.

For the anti-amplification limit the generic answer is QUIC is to use address validation tokens. But of course that only works on successive connections. The trick to give more budget by sending another Initial datagram right away is nice. Though we'd not want to do this unnecessarily, so we'd have to look at whether we can figure out what size we expect the server to need. Ideally not leaving this to configuration but rather from the allowed algorithms. I haven't looked yet at how realistic this is.

However if we decide to do this we should not got for a simple padded datagram. If we are sending some redundant packet anyway we may as well let it carry the same CRYPTO bytes as the first Initial packet. That way if the first packet is lost the 2nd will still be useful and start the handshake instead of having to wait for the tail-loss probe to send it again.

For the pacing I'd have to look at how the congestion controller and pacing both work and interact in more detail. Depending on what the side effects are it might make sense to tweak the existing parameters so that a typical PQC handshake can go out as one burst. However this could backfire: the pacing is supposed to help make it more likely all the packets are delivered rather than get lost on the network or cause buffer-bloat. So it might well be that on average there really isn't anything to be gained here. If you have a network where you expect a certain amount of bandwidth that is higher or have less packet loss I kind of expect the existing initial parameters to be sufficient to tweak this. Have a look at TransportConfig::initial_rtt and related configs to see if you can tune things to not pace already using the existing settings.

[domenkozar]

Follow-up: the prototype now repeats the first Initial’s CRYPTO bytes under normal pacing, with tests covering loss of either packet, reordering, and pacing delay.

I also corrected the benchmark: the old netem limit included packets held for propagation. Transmission queueing and propagation are now separate.

Results are mixed. On the simulated 1 Mbit/s link with four queue slots, repetition improved PQ median latency from 300 to 226 ms. On a capped 250 kbit/s link with one slot, it worsened latency from 495 to 593 ms; competing traffic also favored disabling repetition.

A final PQ-only test with repetition disabled, fixed arrivals, and a rate-limited competing flow found a modest benefit from the existing cap: median 964 → 923 ms, observed p95 1462 → 1275 ms. All 60 handshakes and data checks passed, but these shared-host measurements don’t justify changing defaults.

I’d retain normal pacing and pursue repetition only where a large server flight can be predicted reliably.

Experiments were prepared with AI assistance.

Patches: noq repeated-Initial prototype, iroh integration and harness, and standalone iroh rate-limit API. Base commits and application notes.