Skip to content

nostr: reduce allocations in the NIP-44 v2 message path - #1421

Merged
yukibtc merged 2 commits into
nostrdevkit:masterfrom
JSKitty:nip44-allocations
Aug 4, 2026
Merged

nostr: reduce allocations in the NIP-44 v2 message path#1421
yukibtc merged 2 commits into
nostrdevkit:masterfrom
JSKitty:nip44-allocations

Conversation

@JSKitty

@JSKitty JSKitty commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Description

Follow-up to #1419, stacked on it. The diff is the two commits on top.

Three sources of per-message allocation in the NIP-44 v2 path:

  1. HKDF-expand allocated a Vec for the output and one per block of the PRF chain. expand_into fills a caller-owned buffer, so the 76-byte message keys come off the stack, and T(i-1) | info | i is fed to the engine part by part rather than assembled first. The keyed engine depends only on the PRK, so it's built once and its midstate cloned per block, which 1.x makes possible by putting Clone on HashEngine.
  2. encrypt_to_bytes_with_nonce padded into one buffer, encrypted it, then copied it into a second alongside the version, nonce and MAC. It now lays out [version | nonce | length | plaintext | padding | MAC] once and encrypts the ciphertext region in place.
  3. generate_shared_key parsed the public key twice: xonly(), then from_x_only_public_key(pk, Parity::Even) rebuilding the even-parity compressed form and parsing it again, running the modular square root in ge_set_xo_var both times. Building that form directly parses once.

get_message_keys can no longer fail, so it returns MessageKeys directly and the HkdfLength / TryFromSlice variants go with it.

Measured on an Apple M4, all three trees in one run. "upstream" is master before #1419:

operation upstream after #1419 after this PR total
encrypt 32 B 2786 ns 1088 ns 717 ns -74%
decrypt 32 B 2721 ns 982 ns 772 ns -72%
encrypt 256 B 3734 ns 1770 ns 1396 ns -63%
decrypt 256 B 3643 ns 1630 ns 1419 ns -61%
encrypt 4096 B 19370 ns 12086 ns 11664 ns -40%
decrypt 4096 B 19415 ns 11922 ns 11767 ns -39%
ConversationKey::derive 20218 ns 19703 ns 17547 ns -13%

#1419 does the bulk of it on the hashing, this PR does the rest on the allocations and the ECDH parse. Allocation traffic is all from this PR, since #1419 changed none of it:

operation before after this PR
encrypt 32 B 11 allocs, 448 B 1 alloc, 99 B
decrypt 32 B 8 allocs, 346 B 2 allocs, 66 B
encrypt 1024 B 11 allocs, 3424 B 1 alloc, 1091 B

Nothing here trades memory for speed; both go down.

Notes to the reviewers

Output is unchanged. Same differential harness as #1419 (every padding boundary, MAC and ciphertext tampering, all rejection paths by exact error string, on- and off-curve keys), identical digest c95d027371418aa7d71c0d86ea7cec49.

The in-place payload construction is the riskiest change, so pad is kept as a test oracle and the new payload is asserted byte-equal to the previous pad-encrypt-append form across sixteen plaintext lengths. Also adds the RFC 5869 SHA-256 vectors, which the HKDF helpers had no coverage for.

Verified with the full just precommit set plus cargo build --workspace --all-targets and cargo clippy --all-targets -- -D warnings, on both commits and the tip.

Checklist

  • I followed the contribution guidelines
  • I updated the relevant CHANGELOG.md (if applicable)
  • I understand and can explain all code in this PR

@JSKitty
JSKitty force-pushed the nip44-allocations branch from 4d3fa24 to 0f48c16 Compare August 3, 2026 23:51
@codecov

codecov Bot commented Aug 3, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.90210% with 3 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
nostr/src/nips/nip44/v2.rs 97.77% 2 Missing ⚠️
nostr/src/util/hkdf.rs 97.95% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@JSKitty
JSKitty force-pushed the nip44-allocations branch from 0f48c16 to 2ccb447 Compare August 4, 2026 00:03
@JSKitty

JSKitty commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

Coverage note: the remaining three lines are test-only. Two are the length guards inside pad, which is now a test oracle called only with valid input; the third is a format argument in an assert_eq! that evaluates only on failure.

The two real misses in encrypt_to_bytes_with_nonce are covered as of the last push.

Good for review! 🙏

@yukibtc

yukibtc commented Aug 4, 2026

Copy link
Copy Markdown
Member

@JSKitty, can you rebase on master?

JSKitty added 2 commits August 4, 2026 10:36
Three sources of per-message allocation, none of which the algorithm needs:

HKDF-expand built a `Vec` for the output and another for each block of the
PRF chain. `expand_into` fills a caller-owned buffer instead, so the 76-byte
message keys now come off the stack, and the T(i-1) | info | i concatenation
is fed to the engine part by part rather than assembled in a buffer first.
The keyed engine also depends only on the PRK, so it is built once and its
midstate cloned per block instead of re-absorbing the ipad/opad key schedule
three times per message. Cloning a `HashEngine` is possible because
`bitcoin_hashes` 1.x made `Clone` a supertrait of it.

`encrypt_to_bytes_with_nonce` padded into one buffer, encrypted it, then
copied it into a second buffer alongside the version, nonce and MAC. It now
lays out [version | nonce | length | plaintext | padding | MAC] once and
encrypts the ciphertext region where it already sits. The MAC input is
contiguous in that buffer too, so it needs no second engine input.

`get_message_keys` can no longer fail, so it returns `MessageKeys` directly
and the `HkdfLength` and `TryFromSlice` error variants go away with it.

Per operation on an Apple M4:

| op            | allocs      | bytes         | time            |
|---------------|-------------|---------------|-----------------|
| encrypt 32B   | 11 -> 1     | 448 -> 99     | 1097 -> 710 ns  |
| decrypt 32B   |  8 -> 2     | 346 -> 66     |  987 -> 767 ns  |
| encrypt 1024B | 11 -> 1     | 3424 -> 1091  |                 |

Output is unchanged: a differential harness over every padding boundary,
both tamper paths and all rejection error strings produces an identical
digest, and the payload is asserted byte-equal to the previous
pad-encrypt-append construction across sixteen plaintext lengths.

Adds the RFC 5869 SHA-256 vectors, which the HKDF helpers had no coverage for.
`generate_shared_key` called `PublicKey::xonly()`, which parses the key, and
then `from_x_only_public_key(pk, Parity::Even)`, which rebuilds the
even-parity compressed form and parses it again. Both parses run the modular
square root in `ge_set_xo_var`, the dominant cost of the conversion.

Building the compressed form directly parses once for the same result: both
paths accept exactly the keys whose x coordinate lies on the curve, and both
take the even-parity point.

`ConversationKey::derive` on an Apple M4: 19712 -> 17472 ns (-11%).
@JSKitty
JSKitty force-pushed the nip44-allocations branch from 2ccb447 to 5368934 Compare August 4, 2026 09:37
@JSKitty
JSKitty marked this pull request as ready for review August 4, 2026 09:39

@TheAwiteb TheAwiteb left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks good, but I don't understand cryptography that much

@yukibtc yukibtc left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@yukibtc
yukibtc merged commit f624870 into nostrdevkit:master Aug 4, 2026
43 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants