Skip to content

MaxSize as a const fn#179

Open
jamesmunns wants to merge 4 commits intomainfrom
james/max-size-as-a-const-fn
Open

MaxSize as a const fn#179
jamesmunns wants to merge 4 commits intomainfrom
james/max-size-as-a-const-fn

Conversation

@jamesmunns
Copy link
Owner

No description provided.

@netlify
Copy link

netlify bot commented Oct 31, 2024

Deploy Preview for cute-starship-2d9c9b canceled.

Name Link
🔨 Latest commit 155b157
🔍 Latest deploy log https://app.netlify.com/sites/cute-starship-2d9c9b/deploys/672f80d16fec6d0008f781c5

1
} else {
// CEIL(variants / 7)
(nvars.len() + 6) / 7
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't right, it should be 2^7 not 7, we want to check how many variants fit in how many varint bytes.

Copy link
Collaborator

@max-heller max-heller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple thoughts that have been bouncing around in my head

///
/// You must not rely on this value for safety reasons, as implementations
/// could be wrong.
const MANUAL_MAX_SIZE: Option<usize> = None;
Copy link
Collaborator

@max-heller max-heller Nov 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of having this manual override and a max_size() function that checks it, this could default to the calculated size and be overridden as appropriate:

trait Schema {
    const MAX_SIZE: Option<usize> = max_size_nt(Self::SCHEMA);
}

// Manual max size
impl Schema for heapless::Vec<T, N> {
    const MAX_SIZE: Option<usize> = Some(...);
}

This seems more intuitive to me, since there'd be only one source of truth--T::MAX_SIZE--instead of the separate override and calculator function.

Comment on lines +30 to +31
assert!(nty_eq(nt, Inner::SCHEMA), "Mismatched Outer/Inner types!");
let size_one = max_size::<Inner>();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this dance required to use Inner::MANUAL_MAX_SIZE if it's set (as opposed to using max_size_nt())?

Somewhat relatedly, I'm wondering if it would make sense to embed known max sizes in the schema types somewhere in order for postcard-dyn to be able to take them into account. Currently, from my understanding, a client using postcard-dyn couldn't be told the max size of a heapless::Vec<T, N> because the only information in the serialized schema is the unbounded Seq(T::SCHEMA). If the serialized schema included the max length, then that length could be used to size buffers even without knowing the concrete type.

Concretely, this could take the form of bundling an overall max size with an OwnedNameType:

struct OwnedSchema {
    ty: OwnedNamedType,
    max_size: Option<usize>,
}

Or it could be more tightly coupled with the schema types themselves, something like:

struct NamedType {
    name: &'static str,
    ty: &'static DataModelType,
    max_size: Option<usize>,
}
// In this case, Schema could remain as just a single associated const
trait Schema {
    const SCHEMA: &'static schema::NamedType;
}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or, less flexibly, unbounded data model types (strings, seqs, maps, etc.) could have an optional max length field. Then the max size could always be determined based on the schema. Though that might not play as nicely with schema hashing—are two schemas interchangeable if they have different max lengths for a contained string?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or, less flexibly, unbounded data model types (strings, seqs, maps, etc.) could have an optional max length field. Then the max size could always be determined based on the schema. Though that might not play as nicely with schema hashing—are two schemas interchangeable if they have different max lengths for a contained string?

I'm starting to think this might be the best option:

  1. It's harder to implement in subtly wrong ways (e.g., by assuming integers take the same number of bytes as their in-memory representations, which I've seen in the wild)
  2. It conveys additional information (bounds on the length of collections) that could be calculated based on the max size, but not as obviously
  3. If max size is needed in a postcard-dyn/postcard-rpc context, it's more compact to send N on the wire than N * max_size(T)

@jamesmunns can you think of any cases where max size could not be determined this way?

@max-heller max-heller added the postcard-schema Related to the postcard-schema crate label Feb 13, 2025
@jamesmunns jamesmunns added the Q2 '25 Triage Items tracked as part of https://github.com/jamesmunns/postcard/issues/241 label Jun 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

postcard-schema Related to the postcard-schema crate Q2 '25 Triage Items tracked as part of https://github.com/jamesmunns/postcard/issues/241

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants