Skip to content

Feature Request: Option to preserve original integer width definition #49

Description

@styris-ame

When an integer fits in a smaller marker, zerompk writes the smaller marker. A
u64 holding 5 is written as a one-byte positive fixint, and the same goes for
every other width: u16, u32, i16, and so on all collapse to fixint or a
narrower marker when the runtime value allows. The wire type ends up reflecting
the value, not the Rust type.

This is the same behavior rmp_serde has in
3Hren/msgpack-rust#326.

Why it matters

Clients that map MessagePack markers to types, not the other way around, lose
the width. msgpackr decodes int64/uint64 markers to BigInt and everything
narrower to number. A Rust u64 exported as a TypeScript bigint (for
example through ts-rs) decodes as bigint for large values and number for
small ones, so the field type is unstable across messages. The same mismatch
hits any client that keys off the marker width.

Reproduction

#[derive(zerompk::ToMessagePack)]
struct Row {
    id: u64,
    seq: u32,
}

// id = 5 encodes as [0x05], a positive fixint.
// seq = 5 encodes as [0x05] as well, not a uint32.

Fix

Keep narrowing as the default and add an opt-in preserve-int-width feature that
makes each integer type emit its representative marker for every value: u8/i8 as
uint8/int8, u16/i16 as uint16/int16, and so on through 64-bit.
Floats already emit float32/float64 unconditionally, so they need no change.

Decoding is untouched: read_uN/read_iN already accept every integer width, so
wide output round trips and existing narrow data still decodes.

The write_u*/write_i* primitives stay as-is. They are shared internally for
enum integer tags and c-style enum discriminants, which should remain compact, so
the feature gates the scalar type impls rather than the primitives themselves.

1. Wide writer methods

Add one method per integer type to the Write trait in zerompk/src/write.rs:

fn write_u8_wide(&mut self, u: u8) -> Result<()>;
fn write_u16_wide(&mut self, u: u16) -> Result<()>;
fn write_u32_wide(&mut self, u: u32) -> Result<()>;
fn write_u64_wide(&mut self, u: u64) -> Result<()>;
fn write_i8_wide(&mut self, i: i8) -> Result<()>;
fn write_i16_wide(&mut self, i: i16) -> Result<()>;
fn write_i32_wide(&mut self, i: i32) -> Result<()>;
fn write_i64_wide(&mut self, i: i64) -> Result<()>;

Each body is the widest arm of the matching write_*, applied without the size
check: the representative marker followed by the big-endian bytes. The u64
case on SliceWriter/VecWriter:

fn write_u64_wide(&mut self, u: u64) -> Result<()> {
    let buf = self.take_array::<9>()?;
    let [head, tail @ ..] = buf;
    *head = UINT64_MARKER;
    *tail = u.to_be_bytes();
    Ok(())
}

The smaller widths follow with their own marker and length (UINT8_MARKER and
one byte, UINT16_MARKER and two, etc.), and the signed methods use the
INT*_MARKER constants. IOWriter mirrors the widest arm of its own
write_uN/write_iN.

2. Gate the scalar impls

In zerompk/src/impl.rs, pick the writer per type:

#[cfg(not(feature = "preserve-int-width"))]
impl_scalar!(u64, write_u64, read_u64);
#[cfg(feature = "preserve-int-width")]
impl_scalar!(u64, write_u64_wide, read_u64);
// ... same pairing for u8, u16, u32 and i8, i16, i32, i64

The 64-bit usize/isize branches call write_u64_wide/write_i64_wide under
the feature, and the 32-bit branches call write_u32_wide/write_i32_wide.

3. Cargo feature

[features]
preserve-int-width = []

The flag is write-only and additive, so enabling it changes encoding without
touching any decode path or other feature.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions