Skip to content

Add datatype-aware numeric conversion reads #11

Description

@janosh

Problem

H5Dataset::read_raw::<T> intentionally exposes the stored representation and accepts a target based on element width. A consumer that wants semantic numeric values in one Rust type must instead inspect DatatypeMessage, dispatch across every supported integer or floating-point storage type, call read_raw with that exact type, and convert each value.

For example, reading arbitrary floating-point datasets as f64 requires separate f32 and f64 branches. Reading arbitrary integer datasets as one wide checked type requires branches for signedness and widths from 8 through 64 bits. Generic metadata and scientific-data readers duplicate this dispatch.

Proposed API

Please consider a separate conversion API that leaves read_raw unchanged, for example:

let floats = dataset.read_numeric_as::<f64>()?;
let integers = dataset.read_numeric_as::<i128>()?;

A corresponding hyperslab form would also be useful:

let values = dataset.read_numeric_slice_as::<f64>(&starts, &counts)?;

Requested semantics

  • Inspect the stored datatype class, signedness, byte order, and width rather than only comparing element sizes.
  • Support the standard primitive integer and floating-point datatypes.
  • Perform checked integer conversions and report overflow instead of wrapping.
  • Define float-to-integer and narrowing-float behavior explicitly, or reject those conversions initially.
  • Reuse the existing decoded data path so contiguous, chunked, and filtered datasets behave consistently.
  • Keep read_raw as the zero-conversion representation-level API.

Even an initial pair of focused helpers such as read_f64 and read_i128 would remove substantial downstream datatype dispatch while keeping conversion policy explicit.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions