Problem
write_vlen_strings_slice reclaims superseded global-heap objects, but the recovered space is reusable only by the current writer session. A workload that opens a file, replaces one variable-length string, closes it, and repeats cannot reuse space freed by earlier sessions. The file therefore grows on every update even when alternating between equal-sized values.
Standalone reproducer
use rust_hdf5::H5File;
fn main() -> Result<(), Box<dyn std::error::Error>> {
let path = "/tmp/reopen-vlen.h5";
let _ = std::fs::remove_file(path);
let file = H5File::create(path)?;
file.write_vlen_strings("notes", &["initial"])?;
file.close()?;
for iteration in 0..16 {
let payload = if iteration % 2 == 0 { "a" } else { "b" }.repeat(128 * 1024);
let file = H5File::open_rw(path)?;
file.dataset_writer("notes")?
.write_vlen_strings_slice(0, &[payload.as_str()])?;
file.close()?;
println!("{}", std::fs::metadata(path)?.len());
}
Ok(())
}
After the first large replacement, subsequent equal-sized replacements should ideally reuse the released space and leave the file size roughly flat.
Requested behavior
Please persist or reconstruct reusable free-space information on open_rw, so blocks released by vlen updates can be reused by later writer sessions. This should preserve the existing SWMR safety rule and remain compatible with files written by libhdf5. If persistent reclamation needs an explicit opt-in or compaction step, exposing that would still avoid downstream whole-file rebuilds.
Problem
write_vlen_strings_slicereclaims superseded global-heap objects, but the recovered space is reusable only by the current writer session. A workload that opens a file, replaces one variable-length string, closes it, and repeats cannot reuse space freed by earlier sessions. The file therefore grows on every update even when alternating between equal-sized values.Standalone reproducer
After the first large replacement, subsequent equal-sized replacements should ideally reuse the released space and leave the file size roughly flat.
Requested behavior
Please persist or reconstruct reusable free-space information on
open_rw, so blocks released by vlen updates can be reused by later writer sessions. This should preserve the existing SWMR safety rule and remain compatible with files written by libhdf5. If persistent reclamation needs an explicit opt-in or compaction step, exposing that would still avoid downstream whole-file rebuilds.