mdf4-rs [Compatibility] Use Variable Length mode for DataListBlock to prevent data corruption in Vector CANape/INCA

[EdwardYi2026]

Description:
When a data block exceeds MAX_DT_BLOCK_SIZE (4MB), the library splits it into multiple DT blocks and links them using a DataListBlock initialized with new_equal_length (setting the Equal Length flag, bit 0).

According to the ASAM MDF v4.1 specification, this is completely legal: the spec explicitly states that in "equal length" mode, the very last data block is permitted to be smaller than dl_equal_length. mdf4-rs perfectly follows this rule.

However, we discovered that major industrial MDF readers (e.g., Vector CANape, INCA) appear to have a parsing flaw regarding this specific rule. When they see the Equal Length flag, they aggressively map the records using (dl_equal_length - 24) / record_size for all blocks, ignoring the exception for the last block. Because the last block is smaller, these readers read past the actual block boundary, causing severe timeline compression, misalignment, or garbage data at the end of large files.

(Note: asammdf in Python handles this correctly, but the issue is strictly with commercial C/C++ tools).

Steps to Reproduce (in Vector CANape):

1. Generate an MDF4 file larger than 4MB with mdf4_rs (triggering DT block splitting).
2. Open the .mf4 file in Vector CANape.
3. Observe the timeline/signals near the end of the data; it will become heavily distorted or "squeezed" where the final, smaller DT block is processed.

Proposed Solution (Workaround for flawed readers):
To maximize compatibility with these strict (and slightly flawed) industrial tools, it is safer to write the DataListBlock without the Equal Length flag (Variable Length mode) when the blocks are not strictly identical, thereby providing explicit byte offsets.

We patched this locally by adding a new_variable_length builder and changing finish_data_block. This explicitly spells out the offsets, preventing Vector CANape from making naive multiplication assumptions.

Proposed addition to src/blocks/data_list_block.rs:

    pub fn new_variable_length(data_block_addrs: Vec<u64>, dt_sizes: Vec<u64>) -> Self {
        let link_count = data_block_addrs.len() as u64 + 1;
        let offsets_size = data_block_addrs.len() as u64 * 8;
        let length = 24 + link_count * 8 + 8 + offsets_size; 

        let mut offsets = Vec::with_capacity(data_block_addrs.len());
        let mut current_offset = 0u64;
        for size in dt_sizes {
            offsets.push(current_offset);
            let data_size = size.saturating_sub(24);
            current_offset += data_size;
        }

        Self {
            header: BlockHeader {
                id: "##DL".to_string(),
                reserved: 0,
                length,
                link_count,
            },
            next_dl_addr: 0,
            data_block_count: data_block_addrs.len() as u32,
            data_block_addrs,
            flags: 0, // Variable length flag
            equal_length: None,
            block_offsets: Some(offsets),
        }
    }

In src/writer/data.rs (finish_data_block):

        if dt.dt_ids.len() > 1 {
            let dl_count = self.block_positions.keys().filter(|k| k.starts_with("dl_")).count();
            let dl_id = format!("dl_{}", dl_count);
            // Switch to variable length to enforce safe offsets for all readers
            let dl_block = DataListBlock::new_variable_length(dt.dt_positions.clone(), dt.dt_sizes.clone());
            let dl_bytes = dl_block.to_bytes()?;
            let _pos = self.write_block_with_id(&dl_bytes, &dl_id)?;
            let dg_data_link_offset = 40;
            self.update_block_link(&dt.dg_id, dg_data_link_offset, &dl_id)?;
        }

This workaround guarantees absolute compatibility without violating the standard.