Skip to content

Commit e2014b3

Browse files
committed
perf: compute output_bytes without building ArrayData
Closes #26071 `BaselineMetrics::record_output` called `get_record_batch_memory_size` for every output batch, which materializes `ArrayData` and inserts every buffer address into a hash set. Compute `output_bytes` as the sum of each column's `Array::get_array_memory_size` instead, which never builds `ArrayData`. `get_record_batch_memory_size` is unchanged and remains what memory reservations use, since those need exact deduplicated accounting.
1 parent 81ffb0c commit e2014b3

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

‎datafusion/physical-expr-common/src/metrics/baseline.rs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -359,7 +359,7 @@ impl RecordOutput for RecordBatch {
359359
impl RecordOutput for &RecordBatch {
360360
fn record_output(self, bm: &BaselineMetrics) -> Self {
361361
bm.record_output(self.num_rows());
362-
bm.output_bytes.add(output_bytes(&self));
362+
bm.output_bytes.add(output_bytes(self));
363363
bm.output_batches.add(1);
364364
self
365365
}

0 commit comments

Comments
 (0)