[C++][Parquet] Min Max stats are wrong when the value of a column are both infinity or -infinity #47426

Activity

1. PingLiuPing

   added

   Type: bug

   on Aug 26, 2025
2. github-actions

   added

   Component: C++

   on Aug 26, 2025
3. mapleFU

   changed the title [-][C++] Min Max stats are wrong when the value of a column are both infinity or -infinity[/-] [+][C++][Parquet] Min Max stats are wrong when the value of a column are both infinity or -infinity[/+] on Dec 3, 2025
4. meta-codesync

   added a commit that references this issue on Dec 22, 2025

   fix: Incorrect min max stats when the column value are infinity or -i…

   ...

   25a8810
5. Alan-S-Andrade

   added a commit that references this issue on Jun 2, 2026

   fix: Incorrect min max stats when the column value are infinity or -i…

   ...

   79c442c
6. stackedsax commented on Oct 4, 2026

   

   stackedsax

   on Oct 4, 2026

   Last edited by stackedsax

   More actions

   This was fixed by #50183 (GH-50182), released in Arrow 25.0.0. DefaultMin()/DefaultMax() in cpp/src/parquet/statistics.cc now seed the running min/max with +Inf/-Inf for floating-point types (including FLOAT16) instead of numeric_limits<T>::max()/lowest(), so an all--Inf column now reports [-Inf, -Inf] as expected. TestFloatStatistics.Infinities in statistics_test.cc covers FLOAT, DOUBLE and FLOAT16.

   Quick check on 25.0.0 or later:

   import io, pyarrow as pa, pyarrow.parquet as pq
   inf = float("inf")
   buf = io.BytesIO()
   pq.write_table(pa.table({"c": pa.array([-inf, -inf, -inf])}), buf)
   st = pq.ParquetFile(buf).metadata.row_group(0).column(0).statistics
   print(st.min, st.max)  # -inf -inf

   Note for anyone arriving from facebookincubator/velox#14603: Velox vendors a copy of this file under velox/dwio/parquet/writer/arrow/Statistics.cpp, so it needs to pick up the equivalent change separately.

   @thisisnic I think this can be closed as a duplicate of #50182

   Note: I used Claude Code to help research this, but I read the ticket, the code and the linked commits myself and the conclusion here is mine, not the tool's.

   👍React with 👍1Reacted by Ping Liu
7. thisisnic commented on Oct 4, 2026

   

   thisisnic

   on Oct 4, 2026

   Member

   More actions

   Looks good to me; this code is where it was fixed; closing.

   arrow/cpp/src/parquet/statistics.cc

   Lines 90 to 107 in e85e181

   	// For floating point, seed the running min/max with +/-infinity rather than
   	// the largest/smallest finite value: a finite seed is not an identity for
   	// min/max once the data contains infinities, so an all-+Inf (resp. all--Inf)
   	// column would otherwise never displace the seed and report a finite bound.
   	constexpr static T DefaultMin() {
   	if constexpr (std::is_floating_point_v<T>) {
   	return std::numeric_limits<T>::infinity();
   	} else {
   	return std::numeric_limits<T>::max();
   	}
   	}
   	constexpr static T DefaultMax() {
   	if constexpr (std::is_floating_point_v<T>) {
   	return -std::numeric_limits<T>::infinity();
   	} else {
   	return std::numeric_limits<T>::lowest();
   	}
   	}
8. thisisnic

   closed this as completedon Oct 4, 2026
9. PingLiuPing commented on Oct 8, 2026

   

   PingLiuPing

   on Oct 8, 2026

   Author

   More actions

   Thank you for the fix and summary.