SORTED_HEAP_BYTES_PER_NODE = 192 (rounded up from a measured 151.6 B/node) is the constant the sorted-path row ceiling is computed from. That measurement assumes compressed oops. The server's entrypoint runs with -XX:MaxRAMPercentage=70.0 and no explicit -Xmx, so on any host or pod with more than ~45 GiB of RAM the ergonomic heap exceeds 32 GiB, HotSpot switches UseCompressedOops off, and every reference in the materialized SimpleGroup tree doubles in size.
Measured 2026-09-23 with SortedHeapMeasurement (200k rows, flat 11-field shape), same JVM otherwise:
| heap |
UseCompressedOops |
B/node |
B/row |
-Xmx6g |
on |
152.1 |
1,673 |
-XX:MaxRAM=4g -XX:MaxRAMPercentage=70 (a 4 GiB pod) |
on |
152.0 |
1,672 |
| entrypoint flags on a 48 GiB host (heap 33.6 GiB) |
off (ergonomic) |
216.6–219.3 |
2,383–2,412 |
At 216+ B/node the 192 constant under-counts by ~14%, so a group the planner admits as fitting sortedHeapBytes does not fit, and the refusal that exists to convert an OOM into a counted skip (#118) no longer fires. The same is true on JDK 21 and 25 (JDK 25: 152.1 with compressed oops, 145.7 with -XX:+UseCompactObjectHeaders); this is a heap-size effect, not a JDK one.
Today's maintenance pods are sized below the cliff, so nothing is wrong in production; the hazard is that raising the pod's memory past ~45 GiB to admit bigger sorted groups (the obvious response to heap_budget_exceeded, see #134) silently crosses it.
Fix options
- Pin
-XX:+UseCompressedOops is not possible above 32 GiB; instead cap the heap at 31 GiB explicitly (-Xmx31g or MaxRAM) when the container has more, and let the rest of the container memory go to parquet's direct buffers and the page cache, which is where a compactor wants it anyway.
- Or make the constant a function of
VM.current().objectHeaderSize()/Unsafe.addressSize() at startup: read UseCompressedOops via HotSpotDiagnosticMXBean.getVMOption and use 216 when it is off. Cheap, and it makes the refusal honest on any heap.
- Either way,
SortedHeapMeasurement's KDoc should carry the table above so the next person sizing a pod sees the cliff.
Related: #118 (the OOM this constant guards), #134 (external sort removes the constant entirely), #133.
SORTED_HEAP_BYTES_PER_NODE = 192(rounded up from a measured 151.6 B/node) is the constant the sorted-path row ceiling is computed from. That measurement assumes compressed oops. The server's entrypoint runs with-XX:MaxRAMPercentage=70.0and no explicit-Xmx, so on any host or pod with more than ~45 GiB of RAM the ergonomic heap exceeds 32 GiB, HotSpot switchesUseCompressedOopsoff, and every reference in the materializedSimpleGrouptree doubles in size.Measured 2026-09-23 with
SortedHeapMeasurement(200k rows, flat 11-field shape), same JVM otherwise:-Xmx6g-XX:MaxRAM=4g -XX:MaxRAMPercentage=70(a 4 GiB pod)At 216+ B/node the 192 constant under-counts by ~14%, so a group the planner admits as fitting
sortedHeapBytesdoes not fit, and the refusal that exists to convert an OOM into a counted skip (#118) no longer fires. The same is true on JDK 21 and 25 (JDK 25: 152.1 with compressed oops, 145.7 with-XX:+UseCompactObjectHeaders); this is a heap-size effect, not a JDK one.Today's maintenance pods are sized below the cliff, so nothing is wrong in production; the hazard is that raising the pod's memory past ~45 GiB to admit bigger sorted groups (the obvious response to
heap_budget_exceeded, see #134) silently crosses it.Fix options
-XX:+UseCompressedOopsis not possible above 32 GiB; instead cap the heap at 31 GiB explicitly (-Xmx31gorMaxRAM) when the container has more, and let the rest of the container memory go to parquet's direct buffers and the page cache, which is where a compactor wants it anyway.VM.current().objectHeaderSize()/Unsafe.addressSize()at startup: readUseCompressedOopsviaHotSpotDiagnosticMXBean.getVMOptionand use 216 when it is off. Cheap, and it makes the refusal honest on any heap.SortedHeapMeasurement's KDoc should carry the table above so the next person sizing a pod sees the cliff.Related: #118 (the OOM this constant guards), #134 (external sort removes the constant entirely), #133.