Description
Spark 4.1.0 introduced a new OneRowRelationExec physical plan node via SPARK-52060 (commit f423885f65d).
Changes in Spark 4.1.x
-
New OneRowRelationExec class - A dedicated physical plan node for single-row relations (e.g., SELECT 1, SELECT version())
-
Replaces RDDScanExec - Previously, one-row relations used RDDScanExec with a hard-coded "OneRowRelation" string
-
User-facing change - The plan now shows OneRowRelationExec rather than RDDScanExec
Implementation
Added GpuOneRowRelationExec in spark-rapids that:
- Produces a single
ColumnarBatch with 1 row and 0 columns
- Uses no GPU memory since there are no actual data columns
- Overrides
makeCopy and doCanonicalize to handle Spark's TreeNode reflection
Status
✅ Completed - GPU support for OneRowRelationExec has been implemented in the spark411 shim.
Description
Spark 4.1.0 introduced a new
OneRowRelationExecphysical plan node via SPARK-52060 (commit f423885f65d).Changes in Spark 4.1.x
New
OneRowRelationExecclass - A dedicated physical plan node for single-row relations (e.g.,SELECT 1,SELECT version())Replaces
RDDScanExec- Previously, one-row relations usedRDDScanExecwith a hard-coded "OneRowRelation" stringUser-facing change - The plan now shows
OneRowRelationExecrather thanRDDScanExecImplementation
Added
GpuOneRowRelationExecin spark-rapids that:ColumnarBatchwith 1 row and 0 columnsmakeCopyanddoCanonicalizeto handle Spark's TreeNode reflectionStatus
✅ Completed - GPU support for
OneRowRelationExechas been implemented in the spark411 shim.