Skip to content

feat: add array_filter function - #1320

Closed
davidlghellin wants to merge 39 commits into
lakehq:mainfrom
davidlghellin:feat/lambda
Closed

feat: add array_filter function#1320
davidlghellin wants to merge 39 commits into
lakehq:mainfrom
davidlghellin:feat/lambda

Conversation

@davidlghellin

@davidlghellin davidlghellin commented Feb 5, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Note: This implements filter() as a ScalarUDFImpl. When DataFusion merges apache/datafusion#21679 only the trait interface changes

@github-actions

github-actions Bot commented Feb 5, 2026 •
edited
Loading

Copy link
Copy Markdown

Gold Data Report

Notes
  1. The tables below show the number of true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) in gold data input processing.
  2. A positive input is a valid test case, while a negative input is a test case that is expected to fail.

Commit Information

Commit Revision Branch
After fb89951 refs/pull/1320/merge
Before ab3c52c main

Summary

Commit TP TN FP FN Total
After 2062 197 49 315 2623
Before 2060 197 49 317 2623

Details

Gold Data Metrics
Group File Commit TP TN FP FN Total
spark data_type.json After 48 11 2 3 64
Before 48 11 2 3 64
expression/case.json After 5 0 0 0 5
Before 5 0 0 0 5
expression/cast.json After 4 0 0 0 4
Before 4 0 0 0 4
expression/current.json After 3 0 0 0 3
Before 3 0 0 0 3
expression/date.json After 4 0 1 0 5
Before 4 0 1 0 5
expression/interval.json After 346 4 1 0 351
Before 346 4 1 0 351
expression/large.json After 2 0 0 0 2
Before 2 0 0 0 2
expression/like.json After 29 10 0 0 39
Before 29 10 0 0 39
expression/misc.json After 111 5 1 1 118
Before 111 5 1 1 118
expression/numeric.json After 31 6 1 0 38
Before 31 6 1 0 38
expression/string.json After 18 1 0 0 19
Before 18 1 0 0 19
expression/timestamp.json After 7 0 3 0 10
Before 7 0 3 0 10
expression/window.json After 73 0 1 0 74
Before 73 0 1 0 74
function/agg.json After 154 0 0 32 186
Before 154 0 0 32 186
function/array.json After 44 0 0 0 44
Before 44 0 0 0 44
function/bitwise.json After 15 0 0 0 15
Before 15 0 0 0 15
function/collection.json After 12 0 0 0 12
Before 12 0 0 0 12
function/conditional.json After 15 0 0 0 15
Before 15 0 0 0 15
function/conversion.json After 2 0 0 0 2
Before 2 0 0 0 2
function/csv.json After 3 0 0 2 5
Before 3 0 0 2 5
function/datetime.json After 160 0 0 20 180
Before 160 0 0 20 180
function/generator.json After 12 0 0 1 13
Before 13 0 0 0 13
function/hash.json After 5 0 0 2 7
Before 5 0 0 2 7
function/json.json After 21 0 0 1 22
Before 21 0 0 1 22
function/lambda.json After 4 0 0 27 31
Before 1 0 0 30 31
function/map.json After 11 0 0 0 11
Before 11 0 0 0 11
function/math.json After 124 0 0 0 124
Before 124 0 0 0 124
function/misc.json After 33 0 0 41 74
Before 33 0 0 41 74
function/predicate.json After 72 0 0 7 79
Before 72 0 0 7 79
function/st.json After 2 0 0 5 7
Before 2 0 0 5 7
function/string.json After 175 0 0 30 205
Before 175 0 0 30 205
function/struct.json After 2 0 0 0 2
Before 2 0 0 0 2
function/url.json After 10 0 0 0 10
Before 10 0 0 0 10
function/variant.json After 26 0 0 2 28
Before 26 0 0 2 28
function/window.json After 6 0 0 3 9
Before 6 0 0 3 9
function/xml.json After 10 0 0 7 17
Before 10 0 0 7 17
plan/ddl_alter_table.json After 49 14 3 11 77
Before 49 14 3 11 77
plan/ddl_alter_view.json After 5 1 0 0 6
Before 5 1 0 0 6
plan/ddl_analyze_table.json After 17 6 0 0 23
Before 17 6 0 0 23
plan/ddl_cache.json After 4 0 1 0 5
Before 4 0 1 0 5
plan/ddl_create_index.json After 0 0 0 3 3
Before 0 0 0 3 3
plan/ddl_create_table.json After 28 29 9 39 105
Before 28 29 9 39 105
plan/ddl_delete_from.json After 2 1 0 0 3
Before 2 1 0 0 3
plan/ddl_describe.json After 7 0 0 0 7
Before 7 0 0 0 7
plan/ddl_drop_index.json After 0 0 0 2 2
Before 0 0 0 2 2
plan/ddl_drop_view.json After 5 0 0 0 5
Before 5 0 0 0 5
plan/ddl_insert_into.json After 16 1 1 0 18
Before 16 1 1 0 18
plan/ddl_insert_overwrite.json After 9 0 2 0 11
Before 9 0 2 0 11
plan/ddl_load_data.json After 4 0 0 0 4
Before 4 0 0 0 4
plan/ddl_merge_into.json After 8 4 3 0 15
Before 8 4 3 0 15
plan/ddl_misc.json After 10 0 0 0 10
Before 10 0 0 0 10
plan/ddl_replace_table.json After 24 13 8 39 84
Before 24 13 8 39 84
plan/ddl_select.json After 1 0 0 0 1
Before 1 0 0 0 1
plan/ddl_show_views.json After 7 0 0 0 7
Before 7 0 0 0 7
plan/ddl_uncache.json After 2 0 0 0 2
Before 2 0 0 0 2
plan/ddl_update.json After 2 1 0 0 3
Before 2 1 0 0 3
plan/error_alter_table.json After 0 2 0 0 2
Before 0 2 0 0 2
plan/error_analyze_table.json After 0 1 0 0 1
Before 0 1 0 0 1
plan/error_create_table.json After 0 6 0 0 6
Before 0 6 0 0 6
plan/error_describe.json After 0 1 0 0 1
Before 0 1 0 0 1
plan/error_join.json After 0 2 0 0 2
Before 0 2 0 0 2
plan/error_load_data.json After 0 1 0 0 1
Before 0 1 0 0 1
plan/error_misc.json After 0 14 0 0 14
Before 0 14 0 0 14
plan/error_order_by.json After 1 4 0 0 5
Before 1 4 0 0 5
plan/error_select.json After 0 15 0 0 15
Before 0 15 0 0 15
plan/error_with.json After 0 1 0 0 1
Before 0 1 0 0 1
plan/plan_alter_view.json After 0 2 0 0 2
Before 0 2 0 0 2
plan/plan_create_view.json After 0 2 0 0 2
Before 0 2 0 0 2
plan/plan_explain.json After 0 1 1 0 2
Before 0 1 1 0 2
plan/plan_group_by.json After 9 1 0 1 11
Before 9 1 0 1 11
plan/plan_hint.json After 25 0 3 0 28
Before 25 0 3 0 28
plan/plan_insert_into.json After 3 0 0 0 3
Before 3 0 0 0 3
plan/plan_insert_overwrite.json After 2 0 0 0 2
Before 2 0 0 0 2
plan/plan_join.json After 59 2 1 0 62
Before 59 2 1 0 62
plan/plan_misc.json After 15 4 0 10 29
Before 15 4 0 10 29
plan/plan_order_by.json After 15 5 1 10 31
Before 15 5 1 10 31
plan/plan_select.json After 85 14 5 16 120
Before 85 14 5 16 120
plan/plan_set_operation.json After 17 0 0 0 17
Before 17 0 0 0 17
plan/plan_with.json After 6 0 1 0 7
Before 6 0 1 0 7
plan/unpivot_join.json After 4 0 0 0 4
Before 4 0 0 0 4
plan/unpivot_select.json After 14 6 0 0 20
Before 14 6 0 0 20
table_schema.json After 8 6 0 0 14
Before 8 6 0 0 14

@github-actions

github-actions Bot commented Feb 5, 2026 •
edited
Loading

Copy link
Copy Markdown

Spark 3.5.7 Test Report

Commit Information

Commit Revision Branch
After 7a37f52 refs/pull/1320/merge
Before 9e17a1d refs/heads/main

Test Summary

Suite Commit Failed Passed Skipped Warnings Time (s)
doctest-catalog After 11 13 1 6 4.97
Before 11 13 1 6 4.89
doctest-column After 33 2 5.44
Before 33 2 5.47
doctest-dataframe After 23 81 3 3 7.96
Before 22 82 3 3 7.99
doctest-functions After 48 353 8 7 15.45
Before 48 353 8 7 14.95
test-connect After 134 868 169 692 132.62
Before 134 868 169 699 127.37

Test Details

Error Counts 
(+1)      216 Total
(+12)     117 Total Unique
-------- ---- ----------------------------------------------------------------------------------------------------------
(+1)       24 DocTestFailure
           15 PySparkAssertionError: [DIFFERENT_PANDAS_DATAFRAME] DataFrames are not almost equal:
           10 handle add artifacts
            6 AssertionError: False is not true
            6 UnsupportedOperationException: PlanNode::CacheTable
            6 UnsupportedOperationException: function: window
            5 AssertionError: AnalysisException not raised
            5 UnsupportedOperationException: function: input_file_name
            4 UnsupportedOperationException: unknown aggregate function: hll_sketch_agg
            3 PySparkNotImplementedError: [NOT_IMPLEMENTED] rdd() is not implemented.
            3 UnsupportedOperationException: handle analyze input files
            3 ValueError: Converting to Python dictionary is not supported when duplicate field names are present
            2 AnalysisException: Could not find config namespace "spark"
(+2)        2 AnalysisException: Internal error: Function 'approx_percentile_cont' failed to match any signature, errors: Error during planning: Function 'approx_percentile_cont' expects 2 arguments but received 3,...
            2 AnalysisException: No table format found for: orc
            2 AnalysisException: not supported: function exists
            2 AnalysisException: not supported: list functions
            2 AssertionError
            2 AssertionError: 0 not greater than or equal to 1
            2 AssertionError: Lists differ: [Row([22 chars](key=1, value='1'), Row(key=10, value='10'), R[2402 chars]99')] != [Row([22 chars](key=0, value='0'), Row(key=1, value='1'), Row[4882 chars]99')]
            2 IllegalArgumentException: expected value at line 1 column 1
            2 IllegalArgumentException: invalid argument: found FUNCTION at 7:15 expected 'DATABASE', 'SCHEMA', 'NAMESPACE', 'OR', 'TEMP', 'TEMPORARY', 'EXTERNAL', 'TABLE', 'GLOBAL', or 'VIEW'
            2 UnsupportedOperationException: Aggregate can not be used as a sliding accumulator because `retract_batch` is not implemented: avg@bjwplt75b8s8lnn9byry0pm4s(#9) PARTITION BY [#8] ORDER BY [#9 ASC NULLS...
            2 UnsupportedOperationException: approx quantile
            2 UnsupportedOperationException: collect metrics
            2 UnsupportedOperationException: freq items
            2 UnsupportedOperationException: handle analyze same semantics
(+2)        2 UnsupportedOperationException: lambda function: exists
(+2)        2 UnsupportedOperationException: named lambda variable
            2 UnsupportedOperationException: pivot
            2 UnsupportedOperationException: user defined data type should only exist in a field
            2 UnsupportedOperationException: with watermark
            2 handle artifact statuses
(+1)        1 AnalysisException: Cannot cast string 'a' to value of Float64 type
            1 AnalysisException: Cannot cast string 'abc' to value of Float64 type
            1 AnalysisException: Cannot cast value 'abc' to value of Boolean type
            1 AnalysisException: Error parsing timestamp from '2023-01-01' using format '%d-%m-%Y': input contains invalid characters
            1 AnalysisException: Failed to parse placeholder id: cannot parse integer from empty string
            1 AnalysisException: Found unsupported option type when parsing options: allowUnquotedFieldNames
            1 AnalysisException: Table already exists: tbl1
            1 AnalysisException: Temporary View not found: tab2
(+1)        1 AnalysisException: UNION queries have different number of columns: left has 2 columns whereas right has 3 columns
(+1)        1 AssertionError: "Database 'memory:2f73a5ac-784b-4708-96f7-8d50c81a3b84' dropped." does not match "No table format found for: jdbc"
(+1)        1 AssertionError: "Database 'memory:9771016f-8798-479c-b49a-b5886dbe43ff' dropped." does not match "No table format found for: jdbc"
            1 AssertionError: '+---[17 chars]-----+\n|                        x|\n+--------[132 chars]-+\n' != '+---[17 chars]----------+\n|update_fields(x, WithField(e))|\[167 chars]-+\n'
            1 AssertionError: 1 != 0
            1 AssertionError: 2 != 6
            1 AssertionError: AnalysisException not raised by <lambda>
            1 AssertionError: ArrayIndexOutOfBoundsException not raised
            1 AssertionError: Exception not raised
            1 AssertionError: Exception not raised by <lambda>
            1 AssertionError: Lists differ: [(1, 2), (3, 4), (None, 5), (0, 0)] != [(1, 2), (3, 4), (None, 5), (None, None)]
            1 AssertionError: Lists differ: [Row(id=90, name='90'), Row(id=91, name='91'), Ro[176 chars]99')] != [Row(id=15, name='15'), Row(id=16, name='16'), Ro[176 chars]24')]
            1 AssertionError: Lists differ: [Row(ln(id)=0.0, ln(id)=0.0, struct(id, name)=Row(id=[1232 chars]0'))] != [Row(ln(id)=4.31748811353631, ln(id)=4.31748811353631[1312 chars]4'))]
            1 AssertionError: Lists differ: [Row(name='Andy', age=30), Row(name='Andy', ag[374 chars]one)] != [Row(age=19, name='Justin'), Row(age=19, name=[374 chars]el')]
            1 AssertionError: Lists differ: [Row(name='Andy', age=30), Row(name='Justin', [34 chars]one)] != [Row(_corrupt_record=' "age":19}\n', name=None[104 chars]el')]
            1 AssertionError: Row(point='[1.0, 2.0]', pypoint='[3.0, 4.0]') != Row(point='(1.0, 2.0)', pypoint='[3.0, 4.0]')
            1 AssertionError: StorageLevel(False, True, True, False, 1) != StorageLevel(False, False, False, False, 1)
            1 AssertionError: Struc[15 chars]eld('a', NullType(), True), StructField('b', L[51 chars]ue)]) != Struc[15 chars]eld('b', LongType(), True), StructField('c', S[15 chars]ue)])
            1 AssertionError: Struc[30 chars]estampType(), True), StructField('val', IntegerType(), True)]) != Struc[30 chars]estampType(), True), StructField('val', IntegerType(), False)])
            1 AssertionError: Struc[32 chars]e(), False), StructField('b', DoubleType(), Fa[158 chars]ue)]) != Struc[32 chars]e(), True), StructField('b', DoubleType(), Tru[154 chars]ue)])
            1 AssertionError: Struc[40 chars]ue), StructField('val', ArrayType(DoubleType(), False), True)]) != Struc[40 chars]ue), StructField('val', PythonOnlyUDT(), True)])
            1 AssertionError: YearMonthIntervalType(0, 1) != YearMonthIntervalType(0, 0)
            1 AssertionError: [1.0, 2.0] != ExamplePoint(1.0,2.0)
            1 AssertionError: dtype('<M8[us]') != 'datetime64[ns]'
            1 AttributeError: 'DataFrame' object has no attribute '_ipython_key_completions_'
            1 AttributeError: 'DataFrame' object has no attribute '_joinAsOf'
            1 IllegalArgumentException: invalid argument: field not found in input schema: col1
(+1)        1 IllegalArgumentException: invalid argument: map_filter() first argument must be an array, got Map(Field { name: "entries", data_type: Struct([Field { name: "key", data_type: Utf8 }, Field { name: "val...
(+1)        1 IllegalArgumentException: invalid argument: map_zip_with() first argument must be an array, got Map(Field { name: "entries", data_type: Struct([Field { name: "key", data_type: Utf8 }, Field { name: "v...
(+1)        1 IllegalArgumentException: invalid argument: transform_keys() first argument must be an array, got Map(Field { name: "entries", data_type: Struct([Field { name: "key", data_type: Utf8 }, Field { name: ...
(+1)        1 IllegalArgumentException: invalid argument: transform_values() first argument must be an array, got Map(Field { name: "entries", data_type: Struct([Field { name: "key", data_type: Utf8 }, Field { name...
            1 PySparkNotImplementedError: [NOT_IMPLEMENTED] foreach() is not implemented.
            1 PySparkNotImplementedError: [NOT_IMPLEMENTED] foreachPartition() is not implemented.
            1 PySparkNotImplementedError: [NOT_IMPLEMENTED] localCheckpoint() is not implemented.
            1 PySparkNotImplementedError: [NOT_IMPLEMENTED] sparkContext() is not implemented.
            1 PySparkNotImplementedError: [NOT_IMPLEMENTED] toJSON() is not implemented.
            1 PythonException:  AttributeError: 'NoneType' object has no attribute 'partitionId'
            1 PythonException:  AttributeError: 'list' object has no attribute 'x'
            1 PythonException:  AttributeError: 'list' object has no attribute 'y'
            1 SparkRuntimeException: Cast error: Cannot cast string '1997/02/28 10:30:00' to value of Date32 type
            1 SparkRuntimeException: Invalid argument error: 83.140 is too large to store in a Decimal128 of precision 4. Max is 9.999
            1 SparkRuntimeException: Invalid argument error: column types must match schema types, expected Int64 but found List(Int64) at column index 1
            1 SparkRuntimeException: Invalid argument error: column types must match schema types, expected LargeUtf8 but found Utf8 at column index 0
            1 SparkRuntimeException: Json error: Not valid JSON: EOF while parsing a list at line 1 column 1
            1 SparkRuntimeException: Json error: Not valid JSON: expected value at line 1 column 2
            1 SparkRuntimeException: Parser error: Error while parsing value '0
            1 UnsupportedOperationException: Aggregate can not be used as a sliding accumulator because `retract_batch` is not implemented: avg@bjwplt75b8s8lnn9byry0pm4s(#9) PARTITION BY [#8] ORDER BY [#9 ASC NULLS...
            1 UnsupportedOperationException: Aggregate can not be used as a sliding accumulator because `retract_batch` is not implemented: avg@bjwplt75b8s8lnn9byry0pm4s(plus_one@5u7wzy1x1zf4loxm9n1r277lw(#9)) PART...
            1 UnsupportedOperationException: PlanNode::ClearCache
            1 UnsupportedOperationException: PlanNode::IsCached
            1 UnsupportedOperationException: PlanNode::RecoverPartitions
            1 UnsupportedOperationException: SHOW FUNCTIONS
            1 UnsupportedOperationException: Support for 'approx_distinct' for data type Float64 is not implemented
(+1)        1 UnsupportedOperationException: Unsupported CAST from List(non-null Struct("0": Int64, "1": Int64)) to List(non-null Struct("col0": Int64, "col1": Int64))
            1 UnsupportedOperationException: apply in pandas with state
            1 UnsupportedOperationException: bucketing for writing listing table format
            1 UnsupportedOperationException: deduplicate within watermark
            1 UnsupportedOperationException: function: input_file_block_length
            1 UnsupportedOperationException: function: input_file_block_start
            1 UnsupportedOperationException: function: reflect
            1 UnsupportedOperationException: function: session_window
            1 UnsupportedOperationException: function: to_char
            1 UnsupportedOperationException: function: to_csv
            1 UnsupportedOperationException: function: to_varchar
            1 UnsupportedOperationException: handle analyze semantic hash
(+1)        1 UnsupportedOperationException: lambda function: aggregate
(+1)        1 UnsupportedOperationException: lambda function: array_sort with comparator
(+1)        1 UnsupportedOperationException: lambda function: forall
(+1)        1 UnsupportedOperationException: lambda function: reduce
(+1)        1 UnsupportedOperationException: lambda function: transform
(+1)        1 UnsupportedOperationException: lambda function: zip_with
            1 UnsupportedOperationException: unknown aggregate function: count_min_sketch
            1 UnsupportedOperationException: unknown aggregate function: grouping_id
            1 UnsupportedOperationException: unknown function: distributed_sequence_id
            1 ValueError: The column label 'id' is not unique.
            1 ValueError: The column label 'struct' is not unique.
(-1)        0 AnalysisException: UNION queries have different number of columns: left has 3 columns whereas right has 2 columns
(-1)        0 AssertionError: "Database 'memory:24697781-ba52-4867-aa11-fb5dcae96c7a' dropped." does not match "No table format found for: jdbc"
(-1)        0 AssertionError: "Database 'memory:81f1aa49-b7d0-4a60-85d1-d06c443fdc08' dropped." does not match "No table format found for: jdbc"
(-1)        0 SparkRuntimeException: Csv error: incorrect number of fields for line 1, expected 0 got more than 0
(-15)       0 UnsupportedOperationException: lambda function
(-2)        0 received metadata size exceeds soft limit (9851 vs. 8192);  :status:42B content-type:60B grpc-status:45B grpc-message:4430B grpc-status-details-bin:3920B date:65B ValueError: Code in Status proto (Sta...
Passed Tests Diff 
--- before.txt	2026-05-17 20:30:21.501299970 +0000
+++ after.txt	2026-05-17 20:30:21.672301076 +0000
@@ -104 +103,0 @@
-pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.summary
@@ -255,0 +255 @@
+pyspark/sql/functions.py::pyspark.sql.functions.filter
@@ -407 +406,0 @@
-pyspark/sql/functions.py::pyspark.sql.functions.stack
@@ -508,0 +508 @@
+pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_csv
@@ -903 +902,0 @@
-pyspark/sql/tests/connect/test_parity_functions.py::FunctionsParityTests::test_lit_list
Failed Tests 
pyspark/sql/catalog.py::pyspark.sql.catalog.Catalog.cacheTable
pyspark/sql/catalog.py::pyspark.sql.catalog.Catalog.clearCache
pyspark/sql/catalog.py::pyspark.sql.catalog.Catalog.createTable
pyspark/sql/catalog.py::pyspark.sql.catalog.Catalog.functionExists
pyspark/sql/catalog.py::pyspark.sql.catalog.Catalog.getFunction
pyspark/sql/catalog.py::pyspark.sql.catalog.Catalog.isCached
pyspark/sql/catalog.py::pyspark.sql.catalog.Catalog.listFunctions
pyspark/sql/catalog.py::pyspark.sql.catalog.Catalog.recoverPartitions
pyspark/sql/catalog.py::pyspark.sql.catalog.Catalog.refreshByPath
pyspark/sql/catalog.py::pyspark.sql.catalog.Catalog.refreshTable
pyspark/sql/catalog.py::pyspark.sql.catalog.Catalog.uncacheTable
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame._ipython_key_completions_
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame._joinAsOf
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.checkpoint
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.coalesce
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.colRegex
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.dropDuplicatesWithinWatermark
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.explain
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.foreach
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.foreachPartition
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.hint
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.inputFiles
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.localCheckpoint
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.observe
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.randomSplit
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.repartition
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.repartitionByRange
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.sameSemantics
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.sampleBy
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.storageLevel
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.summary
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.toJSON
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.withWatermark
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrameStatFunctions.sampleBy
pyspark/sql/functions.py::pyspark.sql.functions.aggregate
pyspark/sql/functions.py::pyspark.sql.functions.approx_percentile
pyspark/sql/functions.py::pyspark.sql.functions.array_intersect
pyspark/sql/functions.py::pyspark.sql.functions.array_position
pyspark/sql/functions.py::pyspark.sql.functions.array_sort
pyspark/sql/functions.py::pyspark.sql.functions.count_min_sketch
pyspark/sql/functions.py::pyspark.sql.functions.exists
pyspark/sql/functions.py::pyspark.sql.functions.first
pyspark/sql/functions.py::pyspark.sql.functions.forall
pyspark/sql/functions.py::pyspark.sql.functions.grouping_id
pyspark/sql/functions.py::pyspark.sql.functions.hll_sketch_agg
pyspark/sql/functions.py::pyspark.sql.functions.hll_sketch_estimate
pyspark/sql/functions.py::pyspark.sql.functions.hll_union
pyspark/sql/functions.py::pyspark.sql.functions.hll_union_agg
pyspark/sql/functions.py::pyspark.sql.functions.input_file_block_length
pyspark/sql/functions.py::pyspark.sql.functions.input_file_block_start
pyspark/sql/functions.py::pyspark.sql.functions.input_file_name
pyspark/sql/functions.py::pyspark.sql.functions.kurtosis
pyspark/sql/functions.py::pyspark.sql.functions.map_entries
pyspark/sql/functions.py::pyspark.sql.functions.map_filter
pyspark/sql/functions.py::pyspark.sql.functions.map_zip_with
pyspark/sql/functions.py::pyspark.sql.functions.mode
pyspark/sql/functions.py::pyspark.sql.functions.monotonically_increasing_id
pyspark/sql/functions.py::pyspark.sql.functions.percentile
pyspark/sql/functions.py::pyspark.sql.functions.percentile_approx
pyspark/sql/functions.py::pyspark.sql.functions.randn
pyspark/sql/functions.py::pyspark.sql.functions.reduce
pyspark/sql/functions.py::pyspark.sql.functions.reflect
pyspark/sql/functions.py::pyspark.sql.functions.regexp_extract
pyspark/sql/functions.py::pyspark.sql.functions.regexp_instr
pyspark/sql/functions.py::pyspark.sql.functions.regr_avgy
pyspark/sql/functions.py::pyspark.sql.functions.regr_intercept
pyspark/sql/functions.py::pyspark.sql.functions.regr_r2
pyspark/sql/functions.py::pyspark.sql.functions.regr_slope
pyspark/sql/functions.py::pyspark.sql.functions.regr_sxy
pyspark/sql/functions.py::pyspark.sql.functions.regr_syy
pyspark/sql/functions.py::pyspark.sql.functions.schema_of_json
pyspark/sql/functions.py::pyspark.sql.functions.session_window
pyspark/sql/functions.py::pyspark.sql.functions.stack
pyspark/sql/functions.py::pyspark.sql.functions.to_char
pyspark/sql/functions.py::pyspark.sql.functions.to_csv
pyspark/sql/functions.py::pyspark.sql.functions.to_varchar
pyspark/sql/functions.py::pyspark.sql.functions.transform
pyspark/sql/functions.py::pyspark.sql.functions.transform_keys
pyspark/sql/functions.py::pyspark.sql.functions.transform_values
pyspark/sql/functions.py::pyspark.sql.functions.window
pyspark/sql/functions.py::pyspark.sql.functions.window_time
pyspark/sql/functions.py::pyspark.sql.functions.zip_with
pyspark/sql/tests/connect/client/test_artifact.py::ArtifactTests::test_add_archive
pyspark/sql/tests/connect/client/test_artifact.py::ArtifactTests::test_add_file
pyspark/sql/tests/connect/client/test_artifact.py::ArtifactTests::test_add_pyfile
pyspark/sql/tests/connect/client/test_artifact.py::ArtifactTests::test_add_zipped_package
pyspark/sql/tests/connect/client/test_artifact.py::ArtifactTests::test_basic_requests
pyspark/sql/tests/connect/client/test_artifact.py::ArtifactTests::test_cache_artifact
pyspark/sql/tests/connect/client/test_artifact.py::ArtifactTests::test_copy_from_local_to_fs
pyspark/sql/tests/connect/client/test_artifact.py::LocalClusterArtifactTests::test_add_archive
pyspark/sql/tests/connect/client/test_artifact.py::LocalClusterArtifactTests::test_add_file
pyspark/sql/tests/connect/client/test_artifact.py::LocalClusterArtifactTests::test_add_pyfile
pyspark/sql/tests/connect/client/test_artifact.py::LocalClusterArtifactTests::test_add_zipped_package
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_collect
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_collect_timestamp
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_column_regexp
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_create_global_temp_view
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_deduplicate_within_watermark_in_batch
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_describe
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_explain_string
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_grouped_data
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_hint
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_input_files
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_join_hint
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_json
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_multi_paths
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_numeric_aggregation
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_observe
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_orc
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_random_split
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_same_semantics
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_schema
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_semantic_hash
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_simple_udt_from_read
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_sql_with_command
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_sql_with_pos_args
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_stat_approx_quantile
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_stat_freq_items
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_stat_sample_by
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_streaming_local_relation
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_tail
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_to
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_with_local_list
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_with_local_ndarray
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectBasicTests::test_write_operations
pyspark/sql/tests/connect/test_connect_basic.py::SparkConnectSessionTests::test_error_stack_trace
pyspark/sql/tests/connect/test_connect_column.py::SparkConnectColumnTests::test_column_arithmetic_ops
pyspark/sql/tests/connect/test_connect_column.py::SparkConnectColumnTests::test_column_field_ops
pyspark/sql/tests/connect/test_connect_column.py::SparkConnectColumnTests::test_decimal
pyspark/sql/tests/connect/test_connect_column.py::SparkConnectColumnTests::test_distributed_sequence_id
pyspark/sql/tests/connect/test_connect_function.py::SparkConnectFunctionTests::test_aggregation_functions
pyspark/sql/tests/connect/test_connect_function.py::SparkConnectFunctionTests::test_collection_functions
pyspark/sql/tests/connect/test_connect_function.py::SparkConnectFunctionTests::test_csv_functions
pyspark/sql/tests/connect/test_connect_function.py::SparkConnectFunctionTests::test_date_ts_functions
pyspark/sql/tests/connect/test_connect_function.py::SparkConnectFunctionTests::test_generator_functions
pyspark/sql/tests/connect/test_connect_function.py::SparkConnectFunctionTests::test_json_functions
pyspark/sql/tests/connect/test_connect_function.py::SparkConnectFunctionTests::test_lambda_functions
pyspark/sql/tests/connect/test_connect_function.py::SparkConnectFunctionTests::test_map_collection_functions
pyspark/sql/tests/connect/test_connect_function.py::SparkConnectFunctionTests::test_math_functions
pyspark/sql/tests/connect/test_connect_function.py::SparkConnectFunctionTests::test_nested_lambda_function
pyspark/sql/tests/connect/test_connect_function.py::SparkConnectFunctionTests::test_normal_functions
pyspark/sql/tests/connect/test_connect_function.py::SparkConnectFunctionTests::test_string_functions_multi_args
pyspark/sql/tests/connect/test_connect_function.py::SparkConnectFunctionTests::test_time_window_functions
pyspark/sql/tests/connect/test_connect_function.py::SparkConnectFunctionTests::test_udf
pyspark/sql/tests/connect/test_connect_function.py::SparkConnectFunctionTests::test_udtf
pyspark/sql/tests/connect/test_connect_function.py::SparkConnectFunctionTests::test_window_functions
pyspark/sql/tests/connect/test_parity_arrow.py::ArrowParityTests::test_createDataFrame_duplicate_field_names
pyspark/sql/tests/connect/test_parity_arrow.py::ArrowParityTests::test_pandas_self_destruct
pyspark/sql/tests/connect/test_parity_arrow.py::ArrowParityTests::test_toPandas_duplicate_field_names
pyspark/sql/tests/connect/test_parity_arrow_python_udf.py::ArrowPythonUDFParityTests::test_udf_with_input_file_name
pyspark/sql/tests/connect/test_parity_arrow_python_udf.py::UDFParityTests::test_udf_with_input_file_name
pyspark/sql/tests/connect/test_parity_catalog.py::CatalogParityTests::test_function_exists
pyspark/sql/tests/connect/test_parity_catalog.py::CatalogParityTests::test_get_function
pyspark/sql/tests/connect/test_parity_catalog.py::CatalogParityTests::test_list_functions
pyspark/sql/tests/connect/test_parity_catalog.py::CatalogParityTests::test_list_tables
pyspark/sql/tests/connect/test_parity_catalog.py::CatalogParityTests::test_refresh_table
pyspark/sql/tests/connect/test_parity_catalog.py::CatalogParityTests::test_table_cache
pyspark/sql/tests/connect/test_parity_dataframe.py::DataFrameParityTests::test_cache_dataframe
pyspark/sql/tests/connect/test_parity_dataframe.py::DataFrameParityTests::test_cache_table
pyspark/sql/tests/connect/test_parity_dataframe.py::DataFrameParityTests::test_duplicate_field_names
pyspark/sql/tests/connect/test_parity_dataframe.py::DataFrameParityTests::test_extended_hint_types
pyspark/sql/tests/connect/test_parity_dataframe.py::DataFrameParityTests::test_freqItems
pyspark/sql/tests/connect/test_parity_dataframe.py::DataFrameParityTests::test_generic_hints
pyspark/sql/tests/connect/test_parity_dataframe.py::DataFrameParityTests::test_input_files
pyspark/sql/tests/connect/test_parity_dataframe.py::DataFrameParityTests::test_to
pyspark/sql/tests/connect/test_parity_dataframe.py::DataFrameParityTests::test_to_pandas
pyspark/sql/tests/connect/test_parity_datasources.py::DataSourcesParityTests::test_checking_csv_header
pyspark/sql/tests/connect/test_parity_datasources.py::DataSourcesParityTests::test_encoding_json
pyspark/sql/tests/connect/test_parity_datasources.py::DataSourcesParityTests::test_ignore_column_of_all_nulls
pyspark/sql/tests/connect/test_parity_datasources.py::DataSourcesParityTests::test_jdbc
pyspark/sql/tests/connect/test_parity_datasources.py::DataSourcesParityTests::test_jdbc_format
pyspark/sql/tests/connect/test_parity_datasources.py::DataSourcesParityTests::test_linesep_json
pyspark/sql/tests/connect/test_parity_datasources.py::DataSourcesParityTests::test_multiline_json
pyspark/sql/tests/connect/test_parity_datasources.py::DataSourcesParityTests::test_read_multiple_orc_file
pyspark/sql/tests/connect/test_parity_errors.py::ErrorsParityTests::test_array_index_out_of_bounds_exception
pyspark/sql/tests/connect/test_parity_errors.py::ErrorsParityTests::test_date_time_exception
pyspark/sql/tests/connect/test_parity_errors.py::ErrorsParityTests::test_number_format_exception
pyspark/sql/tests/connect/test_parity_errors.py::ErrorsParityTests::test_spark_runtime_exception
pyspark/sql/tests/connect/test_parity_functions.py::FunctionsParityTests::test_approxQuantile
pyspark/sql/tests/connect/test_parity_functions.py::FunctionsParityTests::test_functions_broadcast
pyspark/sql/tests/connect/test_parity_functions.py::FunctionsParityTests::test_inline
pyspark/sql/tests/connect/test_parity_functions.py::FunctionsParityTests::test_input_file_name_udf
pyspark/sql/tests/connect/test_parity_functions.py::FunctionsParityTests::test_lit_list
pyspark/sql/tests/connect/test_parity_functions.py::FunctionsParityTests::test_nested_higher_order_function
pyspark/sql/tests/connect/test_parity_functions.py::FunctionsParityTests::test_window_time
pyspark/sql/tests/connect/test_parity_pandas_grouped_map.py::GroupedApplyInPandasTests::test_grouped_over_window
pyspark/sql/tests/connect/test_parity_pandas_grouped_map.py::GroupedApplyInPandasTests::test_grouped_over_window_with_key
pyspark/sql/tests/connect/test_parity_pandas_grouped_map_with_state.py::GroupedApplyInPandasWithStateTests::test_apply_in_pandas_with_state_python_worker_random_failure
pyspark/sql/tests/connect/test_parity_pandas_map.py::MapInPandasParityTests::test_large_variable_types
pyspark/sql/tests/connect/test_parity_pandas_udf_scalar.py::PandasUDFScalarParityTests::test_scalar_iter_udf_init
pyspark/sql/tests/connect/test_parity_pandas_udf_scalar.py::PandasUDFScalarParityTests::test_vectorized_udf_check_config
pyspark/sql/tests/connect/test_parity_pandas_udf_scalar.py::PandasUDFScalarParityTests::test_vectorized_udf_invalid_length
pyspark/sql/tests/connect/test_parity_pandas_udf_window.py::PandasUDFWindowParityTests::test_bounded_mixed
pyspark/sql/tests/connect/test_parity_pandas_udf_window.py::PandasUDFWindowParityTests::test_bounded_simple
pyspark/sql/tests/connect/test_parity_pandas_udf_window.py::PandasUDFWindowParityTests::test_shrinking_window
pyspark/sql/tests/connect/test_parity_pandas_udf_window.py::PandasUDFWindowParityTests::test_sliding_window
pyspark/sql/tests/connect/test_parity_readwriter.py::ReadwriterParityTests::test_bucketed_write
pyspark/sql/tests/connect/test_parity_readwriter.py::ReadwriterParityTests::test_insert_into
pyspark/sql/tests/connect/test_parity_readwriter.py::ReadwriterParityTests::test_save_and_load
pyspark/sql/tests/connect/test_parity_readwriter.py::ReadwriterParityTests::test_save_and_load_builder
pyspark/sql/tests/connect/test_parity_readwriter.py::ReadwriterV2ParityTests::test_create_without_provider
pyspark/sql/tests/connect/test_parity_readwriter.py::ReadwriterV2ParityTests::test_table_overwrite
pyspark/sql/tests/connect/test_parity_types.py::TypesParityTests::test_cast_to_string_with_udt
pyspark/sql/tests/connect/test_parity_types.py::TypesParityTests::test_cast_to_udt_with_udt
pyspark/sql/tests/connect/test_parity_types.py::TypesParityTests::test_complex_nested_udt_in_df
pyspark/sql/tests/connect/test_parity_types.py::TypesParityTests::test_negative_decimal
pyspark/sql/tests/connect/test_parity_types.py::TypesParityTests::test_parquet_with_udt
pyspark/sql/tests/connect/test_parity_types.py::TypesParityTests::test_udf_with_udt
pyspark/sql/tests/connect/test_parity_types.py::TypesParityTests::test_udt_with_none
pyspark/sql/tests/connect/test_parity_types.py::TypesParityTests::test_yearmonth_interval_type
pyspark/sql/tests/connect/test_parity_udf.py::UDFParityTests::test_udf_with_input_file_name
pyspark/sql/tests/connect/test_parity_udtf.py::ArrowUDTFParityTests::test_udtf_arrow_sql_conf
pyspark/sql/tests/connect/test_parity_udtf.py::ArrowUDTFParityTests::test_udtf_with_table_argument_multiple
pyspark/sql/tests/connect/test_parity_udtf.py::UDTFParityTests::test_udtf_with_table_argument_multiple
pyspark/sql/tests/connect/test_utils.py::ConnectUtilsTests::test_assert_approx_equal_decimaltype_custom_rtol_pass
pyspark/sql/tests/connect/test_utils.py::ConnectUtilsTests::test_assert_equal_nested_struct_str_duplicate

@github-actions

github-actions Bot commented Feb 5, 2026 •
edited
Loading

Copy link
Copy Markdown

Ibis Test Report

Commit Information

Commit Revision Branch
After 7a37f52 refs/pull/1320/merge
Before 9e17a1d refs/heads/main

Test Summary

Suite Commit Failed Passed Skipped Warnings Time (s)
test-ibis After 73 1515 166 4518 246.57
Before 80 1507 166 4522 185.68

Test Details

Error Counts 
(-7)       74 Total
(+8)       39 Total Unique
-------- ---- ----------------------------------------------------------------------------------------------------------
(+13)      13 UnsupportedOperationException: lambda function: transform
            8 SparkRuntimeException: MERGE planning expects a pre-expanded logical plan (RowLevelWriteNode). Ensure expand_row_level_op is enabled; MERGE is currently only supported for lakehouse tables.
            5 IllegalArgumentException: invalid argument: found TRUNCATE at 0:8 expected something else, ';', statement, or end of input
            4 AssertionError: Series are different
(+4)        4 UnsupportedOperationException: lambda function: aggregate
(+2)        2 AnalysisException: Internal error: Function 'approx_percentile_cont' failed to match any signature, errors: Error during planning: Function 'approx_percentile_cont' requires Float64, but received List...
            2 AssertionError
            2 AssertionError: DataFrame.iloc[:, 0] (column name="result_col") are different
            2 AssertionError: DataFrame.iloc[:, 0] (column name="year_id") are different
            2 IllegalArgumentException: invalid argument: found PARTITIONS at 5:15 expected 'DATABASES', 'SCHEMAS', 'NAMESPACES', 'CATALOGS', 'TABLES', 'TABLE', 'CREATE', 'COLUMNS', 'VIEWS', or 'FUNCTIONS'
            2 assert ibis.Schema {... float64\n} == ibis.Schema {... float64\n} Full diff: ibis.Schema { carat float64 cut string color string clarity string depth float64 table float64 - price int32 ? ^^ + price i...
            1 AnalysisException: Catalog not found: local
(+1)        1 AnalysisException: Database not found: ibis_database_ldovegbo5vcyfhryhcjsczgh6m
            1 AnalysisException: error in SQL parser: found : at 2:3 expected '0', '1', '2', '3', '4', '5', '6', '7', '8', or '9'
            1 AssertionError: DataFrame are different
            1 AssertionError: DataFrame.iloc[:, 0] (column name="id") are different
            1 AssertionError: DataFrame.iloc[:, 0] (column name="playerID") are different
            1 AssertionError: DataFrame.iloc[:, 1] (column name="collect_udf") are different
            1 AssertionError: Series NA mask are different
(+1)        1 AssertionError: assert 'ibis_cached_n62hgq4qm5gsfpxsimj563kqki' not in ['array_types', 'astronauts', 'awards_players', 'basic_table', 'batting', 'complicated', ...]
            1 Failed: DID NOT RAISE <class 'pyspark.errors.exceptions.base.AnalysisException'>
            1 IllegalArgumentException: invalid argument: input schema for INSERT has 1 fields, but table schema has 2 fields
            1 SparkRuntimeException: Cast error: Casting from Date32 to Float64 not supported
            1 SparkRuntimeException: Error during planning: expr type Struct("StructColumn({'x': xs, 'y': ys})": non-null Struct("x": non-null Int32, "y": non-null Int32)) can't cast to Struct("x": Int64, "y": Int6...
(+1)        1 SparkRuntimeException: This feature is not implemented: Unsupported CAST from List(non-null Struct("0": Int32, "1": Int64)) to List(Struct("f1": Int64, "f2": Int64))
(+1)        1 SparkRuntimeException: This feature is not implemented: Unsupported CAST from List(non-null Struct("0": Int64, "1": Int32)) to List(Struct("f1": Int64, "f2": Int64))
(+1)        1 SparkRuntimeException: This feature is not implemented: Unsupported CAST from List(non-null Struct("0": Int64, "1": Int64)) to List(Struct("f1": Int64, "f2": Int64))
            1 TypeError: Cannot convert pyarrow.lib.ChunkedArray to pyarrow.lib.Array
            1 UnsupportedOperationException: CommandNode::AnalyzeTable
            1 UnsupportedOperationException: Physical plan does not support logical expression AggregateFunction(AggregateFunction { func: AggregateUDF { inner: ArrayAgg { signature: Signature { type_signature: Any...
            1 UnsupportedOperationException: Physical plan does not support logical expression InSubquery(InSubquery { expr: Column(Column { relation: Some(Bare { table: "t0" }), name: "#0" }), subquery: <subquery>...
            1 UnsupportedOperationException: Physical plan does not support logical expression InSubquery(InSubquery { expr: Column(Column { relation: Some(Bare { table: "t0" }), name: "#1" }), subquery: <subquery>...
(+1)        1 UnsupportedOperationException: Unsupported CAST from List(non-null Struct("array1": Int64, "array2": Int64)) to List(Struct("f1": Int64, "f2": Int64))
(+1)        1 UnsupportedOperationException: Unsupported CAST from List(non-null Struct("x": Int64, "x": Int64)) to List(Struct("f1": Int64, "f2": Int64))
(+1)        1 UnsupportedOperationException: Unsupported CAST from List(non-null Struct("x": Int64, "x": Int64, "x": Int64, "x": Int64, "x": Int64, "x": Int64)) to List(Struct("f1": Int64, "f2": Int64, "f3": Int64,...
            1 UnsupportedOperationException: unsupported ALTER TABLE operation
(+1)        1 assert 45.45000000000001 == 45.45
            1 assert frozenset({None}) == frozenset({None, 47}) Extra items in the right set: 47 Full diff: frozenset({ None, - 47, })
            1 assert {0.0, 1.0, 2.0, 3.0} == {1, 2, 3} Extra items in the left set: 0.0 Full diff: { + 0.0, - 1, + 1.0, ? ++ - 2, + 2.0, ? ++ - 3, + 3.0, ? ++ }
(-1)        0 AnalysisException: Database not found: ibis_database_xebvxzilije2ncsfjs6bjnyqii
(-2)        0 AnalysisException: Failed to coerce arguments to satisfy a call to 'approx_percentile_cont' function: coercion from Float64, List(Decimal128(2, 2)) to the signature OneOf([Exact([Int8, Float64]), Exac...
(-1)        0 AssertionError: assert 'ibis_cached_u54u4tdsbrd35ibklbvwrjyck4' not in ['array_types', 'astronauts', 'awards_players', 'basic_table', 'batting', 'complicated', ...]
(-31)       0 UnsupportedOperationException: lambda function
Passed Tests Diff 
--- before.txt	2026-05-17 20:31:52.901389910 +0000
+++ after.txt	2026-05-17 20:31:53.097452394 +0000
@@ -198,0 +199,14 @@
+ibis/backends/tests/test_array.py::test_array_filter[pyspark-deferred-no_nulls]
+ibis/backends/tests/test_array.py::test_array_filter[pyspark-deferred-nulls]
+ibis/backends/tests/test_array.py::test_array_filter[pyspark-lambda-no_nulls]
+ibis/backends/tests/test_array.py::test_array_filter[pyspark-lambda-nulls]
+ibis/backends/tests/test_array.py::test_array_filter[pyspark-partial-no_nulls]
+ibis/backends/tests/test_array.py::test_array_filter[pyspark-partial-nulls]
+ibis/backends/tests/test_array.py::test_array_filter_with_index[pyspark-lambda-no_nulls]
+ibis/backends/tests/test_array.py::test_array_filter_with_index[pyspark-lambda-nulls]
+ibis/backends/tests/test_array.py::test_array_filter_with_index[pyspark-partial-no_nulls]
+ibis/backends/tests/test_array.py::test_array_filter_with_index[pyspark-partial-nulls]
+ibis/backends/tests/test_array.py::test_array_filter_with_index_lambda[pyspark-lambda-no_nulls]
+ibis/backends/tests/test_array.py::test_array_filter_with_index_lambda[pyspark-lambda-nulls]
+ibis/backends/tests/test_array.py::test_array_filter_with_index_lambda[pyspark-partial-no_nulls]
+ibis/backends/tests/test_array.py::test_array_filter_with_index_lambda[pyspark-partial-nulls]
@@ -320,6 +333,0 @@
-ibis/backends/tests/test_array.py::test_zip[pyspark-1]
-ibis/backends/tests/test_array.py::test_zip[pyspark-5]
-ibis/backends/tests/test_array.py::test_zip_null[pyspark-non-null-zip-null]
-ibis/backends/tests/test_array.py::test_zip_null[pyspark-null-zip-non-null]
-ibis/backends/tests/test_array.py::test_zip_null[pyspark-null-zip-null]
-ibis/backends/tests/test_array.py::test_zip_unnest_lift[pyspark]
@@ -1507 +1514,0 @@
-ibis/backends/tests/test_vectorized_udf.py::test_reduction_udf[pyspark]
Failed Tests 
ibis/backends/pyspark/tests/test_basic.py::test_group_by
ibis/backends/pyspark/tests/test_client.py::test_catalog_db_args
ibis/backends/pyspark/tests/test_client.py::test_create_table_with_partition_and_catalog
ibis/backends/pyspark/tests/test_client.py::test_create_table_with_partition_no_catalog
ibis/backends/pyspark/tests/test_ddl.py::test_compute_stats
ibis/backends/pyspark/tests/test_ddl.py::test_drop_non_empty_database
ibis/backends/pyspark/tests/test_ddl.py::test_insert_table
ibis/backends/pyspark/tests/test_ddl.py::test_truncate_table
ibis/backends/tests/test_aggregation.py::test_aggregate_list_like[pyspark-list]
ibis/backends/tests/test_aggregation.py::test_aggregate_list_like[pyspark-ndarray]
ibis/backends/tests/test_aggregation.py::test_aggregate_mixed_udf[pyspark]
ibis/backends/tests/test_aggregation.py::test_approx_quantile[pyspark-True-False]
ibis/backends/tests/test_aggregation.py::test_approx_quantile[pyspark-True-True]
ibis/backends/tests/test_aggregation.py::test_date_quantile[pyspark]
ibis/backends/tests/test_aggregation.py::test_group_concat_over_window[pyspark]
ibis/backends/tests/test_array.py::test_array_agg_numeric[pyspark-no-nulls-means]
ibis/backends/tests/test_array.py::test_array_agg_numeric[pyspark-no-nulls-sums]
ibis/backends/tests/test_array.py::test_array_agg_numeric[pyspark-nulls-means]
ibis/backends/tests/test_array.py::test_array_agg_numeric[pyspark-nulls-sums]
ibis/backends/tests/test_array.py::test_array_map[pyspark-deferred-no_nulls]
ibis/backends/tests/test_array.py::test_array_map[pyspark-deferred-nulls]
ibis/backends/tests/test_array.py::test_array_map[pyspark-lambda-no_nulls]
ibis/backends/tests/test_array.py::test_array_map[pyspark-lambda-nulls]
ibis/backends/tests/test_array.py::test_array_map[pyspark-partial-no_nulls]
ibis/backends/tests/test_array.py::test_array_map[pyspark-partial-nulls]
ibis/backends/tests/test_array.py::test_array_map_with_conflicting_names[pyspark]
ibis/backends/tests/test_array.py::test_array_map_with_index[pyspark-lambda-no_nulls]
ibis/backends/tests/test_array.py::test_array_map_with_index[pyspark-lambda-nulls]
ibis/backends/tests/test_array.py::test_array_map_with_index[pyspark-partial-no_nulls]
ibis/backends/tests/test_array.py::test_array_map_with_index[pyspark-partial-nulls]
ibis/backends/tests/test_array.py::test_complex_array_map[pyspark]
ibis/backends/tests/test_array.py::test_table_unnest_column_expr[pyspark]
ibis/backends/tests/test_array.py::test_zip[pyspark-1]
ibis/backends/tests/test_array.py::test_zip[pyspark-5]
ibis/backends/tests/test_array.py::test_zip_null[pyspark-non-null-zip-null]
ibis/backends/tests/test_array.py::test_zip_null[pyspark-null-zip-non-null]
ibis/backends/tests/test_array.py::test_zip_null[pyspark-null-zip-null]
ibis/backends/tests/test_array.py::test_zip_unnest_lift[pyspark]
ibis/backends/tests/test_client.py::test_create_table_overwrite_temp[pyspark-no temp, overwrite]
ibis/backends/tests/test_client.py::test_insert_into_table_missing_columns[pyspark]
ibis/backends/tests/test_client.py::test_insert_overwrite_from_dataframe[pyspark]
ibis/backends/tests/test_client.py::test_insert_overwrite_from_expr[pyspark]
ibis/backends/tests/test_client.py::test_insert_overwrite_from_list[pyspark]
ibis/backends/tests/test_client.py::test_rename_table[pyspark]
ibis/backends/tests/test_client.py::test_upsert_from_dataframe[pyspark]
ibis/backends/tests/test_client.py::test_upsert_from_expr[pyspark-False]
ibis/backends/tests/test_client.py::test_upsert_from_expr[pyspark-True]
ibis/backends/tests/test_client.py::test_upsert_from_memtable[pyspark-sch0-expectation0]
ibis/backends/tests/test_client.py::test_upsert_from_memtable[pyspark-sch1-expectation1]
ibis/backends/tests/test_client.py::test_upsert_from_memtable[pyspark-sch2-expectation2]
ibis/backends/tests/test_client.py::test_upsert_from_memtable[pyspark-sch3-expectation3]
ibis/backends/tests/test_client.py::test_upsert_from_memtable[pyspark-sch4-expectation4]
ibis/backends/tests/test_export.py::test_table_to_csv[pyspark]
ibis/backends/tests/test_expr_caching.py::test_persist_expression_contextmanager[pyspark]
ibis/backends/tests/test_expr_caching.py::test_persist_expression_release[pyspark]
ibis/backends/tests/test_expr_caching.py::test_persist_expression_repeated_cache[pyspark]
ibis/backends/tests/test_generic.py::test_isin_uncorrelated[pyspark]
ibis/backends/tests/test_generic.py::test_isin_uncorrelated_simple[pyspark]
ibis/backends/tests/test_io.py::test_read_csv[pyspark-default]
ibis/backends/tests/test_io.py::test_read_csv[pyspark-file_name]
ibis/backends/tests/test_join.py::test_join_with_pandas[pyspark]
ibis/backends/tests/test_json.py::test_json_getitem_array[pyspark]
ibis/backends/tests/test_struct.py::test_field_overwrite_always_prefers_unpacked[pyspark]
ibis/backends/tests/test_struct.py::test_isin_struct[pyspark]
ibis/backends/tests/test_struct.py::test_single_field[pyspark-a]
ibis/backends/tests/test_struct.py::test_single_field[pyspark-b]
ibis/backends/tests/test_struct.py::test_single_field[pyspark-c]
ibis/backends/tests/test_temporal.py::test_delta[pyspark-time]
ibis/backends/tests/test_udf.py::test_vectorized_udf[pyspark-add_one_pandas]
ibis/backends/tests/test_udf.py::test_vectorized_udf[pyspark-add_one_pyarrow]
ibis/backends/tests/test_vectorized_udf.py::test_reduction_udf[pyspark]
ibis/backends/tests/test_window.py::test_ungrouped_unbounded_window[pyspark-unordered-lag]
ibis/backends/tests/test_window.py::test_ungrouped_unbounded_window[pyspark-unordered-lead]

@codecov

codecov Bot commented Feb 5, 2026 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 51.55945% with 994 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
crates/sail-execution/src/codec.rs 39.15% 101 Missing ⚠️
...ates/sail-plan/src/resolver/expression/function.rs 70.26% 91 Missing ⚠️
...il-function/src/higher_order/spark_array_filter.rs 62.28% 86 Missing ⚠️
...nction/src/scalar/array/spark_array_filter_expr.rs 80.59% 65 Missing ⚠️
crates/sail-cache/src/file_statistics_cache.rs 40.00% 27 Missing ⚠️
crates/sail-plan/src/resolver/expression/cast.rs 30.76% 27 Missing ⚠️
...ail-delta-lake/src/physical_plan/dv_writer_exec.rs 0.00% 22 Missing ⚠️
...il-delta-lake/src/physical_plan/log_replay_exec.rs 0.00% 22 Missing ⚠️
...il-delta-lake/src/physical_plan/relaxed_tz_exec.rs 21.42% 22 Missing ⚠️
crates/sail-plan/src/function/scalar/lambda.rs 84.25% 17 Missing ⚠️
... and 59 more

❗ There is a different number of reports uploaded between BASE (ab3c52c) and HEAD (972d0cd). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (ab3c52c) HEAD (972d0cd)
catalog-integration-tests 1 0 
@@            Coverage Diff             @@
##             main    #1320      +/-   ##
==========================================
- Coverage   76.60%   70.35%   -6.25%     
==========================================
  Files         948      950       +2     
  Lines      141850   153857   +12007     
==========================================
- Hits       108668   108253     -415     
- Misses      33182    45604   +12422
Flag Coverage Δ *Carryforward flag
catalog-integration-tests ?
ibis-tests 16.56% <ø> (-0.02%) ⬇️ Carriedforward from 7b2c1db
python-unit-tests 57.87% <50.47%> (-0.20%) ⬇️
rust-slow-tests 44.50% <ø> (-0.03%) ⬇️ Carriedforward from 7b2c1db
rust-unit-tests 36.98% <41.13%> (-2.91%) ⬇️
spark-tests 31.04% <ø> (-0.18%) ⬇️ Carriedforward from 7b2c1db

*This pull request uses carry forward flags. Click here to find out more.

Files with missing lines Coverage Δ
crates/sail-catalog-system/src/planner.rs 100.00% <ø> (ø)
crates/sail-catalog-system/src/predicate.rs 93.27% <100.00%> (ø)
crates/sail-catalog-system/src/table_source.rs 73.17% <ø> (-26.83%) ⬇️
crates/sail-common-datafusion/src/error.rs 43.95% <ø> (-4.24%) ⬇️
...ail-common-datafusion/src/rename/table_provider.rs 25.00% <ø> (-13.60%) ⬇️
...tes/sail-common-datafusion/src/streaming/source.rs 57.69% <ø> (-29.15%) ⬇️
...sail-data-source/src/formats/binary/file_format.rs 40.00% <ø> (-15.82%) ⬇️
...l-data-source/src/formats/python/table_provider.rs 56.25% <ø> (-30.42%) ⬇️
...s/sail-data-source/src/formats/text/file_format.rs 40.96% <ø> (-14.78%) ⬇️
crates/sail-data-source/src/formats/text/writer.rs 51.20% <ø> (-14.11%) ⬇️
... and 223 more

... and 32 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Apr 10, 2026 •
edited
Loading

Copy link
Copy Markdown

Spark 4.1.1 Test Report

Commit Information

Commit Revision Branch
After 7a37f52 refs/pull/1320/merge
Before 9e17a1d refs/heads/main

Test Summary

Suite Commit Failed Passed Skipped Warnings Time (s)
doctest-catalog After 11 13 1 5.98
Before 11 13 1 5.91
doctest-column After 36 6.90
Before 36 7.26
doctest-dataframe After 31 88 3 2 10.41
Before 30 89 3 2 11.18
doctest-functions After 113 372 10 5 26.55
Before 113 372 10 5 27.84
test-connect After 742 1766 292 487 259.76
Before 745 1763 292 483 255.61

Test Details

Error Counts 
(-2)      897 Total
(+13)     338 Total Unique
-------- ---- ----------------------------------------------------------------------------------------------------------
           60 IllegalArgumentException: missing argument: Python UDF output type
           49 UnsupportedOperationException: unresolved table valued function
(+1)       44 DocTestFailure
           25 IllegalArgumentException: invalid argument: only GroupedMapArrow/GroupedMapPandas UDF is supported in GroupedMap
           22 UnsupportedOperationException: handle add artifacts
           20 AssertionError: 1 != 0 : dict_keys([])
           17 AssertionError: Exception not raised
           14 UnsupportedOperationException: unknown function: kll_sketch_agg_bigint
           13 PySparkAssertionError: [DIFFERENT_PANDAS_DATAFRAME] DataFrames are not almost equal:
(-2)       11 PySparkAssertionError: [DIFFERENT_ROWS] Results do not match: ( 100.00000 % )
           10 SparkRuntimeException: Python error: [test::partitions] NotImplementedError: 
           10 UnsupportedOperationException: unsupported subquery type
           10 UnsupportedOperationException: with watermark
            9 AssertionError: 3 != 0 : []
            9 IllegalArgumentException: expected value at line 1 column 1
            9 IllegalArgumentException: invalid argument: found range at 40:45 expected '->', '.', '(', '[', '::', 'ESCAPE', 'IS', 'NOT', 'IN', '*', '/', '%', 'DIV', '+', '-', '||', '>>>', '>>', '<<', '&', '^', '|'...
            9 UnsupportedOperationException: Physical plan does not support logical expression OuterReferenceColumn(Field { name: "#1", data_type: Int64, nullable: true }, Column { relation: None, name: "#1" })
            9 UnsupportedOperationException: collect metrics
            8 AssertionError
            8 UnsupportedOperationException: unknown function: kll_sketch_agg_double
            8 UnsupportedOperationException: unknown function: kll_sketch_agg_float
(-1)        7 AssertionError: False is not true
            7 UnsupportedOperationException: function: input_file_name
            7 UnsupportedOperationException: unknown function: theta_sketch_agg
            7 UnsupportedOperationException: user defined data type should only exist in a field
            6 AnalysisException: AssertionError: assert False is True
            6 AssertionError: 1 != 0
            6 IllegalArgumentException: data did not match any variant of untagged enum JsonDataType
            6 PythonException:  ValueError: 'toJson' is not in list PySparkAttributeError: [ATTRIBUTE_NOT_SUPPORTED] Attribute `toJson` is not supported.
            6 PythonException:  ValueError: invalid literal for int() with base 10: 'x'
            6 UnsupportedOperationException: PlanNode::CacheTable
            6 UnsupportedOperationException: direct shuffle partition ID expression
            6 UnsupportedOperationException: function: window
            5 AssertionError: AnalysisException not raised
            5 AssertionError: `query_context_type` is required when QueryContext exists. QueryContext: [].
            5 IllegalArgumentException: invalid argument: output mode not supported in group map
            4 AnalysisException: Temporary View not found: t2
            4 AssertionError: 3 != 0 : dict_keys([])
            4 AssertionError: unexpectedly None
            4 SparkRuntimeException: Invalid argument error: column types must match schema types, expected Int32 but found Int64 at column index 0
            4 UnsupportedOperationException: approx quantile
            4 UnsupportedOperationException: freq items
            4 UnsupportedOperationException: transpose
            4 UnsupportedOperationException: unknown aggregate function: hll_sketch_agg
            4 clone session
            3 AnalysisException: Exception: The second argument must be a table argument
            3 AnalysisException: Failed to parse placeholder id: cannot parse integer from empty string
            3 AnalysisException: Invalid Python user-defined table function return type. Expect a struct type, but got Int32.
            3 AssertionError: 0 not greater than or equal to 1
            3 AssertionError: DayTimeIntervalType(0, 3) != DayTimeIntervalType(1, 3)
            3 AssertionError: Lists differ: [Row(<lambda>(struct_of_v)="Row(metadata=b'\\x[787 chars]')")] != [Row(udf='0'), Row(udf='1'), Row(udf='2'), Row[89 chars]'9')]
            3 AssertionError: Lists differ: [Row(udf="Row(metadata=b'\\x01\\x00\\x00', value=b[603 chars]')")] != [Row(udf='0'), Row(udf='1'), Row(udf='2'), Row(udf[85 chars]'9')]
            3 AssertionError: Lists differ: [Row(udf="Row(metadata=b'\\x01\\x01\\x00\\x01a', v[875 chars]')")] != [Row(udf='{"a":"a"}'), Row(udf='{"a":"b"}'), Row(u[165 chars]"}')]
            3 AssertionError: Struc[49 chars]valType(0, 3), True), StructField('name', StringType(), True)]) != Struc[49 chars]valType(1, 3), True), StructField('name', StringType(), True)])
            3 IllegalArgumentException: invalid argument: found PARTITION at 281:290 expected '.', '[', '::', 'ESCAPE', 'IS', 'NOT', 'IN', '*', '/', '%', 'DIV', '+', '-', '||', '>>>', '>>', '<<', '&', '^', '|', '!=...
            3 IllegalArgumentException: invalid argument: found PARTITION at 295:304 expected '.', '[', '::', 'ESCAPE', 'IS', 'NOT', 'IN', '*', '/', '%', 'DIV', '+', '-', '||', '>>>', '>>', '<<', '&', '^', '|', '!=...
            3 IllegalArgumentException: invalid argument: found PARTITION at 59:68 expected '.', '[', '::', 'ESCAPE', 'IS', 'NOT', 'IN', '*', '/', '%', 'DIV', '+', '-', '||', '>>>', '>>', '<<', '&', '^', '|', '!=',...
            3 IllegalArgumentException: invalid argument: found WITH at 171:175 expected '.', '[', '::', 'ESCAPE', 'IS', 'NOT', 'IN', '*', '/', '%', 'DIV', '+', '-', '||', '>>>', '>>', '<<', '&', '^', '|', '!=', '!...
            3 IllegalArgumentException: invalid argument: found WITH at 279:283 expected '.', '[', '::', 'ESCAPE', 'IS', 'NOT', 'IN', '*', '/', '%', 'DIV', '+', '-', '||', '>>>', '>>', '<<', '&', '^', '|', '!=', '!...
            3 PythonException:  Exception: self._partition_col was 1 but the row value was 2 PySparkRuntimeError: [UDTF_EXEC_ERROR] User defined table function encountered an error in the 'eval' method: self._parti...
            3 PythonException:  PySparkRuntimeError: [UDTF_CONSTRUCTOR_INVALID_NO_ANALYZE_METHOD] Failed to evaluate the user-defined table function '' because its constructor is invalid: the function does not impl...
            3 UnsupportedOperationException: Physical plan does not support logical expression OuterReferenceColumn(Field { name: "#0", data_type: Int64 }, Column { relation: None, name: "#0" })
            3 UnsupportedOperationException: Physical plan does not support logical expression OuterReferenceColumn(Field { name: "#2", data_type: Int32, nullable: true }, Column { relation: Some(Bare { table: "t" ...
            3 UnsupportedOperationException: cached remote relation
            3 UnsupportedOperationException: handle analyze input files
            3 UnsupportedOperationException: pivot
            3 UnsupportedOperationException: table argument options in subquery expression
            3 UnsupportedOperationException: unknown function: distributed_sequence_id
            3 ValueError: Converting to Python dictionary is not supported when duplicate field names are present
(+2)        2 AnalysisException: Internal error: Function 'approx_percentile_cont' failed to match any signature, errors: Error during planning: Function 'approx_percentile_cont' expects 2 arguments but received 3,...
            2 AnalysisException: No table format found for: orc
            2 AnalysisException: Spark `variant_get` function: path must be a constant string
            2 AnalysisException: ambiguous attribute: ObjectName([Identifier("id")])
            2 AnalysisException: not supported: function exists
            2 AnalysisException: not supported: list functions
            2 AssertionError: "ARROW_TYPE_MISMATCH.*SQL_MAP_ARROW_ITER_UDF" does not match "Invalid argument error: column types must match schema types, expected Int32 but found Int64 at column index 0"
            2 AssertionError: AnalysisException not raised by <lambda>
            2 AssertionError: Lists differ: [Row(<lambda>(struct_of_v)="Row(metadata=bytea[1007 chars]))")] != [Row(udf='0'), Row(udf='1'), Row(udf='2'), Row[89 chars]'9')]
            2 AssertionError: Lists differ: [Row(udf="Row(metadata=bytearray(b'\\x01\\x00\\x00[823 chars]))")] != [Row(udf='0'), Row(udf='1'), Row(udf='2'), Row(udf[85 chars]'9')]
            2 AssertionError: Lists differ: [Row(udf="Row(metadata=bytearray(b'\\x01\\x01\\x00[1095 chars]))")] != [Row(udf='{"a":"a"}'), Row(udf='{"a":"b"}'), Row(u[165 chars]"}')]
            2 AssertionError: Lists differ: [Row(udf="{'metadata': b'\\x01\\x00\\x00', 'value'[633 chars]'}")] != [Row(udf='0'), Row(udf='1'), Row(udf='2'), Row(udf[85 chars]'9')]
            2 AssertionError: StructType([StructField('metadata', Binar[55 chars]se)]) != VariantType()
            2 AssertionError: {'foo': 'bar'} != {}
            2 IllegalArgumentException: invalid argument: found FUNCTION at 7:15 expected 'DATABASE', 'SCHEMA', 'NAMESPACE', 'OR', 'TEMP', 'TEMPORARY', 'EXTERNAL', 'TABLE', 'GLOBAL', or 'VIEW'
            2 IllegalArgumentException: invalid argument: found PARTITION at 97:106 expected '.', '[', '::', 'ESCAPE', 'IS', 'NOT', 'IN', '*', '/', '%', 'DIV', '+', '-', '||', '>>>', '>>', '<<', '&', '^', '|', '!='...
            2 IllegalArgumentException: invalid argument: initial input not supported in group map
            2 PySparkAssertionError: [DIFFERENT_ROWS] Results do not match: ( 99.50000 % )
            2 PythonException:  AssertionError: assert None is not None
            2 PythonException:  AttributeError: 'NoneType' object has no attribute 'cpus'
            2 PythonException:  PySparkValueError: Exception thrown when converting pandas.Series (int64) with name 'decimal_result' to Arrow Array (decimal128(10, 2)).
            2 PythonException:  PySparkValueError: Exception thrown when converting pandas.Series (int8) with name 'None' to Arrow Array (decimal128(10, 0)).
            2 PythonException:  PySparkValueError: Exception thrown when converting pandas.Series (object) with name 'None' to Arrow Array (int32).
            2 PythonException:  PySparkValueError: Rows cannot be of type VariantVal
            2 PythonException:  TypeError: int() argument must be a string, a bytes-like object or a real number, not 'dict'
            2 SparkRuntimeException: Error during planning: Correlated scalar subquery must be aggregated to return at most one row
            2 SparkRuntimeException: Invalid argument error: column types must match schema types, expected Int32 but found Utf8 at column index 0
            2 SparkRuntimeException: Invalid argument error: column types must match schema types, expected LargeUtf8 but found Utf8 at column index 0
            2 SparkRuntimeException: Python error: [TestDataSource::partitions] NotImplementedError: 
            2 SparkRuntimeException: Python error: [my-json::partitions] AttributeError: 'pyarrow.lib.Schema' object has no attribute 'fieldNames'
            2 TypeError: 'NoneType' object is not iterable
            2 UnsupportedOperationException: Aggregate can not be used as a sliding accumulator because `retract_batch` is not implemented: avg@42nxldcwlikpshede68hki4hd(#9) PARTITION BY [#8] ORDER BY [#9 ASC NULLS...
            2 UnsupportedOperationException: Aggregate can not be used as a sliding accumulator because `retract_batch` is not implemented: avg@9iphoaj9v2becke5sqyqokbpc(#9) PARTITION BY [#8] ORDER BY [#9 ASC NULLS...
            2 UnsupportedOperationException: Aggregate can not be used as a sliding accumulator because `retract_batch` is not implemented: mean_udf@56f23o3q96gbgh8wooin3wlez(#3) PARTITION BY [#2] ORDER BY [#3 ASC ...
            2 UnsupportedOperationException: Aggregate can not be used as a sliding accumulator because `retract_batch` is not implemented: mean_udf@6blgh7qrs537jqp40qew4vlxb(#3) PARTITION BY [#2] ORDER BY [#3 ASC ...
            2 UnsupportedOperationException: CLUSTER BY for write
            2 UnsupportedOperationException: Physical plan does not support logical expression OuterReferenceColumn(Field { name: "#0", data_type: Int32, nullable: true }, Column { relation: Some(Bare { table: "t1"...
(+1)        2 UnsupportedOperationException: Physical plan does not support logical expression OuterReferenceColumn(Field { name: "#0", data_type: List(Field { data_type: Float64 }), nullable: true, metadata: {"udt...
            2 UnsupportedOperationException: Physical plan does not support logical expression Wildcard { qualifier: None, options: WildcardOptions { ilike: None, exclude: None, except: None, replace: None, rename:...
            2 UnsupportedOperationException: Unsupported CAST from Struct("metadata": non-null Binary, "value": non-null Binary) to Int32
            2 UnsupportedOperationException: cast Time64(Nanosecond) to Spark data type
            2 UnsupportedOperationException: create resource profile command
            2 UnsupportedOperationException: function: from_xml
            2 UnsupportedOperationException: function: st_setsrid
            2 UnsupportedOperationException: function: st_srid
            2 UnsupportedOperationException: function: try_make_interval
            2 UnsupportedOperationException: function: try_parse_json
            2 UnsupportedOperationException: handle analyze same semantics
(+2)        2 UnsupportedOperationException: lambda function: exists
(+2)        2 UnsupportedOperationException: named lambda variable
            2 UnsupportedOperationException: named window function arguments
            2 UnsupportedOperationException: unknown function: try_to_date
            2 UnsupportedOperationException: unknown function: try_to_time
            2 UnsupportedOperationException: wildcard with plan ID
            2 handle artifact statuses
(+1)        1 AnalysisException: Cannot cast string 'a' to value of Float64 type
            1 AnalysisException: Cannot cast string 'abc' to value of Float64 type
            1 AnalysisException: Cannot cast value 'abc' to value of Boolean type
            1 AnalysisException: Could not find config namespace "mapred"
            1 AnalysisException: Could not find config namespace "spark"
            1 AnalysisException: Database not found: testcat
            1 AnalysisException: Error parsing timestamp from '082017' using format '%m%Y': input is not enough for unique date and time
            1 AnalysisException: Error parsing timestamp from '2014-31-12' using format '%Y-%d-%pa': input contains invalid characters
            1 AnalysisException: Error parsing timestamp from '2023-01-01' using format '%d-%m-%Y': input contains invalid characters
            1 AnalysisException: Failed to parse date '02-29' with format '%m-%d': input is not enough for unique date and time
            1 AnalysisException: Found unsupported option type when parsing options: allowUnquotedFieldNames
            1 AnalysisException: Invalid partition id 1 in write result (expected < 1)
            1 AnalysisException: Invalid partition id 2 in write result (expected < 1)
            1 AnalysisException: No table format found for: xml
            1 AnalysisException: Spark `try_variant_get` function: path must be a constant string
            1 AnalysisException: Table already exists: tbl1
            1 AnalysisException: Temporary View not found: tab2
            1 AnalysisException: UNION queries have different number of columns: left has 3 columns whereas right has 2 columns
            1 AnalysisException: Write failed for partition 0: External error: Python error: [TestJsonWriter::write] AttributeError: 'NoneType' object has no attribute 'partitionId'
            1 AnalysisException: Write failed for partition 1: External error: Python error: [TestJsonWriter::write] AttributeError: 'NoneType' object has no attribute 'partitionId'
            1 AnalysisException: ambiguous attribute: ObjectName([Identifier("b")])
            1 AnalysisException: ambiguous attribute: ObjectName([Identifier("i")])
            1 AnalysisException: attribute ObjectName([Identifier("t")]) is missing from the schema: cannot resolve attribute
            1 AnalysisException: attribute ObjectName([Identifier("x")]) is missing from the schema: cannot resolve attribute
            1 AnalysisException: element_at expects List or Map type as first argument, got Null
            1 AnalysisException: foobar
            1 AnalysisException: join condition should not be empty
            1 AnalysisException: one value expected: [Column(Column { relation: None, name: "#0" }), Literal(Int32(123), None)]
(+1)        1 AnalysisException: one value expected: [Column(Column { relation: None, name: "#1" }), Literal(Int64(3176347237897044447), None)]
(+1)        1 AnalysisException: one value expected: [Column(Column { relation: None, name: "#1" }), Literal(Int64(5164102944296940640), None)]
            1 AnalysisException: to_time format argument 2 must be a scalar, not an array
            1 AnalysisException: too big
            1 AnalysisException: zero values expected: [Literal(Int32(123), None)]
            1 AssertionError: "'path' is not specified." does not match "missing path in listing table options"
            1 AssertionError: "ARROW_TYPE_MISMATCH.*SQL_MAP_ARROW_ITER_UDF" does not match "Invalid argument error: column types must match schema types, expected Struct("b": Int32) but found Struct("a": Int64, "b"...
            1 AssertionError: "DATA_SOURCE_EXTRANEOUS_FILTERS" does not match "Python error: [test::partitions] AssertionError: assert False
            1 AssertionError: "DATA_SOURCE_PUSHDOWN_DISABLED" does not match "Python error: [<reader>::read] AssertionError: assert False
(+1)        1 AssertionError: "Database 'memory:563c524a-b3bb-4557-9f05-5cbbab7bcf7e' dropped." does not match "No table format found for: jdbc"
(+1)        1 AssertionError: "Database 'memory:870b163f-be3f-48e4-a2cc-659d7a207580' dropped." does not match "No table format found for: jdbc"
            1 AssertionError: "Invalid return type" does not match " AttributeError: 'Series' object has no attribute 'columns'
            1 AssertionError: "Python worker process terminated due to idle timeout \(timeout: 1 seconds\)" does not match " PySparkRuntimeError: [UDTF_INVALID_OUTPUT_ROW_TYPE] The type of an individual output row ...
            1 AssertionError: "UNRESOLVED_COLUMN.WITH_SUGGESTION" does not match "attribute ObjectName([Identifier("b")]) is missing from the schema: cannot resolve attribute"
            1 AssertionError: "is null" does not match " ArrowException: Invalid argument error: Column 'a' is declared as non-nullable but contains null values
            1 AssertionError: "requirement failed: Cogroup keys must have same size: 2 != 1" does not match "invalid argument: child plan grouping expressions must have the same length"
            1 AssertionError: '+---[17 chars]-----+\n|                        x|\n+--------[132 chars]-+\n' != '+---[17 chars]----------+\n|update_fields(x, WithField(e))|\[167 chars]-+\n'
            1 AssertionError: '+---[23 chars]---+-----+\n|  1|    1|\n+---+-----+\nonly showing top 1 row' != '+---[23 chars]---+-----+\n|  1|    1|\n+---+-----+\nonly showing top 1 row\n'
(+1)        1 AssertionError: 'INVALID_CLONE_SESSION_REQUEST.TARGET_SESSION_ID_FORMAT' not found in '<_InactiveRpcError of RPC that terminated with:\n\tstatus = StatusCode.UNIMPLEMENTED\n\tdetails = "clone session"...
            1 AssertionError: 'ST_INVALID_SRID_VALUE' != None : Expected error class was 'ST_INVALID_SRID_VALUE', got 'None'.
            1 AssertionError: 'UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.UNSUPPORTED_IN_EXISTS_SUBQUERY' != None : Expected error class was 'UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.UNSUPPORTED_IN_EXISTS_SUBQUERY', ...
            1 AssertionError: 'a NULL, b BOOLEAN, c BINARY' != 'a VOID,b BOOLEAN,c BINARY'
            1 AssertionError: 'bytearray' != 'bytes'
            1 AssertionError: 0 not greater than 0
            1 AssertionError: 0.6363787615254752 != 0.9531453492357947 : Column<'rand(1)'>
            1 AssertionError: 2 != 6
            1 AssertionError: 6 != 0 : []
            1 AssertionError: ArrayIndexOutOfBoundsException not raised
            1 AssertionError: Exception not raised by <lambda>
            1 AssertionError: Lists differ: [(1, 2), (3, 4), (None, 5), (0, 0)] != [(1, 2), (3, 4), (None, 5), (None, None)]
            1 AssertionError: Lists differ: [Row([22 chars]e(2018, 12, 31, 16, 0), aware=datetime.datetim[16 chars] 0))] != [Row([22 chars]e(2019, 1, 1, 0, 0), aware=datetime.datetime(2[13 chars] 0))]
            1 AssertionError: Lists differ: [Row([259 chars]681098, ln(id)=1.0986122886681098, struct(id, [975 chars]0'))] != [Row([259 chars]681096, ln(id)=1.0986122886681096, struct(id, [975 chars]0'))]
            1 AssertionError: Lists differ: [Row([47 chars] 3, 6: 3}), Row(map={4: 3, 6: 3}), Row(map={4: 5, 6: 3})] != [Row([47 chars] 3, 6: 3, 5: 5}), Row(map={4: 3, 6: 3}), Row(map={4: 3, 6: 3})]
            1 AssertionError: Lists differ: [Row(id=90, name='90'), Row(id=91, name='91'), Ro[176 chars]99')] != [Row(id=15, name='15'), Row(id=16, name='16'), Ro[176 chars]24')]
            1 AssertionError: Lists differ: [Row(name='Andy', age=30), Row(name='Andy', ag[374 chars]one)] != [Row(age=19, name='Justin'), Row(age=19, name=[374 chars]el')]
            1 AssertionError: Lists differ: [Row(name='Andy', age=30), Row(name='Justin', [34 chars]one)] != [Row(_corrupt_record=' "age":19}\n', name=None[104 chars]el')]
            1 AssertionError: Lists differ: [Row(udf=217.5)] != [Row(udf=1)]
            1 AssertionError: Row(name='Bob', age=27, height=66.0) != Row(name='Alice', age=10, height=10.0)
            1 AssertionError: Row(point='[1.0, 2.0]', pypoint='[3.0, 4.0]') != Row(point='(1.0, 2.0)', pypoint='[3.0, 4.0]')
            1 AssertionError: SparkConnectGrpcException not raised
            1 AssertionError: StorageLevel(False, True, True, False, 1) != StorageLevel(False, False, False, False, 1)
            1 AssertionError: Struc[15 chars]eld('a', NullType(), True), StructField('b', L[51 chars]ue)]) != Struc[15 chars]eld('b', LongType(), True), StructField('c', S[15 chars]ue)])
            1 AssertionError: Struc[23 chars]st', StructType([StructField('metadata', Binar[194 chars]ue)]) != Struc[23 chars]st', VariantType(), True), StructField('last',[18 chars]ue)])
            1 AssertionError: Struc[30 chars]estampType(), True), StructField('val', IntegerType(), True)]) != Struc[30 chars]estampType(), True), StructField('val', IntegerType(), False)])
            1 AssertionError: Struc[32 chars]e(), False), StructField('b', DoubleType(), Fa[158 chars]ue)]) != Struc[32 chars]e(), True), StructField('b', DoubleType(), Tru[154 chars]ue)])
            1 AssertionError: Struc[40 chars]ue), StructField('val', ArrayType(DoubleType(), False), True)]) != Struc[40 chars]ue), StructField('val', PythonOnlyUDT(), True)])
            1 AssertionError: True is not false : Default URL is not secure
            1 AssertionError: VariantType() != StructType([StructField('metadata', Binar[55 chars]se)])
            1 AssertionError: YearMonthIntervalType(0, 1) != YearMonthIntervalType(0, 0)
            1 AssertionError: [1.0, 2.0] != ExamplePoint(1.0,2.0)
            1 AttributeError: 'NoneType' object has no attribute 'extract_graph'
            1 AttributeError: 'NoneType' object has no attribute 'toText'
            1 FileNotFoundError: [Errno 2] No such file or directory: '/home/runner/work/sail/sail/.venvs/test-spark.spark-4.1.1/lib/python3.11/site-packages/pyspark/data/artifact-tests/junitLargeJar.jar'
(+1)        1 FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpgc95738q'
            1 IllegalArgumentException: invalid argument: empty data type
            1 IllegalArgumentException: invalid argument: expecting column to drop
            1 IllegalArgumentException: invalid argument: field not found in input schema: col1
            1 IllegalArgumentException: invalid argument: found PARTITION at 100:109 expected '.', '[', '::', 'ESCAPE', 'IS', 'NOT', 'IN', '*', '/', '%', 'DIV', '+', '-', '||', '>>>', '>>', '<<', '&', '^', '|', '!=...
            1 IllegalArgumentException: invalid argument: found PARTITION at 103:112 expected '.', '[', '::', 'ESCAPE', 'IS', 'NOT', 'IN', '*', '/', '%', 'DIV', '+', '-', '||', '>>>', '>>', '<<', '&', '^', '|', '!=...
            1 IllegalArgumentException: invalid argument: found PARTITION at 108:117 expected '.', '[', '::', 'ESCAPE', 'IS', 'NOT', 'IN', '*', '/', '%', 'DIV', '+', '-', '||', '>>>', '>>', '<<', '&', '^', '|', '!=...
            1 IllegalArgumentException: invalid argument: found PARTITION at 111:120 expected '.', '[', '::', 'ESCAPE', 'IS', 'NOT', 'IN', '*', '/', '%', 'DIV', '+', '-', '||', '>>>', '>>', '<<', '&', '^', '|', '!=...
            1 IllegalArgumentException: invalid argument: found PARTITION at 117:126 expected '.', '[', '::', 'ESCAPE', 'IS', 'NOT', 'IN', '*', '/', '%', 'DIV', '+', '-', '||', '>>>', '>>', '<<', '&', '^', '|', '!=...
            1 IllegalArgumentException: invalid argument: found PARTITION at 118:127 expected '.', '[', '::', 'ESCAPE', 'IS', 'NOT', 'IN', '*', '/', '%', 'DIV', '+', '-', '||', '>>>', '>>', '<<', '&', '^', '|', '!=...
            1 IllegalArgumentException: invalid argument: found PARTITION at 53:62 expected '.', '[', '::', 'ESCAPE', 'IS', 'NOT', 'IN', '*', '/', '%', 'DIV', '+', '-', '||', '>>>', '>>', '<<', '&', '^', '|', '!=',...
            1 IllegalArgumentException: invalid argument: found PARTITION at 69:78 expected '.', '[', '::', 'ESCAPE', 'IS', 'NOT', 'IN', '*', '/', '%', 'DIV', '+', '-', '||', '>>>', '>>', '<<', '&', '^', '|', '!=',...
            1 IllegalArgumentException: invalid argument: found PARTITION at 90:99 expected '.', '[', '::', 'ESCAPE', 'IS', 'NOT', 'IN', '*', '/', '%', 'DIV', '+', '-', '||', '>>>', '>>', '<<', '&', '^', '|', '!=',...
            1 IllegalArgumentException: invalid argument: found WITH at 115:119 expected '.', '[', '::', 'ESCAPE', 'IS', 'NOT', 'IN', '*', '/', '%', 'DIV', '+', '-', '||', '>>>', '>>', '<<', '&', '^', '|', '!=', '!...
            1 IllegalArgumentException: invalid argument: found abc at 0:3 expected something else, ';', statement, or end of input
            1 IllegalArgumentException: invalid argument: found collate at 13:20 expected string, '.', '[', '::', 'ESCAPE', 'IS', 'NOT', 'IN', '*', '/', '%', 'DIV', '+', '-', '||', '>>>', '>>', '<<', '&', '^', '|',...
            1 IllegalArgumentException: invalid argument: grouping sets with grouping expressions
            1 IllegalArgumentException: invalid argument: invalid user-defined window function type
(+1)        1 IllegalArgumentException: invalid argument: map_filter() first argument must be an array, got Map(Field { name: "entries", data_type: Struct([Field { name: "key", data_type: Utf8 }, Field { name: "val...
(+1)        1 IllegalArgumentException: invalid argument: map_zip_with() first argument must be an array, got Map(Field { name: "entries", data_type: Struct([Field { name: "key", data_type: Utf8 }, Field { name: "v...
            1 IllegalArgumentException: invalid argument: table does not exist: ObjectName([Identifier("test_table")])
(+1)        1 IllegalArgumentException: invalid argument: transform_keys() first argument must be an array, got Map(Field { name: "entries", data_type: Struct([Field { name: "key", data_type: Utf8 }, Field { name: ...
(+1)        1 IllegalArgumentException: invalid argument: transform_values() first argument must be an array, got Map(Field { name: "entries", data_type: Struct([Field { name: "key", data_type: Utf8 }, Field { name...
            1 IndexError: list index out of range
(+1)        1 PySparkAssertionError: Received incorrect server side session identifier for request. Please create a new Spark Session to reconnect. (6ae4fae5-f170-4da9-b37e-5ad6a663c131 != d01d5a58-3774-4dd4-9458-f...
(+1)        1 PySparkAssertionError: Received incorrect server side session identifier for request. Please create a new Spark Session to reconnect. (b4c2e44a-534a-4a7e-bdd0-aede80e9d643 != d89fa1d5-738c-4f51-9b8e-b...
            1 PySparkNotImplementedError: [NOT_IMPLEMENTED] Invalid return type with grouped aggregate Pandas UDFs: StructType([StructField('metadata', BinaryType(), False), StructField('value', BinaryType(), False...
            1 PySparkNotImplementedError: [NOT_IMPLEMENTED] toJSON() is not implemented.
            1 PythonException: 
            1 PythonException:  AssertionError: Undefined error message parameter for error class: UDTF_ARROW_TYPE_CONVERSION_ERROR. Parameters: {'data': "[('x',)]", 'schema': 'struct<a:int>', 'arrow_schema': 'stru...
            1 PythonException:  AssertionError: Undefined error message parameter for error class: UDTF_ARROW_TYPE_CONVERSION_ERROR. Parameters: {'data': '[(ExamplePoint(10.0,20.0),)]', 'schema': 'struct<point:arra...
            1 PythonException:  AssertionError: Undefined error message parameter for error class: UDTF_ARROW_TYPE_CONVERSION_ERROR. Parameters: {'data': '[(ExamplePoint(10.0,20.0),)]', 'schema': 'struct<udt:array<...
            1 PythonException:  AttributeError: 'NoneType' object has no attribute 'partitionId'
            1 PythonException:  AttributeError: 'Series' object has no attribute 'columns'
            1 PythonException:  AttributeError: 'list' object has no attribute 'x'
            1 PythonException:  AttributeError: 'list' object has no attribute 'y'
            1 PythonException:  KeyError: 'v'
            1 PythonException:  PySparkRuntimeError: [SCHEMA_MISMATCH_FOR_PANDAS_UDF] Result vector from pandas_udf was not the required length: expected 10, got 2.
            1 PythonException:  PySparkRuntimeError: [UDTF_ARROW_TYPE_CAST_ERROR] Cannot convert the output value of the column 'point' with type 'object' to the specified return type of the column: 'list<item: dou...
            1 PythonException:  PySparkRuntimeError: [UDTF_ARROW_TYPE_CAST_ERROR] Cannot convert the output value of the column 'udt' with type 'object' to the specified return type of the column: 'list<item: doubl...
            1 PythonException:  PySparkRuntimeError: [UDTF_ARROW_TYPE_CAST_ERROR] Cannot convert the output value of the column 'v' with type 'object' to the specified return type of the column: 'struct<metadata: b...
            1 PythonException:  PySparkRuntimeError: [UDTF_ARROW_TYPE_CAST_ERROR] Cannot convert the output value of the column 'v' with type 'object' to the specified return type of the column: 'struct<v1: struct<...
            1 PythonException:  PySparkValueError: Exception thrown when converting pandas.Series (int64) with name 'None' to Arrow Array (decimal128(10, 2)).
            1 PythonException:  PySparkValueError: Exception thrown when converting pandas.Series (int8) with name 'int8' to Arrow Array (decimal128(10, 0)).
            1 PythonException:  PySparkValueError: Exception thrown when converting pandas.Series (int8) with name 'value' to Arrow Array (decimal128(10, 0)).
            1 PythonException:  PySparkValueError: Exception thrown when converting pandas.Series (object) with name 'value' to Arrow Array (int32).
            1 PythonException:  TypeError: 'str' object cannot be interpreted as an integer
            1 PythonException:  TypeError: cannot unpack non-iterable NoneType object
            1 PythonException:  TypeError: net.razorvine.pickle.PickleException: expected zero arguments for construction of ClassDict (for pyspark.sql.types._create_row).
            1 PythonException:  ValueError: invalid literal for int() with base 10: 'z'
            1 SparkRuntimeException: Compute error: Cannot perform a binary operation on arrays of different length
            1 SparkRuntimeException: Exception: path is not specified
            1 SparkRuntimeException: Execution error: Schema field count mismatch: expected 1 fields, got 2
            1 SparkRuntimeException: Internal error: Cannot run range queries on datatype: Time64(µs).
(+1)        1 SparkRuntimeException: Invalid argument error: Column '#3' is declared as non-nullable but contains null values
            1 SparkRuntimeException: Invalid argument error: column types must match schema types, expected Int64 but found List(Int64) at column index 1
            1 SparkRuntimeException: Invalid argument error: column types must match schema types, expected List(Int32) but found List(Int32, field: 'element') at column index 0
            1 SparkRuntimeException: Invalid argument error: column types must match schema types, expected List(Struct("metadata": non-null Binary, "value": non-null Binary), metadata: {"ARROW:extension:name": "ar...
            1 SparkRuntimeException: Invalid argument error: column types must match schema types, expected List(Utf8) but found List(Utf8, field: 'element') at column index 0
            1 SparkRuntimeException: Invalid argument error: column types must match schema types, expected Struct("v1": Struct("metadata": non-null Binary, "value": non-null Binary), metadata: {"ARROW:extension:na...
            1 SparkRuntimeException: Invalid argument error: column types must match schema types, expected Utf8 but found Float64 at column index 1
            1 SparkRuntimeException: Invalid argument error: must either specify a row count or at least one column
            1 SparkRuntimeException: Invalid argument error: number of columns(3) must match number of fields(2) in schema
            1 SparkRuntimeException: Json error: Not valid JSON: EOF while parsing a list at line 1 column 1
            1 SparkRuntimeException: Json error: Not valid JSON: expected value at line 1 column 2
            1 SparkRuntimeException: No field named t1."#0". Did you mean '#2'?.
            1 SparkRuntimeException: Parser error: Error while parsing value '0
            1 SparkRuntimeException: Parser error: Invalid timezone "+7:30": failed to parse timezone
            1 SparkRuntimeException: Python error: [TestArrowStreamWriter::writer] PySparkNotImplementedError: [NOT_IMPLEMENTED] writer is not implemented.
            1 SparkRuntimeException: Python error: [TestDataSource::writer] PySparkNotImplementedError: [NOT_IMPLEMENTED] writer is not implemented.
            1 SparkRuntimeException: Python error: [my-json::writer] AttributeError: 'pyarrow.lib.Schema' object has no attribute 'fieldNames'
            1 SparkRuntimeException: Python error: [test::partitions] AssertionError: assert False
            1 SparkRuntimeException: Python error: [testdatasourcepyarrow::partitions] PySparkNotImplementedError: [NOT_IMPLEMENTED] reader is not implemented.
            1 SparkRuntimeException: Schema error: Unsupported type in DDL schema: List { data_type: Int32, nullable: true }. Use PyArrow Schema for complex types.
            1 SparkRuntimeException: Schema error: Unsupported type in DDL schema: Struct { fields: Fields([Field { name: "a", data_type: Int32, nullable: true, metadata: [] }, Field { name: "b", data_type: Int32, ...
            1 SparkRuntimeException: Schema error: Unsupported type in DDL schema: Struct { fields: Fields([Field { name: "y", data_type: Int32, nullable: true, metadata: [] }]) }. Use PyArrow Schema for complex ty...
            1 SparkRuntimeException: Schema error: Unsupported type in DDL schema: Variant. Use PyArrow Schema for complex types.
            1 SparkRuntimeException: This feature is not implemented: Data type Decimal128(38, 18) not supported in row-based write path. Use DataSourceArrowWriter for full type support.
            1 UnsupportedOperationException: Aggregate can not be used as a sliding accumulator because `retract_batch` is not implemented: avg@42nxldcwlikpshede68hki4hd(#9) PARTITION BY [#8] ORDER BY [#9 ASC NULLS...
            1 UnsupportedOperationException: Aggregate can not be used as a sliding accumulator because `retract_batch` is not implemented: avg@42nxldcwlikpshede68hki4hd(plus_one@37pn5wx7sqmlbvfr2ebnbd6a0(#9)) PART...
            1 UnsupportedOperationException: Aggregate can not be used as a sliding accumulator because `retract_batch` is not implemented: avg@9iphoaj9v2becke5sqyqokbpc(#9) PARTITION BY [#8] ORDER BY [#9 ASC NULLS...
            1 UnsupportedOperationException: Aggregate can not be used as a sliding accumulator because `retract_batch` is not implemented: avg@9iphoaj9v2becke5sqyqokbpc(plus_one@8f9fwaevnfdj031wzemmraerh(#9)) PART...
            1 UnsupportedOperationException: Aggregate can not be used as a sliding accumulator because `retract_batch` is not implemented: weighted_mean@1h5j8ehtp238nduw020pdafyp(#9, #10) PARTITION BY [#8] ORDER B...
            1 UnsupportedOperationException: Aggregate can not be used as a sliding accumulator because `retract_batch` is not implemented: weighted_mean@1z16p7g8orl8a73u97f1704fz(#9, #10) PARTITION BY [#8] ORDER B...
            1 UnsupportedOperationException: Aggregate can not be used as a sliding accumulator because `retract_batch` is not implemented: weighted_mean@5lgtucrfqmvjc9hq9885z0x6d(#9, #10) PARTITION BY [#8] ORDER B...
            1 UnsupportedOperationException: Aggregate can not be used as a sliding accumulator because `retract_batch` is not implemented: weighted_mean@bqv855sdysxo4klnvnyi7f0v3(#9, #10) PARTITION BY [#8] ORDER B...
            1 UnsupportedOperationException: LATERAL table function with criteria
(-1)        1 UnsupportedOperationException: Physical plan does not support logical expression OuterReferenceColumn(Field { name: "#0", data_type: List(Field { data_type: Float64 }), nullable: true, metadata: {"udt...
            1 UnsupportedOperationException: Physical plan does not support logical expression OuterReferenceColumn(Field { name: "#1", data_type: Int32, nullable: true }, Column { relation: Some(Bare { table: "t1"...
(-1)        1 UnsupportedOperationException: Physical plan does not support logical expression ScalarSubquery(<subquery>)
(+1)        1 UnsupportedOperationException: Physical plan does not support undecorrelated Subquery
            1 UnsupportedOperationException: PlanNode::ClearCache
            1 UnsupportedOperationException: PlanNode::IsCached
            1 UnsupportedOperationException: PlanNode::RecoverPartitions
            1 UnsupportedOperationException: SHOW FUNCTIONS
            1 UnsupportedOperationException: Support for 'approx_distinct' for data type Float64 is not implemented
            1 UnsupportedOperationException: Support for 'approx_distinct' for data type Struct("name": Utf8, "value": Int64) is not implemented
(+1)        1 UnsupportedOperationException: Unsupported CAST from List(non-null Struct("0": Int64, "1": Int64)) to List(non-null Struct("col0": Int64, "col1": Int64))
(+1)        1 UnsupportedOperationException: Unsupported CAST from Struct("c": Int64, "d": Float64) to Struct("a": Int64, "b": Float64)
            1 UnsupportedOperationException: apply in pandas with state
            1 UnsupportedOperationException: as of join
            1 UnsupportedOperationException: bucketing for writing listing table format
            1 UnsupportedOperationException: deduplicate within watermark
            1 UnsupportedOperationException: function: collate
            1 UnsupportedOperationException: function: collation
            1 UnsupportedOperationException: function: input_file_block_length
            1 UnsupportedOperationException: function: input_file_block_start
            1 UnsupportedOperationException: function: java_method
            1 UnsupportedOperationException: function: reflect
            1 Unsuppo

(truncated)

Passed Tests Diff 
--- before.txt	2026-05-17 20:32:55.869311928 +0000
+++ after.txt	2026-05-17 20:32:56.491319122 +0000
@@ -113 +112,0 @@
-pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.summary
@@ -268,0 +268 @@
+pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.filter
@@ -430 +429,0 @@
-pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.stack
@@ -1657 +1655,0 @@
-pyspark/sql/tests/connect/test_parity_functions.py::FunctionsParityTests::test_lit_list
@@ -1800,0 +1799 @@
+pyspark/sql/tests/connect/test_parity_subquery.py::SubqueryParityTests::test_exists_subquery
@@ -1809,0 +1809,2 @@
+pyspark/sql/tests/connect/test_parity_subquery.py::SubqueryParityTests::test_subquery_in_repartition
+pyspark/sql/tests/connect/test_parity_subquery.py::SubqueryParityTests::test_uncorrelated_scalar_subquery_with_view
@@ -1944,0 +1946 @@
+pyspark/sql/tests/connect/test_parity_udtf.py::ArrowUDTFParityTests::test_numeric_output_type_casting
Failed Tests 
pyspark/sql/catalog.py::pyspark.sql.catalog.Catalog.cacheTable
pyspark/sql/catalog.py::pyspark.sql.catalog.Catalog.clearCache
pyspark/sql/catalog.py::pyspark.sql.catalog.Catalog.createTable
pyspark/sql/catalog.py::pyspark.sql.catalog.Catalog.functionExists
pyspark/sql/catalog.py::pyspark.sql.catalog.Catalog.getFunction
pyspark/sql/catalog.py::pyspark.sql.catalog.Catalog.isCached
pyspark/sql/catalog.py::pyspark.sql.catalog.Catalog.listFunctions
pyspark/sql/catalog.py::pyspark.sql.catalog.Catalog.recoverPartitions
pyspark/sql/catalog.py::pyspark.sql.catalog.Catalog.refreshByPath
pyspark/sql/catalog.py::pyspark.sql.catalog.Catalog.refreshTable
pyspark/sql/catalog.py::pyspark.sql.catalog.Catalog.uncacheTable
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame._joinAsOf
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.approxQuantile
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.asTable
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.cache
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.coalesce
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.colRegex
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.dropDuplicatesWithinWatermark
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.explain
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.freqItems
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.groupingSets
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.hint
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.inputFiles
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.lateralJoin
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.localCheckpoint
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.mapInArrow
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.observe
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.pandas_api
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.persist
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.randomSplit
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.repartitionById
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.repartitionByRange
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.sameSemantics
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.sampleBy
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.storageLevel
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.summary
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.toJSON
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.transpose
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrame.withWatermark
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrameStatFunctions.approxQuantile
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrameStatFunctions.freqItems
pyspark/sql/dataframe.py::pyspark.sql.dataframe.DataFrameStatFunctions.sampleBy
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.aggregate
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.approx_count_distinct
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.approx_percentile
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.array_sort
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.assert_true
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.collation
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.corr
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.cosh
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.count_if
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.count_min_sketch
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.degrees
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.exists
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.exp
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.first
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.forall
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.from_json
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.from_xml
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.get_json_object
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.grouping_id
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.hash
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.histogram_numeric
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.hll_sketch_agg
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.hll_sketch_estimate
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.hll_union
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.hll_union_agg
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.hour
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.inline_outer
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.input_file_block_length
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.input_file_block_start
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.input_file_name
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.java_method
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.kll_sketch_agg_bigint
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.kll_sketch_agg_double
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.kll_sketch_agg_float
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.kll_sketch_get_n_bigint
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.kll_sketch_get_n_double
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.kll_sketch_get_n_float
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.kll_sketch_get_quantile_bigint
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.kll_sketch_get_quantile_double
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.kll_sketch_get_quantile_float
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.kll_sketch_get_rank_bigint
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.kll_sketch_get_rank_double
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.kll_sketch_get_rank_float
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.kll_sketch_merge_bigint
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.kll_sketch_merge_double
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.kll_sketch_merge_float
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.kll_sketch_to_string_bigint
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.kll_sketch_to_string_double
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.kll_sketch_to_string_float
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.kurtosis
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.log2
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.map_filter
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.map_zip_with
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.minute
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.mode
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.monotonically_increasing_id
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.percentile
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.percentile_approx
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.randn
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.randstr
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.reduce
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.reflect
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.regexp_extract
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.regexp_extract_all
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.schema_of_csv
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.schema_of_json
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.schema_of_xml
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.second
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.session_window
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.shuffle
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.sin
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.spark_partition_id
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.st_setsrid
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.st_srid
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.stack
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.tan
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.theta_difference
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.theta_intersection
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.theta_intersection_agg
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.theta_sketch_agg
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.theta_sketch_estimate
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.theta_union
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.theta_union_agg
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.timestamp_add
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.timestamp_diff
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.to_char
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.to_csv
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.to_json
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.to_time
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.to_varchar
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.to_xml
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.transform
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.transform_keys
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.transform_values
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.try_add
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.try_avg
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.try_make_interval
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.try_make_timestamp
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.try_parse_json
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.try_parse_url
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.try_reflect
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.try_subtract
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.try_to_date
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.try_to_time
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.try_variant_get
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.udf
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.unwrap_udt
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.uuid
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.variant_get
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.window
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.window_time
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.xxhash64
pyspark/sql/functions/builtin.py::pyspark.sql.functions.builtin.zip_with
pyspark/sql/tests/connect/arrow/test_parity_arrow.py::ArrowParityTests::test_createDataFrame_pandas_duplicate_field_names
pyspark/sql/tests/connect/arrow/test_parity_arrow.py::ArrowParityTests::test_pandas_self_destruct
pyspark/sql/tests/connect/arrow/test_parity_arrow.py::ArrowParityTests::test_toPandas_duplicate_field_names
pyspark/sql/tests/connect/arrow/test_parity_arrow_cogrouped_map.py::CogroupedMapInArrowParityTests::test_cogroup_apply_in_arrow_with_logging
pyspark/sql/tests/connect/arrow/test_parity_arrow_grouped_map.py::ApplyInArrowParityTests::test_apply_in_arrow
pyspark/sql/tests/connect/arrow/test_parity_arrow_grouped_map.py::ApplyInArrowParityTests::test_apply_in_arrow_batching
pyspark/sql/tests/connect/arrow/test_parity_arrow_grouped_map.py::ApplyInArrowParityTests::test_apply_in_arrow_column_order
pyspark/sql/tests/connect/arrow/test_parity_arrow_grouped_map.py::ApplyInArrowParityTests::test_apply_in_arrow_empty_groupby
pyspark/sql/tests/connect/arrow/test_parity_arrow_grouped_map.py::ApplyInArrowParityTests::test_apply_in_arrow_iter_with_logging
pyspark/sql/tests/connect/arrow/test_parity_arrow_grouped_map.py::ApplyInArrowParityTests::test_apply_in_arrow_not_returning_arrow_table
pyspark/sql/tests/connect/arrow/test_parity_arrow_grouped_map.py::ApplyInArrowParityTests::test_apply_in_arrow_partial_iteration
pyspark/sql/tests/connect/arrow/test_parity_arrow_grouped_map.py::ApplyInArrowParityTests::test_apply_in_arrow_returning_empty_dataframe
pyspark/sql/tests/connect/arrow/test_parity_arrow_grouped_map.py::ApplyInArrowParityTests::test_apply_in_arrow_returning_wrong_column_names
pyspark/sql/tests/connect/arrow/test_parity_arrow_grouped_map.py::ApplyInArrowParityTests::test_apply_in_arrow_returning_wrong_types
pyspark/sql/tests/connect/arrow/test_parity_arrow_grouped_map.py::ApplyInArrowParityTests::test_apply_in_arrow_returning_wrong_types_positional_assignment
pyspark/sql/tests/connect/arrow/test_parity_arrow_grouped_map.py::ApplyInArrowParityTests::test_apply_in_arrow_with_key
pyspark/sql/tests/connect/arrow/test_parity_arrow_grouped_map.py::ApplyInArrowParityTests::test_apply_in_arrow_with_logging
pyspark/sql/tests/connect/arrow/test_parity_arrow_grouped_map.py::ApplyInArrowParityTests::test_negative_and_zero_batch_size
pyspark/sql/tests/connect/arrow/test_parity_arrow_grouped_map.py::ApplyInArrowParityTests::test_positional_assignment_conf
pyspark/sql/tests/connect/arrow/test_parity_arrow_map.py::ArrowMapParityTests::test_map_in_arrow_with_barrier_mode
pyspark/sql/tests/connect/arrow/test_parity_arrow_map.py::ArrowMapParityTests::test_map_in_arrow_with_logging
pyspark/sql/tests/connect/arrow/test_parity_arrow_map.py::ArrowMapParityTests::test_nested_extraneous_field
pyspark/sql/tests/connect/arrow/test_parity_arrow_map.py::ArrowMapParityTests::test_nullability_widen
pyspark/sql/tests/connect/arrow/test_parity_arrow_map.py::ArrowMapParityTests::test_top_level_wrong_order
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityLegacyTests::test_chained_udfs_with_variant
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityLegacyTests::test_day_time_interval_in_struct
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityLegacyTests::test_day_time_interval_type_casting
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityLegacyTests::test_multiple_udfs_with_logging
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityLegacyTests::test_udf_cache
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityLegacyTests::test_udf_kill_on_timeout
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityLegacyTests::test_udf_with_char_varchar_return_type
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityLegacyTests::test_udf_with_collated_string_types
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityLegacyTests::test_udf_with_complex_variant_input
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityLegacyTests::test_udf_with_input_file_name
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityLegacyTests::test_udf_with_logging
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityLegacyTests::test_udf_with_pyspark_logger
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityLegacyTests::test_udf_with_udt
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityLegacyTests::test_udf_with_variant_input
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityNonLegacyTests::test_chained_udfs_with_variant
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityNonLegacyTests::test_day_time_interval_in_struct
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityNonLegacyTests::test_day_time_interval_type_casting
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityNonLegacyTests::test_multiple_udfs_with_logging
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityNonLegacyTests::test_udf_cache
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityNonLegacyTests::test_udf_kill_on_timeout
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityNonLegacyTests::test_udf_with_char_varchar_return_type
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityNonLegacyTests::test_udf_with_collated_string_types
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityNonLegacyTests::test_udf_with_complex_variant_input
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityNonLegacyTests::test_udf_with_input_file_name
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityNonLegacyTests::test_udf_with_logging
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityNonLegacyTests::test_udf_with_pyspark_logger
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityNonLegacyTests::test_udf_with_udt
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityNonLegacyTests::test_udf_with_variant_input
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityTests::test_chained_udfs_with_variant
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityTests::test_day_time_interval_in_struct
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityTests::test_day_time_interval_type_casting
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityTests::test_multiple_udfs_with_logging
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityTests::test_udf_cache
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityTests::test_udf_kill_on_timeout
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityTests::test_udf_with_char_varchar_return_type
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityTests::test_udf_with_collated_string_types
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityTests::test_udf_with_complex_variant_input
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityTests::test_udf_with_input_file_name
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityTests::test_udf_with_logging
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityTests::test_udf_with_pyspark_logger
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityTests::test_udf_with_udt
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::ArrowPythonUDFParityTests::test_udf_with_variant_input
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::UDFParityTests::test_chained_udfs_with_variant
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::UDFParityTests::test_multiple_udfs_with_logging
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::UDFParityTests::test_udf_binary_type
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::UDFParityTests::test_udf_binary_type_in_nested_structures
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::UDFParityTests::test_udf_cache
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::UDFParityTests::test_udf_kill_on_timeout
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::UDFParityTests::test_udf_with_char_varchar_return_type
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::UDFParityTests::test_udf_with_collated_string_types
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::UDFParityTests::test_udf_with_complex_variant_input
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::UDFParityTests::test_udf_with_input_file_name
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::UDFParityTests::test_udf_with_logging
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::UDFParityTests::test_udf_with_pyspark_logger
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::UDFParityTests::test_udf_with_udt
pyspark/sql/tests/connect/arrow/test_parity_arrow_python_udf.py::UDFParityTests::test_udf_with_variant_input
pyspark/sql/tests/connect/arrow/test_parity_arrow_udf.py::ArrowPythonUDFParityTests::test_arrow_udf_basic_with_return_type_string
pyspark/sql/tests/connect/arrow/test_parity_arrow_udf.py::ArrowPythonUDFParityTests::test_arrow_udf_wrong_arg
pyspark/sql/tests/connect/arrow/test_parity_arrow_udf_grouped_agg.py::GroupedAggArrowPythonUDFParityTests::test_grouped_agg_arrow_udf_with_logging
pyspark/sql/tests/connect/arrow/test_parity_arrow_udf_grouped_agg.py::GroupedAggArrowPythonUDFParityTests::test_input_output_variant
pyspark/sql/tests/connect/arrow/test_parity_arrow_udf_scalar.py::ScalarArrowPythonUDFParityTests::test_arrow_udf_output_variant
pyspark/sql/tests/connect/arrow/test_parity_arrow_udf_scalar.py::ScalarArrowPythonUDFParityTests::test_scalar_arrow_udf_with_logging
pyspark/sql/tests/connect/arrow/test_parity_arrow_udf_scalar.py::ScalarArrowPythonUDFParityTests::test_scalar_iter_arrow_udf_with_logging
pyspark/sql/tests/connect/arrow/test_parity_arrow_udf_window.py::WindowArrowPythonUDFParityTests::test_bounded_mixed
pyspark/sql/tests/connect/arrow/test_parity_arrow_udf_window.py::WindowArrowPythonUDFParityTests::test_bounded_simple
pyspark/sql/tests/connect/arrow/test_parity_arrow_udf_window.py::WindowArrowPythonUDFParityTests::test_complex_window_collect_as_map
pyspark/sql/tests/connect/arrow/test_parity_arrow_udf_window.py::WindowArrowPythonUDFParityTests::test_kwargs
pyspark/sql/tests/connect/arrow/test_parity_arrow_udf_window.py::WindowArrowPythonUDFParityTests::test_named_arguments
pyspark/sql/tests/connect/arrow/test_parity_arrow_udf_window.py::WindowArrowPythonUDFParityTests::test_named_arguments_negative
pyspark/sql/tests/connect/arrow/test_parity_arrow_udf_window.py::WindowArrowPythonUDFParityTests::test_shrinking_window
pyspark/sql/tests/connect/arrow/test_parity_arrow_udf_window.py::WindowArrowPythonUDFParityTests::test_sliding_window
pyspark/sql/tests/connect/arrow/test_parity_arrow_udf_window.py::WindowArrowPythonUDFParityTests::test_time_min
pyspark/sql/tests/connect/arrow/test_parity_arrow_udf_window.py::WindowArrowPythonUDFParityTests::test_window_arrow_udf_with_logging
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_error_mismatched_schema
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_lateral_join_disallowed
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_lateral_join_with_table_argument_disallowed
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_partition_by_all_columns
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_partition_by_single_partition_multiple_input_partitions
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_partition_column_removal
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_table_argument_with_regular_udtf_lateral_join_allowed
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_table_partition_by_multiple_columns
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_table_partition_by_single_column
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_type_coercion_long_to_int
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_type_coercion_string_to_int
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_type_coercion_string_to_int_safe
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_type_corecion_int64_to_int32_safe
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_with_empty_column_result
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_with_logging
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_with_named_arguments
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_with_partition_by
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_with_partition_by_and_order_by
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_with_partition_by_and_terminate
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_with_partition_by_empty_input_batch
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_with_partition_by_null_values
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_with_partition_by_skip_rest_of_input
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_with_scalar_first_table_second
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_with_table_and_struct_arguments
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_with_table_argument_and_scalar
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_with_table_argument_basic
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_with_table_argument_in_middle
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_arrow_udtf_with_table_argument_then_lateral_join_allowed
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_return_type_coercion_multiple_columns
pyspark/sql/tests/connect/arrow/test_parity_arrow_udtf.py::ArrowUDTFParityTests::test_return_type_coercion_success
pyspark/sql/tests/connect/client/test_artifact.py::ArtifactTests::test_add_archive
pyspark/sql/tests/connect/client/test_artifact.py::ArtifactTests::test_add_file
pyspark/sql/tests/connect/client/test_artifact.py::ArtifactTests::test_add_pyfile
pyspark/sql/tests/connect/client/test_artifact.py::ArtifactTests::test_add_zipped_package
pyspark/sql/tests/connect/client/test_artifact.py::ArtifactTests::test_artifacts_cannot_be_overwritten
pyspark/sql/tests/connect/client/test_artifact.py::ArtifactTests::test_cache_artifact
pyspark/sql/tests/connect/client/test_artifact.py::ArtifactTests::test_copy_from_local_to_fs
pyspark/sql/tests/connect/client/test_artifact.py::ArtifactTests::test_single_chunked_and_chunked_artifact
pyspark/sql/tests/connect/client/test_artifact_localcluster.py::LocalClusterArtifactTests::test_add_archive
pyspark/sql/tests/connect/client/test_artifact_localcluster.py::LocalClusterArtifactTests::test_add_file
pyspark/sql/tests/connect/client/test_artifact_localcluster.py::LocalClusterArtifactTests::test_add_pyfile
pyspark/sql/tests/connect/client/test_artifact_localcluster.py::LocalClusterArtifactTests::test_add_zipped_package
pyspark/sql/tests/connect/client/test_artifact_localcluster.py::LocalClusterArtifactTests::test_artifacts_cannot_be_overwritten
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_grouped_map_with_state.py::GroupedApplyInPandasWithStateTests::test_apply_in_pandas_with_state_basic
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_grouped_map_with_state.py::GroupedApplyInPandasWithStateTests::test_apply_in_pandas_with_state_basic_fewer_data
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_grouped_map_with_state.py::GroupedApplyInPandasWithStateTests::test_apply_in_pandas_with_state_basic_more_data
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_grouped_map_with_state.py::GroupedApplyInPandasWithStateTests::test_apply_in_pandas_with_state_basic_no_state
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_grouped_map_with_state.py::GroupedApplyInPandasWithStateTests::test_apply_in_pandas_with_state_basic_no_state_no_data
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_grouped_map_with_state.py::GroupedApplyInPandasWithStateTests::test_apply_in_pandas_with_state_basic_with_null
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_grouped_map_with_state.py::GroupedApplyInPandasWithStateTests::test_apply_in_pandas_with_state_int_to_decimal_coercion
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_grouped_map_with_state.py::GroupedApplyInPandasWithStateTests::test_apply_in_pandas_with_state_python_worker_random_failure
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state.py::TransformWithStateInPandasParityTests::test_composite_output_schema
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state.py::TransformWithStateInPandasParityTests::test_schema_evolution_fails
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state.py::TransformWithStateInPandasParityTests::test_transform_with_state_basic
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state.py::TransformWithStateInPandasParityTests::test_transform_with_state_batch_query
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state.py::TransformWithStateInPandasParityTests::test_transform_with_state_batch_query_initial_state
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state.py::TransformWithStateInPandasParityTests::test_transform_with_state_chaining_ops
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state.py::TransformWithStateInPandasParityTests::test_transform_with_state_event_time
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state.py::TransformWithStateInPandasParityTests::test_transform_with_state_in_pandas_composite_type
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state.py::TransformWithStateInPandasParityTests::test_transform_with_state_int_to_decimal_coercion
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state.py::TransformWithStateInPandasParityTests::test_transform_with_state_large_values
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state.py::TransformWithStateInPandasParityTests::test_transform_with_state_non_contiguous_grouping_cols
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state.py::TransformWithStateInPandasParityTests::test_transform_with_state_non_contiguous_grouping_cols_with_init_state
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state.py::TransformWithStateInPandasParityTests::test_transform_with_state_proc_timer
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state.py::TransformWithStateInPandasParityTests::test_transform_with_state_query_restarts
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state.py::TransformWithStateInPandasParityTests::test_transform_with_state_with_bytes_limit
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state.py::TransformWithStateInPandasParityTests::test_transform_with_state_with_timers_single_partition
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state.py::TransformWithStateInPandasParityTests::test_transform_with_state_with_wmark_and_non_event_time
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state_state_variable.py::TransformWithStateInPandasStateVariableParityTests::test_transform_with_list_state_metadata
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state_state_variable.py::TransformWithStateInPandasStateVariableParityTests::test_transform_with_map_state_metadata
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state_state_variable.py::TransformWithStateInPandasStateVariableParityTests::test_transform_with_map_state_metadata_with_init_state
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state_state_variable.py::TransformWithStateInPandasStateVariableParityTests::test_transform_with_state_basic
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state_state_variable.py::TransformWithStateInPandasStateVariableParityTests::test_transform_with_state_init_state
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state_state_variable.py::TransformWithStateInPandasStateVariableParityTests::test_transform_with_state_init_state_with_extra_transformation
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state_state_variable.py::TransformWithStateInPandasStateVariableParityTests::test_transform_with_state_init_state_with_timers
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state_state_variable.py::TransformWithStateInPandasStateVariableParityTests::test_transform_with_state_list_state
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state_state_variable.py::TransformWithStateInPandasStateVariableParityTests::test_transform_with_state_list_state_large_list
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state_state_variable.py::TransformWithStateInPandasStateVariableParityTests::test_transform_with_state_list_state_large_ttl
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state_state_variable.py::TransformWithStateInPandasStateVariableParityTests::test_transform_with_state_map_state
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state_state_variable.py::TransformWithStateInPandasStateVariableParityTests::test_transform_with_state_map_state_large_ttl
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state_state_variable.py::TransformWithStateInPandasStateVariableParityTests::test_transform_with_state_non_exist_value_state
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state_state_variable.py::TransformWithStateInPandasStateVariableParityTests::test_transform_with_state_restart_with_multiple_rows_init_state
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state_state_variable.py::TransformWithStateInPandasStateVariableParityTests::test_transform_with_state_with_timers_single_partition
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state_state_variable.py::TransformWithStateInPandasStateVariableParityTests::test_transform_with_value_state_metadata
pyspark/sql/tests/connect/pandas/streaming/test_parity_pandas_transform_with_state_state_variable.py::TransformWithStateInPandasStateVariableParityTests::test_value_state_ttl_basic
pyspark/sql/tests/connect/pandas/streaming/test_parity_transform_with_state.py::TransformWithStateInPySparkParityTests::test_composite_output_schema
pyspark/sql/tests/connect/pandas/streaming/test_parity_transform_with_state.py::TransformWithStateInPySparkParityTests::test_schema_evolution_fails
pyspark/sql/tests/connect/pandas/streaming/test_parity_transform_with_state.py::TransformWithStateInPySparkParityTests::test_transform_with_state_basic
pyspark/sql/tests/connect/pandas/streaming/test_parity_transform_with_state.py::TransformWithStateInPySparkParityTests::test_transform_with_state_batch_query
pyspark/sql/tests/connect/pandas/streaming/test_parity_transform_with_state.py::TransformWithStateInPySparkParityTests::test_transform_with_state_batch_query_initial_state
pyspark/sql/tests/connect/pandas/streaming/test_parity_transform_with_state.py::TransformWithStateInPySparkParityTests::test_transform_with_state_chaining_ops
pyspark/sql/tests/connect/pandas/streaming/test_parity_transform_with_state.py::TransformWithStateInPySparkParityTests::test_transform_with_state_event_time
pyspark/sql/tests/connect/pandas/streaming/test_parity_transform_with_state.py::TransformWithStateInPySparkParityTests::test_transform_with_state_in_pandas_composite_type
pyspark/sql/tests/connect/pandas/streaming/test_parity_transform_with_state.py::TransformWithStateInPySparkParityTests::test_transform_with_state_large_values
pyspark/sql/tests/connect/pandas/streaming/test_parity_transform_with_state.py::TransformWithStateInPySparkParityTests::test_transform_with_state_non_contiguous_grouping_cols
pyspark/sql/tests/connect/pandas/streaming/test_parity_transform_with_state.py::TransformWithStateInPySparkParityTests::test_transform_with_state_non_contiguous_grouping_cols_with_init_state
pyspark/sql/tests/connect/pandas/streaming/test_parity_transform_with_state.py::TransformWithStateInPySparkParityTests::test_transform_with_state_proc_timer
pyspark/sql/tests/connect/pandas/streaming/test_parity_transform_with_state.py::TransformWithStateInPySparkParityTests::test_transform_with_state_query_restarts
pyspark/sql/tests/connect/pandas/streaming/test_parity_transform_with_state.py::TransformWithStateInPySparkParityTests::test_transform_with_state_with_timers_single_partition
pyspark/sql/tests/connect/pandas/streaming/test_parity_transform_with_state.py::TransformWithStateInPySparkParityTests::test_transform_with_state_with_wmark_and_non_event_time
pyspark/sql/tests/connect/pandas/streaming/test_parity_transform_with_state_state_variable.py::TransformWithStateInPySparkStateVariableParityTests::test_transform_with_list_state_metadata
pyspark/sql/tests/connect/pandas/streaming/test_parity_transform_with_state_state_variable.py::TransformWithStateInPySparkStateVariableParityTests::test_transform_with_map_state_metadata
pyspark/sql/tests/connect/pandas/streaming/test_parity_transform_with_state_state_variable.py::TransformWithStateInPySparkStateVariableParityTests::test_transform_with_map_state_metadata_with_init_state
pyspark/sql/tests/connect/pandas/streaming/test_parity_transform_with_state_state_variable.py::TransformWithStateInPySparkStateVariableParityTests::test_transform_with_state_basic
pyspark/sql/tests/connect/pandas/streaming/test_parity_transform_with_state_state_variable.py::TransformWithStateInPySparkStateVariableParityTests::test_transform_with_state_init_state
pyspark/sql/tests/connect/pandas/streaming/test_parity_transform_with_state_state_variable.py::TransformWithStateInPySparkStateVariableParityTests::test_

(truncated)

@davidlghellin davidlghellin mentioned this pull request Apr 29, 2026
Open
10 of 12 tasks
@davidlghellin

davidlghellin commented Apr 30, 2026

Copy link
Copy Markdown
Contributor Author

apache/datafusion#21895

@davidlghellin

davidlghellin commented Jun 11, 2026

Copy link
Copy Markdown
Contributor Author

DF 54 is merge
Closing this WIP — too many upstream merge conflicts. Re-opening a clean PR from main with just the array_filter lambda work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@davidlghellin