[Bug]: skew() and kurtosis() lose the dtype on an all-null or empty column #4044

Describe the bug

skew() and kurtosis() over a column with no non-null values come back with a different dtype on every backend: String on pandas, Unknown on pyarrow, Float64 on Polars and DuckDB. On pandas it is not only cosmetic — chaining .is_nan() onto the result raises, so the result of a reduction over an emptied column cannot be used where the same reduction over a populated column can.

kurtosis behaves identically. An empty column produced with .filter(nw.col("a") < 0) gives the same result as an all-null one.

Cause

skew and kurtosis return a bare Python None when the column has no non-null values:

• src/narwhals/_pandas_like/series.py:574 and :588
• src/narwhals/_arrow/series.py:408 and :422

That None is turned into a one-row column by series._from_scalar(...) in src/narwhals/_compliant/expr.py:382, and the dtype is inferred from it.

This is narwhals' own substitution rather than something a backend returns. pandas itself gives a typed nan for the same input, and pc.mean gives a typed null:

Why the tests do not catch it

test_skew_expr already covers ([], None), but assert_equal_data compares values and not dtypes, and treats nan and None as equal (tests/utils.py:160-163). The empty branch is exercised on every run; only its type is never looked at.

Proposed fix

Two halves, which can land separately.

pandas-like — src/narwhals/_pandas_like/series.py, in both skew and kurtosis:

         if len(ser_not_null) == 0:
-            return None
+            return float("nan")

This alone fixes all three pandas backends. It does not depend on anything else.

pyarrow — src/narwhals/_arrow/series.py, in both skew and kurtosis:

         if len(ser_not_null) == 0:
-            return None
+            return maybe_extract_py_scalar(lit(None, pa.float64()), _return_py_scalar)

Going through maybe_extract_py_scalar means the Series API is unchanged: with _return_py_scalar=True it still returns Python None, and only the expression path receives the typed scalar.

This half is a no-op without #4043 — I checked: _from_scalar currently calls .as_py() on the scalar, so the type is discarded again and the dtype stays Unknown. It only takes effect once #4043 (or its narrower variant) lands.

Behaviour change on pandas

Series.skew() and Series.kurtosis() on an empty or all-null column change from None to nan on pandas-like backends. On the same backends mean and std already return nan (numpy-backed) or <NA> (nullable), so skew/kurtosis are currently the odd ones out:

test_skew_series's ([], None) case still passes, because assert_equal_data treats nan and None as equal.

I think this is the right trade: it makes skew/kurtosis agree with mean and std on the same backend, rather than agreeing with Polars on a value that pandas cannot represent anyway. But it is a visible change to a public return value, so I would rather you decide than assume.

Verification

Constructors: pandas,pandas[nullable],pandas[pyarrow],pyarrow,polars[eager],polars[lazy],dask,duckdb. The single xpass is the pre-existing list/contains_test.py::test_contains_chunked[polars[eager]] ("Polars 1.44 mixes up chunks that are slices of the same list array").

The numbers above were produced in a Linux container rather than on my own machine.

Related

This is the same family as #4031, which fixed str.len_chars returning a non-numeric dtype on an all-null object column, and tested it with:

assert result.schema["a"].is_numeric()

I would use the same shape of test here: assert the dtype of a reduction over an all-null column equals the dtype of the same reduction over the populated column.

It is also adjacent to #4043, which fixes the pyarrow _from_scalar type loss that this one depends on, but it is a separate cause — skew/kurtosis discard the type before reaching _from_scalar, so #4043 does not fix them.

I would like to pick this up myself either way — tell me which direction you want and I will open the PR. I plan to wait until #4043 lands before opening it, since the pyarrow half has no effect without it; if you would rather have the pandas half on its own first, I can open that immediately.

Steps or code to reproduce the bug

Against main at b89e14a:

import narwhals as nw
import pandas as pd, pyarrow as pa, polars as pl, duckdb

def all_null(frame):
    return nw.from_native(frame).with_columns(a=nw.when(nw.col("a") < 0).then(nw.col("a")))

data = {"a": [1.0, 2.0, 3.0, 4.0]}
frames = {
    "pandas": pd.DataFrame(data),
    "pandas[nullable]": pd.DataFrame(data).convert_dtypes(dtype_backend="numpy_nullable"),
    "pandas[pyarrow]": pd.DataFrame(data).convert_dtypes(dtype_backend="pyarrow"),
    "pyarrow": pa.table(data),
    "polars": pl.DataFrame(data),
    "duckdb": duckdb.sql("select * from (values (1.0),(2.0),(3.0),(4.0)) t(a)"),
}
for name, frame in frames.items():
    dtype = all_null(frame).select(nw.col("a").skew()).lazy().collect().collect_schema()["a"]
    print(f"{name:18} {dtype}")

And, for the downstream effect on pandas:

all_null(pd.DataFrame(data)).select(nw.col("a").skew().is_nan())

Expected results

Float64 on every backend, as Polars and DuckDB already give, and .is_nan() usable on the result.

skew and kurtosis produce Float64 for every input dtype, so there is no question of what the empty result should be typed as:

Actual results

And on the three pandas backends:

narwhals.exceptions.InvalidOperationError: `.is_nan` only supported for numeric dtype and not String, did you mean `.is_null`?

On pyarrow .is_nan() does not raise, but the column dtype is still Unknown.

Please run narwhals.show_versions() and enter the output below.

System:
    python: 3.11.15 (main, Mar  3 2026, 09:26:23) [GCC 13.3.0]
executable: /home/claude/nwsrc/.venv/bin/python
   machine: Linux-6.18.44-fc-v80-x86_64-with-glibc2.39

Python dependencies:
     narwhals: 2.27.0
        numpy: 2.4.6
       pandas: 3.0.6
        modin:
         cudf:
      pyarrow: 25.0.1
      pyspark:
       polars: 1.44.2
         dask: 2026.8.0
       duckdb: 1.5.6
         ibis:
     sqlframe:

Relevant log output