fix: return null from `mean_horizontal` for an all-null row on PyArrow, DuckDB, Ibis and Spark-like by SausageMan99 · Pull Request #4059 · narwhals-dev/narwhals

SausageMan99 · 2026-10-11T12:48:05Z

Description

For a row where every input is null, the count of non-null values is 0, so mean_horizontal computed 0 / 0 and returned NaN on PyArrow, DuckDB, Ibis and Spark-like, where NaN is not null. It now returns null there, like pandas and Polars.

The issue mentions PyArrow and DuckDB; the new test showed the same result on Ibis and SQLFrame, so those are fixed too.

What type of PR is this? (check all applicable)

Related issues

Closes [Bug]: mean_horizontal returns NaN instead of null for all-null rows on PyArrow and DuckDB #4056

AI assistance

No AI tools were used for this PR.
AI tools were used. (Claude Code, Claude Opus 5.5: investigation, fix and tests; I reviewed the diff and ran the suite locally.)

Checklist

Code follows style guide (ruff)
Tests added
Documented the changes
If this is your first PR to narwhals, attach a screenshot of pytest passing locally (not CI):

Local run (template command + ibis)

Required test coverage of 80.0% reached. Total coverage: 100.00%
17716 passed, 275 skipped, 1090 xfailed

🤖 Generated with Claude Code

Summary by CodeRabbit

Bug Fixes
- Horizontal mean now returns null when all values in a row are null, while continuing to calculate the average for rows with non-null values.

…w, DuckDB, Ibis and Spark-like Closes narwhals-dev#4056 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

coderabbitai · 2026-10-11T12:48:23Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults
Review profile: CHILL
Plan: Advanced
Run ID: fb258be1-480d-42b7-a9de-561cd0a53482

📥 Commits

Reviewing files that changed from the base of the PR and between 4d6f7b0 and 143e04b.

📒 Files selected for processing (5)

src/narwhals/_arrow/namespace.py
src/narwhals/_duckdb/namespace.py
src/narwhals/_ibis/namespace.py
src/narwhals/_spark_like/namespace.py
tests/expr_and_series/mean_horizontal_test.py

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.

📝 Walkthrough

Walkthrough

mean_horizontal now returns null when all inputs are null in the Arrow, DuckDB, Ibis, and Spark-like implementations. A test checks the all-null result, its null status, and the average for a row with values.

Changes

Horizontal mean null handling

Layer / File(s)	Summary
Guard horizontal mean for all-null rows `src/narwhals/_arrow/namespace.py`, `src/narwhals/_duckdb/namespace.py`, `src/narwhals/_ibis/namespace.py`, `src/narwhals/_spark_like/namespace.py`, `tests/expr_and_series/mean_horizontal_test.py`	The four backend implementations return null when the non-null input count is zero. The test checks that all-null inputs produce null and that inputs `[1.0, 3.0]` produce `2.0`.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Medium

Suggested reviewers: fbruzzesi

Merge Risk | ⚪ Minimal · up to 143e0

Merge Risk: ⚪ Minimal · up to 143e0

The all-null horizontal mean behavior is covered across the changed backend paths, and no actionable merge blocker remains.

Security Architecture Review

Security architecture risk: ⚪ Minimal · up to 143e0

The change aligns all-null horizontal means with null semantics while preserving the existing operation and execution paths. No introduced or worsened security risk was identified in the reviewed changes.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

inferred — The changed behavior applies to data already evaluated through the existing horizontal-mean operation in the four affected backends. The inspected comparison does not expand input acceptance, introduce callers, or grant access to additional assets or execution environments.

Trust Boundaries and Controls

inferred — The changed branches depend on a computed non-null count and produce backend-native arithmetic or conditional expressions. Within the inspected methods, input values remain data; the change adds no string-based command construction, credential access, or authorization bypass.

Pre-merge checks | 4 | 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 10 functions across 5 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check		The title clearly and concisely identifies the main fix: returning null for all-null rows in mean_horizontal across the affected backends.
Description check		The description explains the bug, affected backends, expected behavior, linked issue, AI assistance, testing, and checklist status. It is sufficiently complete for this change.
Linked Issues check		Issue `#4056` requires `mean_horizontal` to return null for an all-null row on PyArrow and DuckDB, with `is_null()` equal to `True`. The PyArrow implementation replaces the zero-count result with null. …
Out of Scope Changes check		The changes stay within the linked bug scope. The Ibis and Spark-like updates correct the same all-null `mean_horizontal` behavior described in the PR. The added test verifies the required null value …

Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create a new PR

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

FBruzzesi · 2026-10-11T21:38:16Z

+        nw.col("mean"), is_null=nw.col("mean").is_null()
+    )
+    expected = {"mean": [2.0, None], "is_null": [False, True]}
+    assert_equal_data(result, expected)


Could we extend this a bit?

an integer case, since that takes a different division path on several backends;

a check that the output is still Float64 when every row is null, as in Polars.

This needs import pytest at the top of the file. The filter is there to get an all-null column that keeps its dtype: building all-None columns directly fails on Ibis.

Suggested change

assert_equal_data(result, expected)

@pytest.mark.parametrize(

"data", [{"a": [1.0, None], "b": [3.0, None]}, {"a": [1, None], "b": [3, None]}]

)

def test_meanh_all_null_row_is_null(

constructor: Constructor, data: dict[str, list[float | None]]

) -> None:

df = nw.from_native(constructor(data))

mean = nw.mean_horizontal("a", "b")

result = df.select(mean=mean, is_null=mean.is_null())

expected = {"mean": [2.0, None], "is_null": [False, True]}

assert_equal_data(result, expected)

result = df.filter(nw.col("a").is_null()).select(mean=mean)

assert result.collect_schema() == {"mean": nw.Float64()}

FBruzzesi

Thanks for the PR @SausageMan99, and nice catch extending the fix to Ibis and Spark-like beyond what the issue reported!

Two things before we can merge:

1. pandas-like still returns NaN on pandas<3 with nullable and pyarrow-backed dtypes. The new test fails on pandas[nullable] and pandas[pyarrow] with pandas 2.x (I checked 2.2.3 and 2.3.3): on those dtypes 0 / 0 gives NaN, which is not null. pandas 3 returns <NA> there, which is likely why it passed for you locally. CI runs these constructors on pandas 2.x, so it would fail there.

#4060 had the fix for this one, in src/narwhals/_pandas_like/namespace.py::mean_horizontal:

non_na = (1 - s.is_null() for s in expr_results)
count = reduce(operator.add, non_na)
return [(reduce(operator.add, series) / count).zip_with(count > 0, None)]

Could you port it, and credit @ioseba with a trailer in the commit message?

Co-authored-by: I. Alonso <11636356+ioseba@users.noreply.github.com>

2. Test: pin the output dtype. See the inline comment. When every row is null, the output should still be Float64, as in Polars. That already holds on every backend with this PR, and the test should lock it in.

Optional: now that all backends agree, we could make the guarantee explicit in the docstring, in both src/narwhals/functions.py and src/narwhals/stable/v2/__init__.py:

Null values are ignored. If all values in a row are null, the result is null.

With both changes, the test passes locally on pandas, pandas[nullable], pandas[pyarrow], polars (eager and lazy), pyarrow, duckdb, sqlframe, ibis, dask and modin.

fix: return null from mean_horizontal for an all-null row on PyArro…

143e04b

…w, DuckDB, Ibis and Spark-like Closes narwhals-dev#4056 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

FBruzzesi mentioned this pull request Oct 11, 2026

fix: return null for all-null rows in mean_horizontal (#4056) #4060

Closed

6 tasks done

FBruzzesi reviewed Oct 11, 2026

View reviewed changes

FBruzzesi requested changes Oct 11, 2026

View reviewed changes

FBruzzesi added the fix label Oct 11, 2026

Navigation Menu

Repository navigation

fix: return null from `mean_horizontal` for an all-null row on PyArrow, DuckDB, Ibis and Spark-like - #4059

fix: return null from `mean_horizontal` for an all-null row on PyArrow, DuckDB, Ibis and Spark-like#4059
SausageMan99 wants to merge 1 commit into
narwhals-dev:mainfrom
SausageMan99:fix/4056-mean-horizontal-all-null

SausageMan99 commented Oct 11, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Oct 11, 2026 •

edited

Loading

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

FBruzzesi Oct 11, 2026

Uh oh!

FBruzzesi left a comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

-    assert_equal_data(result, expected)
+@pytest.mark.parametrize(
+    "data", [{"a": [1.0, None], "b": [3.0, None]}, {"a": [1, None], "b": [3, None]}]
+)
+def test_meanh_all_null_row_is_null(
+    constructor: Constructor, data: dict[str, list[float | None]]
+) -> None:
+    df = nw.from_native(constructor(data))
+    mean = nw.mean_horizontal("a", "b")
+    result = df.select(mean=mean, is_null=mean.is_null())
+    expected = {"mean": [2.0, None], "is_null": [False, True]}
+    assert_equal_data(result, expected)
+    result = df.filter(nw.col("a").is_null()).select(mean=mean)
+    assert result.collect_schema() == {"mean": nw.Float64()}

Conversation

SausageMan99 commented Oct 11, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

What type of PR is this? (check all applicable)

Related issues

AI assistance

Checklist

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Oct 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

FBruzzesi Oct 11, 2026

Choose a reason for hiding this comment

Uh oh!

FBruzzesi left a comment

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

SausageMan99 commented Oct 11, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Oct 11, 2026 •

edited

Loading