Skip to content

Internal error (nullability mismatch) when aggregating over CASE WHEN a < b THEN a ELSE ... END #26284

Open

Description

@acbokade

Describe the bug

Aggregating over a projected CASE expression whose WHEN compares two columns fails with an internal error:

Internal error: Physical input schema should be the same as the one converted from logical input schema. Differences: 
	- field nullability at index 0 [x]: (physical) true vs (logical) false.
This issue was likely caused by a bug in DataFusion's code. ...

The logical schema marks CASE WHEN a < b THEN a ELSE 1 END as non-nullable (THEN a is only reached when a < b is true, so a isn't NULL there), but the physical CaseExpr reports it as nullable. The physical planner compares the two schemas for an aggregate's input, so queries that compute the expression below an aggregate fail, for example DISTINCT, UNION, or an aggregate over a subquery.

To Reproduce

CREATE TABLE n(a INT, b INT) AS VALUES (NULL, 1), (0, 5), (7, 2);

SELECT DISTINCT CASE WHEN a < b THEN a ELSE 1 END AS x FROM n;

These fail the same way:

SELECT sum(x) FROM (SELECT CASE WHEN a < b THEN a ELSE 1 END AS x FROM n);
SELECT CASE WHEN a < b THEN a ELSE 1 END AS x FROM n UNION SELECT 2;

With WHEN a > 0 or WHEN a IS NOT NULL in place of a < b, the DISTINCT query works.

Tested on main at 0566b55.

Expected behavior

0 and 1 for the DISTINCT query, 2 for the sum, and 0, 1, 2 for the UNION, as in PostgreSQL.

Additional context

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions