Skip to content

from_csv: wrong results in permissive modes and failure with a variant schema #6800

Open
Open
from_csv: wrong results in permissive modes and failure with a variant schema#6800
Labels
area:expressionsExpression evaluationbugSomething isn't workingcorrectnessexpressionSpark expression relatedspark sql testsSpark SQL test failures

Description

@andygrove
andygrove
opened on Oct 8, 2026

Found while enabling spark.comet.exec.localTableScan.enabled by default in draft PR #6780 (Spark 4.1 SQL tests and [scans] job). Part of #4347.

CsvFunctionsSuite "from_csv invalid csv - check modes" (results differ), "from_csv with variant" (ArrayIndexOutOfBoundsException in GenericInternalRow). Not yet diagnosed.

Activity

  1. andygrove
    added
    expressionSpark expression related
    spark sql testsSpark SQL test failures
    on Oct 8, 2026
  2. andygrove
    added a parent issue on Oct 8, 2026
  3. github-actions
    added
    requires-triage
    on Oct 8, 2026
  4. andygrove
    added
    bugSomething isn't working
    priority:criticalData corruption, silent wrong results, security issues
    area:expressionsExpression evaluation
    and removed
    requires-triage
    on Oct 12, 2026
  5. kazuyukitanimura
    added
    correctness
    and removed
    priority:criticalData corruption, silent wrong results, security issues
    on Oct 12, 2026

  6. kazuyukitanimura commented on Oct 12, 2026

    @kazuyukitanimura
    kazuyukitanimura
    on Oct 12, 2026
    Contributor

    not enabled by default yet

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:expressionsExpression evaluationbugSomething isn't workingcorrectnessexpressionSpark expression relatedspark sql testsSpark SQL test failures

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions