You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "koert kuipers (Jira)" <ji...@apache.org> on 2023/07/06 17:09:00 UTC

[jira] [Created] (SPARK-44323) Scala None shows up as null for Aggregator BUF or OUT

koert kuipers created SPARK-44323:
-------------------------------------

             Summary: Scala None shows up as null for Aggregator BUF or OUT  
                 Key: SPARK-44323
                 URL: https://issues.apache.org/jira/browse/SPARK-44323
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 3.4.1
            Reporter: koert kuipers


when doing an upgrade from spark 3.3.1 to spark 3.4.1 we suddenly started getting null pointer exceptions in Aggregators (classes extending org.apache.spark.sql.expressions.Aggregator) that use scala Option for BUF and/or OUT. basically None is now showing up as null.

after adding a simple test case and doing a binary search on commits we landed on SPARK-37829 being the cause.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org