You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2020/04/13 05:07:00 UTC

[jira] [Resolved] (SPARK-31403) TreeNode asCode function incorrectly handles null literals

     [ https://issues.apache.org/jira/browse/SPARK-31403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyukjin Kwon resolved SPARK-31403.
----------------------------------
    Resolution: Cannot Reproduce

Seems I can't reproduce from the master. Probably fixed somewhere else:

{code}
import org.apache.spark.sql.catalyst.plans.logical.Project

spark.range(10).createOrReplaceTempView("users")

val plan = (spark
  .sql("select if(isnull(id), null, 2) from users")
  .queryExecution
  .optimizedPlan)

println(plan.asInstanceOf[Project].projectList.head.asCode) 
{code}

> TreeNode asCode function incorrectly handles null literals
> ----------------------------------------------------------
>
>                 Key: SPARK-31403
>                 URL: https://issues.apache.org/jira/browse/SPARK-31403
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.4.4
>            Reporter: Carl Sverre
>            Priority: Minor
>
> In the TreeNode code in Catalyst the asCode function incorrectly handles null literals.  When it tries to render a null literal it will match {{null}} using the third case expression and try to call {{null.toString}} which will raise a NullPointerException.
> I verified this bug exists in Spark 2.4.4 and the same code appears to be in master:
> [https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/trees/TreeNode.scala#L707]
> The fix seems trivial - add an explicit case for null.
> One way to reproduce this is via:
> {code:java}
>       val plan =
>         spark
>           .sql("select if(isnull(id), null, 2) from testdb_jdbc.users")
>           .queryExecution
>           .optimizedPlan
>       println(plan.asInstanceOf[Project].projectList.head.asCode) {code}
> However any other way which generates a Literal with the value null will cause the issue.
> In this case the above SparkSQL will generate the literal: {{Literal(null, IntegerType)}} for the "trueValue" of the if statement.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org