[jira] [Resolved] (SPARK-34857) AtLeastNNonNulls does not show up correctly in explain

["Hyukjin Kwon (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-34857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyukjin Kwon resolved SPARK-34857.
----------------------------------
    Fix Version/s: 3.2.0
       Resolution: Fixed

Issue resolved by pull request 31956
[https://github.com/apache/spark/pull/31956]

> AtLeastNNonNulls does not show up correctly in explain
> ------------------------------------------------------
>
>                 Key: SPARK-34857
>                 URL: https://issues.apache.org/jira/browse/SPARK-34857
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 3.2.0, 3.1.2
>            Reporter: Tim Armstrong
>            Assignee: Tim Armstrong
>            Priority: Minor
>             Fix For: 3.2.0
>
>
> I see this in an explain plan
> {noformat}
> (12) Filter
> Input [3]: [c1#2410L, c2#2419, c3#2422]
> Condition : AtLeastNNulls(n, c1#2410L)
> I expect it to be AtLeastNNonNulls and n to have the actual value.
> {noformat}
> Proposed fix is to change https://github.com/databricks/runtime/blob/09ad623c458c40cbaa0dbb4b9f600d96531d35f5/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/nullExpressions.scala#L384 to
> {noformat}
> override def toString: String = s"AtLeastNNonNulls(${n}, ${children.mkString(",")})"
> {noformat}
> Or maybe it's OK to remove and use a default implementation?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org