You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "viirya (via GitHub)" <gi...@apache.org> on 2023/07/24 05:44:23 UTC
[GitHub] [spark] viirya commented on a diff in pull request #42117: [SPARK-44517][SQL] Respect ignorenulls and child's nullability in first
viirya commented on code in PR #42117:
URL: https://github.com/apache/spark/pull/42117#discussion_r1271772407
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/First.scala:
##########
@@ -60,7 +60,13 @@ case class First(child: Expression, ignoreNulls: Boolean)
this(child, FirstLast.validateIgnoreNullExpr(ignoreNullsExpr, "first"))
}
- override def nullable: Boolean = true
+ override def nullable: Boolean = {
+ if (ignoreNulls) {
+ false
Review Comment:
Hmm, even `ignoreNulls` is true to ignore null values, because `first`'s initial value is null, the evaluated value of `First` is still possibly null if,
1. No inputs, i.e. empty relation
2. All input values are null
I think this maybe why `nullable` is set to true always.
Although we can use `child.nullable` to check if input value could be null (2), but we cannot know if inputs are empty or not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org