[GitHub] [spark] cloud-fan commented on a change in pull request #31797: [SPARK-34692][SQL] Support Not(Int) and Not(InSet) propagate null in predicate

[GitBox <gi...@apache.org>]

cloud-fan commented on a change in pull request #31797:
URL: https://github.com/apache/spark/pull/31797#discussion_r592368015



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ReplaceNullWithFalseInPredicate.scala
##########
@@ -95,6 +95,11 @@ object ReplaceNullWithFalseInPredicate extends Rule[LogicalPlan] {
   private def replaceNullWithFalse(e: Expression): Expression = e match {
     case Literal(null, BooleanType) =>
       FalseLiteral
+    case Not(In(value, list)) if (value +: list).exists(isNullLiteral) =>

Review comment:
       Can we add some comments to explain the rationale?
   
   if `value` is null, the IN result is null and thus `Not(null)` means false. If `value` is not null but the list contains null values, the IN result can either be `true` or `null`, and thus the final result after Not is always false as a predicate.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org