You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Xiaoju Wu (Jira)" <ji...@apache.org> on 2020/05/30 16:01:00 UTC

[jira] [Created] (SPARK-31872) NotNullSafe to get complementary set

Xiaoju Wu created SPARK-31872:
---------------------------------

             Summary: NotNullSafe to get complementary set
                 Key: SPARK-31872
                 URL: https://issues.apache.org/jira/browse/SPARK-31872
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 2.4.0, 2.3.0, 3.0.0
            Reporter: Xiaoju Wu


If we have a filter expression to get subset of rows, and then we want the complementary set, Not(expression) cannot work, since Not is NullIntolerent, if expression.eval(row) is null, filter predicate is false too for Not(expression), "row" will not appear in both subset and complementary set.

So, maybe we need a NotNullSafe implementation to get the complementary set which will result in true if the expression.eval(row) is null.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org