You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dhiren Navani (Jira)" <ji...@apache.org> on 2021/10/31 03:39:00 UTC

[jira] [Created] (SPARK-37171) Addition of forany and forall semantics to Spark Dataframes

Dhiren Navani created SPARK-37171:
-------------------------------------

             Summary: Addition of forany and forall semantics to Spark Dataframes
                 Key: SPARK-37171
                 URL: https://issues.apache.org/jira/browse/SPARK-37171
             Project: Spark
          Issue Type: New Feature
          Components: SQL
    Affects Versions: 3.2.0
            Reporter: Dhiren Navani


Spark dataframes currently do not have semantics of forany and forall.

Scala collections do have these semantics.

It would be nice to have them as an API as there is potential for implementing them in unoptimized form.

 

E.g. forany might be implemented as df.filter(<condition>).count > 0, which is not optimal



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org