You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Allison Wang (Jira)" <ji...@apache.org> on 2021/04/07 01:58:00 UTC

[jira] [Created] (SPARK-34974) Improve the subquery decorrelation framework

Allison Wang created SPARK-34974:
------------------------------------

             Summary: Improve the subquery decorrelation framework
                 Key: SPARK-34974
                 URL: https://issues.apache.org/jira/browse/SPARK-34974
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 3.2.0
            Reporter: Allison Wang


Currently, Spark has limited support for correlated subqueries. It only allows Filter to have outer column references and does not support non-equality predicates when the subquery is aggregated. Please see https://issues.apache.org/jira/browse/SPARK-18455 for more details on how subquery decorrelation works in Spark.

The task here is to implement another technique introduced in "Unnesting Arbitrary Queries" by T. Neumann; A. Kemper
([http://www.btw-2015.de/res/proceedings/Hauptband/Wiss/Neumann-Unnesting_Arbitrary_Querie.pdf]). This allows Spark to have outer query column references other than Filter, decorrelate subqueries with more types of operators, and support correlated non-equality predicates.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org