[GitHub] [spark] allisonwang-db commented on a diff in pull request #43111: [SPARK-36112] [SQL] Support correlated exists subqueries using DecorrelateInnerQuery framework

["allisonwang-db (via GitHub)" <gi...@apache.org>]

allisonwang-db commented on code in PR #43111:
URL: https://github.com/apache/spark/pull/43111#discussion_r1342014577


##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala:
##########
@@ -63,6 +64,18 @@ object RewritePredicateSubquery extends Rule[LogicalPlan] with PredicateHelper {
     Join(outerPlan, dedupSubplan, joinType, condition, JoinHint(None, subHint))
   }
 
+  private def removeDomainJoins(

Review Comment:
   nit: `maybeRewriteDomainJoins`



##########
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##########
@@ -3416,6 +3416,14 @@ object SQLConf {
       .booleanConf
       .createWithDefault(true)
 
+  val DECORRELATE_EXISTS_AND_IN_SUBQUERIES =
+    buildConf("spark.sql.optimizer.decorrelateExistsIn.enabled")

Review Comment:
   Does this flag depend on this `DECORRELATE_INNER_QUERY_ENABLED` flag?



##########
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##########
@@ -3416,6 +3416,14 @@ object SQLConf {
       .booleanConf
       .createWithDefault(true)
 
+  val DECORRELATE_EXISTS_AND_IN_SUBQUERIES =
+    buildConf("spark.sql.optimizer.decorrelateExistsIn.enabled")
+      .internal()
+      .doc("Decorrelate EXISTS and IN subqueries.")
+      .version("4.0.0")
+      .booleanConf
+      .createWithDefault(true)

Review Comment:
   Will this have any query result changes if we enable this by default?



##########
sql/core/src/test/resources/sql-tests/inputs/subquery/exists-subquery/exists-count-bug.sql:
##########
@@ -0,0 +1,21 @@
+create temporary view t1(c1, c2) as values (0, 1), (1, 2);
+create temporary view t2(c1, c2) as values (0, 2), (0, 3);
+create temporary view t3(c1, c2) as values (0, 3), (1, 4), (2, 5);
+
+select * from t1 where exists (select count(*) from t2 where t2.c1 = t1.c1);

Review Comment:
   are results before and after this PR the same for these queries?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org