You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/03/20 08:54:43 UTC
[GitHub] [spark] wangyum opened a new pull request #31907: [SPARK-34807][SQL] Push down filter through window after TransposeWindow
wangyum opened a new pull request #31907:
URL: https://github.com/apache/spark/pull/31907
### What changes were proposed in this pull request?
This pr move `TransposeWindow` ahead of `Operator push down` to make `PushDownPredicates` push down more cases. For example:
```scala
spark.range(10).selectExpr("id AS a", "id AS b", "id AS c", "id AS d").createTempView("t1")
val df = spark.sql(
"""
|SELECT *
| FROM (
| SELECT b,
| sum(d) OVER (PARTITION BY a, b),
| rank() OVER (PARTITION BY a ORDER BY c)
| FROM t1
| ) v1
|WHERE b = 2
|""".stripMargin).explain(true)
```
Before this pr:
```
== Optimized Logical Plan ==
Project [b#221L, sum(d) OVER (PARTITION BY a, b ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)#231L, RANK() OVER (PARTITION BY a ORDER BY c ASC NULLS FIRST ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW)#232]
+- Filter (b#221L = 2)
+- Window [rank(c#222L) windowspecdefinition(a#220L, c#222L ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS RANK() OVER (PARTITION BY a ORDER BY c ASC NULLS FIRST ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW)#232], [a#220L], [c#222L ASC NULLS FIRST]
+- Project [b#221L, a#220L, c#222L, sum(d) OVER (PARTITION BY a, b ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)#231L]
+- Window [sum(d#223L) windowspecdefinition(a#220L, b#221L, specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) AS sum(d) OVER (PARTITION BY a, b ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)#231L], [a#220L, b#221L]
+- Project [id#218L AS b#221L, id#218L AS d#223L, id#218L AS a#220L, id#218L AS c#222L]
+- Range (0, 10, step=1, splits=Some(2))
```
After this pr:
```
== Optimized Logical Plan ==
Project [b#221L, sum(d) OVER (PARTITION BY a, b ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)#231L, RANK() OVER (PARTITION BY a ORDER BY c ASC NULLS FIRST ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW)#232]
+- Window [sum(d#223L) windowspecdefinition(a#220L, b#221L, specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) AS sum(d) OVER (PARTITION BY a, b ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)#231L], [a#220L, b#221L]
+- Project [b#221L, d#223L, a#220L, RANK() OVER (PARTITION BY a ORDER BY c ASC NULLS FIRST ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW)#232]
+- Filter (b#221L = 2)
+- Window [rank(c#222L) windowspecdefinition(a#220L, c#222L ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS RANK() OVER (PARTITION BY a ORDER BY c ASC NULLS FIRST ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW)#232], [a#220L], [c#222L ASC NULLS FIRST]
+- Project [id#218L AS b#221L, id#218L AS d#223L, id#218L AS a#220L, id#218L AS c#222L]
+- Range (0, 10, step=1, splits=Some(2))
```
### Why are the changes needed?
Improve query performance.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Unit test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-804212337
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40938/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-804361362
**[Test build #136354 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136354/testReport)** for PR 31907 at commit [`0843b6b`](https://github.com/apache/spark/commit/0843b6ba6ab8c7a3df55a7bae11774001757ab4c).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-803298516
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40864/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-803314666
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136281/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-803373253
**[Test build #136282 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136282/testReport)** for PR 31907 at commit [`1dc44de`](https://github.com/apache/spark/commit/1dc44de9840b97ee853f42036884f147138fe8e4).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-804145945
**[Test build #136354 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136354/testReport)** for PR 31907 at commit [`0843b6b`](https://github.com/apache/spark/commit/0843b6ba6ab8c7a3df55a7bae11774001757ab4c).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-803298520
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-803282715
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40863/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-803282709
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40863/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-803290821
**[Test build #136282 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136282/testReport)** for PR 31907 at commit [`1dc44de`](https://github.com/apache/spark/commit/1dc44de9840b97ee853f42036884f147138fe8e4).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
wangyum commented on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-808706791
@tanelk Could you create pr for your changes: https://github.com/apache/spark/pull/31907#issuecomment-803521361. I'm close this pr.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] tanelk commented on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
tanelk commented on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-803521361
@wangyum, I'll answer your question (https://github.com/apache/spark/pull/31677#pullrequestreview-616975279) here.
Just to clarify - the #31677 does not fix the issue, you are trying to fix here. But they are very similar. I improved the `CollapseWindow`, the `TransposeWindow` can be improved in a similar way to fix your issue.
I tried a quick change:
```diff
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
index 3e3550d5da..e629ccc268 100644
--- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
+++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
@@ -991,13 +991,24 @@ object TransposeWindow extends Rule[LogicalPlan] {
})
}
+ private def windowsCompatible(w1: Window, w2: Window): Boolean = {
+ w1.references.intersect(w2.windowOutputSet).isEmpty &&
+ w1.expressions.forall(_.deterministic) &&
+ w2.expressions.forall(_.deterministic) &&
+ compatiblePartitions(w1.partitionSpec, w2.partitionSpec)
+ }
+
def apply(plan: LogicalPlan): LogicalPlan = plan transformUp {
- case w1 @ Window(we1, ps1, os1, w2 @ Window(we2, ps2, os2, grandChild))
- if w1.references.intersect(w2.windowOutputSet).isEmpty &&
- w1.expressions.forall(_.deterministic) &&
- w2.expressions.forall(_.deterministic) &&
- compatiblePartitions(ps1, ps2) =>
- Project(w1.output, Window(we2, ps2, os2, Window(we1, ps1, os1, grandChild)))
+ case w1 @ Window(_, _, _, w2 @ Window(_, _, _, grandChild))
+ if windowsCompatible(w1, w2) =>
+ Project(w1.output, w2.copy(child = w1.copy(child = grandChild)))
+
+ case w1 @ Window(_, _, _, Project(pl, w2 @ Window(_, _, _, grandChild)))
+ if windowsCompatible(w1, w2) && w1.references.subsetOf(grandChild.outputSet) =>
+ Project(
+ pl ++ w1.windowOutputSet,
+ w2.copy(child = w1.copy(child = grandChild))
+ )
}
}
```
And it changes the TPC-DS q47 and TPC-DS q57 in the same way your PR does, but I find this change to be more robust.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-804363204
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136354/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] tanelk commented on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
tanelk commented on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-803459084
Reordering the optimizers seems like a bit of a band-aid fix.
If I'm not mistaken the issue is similar to the one I tried to fix in #31677. The `CollapseWindow` rule did not work with `Project` nodes between windows. It seems, that the `TransposeWindow` has a similar problem.
@cloud-fan did comment (https://github.com/apache/spark/pull/31677#issuecomment-800894675), that perhaps we should merge `Project` and `Window` nodes. But that seems like a major refacto of everything related to the logical `Window` nodes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-804363204
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136354/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum closed pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
wangyum closed pull request #31907:
URL: https://github.com/apache/spark/pull/31907
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-804145945
**[Test build #136354 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136354/testReport)** for PR 31907 at commit [`0843b6b`](https://github.com/apache/spark/commit/0843b6ba6ab8c7a3df55a7bae11774001757ab4c).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31907: [SPARK-34807][SQL] Push down filter through window after TransposeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-803275218
**[Test build #136281 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136281/testReport)** for PR 31907 at commit [`4622aeb`](https://github.com/apache/spark/commit/4622aeb2733ddcb22373e2e37af7c172e96c5814).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on a change in pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
wangyum commented on a change in pull request #31907:
URL: https://github.com/apache/spark/pull/31907#discussion_r598090539
##########
File path: sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q47.sf100/explain.txt
##########
@@ -1,62 +1,61 @@
== Physical Plan ==
-TakeOrderedAndProject (58)
-+- * Project (57)
- +- * SortMergeJoin Inner (56)
- :- * Project (48)
- : +- * SortMergeJoin Inner (47)
- : :- * Sort (37)
- : : +- Exchange (36)
- : : +- * Filter (35)
- : : +- Window (34)
- : : +- * Sort (33)
- : : +- Exchange (32)
- : : +- * Project (31)
- : : +- Window (30)
- : : +- * Sort (29)
- : : +- Exchange (28)
- : : +- * HashAggregate (27)
- : : +- Exchange (26)
- : : +- * HashAggregate (25)
- : : +- * Project (24)
- : : +- * SortMergeJoin Inner (23)
- : : :- * Sort (17)
- : : : +- Exchange (16)
- : : : +- * Project (15)
- : : : +- * BroadcastHashJoin Inner BuildRight (14)
- : : : :- * Project (9)
- : : : : +- * BroadcastHashJoin Inner BuildRight (8)
- : : : : :- * Filter (3)
- : : : : : +- * ColumnarToRow (2)
- : : : : : +- Scan parquet default.store_sales (1)
- : : : : +- BroadcastExchange (7)
- : : : : +- * Filter (6)
- : : : : +- * ColumnarToRow (5)
- : : : : +- Scan parquet default.date_dim (4)
- : : : +- BroadcastExchange (13)
- : : : +- * Filter (12)
- : : : +- * ColumnarToRow (11)
- : : : +- Scan parquet default.store (10)
- : : +- * Sort (22)
- : : +- Exchange (21)
- : : +- * Filter (20)
- : : +- * ColumnarToRow (19)
- : : +- Scan parquet default.item (18)
- : +- * Sort (46)
- : +- Exchange (45)
- : +- * Project (44)
- : +- * Filter (43)
- : +- Window (42)
- : +- * Sort (41)
- : +- Exchange (40)
- : +- * HashAggregate (39)
- : +- ReusedExchange (38)
- +- * Sort (55)
- +- Exchange (54)
- +- * Project (53)
- +- * Filter (52)
- +- Window (51)
- +- * Sort (50)
- +- ReusedExchange (49)
+TakeOrderedAndProject (57)
++- * Project (56)
+ +- * SortMergeJoin Inner (55)
+ :- * Project (47)
+ : +- * SortMergeJoin Inner (46)
+ : :- * Sort (36)
+ : : +- Exchange (35)
+ : : +- * Project (34)
+ : : +- * Filter (33)
+ : : +- Window (32)
+ : : +- * Filter (31)
Review comment:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-803297339
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40864/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-803275218
**[Test build #136281 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136281/testReport)** for PR 31907 at commit [`4622aeb`](https://github.com/apache/spark/commit/4622aeb2733ddcb22373e2e37af7c172e96c5814).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-804205292
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40938/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-804212289
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40938/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-803314133
**[Test build #136281 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136281/testReport)** for PR 31907 at commit [`4622aeb`](https://github.com/apache/spark/commit/4622aeb2733ddcb22373e2e37af7c172e96c5814).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-803281412
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40863/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-803298520
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40864/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-803290821
**[Test build #136282 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136282/testReport)** for PR 31907 at commit [`1dc44de`](https://github.com/apache/spark/commit/1dc44de9840b97ee853f42036884f147138fe8e4).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-804212337
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40938/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-803282715
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40863/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31907: [SPARK-34807][SQL] Move TransposeWindow before Operator push down
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31907:
URL: https://github.com/apache/spark/pull/31907#issuecomment-803388197
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136282/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org