You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/07/16 16:00:33 UTC
[GitHub] [spark] wangyum opened a new pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
wangyum opened a new pull request #33397:
URL: https://github.com/apache/spark/pull/33397
### What changes were proposed in this pull request?
Push down limit 1 through `Aggregate` if it is group only. For example:
```sql
create table t1 using parquet as select id from range(100000000L);
create table t2 using parquet as select id from range(100000000L);
create view v1 as select * from t1 union select * from t2;
select * from v1 limit 1;
```
Before this PR | After this PR
-- | --
 | 
### Why are the changes needed?
Improve query performance.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Unit test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on a change in pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
wangyum commented on a change in pull request #33397:
URL: https://github.com/apache/spark/pull/33397#discussion_r672410924
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
##########
@@ -647,6 +647,11 @@ object LimitPushDown extends Rule[LogicalPlan] {
// There is a Project between LocalLimit and Join if they do not have the same output.
case LocalLimit(exp, project @ Project(_, join: Join)) =>
LocalLimit(exp, project.copy(child = pushLocalLimitThroughJoin(exp, join)))
+ // Push down limit 1 through Aggregate if it is group only.
Review comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882565744
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882777963
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141258/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882777963
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141258/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882565744
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #33397:
URL: https://github.com/apache/spark/pull/33397#discussion_r672106833
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
##########
@@ -647,6 +647,11 @@ object LimitPushDown extends Rule[LogicalPlan] {
// There is a Project between LocalLimit and Join if they do not have the same output.
case LocalLimit(exp, project @ Project(_, join: Join)) =>
LocalLimit(exp, project.copy(child = pushLocalLimitThroughJoin(exp, join)))
+ // Push down limit 1 through Aggregate if it is group only.
Review comment:
seems we can also turn Aggregate into Project in this case?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882658234
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-881611263
Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45676/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #33397:
URL: https://github.com/apache/spark/pull/33397#discussion_r672106833
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
##########
@@ -647,6 +647,11 @@ object LimitPushDown extends Rule[LogicalPlan] {
// There is a Project between LocalLimit and Join if they do not have the same output.
case LocalLimit(exp, project @ Project(_, join: Join)) =>
LocalLimit(exp, project.copy(child = pushLocalLimitThroughJoin(exp, join)))
+ // Push down limit 1 through Aggregate if it is group only.
Review comment:
seems we can also turn Aggregate into Project in this case?
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
##########
@@ -647,6 +647,11 @@ object LimitPushDown extends Rule[LogicalPlan] {
// There is a Project between LocalLimit and Join if they do not have the same output.
case LocalLimit(exp, project @ Project(_, join: Join)) =>
LocalLimit(exp, project.copy(child = pushLocalLimitThroughJoin(exp, join)))
+ // Push down limit 1 through Aggregate if it is group only.
Review comment:
can we mention it in the comment?
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
##########
@@ -647,6 +647,11 @@ object LimitPushDown extends Rule[LogicalPlan] {
// There is a Project between LocalLimit and Join if they do not have the same output.
case LocalLimit(exp, project @ Project(_, join: Join)) =>
LocalLimit(exp, project.copy(child = pushLocalLimitThroughJoin(exp, join)))
+ // Push down limit 1 and turn Aggregate into Project through Aggregate if it is group only.
Review comment:
```suggestion
// Push down limit 1 through Aggregate and turn Aggregate into Project if it is group only.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882565744
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882658234
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45772/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-881577532
**[Test build #141165 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141165/testReport)** for PR 33397 at commit [`11c266b`](https://github.com/apache/spark/commit/11c266b4db7c515c24aa9112a07655cb0a09f850).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882658234
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882720292
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45780/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882658234
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45772/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882658234
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on a change in pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
wangyum commented on a change in pull request #33397:
URL: https://github.com/apache/spark/pull/33397#discussion_r672410924
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
##########
@@ -647,6 +647,11 @@ object LimitPushDown extends Rule[LogicalPlan] {
// There is a Project between LocalLimit and Join if they do not have the same output.
case LocalLimit(exp, project @ Project(_, join: Join)) =>
LocalLimit(exp, project.copy(child = pushLocalLimitThroughJoin(exp, join)))
+ // Push down limit 1 through Aggregate if it is group only.
Review comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882565744
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-883378804
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141311/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-883182421
**[Test build #141311 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141311/testReport)** for PR 33397 at commit [`1d60d64`](https://github.com/apache/spark/commit/1d60d648aea63000366aa1a51bb5d62dc29e27dd).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882661289
**[Test build #141266 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141266/testReport)** for PR 33397 at commit [`ec3df29`](https://github.com/apache/spark/commit/ec3df298047861d8b4afd3fb764a4ec853d0d3ea).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #33397:
URL: https://github.com/apache/spark/pull/33397#discussion_r672389513
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
##########
@@ -647,6 +647,11 @@ object LimitPushDown extends Rule[LogicalPlan] {
// There is a Project between LocalLimit and Join if they do not have the same output.
case LocalLimit(exp, project @ Project(_, join: Join)) =>
LocalLimit(exp, project.copy(child = pushLocalLimitThroughJoin(exp, join)))
+ // Push down limit 1 through Aggregate if it is group only.
Review comment:
can we mention it in the comment?
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
##########
@@ -647,6 +647,11 @@ object LimitPushDown extends Rule[LogicalPlan] {
// There is a Project between LocalLimit and Join if they do not have the same output.
case LocalLimit(exp, project @ Project(_, join: Join)) =>
LocalLimit(exp, project.copy(child = pushLocalLimitThroughJoin(exp, join)))
+ // Push down limit 1 and turn Aggregate into Project through Aggregate if it is group only.
Review comment:
```suggestion
// Push down limit 1 through Aggregate and turn Aggregate into Project if it is group only.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882658234
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-883296967
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45825/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882633984
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45772/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-883377174
**[Test build #141311 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141311/testReport)** for PR 33397 at commit [`1d60d64`](https://github.com/apache/spark/commit/1d60d648aea63000366aa1a51bb5d62dc29e27dd).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-883378804
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141311/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-881577532
**[Test build #141165 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141165/testReport)** for PR 33397 at commit [`11c266b`](https://github.com/apache/spark/commit/11c266b4db7c515c24aa9112a07655cb0a09f850).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on a change in pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
wangyum commented on a change in pull request #33397:
URL: https://github.com/apache/spark/pull/33397#discussion_r672410924
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
##########
@@ -647,6 +647,11 @@ object LimitPushDown extends Rule[LogicalPlan] {
// There is a Project between LocalLimit and Join if they do not have the same output.
case LocalLimit(exp, project @ Project(_, join: Join)) =>
LocalLimit(exp, project.copy(child = pushLocalLimitThroughJoin(exp, join)))
+ // Push down limit 1 through Aggregate if it is group only.
Review comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882850318
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141266/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882773242
**[Test build #141258 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141258/testReport)** for PR 33397 at commit [`c630a63`](https://github.com/apache/spark/commit/c630a6344f8df95cea739f71ae81821555bde8d1).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-883294292
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45825/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-881734535
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141165/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882565744
**[Test build #141258 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141258/testReport)** for PR 33397 at commit [`c630a63`](https://github.com/apache/spark/commit/c630a6344f8df95cea739f71ae81821555bde8d1).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-883267292
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45825/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-883296967
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45825/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882840182
**[Test build #141266 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141266/testReport)** for PR 33397 at commit [`ec3df29`](https://github.com/apache/spark/commit/ec3df298047861d8b4afd3fb764a4ec853d0d3ea).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882565744
**[Test build #141258 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141258/testReport)** for PR 33397 at commit [`c630a63`](https://github.com/apache/spark/commit/c630a6344f8df95cea739f71ae81821555bde8d1).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882694557
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45780/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882740641
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45780/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-881611290
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45676/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882607204
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45772/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882740641
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45780/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #33397:
URL: https://github.com/apache/spark/pull/33397#discussion_r672880818
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
##########
@@ -647,6 +647,11 @@ object LimitPushDown extends Rule[LogicalPlan] {
// There is a Project between LocalLimit and Join if they do not have the same output.
case LocalLimit(exp, project @ Project(_, join: Join)) =>
LocalLimit(exp, project.copy(child = pushLocalLimitThroughJoin(exp, join)))
+ // Push down limit 1 and turn Aggregate into Project through Aggregate if it is group only.
Review comment:
```suggestion
// Push down limit 1 through Aggregate and turn Aggregate into Project if it is group only.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882850318
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141266/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum closed pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
wangyum closed pull request #33397:
URL: https://github.com/apache/spark/pull/33397
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-881733744
**[Test build #141165 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141165/testReport)** for PR 33397 at commit [`11c266b`](https://github.com/apache/spark/commit/11c266b4db7c515c24aa9112a07655cb0a09f850).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
wangyum commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-886352350
The benchmark result:
Before this PR | After this PR
-- | --
 | 
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
wangyum commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-883351030
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-882661289
**[Test build #141266 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141266/testReport)** for PR 33397 at commit [`ec3df29`](https://github.com/apache/spark/commit/ec3df298047861d8b4afd3fb764a4ec853d0d3ea).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-881734535
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141165/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #33397:
URL: https://github.com/apache/spark/pull/33397#discussion_r672389513
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
##########
@@ -647,6 +647,11 @@ object LimitPushDown extends Rule[LogicalPlan] {
// There is a Project between LocalLimit and Join if they do not have the same output.
case LocalLimit(exp, project @ Project(_, join: Join)) =>
LocalLimit(exp, project.copy(child = pushLocalLimitThroughJoin(exp, join)))
+ // Push down limit 1 through Aggregate if it is group only.
Review comment:
can we mention it in the comment?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-883182421
**[Test build #141311 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141311/testReport)** for PR 33397 at commit [`1d60d64`](https://github.com/apache/spark/commit/1d60d648aea63000366aa1a51bb5d62dc29e27dd).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33397: [SPARK-36183][SQL] Push down limit 1 through Aggregate if it is group only
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33397:
URL: https://github.com/apache/spark/pull/33397#issuecomment-881611290
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45676/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org