You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/10/20 16:09:23 UTC
[GitHub] [spark] tanelk opened a new pull request #34339: [SPARK-37074][SQL] Push extra predicates through non-join
tanelk opened a new pull request #34339:
URL: https://github.com/apache/spark/pull/34339
### What changes were proposed in this pull request?
In the `Optimizer` partially push some predicates through a non-join nodes, that produce new columns: `Aggregate`, `Generate`, `Window`.
### Why are the changes needed?
Performance improvements
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
New UTs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] closed pull request #34339: [SPARK-37074][SQL] Push extra predicates through non-join
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #34339:
URL: https://github.com/apache/spark/pull/34339
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34339: [SPARK-37074][SQL] Push extra predicates through non-join
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34339:
URL: https://github.com/apache/spark/pull/34339#issuecomment-948058439
**[Test build #144466 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144466/testReport)** for PR 34339 at commit [`0c543b2`](https://github.com/apache/spark/commit/0c543b2ca99649c43b2b3d0ac9fd0eb8f51d790c).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34339: [SPARK-37074][SQL] Push extra predicates through non-join
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34339:
URL: https://github.com/apache/spark/pull/34339#issuecomment-947937820
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48939/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34339: [SPARK-37074][SQL] Push extra predicates through non-join
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34339:
URL: https://github.com/apache/spark/pull/34339#issuecomment-947844487
**[Test build #144466 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144466/testReport)** for PR 34339 at commit [`0c543b2`](https://github.com/apache/spark/commit/0c543b2ca99649c43b2b3d0ac9fd0eb8f51d790c).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34339: [SPARK-37074][SQL] Push extra predicates through non-join
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34339:
URL: https://github.com/apache/spark/pull/34339#issuecomment-948059466
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144466/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34339: [SPARK-37074][SQL] Push extra predicates through non-join
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34339:
URL: https://github.com/apache/spark/pull/34339#issuecomment-948059466
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144466/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34339: [SPARK-37074][SQL] Push extra predicates through non-join
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34339:
URL: https://github.com/apache/spark/pull/34339#issuecomment-947937820
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48939/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34339: [SPARK-37074][SQL] Push extra predicates through non-join
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34339:
URL: https://github.com/apache/spark/pull/34339#issuecomment-947889553
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48939/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] tanelk commented on pull request #34339: [SPARK-37074][SQL] Push extra predicates through non-join
Posted by GitBox <gi...@apache.org>.
tanelk commented on pull request #34339:
URL: https://github.com/apache/spark/pull/34339#issuecomment-948272878
@cloud-fan , could you take a quick look at this?
I'm not sure if it is best to keep it as a separate rule, or should this be just an improvement to the existing `PushPredicateThroughNonJoin` to avoid code duplication.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34339: [SPARK-37074][SQL] Push extra predicates through non-join
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34339:
URL: https://github.com/apache/spark/pull/34339#issuecomment-947921469
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48939/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34339: [SPARK-37074][SQL] Push extra predicates through non-join
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34339:
URL: https://github.com/apache/spark/pull/34339#issuecomment-947844487
**[Test build #144466 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144466/testReport)** for PR 34339 at commit [`0c543b2`](https://github.com/apache/spark/commit/0c543b2ca99649c43b2b3d0ac9fd0eb8f51d790c).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #34339: [SPARK-37074][SQL] Push extra predicates through non-join
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #34339:
URL: https://github.com/apache/spark/pull/34339#issuecomment-1025012531
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org