You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/07/15 11:39:36 UTC
[GitHub] [spark] HeartSaVioR opened a new pull request #33367: [SPARK-36157][SQL][SS] TimeWindow expression: apply filter before project
HeartSaVioR opened a new pull request #33367:
URL: https://github.com/apache/spark/pull/33367
### What changes were proposed in this pull request?
This PR proposes to change the application of the operators for TimeWindow, from project -> filter, to filter -> project.
Currently Spark applies project, and filter, while filter is not dependent on project. That said, if the input rows are going to be filtered out via filter predicate, applying projection on these input rows are simply waste of time.
### Why are the changes needed?
This is a simple improvement requiring changes from a couple of lines.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Existing tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33367: [SPARK-36157][SQL][SS] TimeWindow expression: apply filter before project
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33367:
URL: https://github.com/apache/spark/pull/33367#issuecomment-880845670
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141076/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33367: [SPARK-36157][SQL][SS] TimeWindow expression: apply filter before project
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33367:
URL: https://github.com/apache/spark/pull/33367#issuecomment-880625427
**[Test build #141076 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141076/testReport)** for PR 33367 at commit [`1441e99`](https://github.com/apache/spark/commit/1441e99f23f06cee8df52c03b92613bcb9c65e76).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33367: [SPARK-36157][SQL][SS] TimeWindow expression: apply filter before project
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33367:
URL: https://github.com/apache/spark/pull/33367#issuecomment-880699031
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45590/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33367: [SPARK-36157][SQL][SS] TimeWindow expression: apply filter before project
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33367:
URL: https://github.com/apache/spark/pull/33367#issuecomment-880832048
**[Test build #141076 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141076/testReport)** for PR 33367 at commit [`1441e99`](https://github.com/apache/spark/commit/1441e99f23f06cee8df52c03b92613bcb9c65e76).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33367: [SPARK-36157][SQL][SS] TimeWindow expression: apply filter before project
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33367:
URL: https://github.com/apache/spark/pull/33367#issuecomment-880687979
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45590/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on pull request #33367: [SPARK-36157][SQL][SS] TimeWindow expression: apply filter before project
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on pull request #33367:
URL: https://github.com/apache/spark/pull/33367#issuecomment-881138153
Thanks for reviewing and merging!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #33367: [SPARK-36157][SQL][SS] TimeWindow expression: apply filter before project
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #33367:
URL: https://github.com/apache/spark/pull/33367#issuecomment-880855782
Thanks. Merging to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya closed pull request #33367: [SPARK-36157][SQL][SS] TimeWindow expression: apply filter before project
Posted by GitBox <gi...@apache.org>.
viirya closed pull request #33367:
URL: https://github.com/apache/spark/pull/33367
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33367: [SPARK-36157][SQL][SS] TimeWindow expression: apply filter before project
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33367:
URL: https://github.com/apache/spark/pull/33367#issuecomment-880625427
**[Test build #141076 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141076/testReport)** for PR 33367 at commit [`1441e99`](https://github.com/apache/spark/commit/1441e99f23f06cee8df52c03b92613bcb9c65e76).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33367: [SPARK-36157][SQL][SS] TimeWindow expression: apply filter before project
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33367:
URL: https://github.com/apache/spark/pull/33367#issuecomment-880845670
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141076/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on pull request #33367: [SPARK-36157][SQL][SS] TimeWindow expression: apply filter before project
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on pull request #33367:
URL: https://github.com/apache/spark/pull/33367#issuecomment-880624963
cc. @viirya
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33367: [SPARK-36157][SQL][SS] TimeWindow expression: apply filter before project
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33367:
URL: https://github.com/apache/spark/pull/33367#issuecomment-880661078
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45590/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33367: [SPARK-36157][SQL][SS] TimeWindow expression: apply filter before project
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33367:
URL: https://github.com/apache/spark/pull/33367#issuecomment-880699031
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45590/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org