You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/09/22 22:32:48 UTC
[GitHub] [spark] huaxingao opened a new pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
huaxingao opened a new pull request #34073:
URL: https://github.com/apache/spark/pull/34073
### What changes were proposed in this pull request?
update java doc...
### Why are the changes needed?
to highlight the difference between this new interface `SupportsPushDownV2Filters` and the old one `SupportsPushDownFilters`
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Test not needed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34073:
URL: https://github.com/apache/spark/pull/34073#issuecomment-925495331
**[Test build #143532 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143532/testReport)** for PR 34073 at commit [`3a0052f`](https://github.com/apache/spark/commit/3a0052f2830ae1f31b92f0e2847937a359145477).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34073:
URL: https://github.com/apache/spark/pull/34073#issuecomment-925377369
**[Test build #143523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143523/testReport)** for PR 34073 at commit [`1014995`](https://github.com/apache/spark/commit/1014995820aa9871ed9ac823775dda41d5024299).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34073:
URL: https://github.com/apache/spark/pull/34073#issuecomment-925436696
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48031/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34073:
URL: https://github.com/apache/spark/pull/34073#issuecomment-925495331
**[Test build #143532 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143532/testReport)** for PR 34073 at commit [`3a0052f`](https://github.com/apache/spark/commit/3a0052f2830ae1f31b92f0e2847937a359145477).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34073:
URL: https://github.com/apache/spark/pull/34073#issuecomment-925477621
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143523/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34073:
URL: https://github.com/apache/spark/pull/34073#issuecomment-925632476
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143532/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34073:
URL: https://github.com/apache/spark/pull/34073#issuecomment-925436696
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48031/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34073:
URL: https://github.com/apache/spark/pull/34073#issuecomment-925477621
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143523/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34073:
URL: https://github.com/apache/spark/pull/34073#discussion_r714445044
##########
File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/SupportsPushDownV2Filters.java
##########
@@ -22,23 +22,26 @@
/**
* A mix-in interface for {@link ScanBuilder}. Data sources can implement this interface to
- * push down filters to the data source and reduce the size of the data to be read.
Review comment:
Let's only change the classdoc
```
push down V2 {@link Filter}s to ...
Note that, this interface is preferred over {@link SupportsPushDownFilters}, which uses V1 Filter and is less
efficient due to the internal -> external data conversion.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34073:
URL: https://github.com/apache/spark/pull/34073#issuecomment-925527538
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48040/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34073:
URL: https://github.com/apache/spark/pull/34073#issuecomment-925632476
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143532/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34073:
URL: https://github.com/apache/spark/pull/34073#issuecomment-925524049
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48040/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34073:
URL: https://github.com/apache/spark/pull/34073#issuecomment-925510025
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48040/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34073:
URL: https://github.com/apache/spark/pull/34073#issuecomment-925377369
**[Test build #143523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143523/testReport)** for PR 34073 at commit [`1014995`](https://github.com/apache/spark/commit/1014995820aa9871ed9ac823775dda41d5024299).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
sarutak commented on pull request #34073:
URL: https://github.com/apache/spark/pull/34073#issuecomment-925969333
This change seems to break the build.
```
[error] * internal -> external data conversion.
```
Please let me fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34073:
URL: https://github.com/apache/spark/pull/34073#issuecomment-925527538
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48040/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34073:
URL: https://github.com/apache/spark/pull/34073#issuecomment-925433580
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48031/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34073:
URL: https://github.com/apache/spark/pull/34073#issuecomment-925409330
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48031/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #34073:
URL: https://github.com/apache/spark/pull/34073#issuecomment-925937092
thanks, merging to master!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
huaxingao commented on pull request #34073:
URL: https://github.com/apache/spark/pull/34073#issuecomment-925943856
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #34073:
URL: https://github.com/apache/spark/pull/34073
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on a change in pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
huaxingao commented on a change in pull request #34073:
URL: https://github.com/apache/spark/pull/34073#discussion_r714356932
##########
File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/SupportsPushDownV2Filters.java
##########
@@ -22,23 +22,26 @@
/**
* A mix-in interface for {@link ScanBuilder}. Data sources can implement this interface to
- * push down filters to the data source and reduce the size of the data to be read.
+ * push down data source V2 filters {@link Filter} to the data source and reduce the size of
+ * the data to be read.
*
* @since 3.3.0
*/
@Evolving
public interface SupportsPushDownV2Filters extends ScanBuilder {
/**
- * Pushes down filters, and returns filters that need to be evaluated after scanning.
+ * Pushes down data source V2 filters, and returns V2 filters that need to be evaluated after
+ * scanning.
Review comment:
@cloud-fan Please let me know how you want to document this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34073:
URL: https://github.com/apache/spark/pull/34073#issuecomment-925630937
**[Test build #143532 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143532/testReport)** for PR 34073 at commit [`3a0052f`](https://github.com/apache/spark/commit/3a0052f2830ae1f31b92f0e2847937a359145477).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34073: [SPARK-36760][SQL][FOLLOWUP] Add interface SupportsPushDownV2Filters
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34073:
URL: https://github.com/apache/spark/pull/34073#issuecomment-925476776
**[Test build #143523 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143523/testReport)** for PR 34073 at commit [`1014995`](https://github.com/apache/spark/commit/1014995820aa9871ed9ac823775dda41d5024299).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org