You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/02/11 02:04:41 UTC
[GitHub] [spark] fuwhu opened a new pull request #27535: Add class document
for PruneFileSourcePartitions and PruneHiveTablePartitions
fuwhu opened a new pull request #27535: Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535
### What changes were proposed in this pull request?
Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions.
### Why are the changes needed?
To describe these two classes.
### Does this PR introduce any user-facing change?
no
### How was this patch tested?
no
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #27535: [MINOR][DOC] Add
class document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27535: [MINOR][DOC] Add
class document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584538654
**[Test build #118225 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118225/testReport)** for PR 27535 at commit [`5274b1a`](https://github.com/apache/spark/commit/5274b1adae3ea7cd99ea4bc3bfe51e2a42cd2984).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27535:
[MINOR][DOC] Add class document for PruneFileSourcePartitions and
PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584495646
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27535: [MINOR][DOC] Add
class document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584484101
**[Test build #118208 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118208/testReport)** for PR 27535 at commit [`5274b1a`](https://github.com/apache/spark/commit/5274b1adae3ea7cd99ea4bc3bfe51e2a42cd2984).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27535: [MINOR][DOC] Add class
document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584495209
**[Test build #118194 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118194/testReport)** for PR 27535 at commit [`23813ec`](https://github.com/apache/spark/commit/23813ec9170f5f3ffb97b77727edf65d1f02fb18).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27535: [MINOR][DOC] Add class
document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584516999
**[Test build #118208 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118208/testReport)** for PR 27535 at commit [`5274b1a`](https://github.com/apache/spark/commit/5274b1adae3ea7cd99ea4bc3bfe51e2a42cd2984).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] fuwhu commented on issue #27535: [MINOR][DOC] Add class
document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
fuwhu commented on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584889718
thank you all for help.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27535: Add class document
for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27535: Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584449138
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22956/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27535: [MINOR][DOC] Add
class document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584639936
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118225/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27535: [MINOR][DOC] Add
class document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584539316
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22985/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27535: Add class
document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27535: Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584449138
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22956/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27535:
[MINOR][DOC] Add class document for PruneFileSourcePartitions and
PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584539316
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22985/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27535: [MINOR][DOC] Add class
document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584536623
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27535: [MINOR][DOC] Add class
document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584538654
**[Test build #118225 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118225/testReport)** for PR 27535 at commit [`5274b1a`](https://github.com/apache/spark/commit/5274b1adae3ea7cd99ea4bc3bfe51e2a42cd2984).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27535: [MINOR][DOC] Add
class document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584484362
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22968/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27535:
[MINOR][DOC] Add class document for PruneFileSourcePartitions and
PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584495650
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118194/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] fuwhu commented on a change in pull request #27535:
[MINOR][DOC] Add class document for PruneFileSourcePartitions and
PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
fuwhu commented on a change in pull request #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#discussion_r377451447
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PruneFileSourcePartitions.scala
##########
@@ -26,6 +26,18 @@ import org.apache.spark.sql.catalyst.rules.Rule
import org.apache.spark.sql.execution.datasources.v2.{DataSourceV2ScanRelation, FileScan}
import org.apache.spark.sql.types.StructType
+/**
+ * Prune the partitions of file source based table using partition filters. Currently, this rule
+ * is applied to [[HadoopFsRelation]] and [[DataSourceV2ScanRelation]].
Review comment:
@cloud-fan updated as :
DataSourceV2ScanRelation -> DataSourceV2ScanRelation with FileScan
HadoopFsRelation -> HadoopFsRelation with CatalogFileIndex
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27535:
[MINOR][DOC] Add class document for PruneFileSourcePartitions and
PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584484357
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27535: [MINOR][DOC] Add class
document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584484101
**[Test build #118208 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118208/testReport)** for PR 27535 at commit [`5274b1a`](https://github.com/apache/spark/commit/5274b1adae3ea7cd99ea4bc3bfe51e2a42cd2984).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27535: Add class document
for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27535: Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584449134
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27535:
[MINOR][DOC] Add class document for PruneFileSourcePartitions and
PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584484362
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22968/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27535:
[MINOR][DOC] Add class document for PruneFileSourcePartitions and
PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584517284
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118208/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27535:
[MINOR][DOC] Add class document for PruneFileSourcePartitions and
PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584517282
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27535: [MINOR][DOC] Add
class document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584517284
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118208/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27535: [MINOR][DOC] Add class
document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584639018
**[Test build #118225 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118225/testReport)** for PR 27535 at commit [`5274b1a`](https://github.com/apache/spark/commit/5274b1adae3ea7cd99ea4bc3bfe51e2a42cd2984).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27535: [MINOR][DOC] Add
class document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584495646
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27535: [MINOR][DOC] Add class
document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584655932
thanks, merging to master/3.0!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27535: Add
class document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27535: Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#discussion_r377439182
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PruneFileSourcePartitions.scala
##########
@@ -26,6 +26,18 @@ import org.apache.spark.sql.catalyst.rules.Rule
import org.apache.spark.sql.execution.datasources.v2.{DataSourceV2ScanRelation, FileScan}
import org.apache.spark.sql.types.StructType
+/**
+ * Prune the partitions of file source based table using partition filters. Currently, this rule
+ * is applied to [[HadoopFsRelation]] and [[DataSourceV2ScanRelation]].
Review comment:
`DataSourceV2ScanRelation` -> `DataSourceV2ScanRelation with FileScan`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27535:
[MINOR][DOC] Add class document for PruneFileSourcePartitions and
PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584639936
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118225/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27535: Add
class document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27535: Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#discussion_r377439182
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PruneFileSourcePartitions.scala
##########
@@ -26,6 +26,18 @@ import org.apache.spark.sql.catalyst.rules.Rule
import org.apache.spark.sql.execution.datasources.v2.{DataSourceV2ScanRelation, FileScan}
import org.apache.spark.sql.types.StructType
+/**
+ * Prune the partitions of file source based table using partition filters. Currently, this rule
+ * is applied to [[HadoopFsRelation]] and [[DataSourceV2ScanRelation]].
Review comment:
`DataSourceV2ScanRelation` -> `FileScan`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27535:
[MINOR][DOC] Add class document for PruneFileSourcePartitions and
PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584539308
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27535: [MINOR][DOC] Add
class document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584539308
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27535:
[MINOR][DOC] Add class document for PruneFileSourcePartitions and
PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584639920
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27535: [MINOR][DOC] Add
class document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584448855
**[Test build #118194 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118194/testReport)** for PR 27535 at commit [`23813ec`](https://github.com/apache/spark/commit/23813ec9170f5f3ffb97b77727edf65d1f02fb18).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27535: [MINOR][DOC] Add
class document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584639920
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27535: Add class
document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27535: Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584449134
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27535: [MINOR][DOC] Add
class document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584484357
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27535: Add class document for
PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27535: Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584448855
**[Test build #118194 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118194/testReport)** for PR 27535 at commit [`23813ec`](https://github.com/apache/spark/commit/23813ec9170f5f3ffb97b77727edf65d1f02fb18).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27535: [MINOR][DOC] Add
class document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584495650
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118194/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27535: [MINOR][DOC] Add
class document for PruneFileSourcePartitions and PruneHiveTablePartitions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27535: [MINOR][DOC] Add class document for PruneFileSourcePartitions and PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/27535#issuecomment-584517282
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org