You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/02/17 03:49:33 UTC
[GitHub] [iceberg] yittg opened a new pull request #4146: Support planning with particular executor for table scan
yittg opened a new pull request #4146:
URL: https://github.com/apache/iceberg/pull/4146
parts of #4145
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a change in pull request #4146: Support planning with particular executor for table scan
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #4146:
URL: https://github.com/apache/iceberg/pull/4146#discussion_r809331653
##########
File path: core/src/main/java/org/apache/iceberg/DataTableScan.java
##########
@@ -94,7 +93,7 @@ protected TableScan newRefinedScan(TableOperations ops, Table table, Schema sche
}
if (PLAN_SCANS_WITH_WORKER_POOL && snapshot.dataManifests().size() > 1) {
- manifestGroup = manifestGroup.planWith(ThreadPools.getWorkerPool());
+ manifestGroup = manifestGroup.planWith(context().planExecutor());
Review comment:
I think the logic here needs to be slightly different. If `planWith` was called, then the environment option `PLAN_SCANS_WITH_WORKER_POOL` should be ignored.
```java
if (snapshot.dataManifests().size() > 1 &&
(PLAN_SCANS_WITH_WORKER_POOL || context().planExecutor() != ThreadPools.getWorkerPool())) {
manifestGroup.planWith(context().planExecutor());
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a change in pull request #4146: Support planning with particular executor for table scan
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #4146:
URL: https://github.com/apache/iceberg/pull/4146#discussion_r809333698
##########
File path: api/src/main/java/org/apache/iceberg/TableScan.java
##########
@@ -159,6 +160,15 @@ default TableScan select(String... columns) {
*/
TableScan appendsAfter(long fromSnapshotId);
+ /**
+ * Create a new {@link TableScan} to use a particular executor to plan. The default worker pool will be
+ * used by default.
+ *
+ * @param executorService the provided executor.
Review comment:
Nit: no need to add end punctuation to these descriptions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #4146: Support planning with particular executor for table scan
Posted by GitBox <gi...@apache.org>.
rdblue commented on pull request #4146:
URL: https://github.com/apache/iceberg/pull/4146#issuecomment-1043259837
Thanks, @yittg! Looks really close.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #4146: Support planning with particular executor for table scan
Posted by GitBox <gi...@apache.org>.
rdblue commented on pull request #4146:
URL: https://github.com/apache/iceberg/pull/4146#issuecomment-1046274818
Thanks, @yittg!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue merged pull request #4146: Support planning with particular executor for table scan
Posted by GitBox <gi...@apache.org>.
rdblue merged pull request #4146:
URL: https://github.com/apache/iceberg/pull/4146
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a change in pull request #4146: Support planning with particular executor for table scan
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #4146:
URL: https://github.com/apache/iceberg/pull/4146#discussion_r809333441
##########
File path: api/src/main/java/org/apache/iceberg/TableScan.java
##########
@@ -159,6 +160,15 @@ default TableScan select(String... columns) {
*/
TableScan appendsAfter(long fromSnapshotId);
+ /**
+ * Create a new {@link TableScan} to use a particular executor to plan. The default worker pool will be
+ * used by default.
+ *
+ * @param executorService the provided executor.
+ * @return a table scan which use the provided executor to access manifests.
Review comment:
Typo: should be "that uses" not "which use"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] yittg commented on pull request #4146: Support planning with particular executor for table scan
Posted by GitBox <gi...@apache.org>.
yittg commented on pull request #4146:
URL: https://github.com/apache/iceberg/pull/4146#issuecomment-1043758954
@rdblue I've appended some fix changes, PTAL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org