You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2020/10/28 22:04:11 UTC
[GitHub] [iceberg] rdblue opened a new pull request #1678: Do not create an Evaluator in ManifestGroup unless the file filter is used
rdblue opened a new pull request #1678:
URL: https://github.com/apache/iceberg/pull/1678
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] HeartSaVioR commented on a change in pull request #1678: Do not create an Evaluator in ManifestGroup unless the file filter is used
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on a change in pull request #1678:
URL: https://github.com/apache/iceberg/pull/1678#discussion_r514090835
##########
File path: core/src/main/java/org/apache/iceberg/ManifestGroup.java
##########
@@ -255,7 +260,7 @@ ManifestGroup planWith(ExecutorService newExecutorService) {
entry -> entry.status() != ManifestEntry.Status.EXISTING);
}
- if (fileFilter != null && fileFilter != Expressions.alwaysTrue()) {
+ if (evaluator != null) {
Review comment:
Would it work if we simply define the evaluator only needed, like below?
```
if (fileFilter != null && fileFilter != Expressions.alwaysTrue()) {
Evaluator evaluator = new Evaluator(DataFile.getType(EMPTY_STRUCT), fileFilter, caseSensitive);
entries = CloseableIterable.filter(entries,
entry -> evaluator.eval((GenericDataFile) entry.file()));
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a change in pull request #1678: Do not create an Evaluator in ManifestGroup unless the file filter is used
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #1678:
URL: https://github.com/apache/iceberg/pull/1678#discussion_r514439406
##########
File path: core/src/main/java/org/apache/iceberg/ManifestGroup.java
##########
@@ -255,7 +260,7 @@ ManifestGroup planWith(ExecutorService newExecutorService) {
entry -> entry.status() != ManifestEntry.Status.EXISTING);
}
- if (fileFilter != null && fileFilter != Expressions.alwaysTrue()) {
+ if (evaluator != null) {
Review comment:
We instantiate it above so that we only create one and share it for all of the evaluations. It just cuts down on Evaluator creation and garbage collection.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] aokolnychyi merged pull request #1678: Do not create an Evaluator in ManifestGroup unless the file filter is used
Posted by GitBox <gi...@apache.org>.
aokolnychyi merged pull request #1678:
URL: https://github.com/apache/iceberg/pull/1678
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org