You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/07/19 10:36:41 UTC
[GitHub] [iceberg] nastra opened a new pull request, #5306: API: CloseableIterable.concat evaluates first item twice
nastra opened a new pull request, #5306:
URL: https://github.com/apache/iceberg/pull/5306
While working on Scan reporting (#5268) I noticed that this entire chain https://github.com/apache/iceberg/blob/eff879daa492641ac8bc829e7c4625dcdfac05fa/core/src/main/java/org/apache/iceberg/ManifestGroup.java#L218-L237 would be evaluated twice.
It turns out we eventually use `CloseableIterable.concat` with those
`CloseableIterables` in
https://github.com/apache/iceberg/blob/eff879daa492641ac8bc829e7c4625dcdfac05fa/core/src/main/java/org/apache/iceberg/ManifestGroup.java#L198.
When `CloseableIterable.concat` would check for emptyness via
```
if (!iterables.hasNext()) {
return empty();
}
```
it would actually evaluate the first chain of Iterables twice.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] nastra commented on a diff in pull request #5306: API: CloseableIterable.concat evaluates first item twice
Posted by GitBox <gi...@apache.org>.
nastra commented on code in PR #5306:
URL: https://github.com/apache/iceberg/pull/5306#discussion_r924747190
##########
api/src/main/java/org/apache/iceberg/io/CloseableIterable.java:
##########
@@ -148,8 +143,6 @@ public CloseableIterator<E> iterator() {
private ConcatCloseableIterator(Iterable<CloseableIterable<E>> inputs) {
this.iterables = inputs.iterator();
- this.currentIterable = iterables.next();
- this.currentIterator = currentIterable.iterator();
Review Comment:
The init code here already implied that there would be at least one element, so this was necessary to make the code work even with empty iterables
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue merged pull request #5306: API: CloseableIterable.concat evaluates first item twice
Posted by GitBox <gi...@apache.org>.
rdblue merged PR #5306:
URL: https://github.com/apache/iceberg/pull/5306
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5306: API: CloseableIterable.concat evaluates first item twice
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5306:
URL: https://github.com/apache/iceberg/pull/5306#discussion_r924741448
##########
api/src/main/java/org/apache/iceberg/io/CloseableIterable.java:
##########
@@ -148,8 +143,6 @@ public CloseableIterator<E> iterator() {
private ConcatCloseableIterator(Iterable<CloseableIterable<E>> inputs) {
this.iterables = inputs.iterator();
- this.currentIterable = iterables.next();
- this.currentIterator = currentIterable.iterator();
Review Comment:
Why was it necessary to change how these are initialized? Isn't the problem that `iterator` is called twice on the `iterable` passed to `concat`? If that is avoided, then I think this can stay the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #5306: API: CloseableIterable.concat evaluates first item twice
Posted by GitBox <gi...@apache.org>.
rdblue commented on PR #5306:
URL: https://github.com/apache/iceberg/pull/5306#issuecomment-1189744433
Thanks, @nastra!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org