[GitHub] [iceberg] uncleGen commented on a diff in pull request #4596: Use bounded queue to avoid consuming too much memory

[GitBox <gi...@apache.org>]

uncleGen commented on code in PR #4596:
URL: https://github.com/apache/iceberg/pull/4596#discussion_r855885750


##########
core/src/main/java/org/apache/iceberg/util/ParallelIterable.java:
##########
@@ -65,10 +67,10 @@ private ParallelIterator(Iterable<? extends Iterable<T>> iterables,
             try (Closeable ignored = (iterable instanceof Closeable) ?
                 (Closeable) iterable : () -> { }) {
               for (T item : iterable) {
-                queue.add(item);
+                queue.put(item);
               }
-            } catch (IOException e) {
-              throw new RuntimeIOException(e, "Failed to close iterable");
+            } catch (IOException | InterruptedException e) {
+              throw new RuntimeException("Failed to close iterable", e);

Review Comment:
   `InterruptedException` is thrown by `BlockingQueue.put()`.  Using `put()` but not `add()` is because `put()` will , waiting if necessary for space to become available.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org