[GitHub] [iceberg] jackye1995 commented on a diff in pull request #5379: AWS: Use executor service by default when performing batch deletion of files

[GitBox <gi...@apache.org>]

jackye1995 commented on code in PR #5379:
URL: https://github.com/apache/iceberg/pull/5379#discussion_r938247269


##########
aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIO.java:
##########
@@ -167,6 +168,29 @@ public Map<String, String> properties() {
    */
   @Override
   public void deleteFiles(Iterable<String> paths) throws BulkDeletionFailureException {
+    deleteFilesWithBatchSize(paths, awsProperties.s3FileIoDeleteBatchSize());
+  }
+
+  /**
+   * Deletes the given paths in a batched manner with the given executor service
+   *
+   * @param paths paths to delete
+   * @param svc executor service to use for the deletion
+   */
+  @Override
+  public void deleteFiles(Iterable<String> paths, ExecutorService svc)
+      throws BulkDeletionFailureException {
+    Iterable<List<String>> pathBatches =
+        Iterables.partition(paths, awsProperties.s3FileIoDeleteBatchSize());
+    Tasks.foreach(pathBatches)
+        .noRetry()

Review Comment:
   yes I agree, then this looks good to me



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org