[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #4704: Spark: Support parallel delete files when abort

[GitBox <gi...@apache.org>]

RussellSpitzer commented on code in PR #4704:
URL: https://github.com/apache/iceberg/pull/4704#discussion_r869414980


##########
spark/v3.2/spark/src/main/java/org/apache/iceberg/spark/source/SparkWrite.java:
##########
@@ -213,6 +214,7 @@ private void abort(WriterCommitMessage[] messages) {
     if (cleanupOnAbort) {
       Map<String, String> props = table.properties();
       Tasks.foreach(files(messages))
+          .executeWith(ThreadPools.getWorkerPool())

Review Comment:
   Seems fine to me, although I really wish we had a more discrete pool for things like this. I find that our worker pool is hard to configure for end users.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org